CN115114250A - Log storage method and system, electronic equipment and storage medium - Google Patents

Log storage method and system, electronic equipment and storage medium Download PDF

Info

Publication number
CN115114250A
CN115114250A CN202210761297.8A CN202210761297A CN115114250A CN 115114250 A CN115114250 A CN 115114250A CN 202210761297 A CN202210761297 A CN 202210761297A CN 115114250 A CN115114250 A CN 115114250A
Authority
CN
China
Prior art keywords
log
storage
application
mode
stored
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210761297.8A
Other languages
Chinese (zh)
Inventor
戴娇
李光祝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Changan Automobile Co Ltd
Original Assignee
Chongqing Changan Automobile Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Changan Automobile Co Ltd filed Critical Chongqing Changan Automobile Co Ltd
Priority to CN202210761297.8A priority Critical patent/CN115114250A/en
Publication of CN115114250A publication Critical patent/CN115114250A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3476Data logging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/1805Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815Journaling file systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention provides a log storage method, a log storage system, an electronic device and a storage medium, wherein a log to be stored is obtained firstly, and then log specification is carried out on the log to be stored to obtain a specification log; carrying out data source distinguishing on the normalized logs, and determining the log types of the normalized logs; and finally, matching a log transmission mode according to the log type, and uploading the standard log to a preset storage area based on the matched log transmission mode, so that the standard storage is carried out on the log to be stored. This application can define the log standard that the car end was used, can solve prior art problem that is irregular when saving the log. Simultaneously, the application can respectively carry out more detailed, unified and specific specifications on the log format, the log grade, the log file storage, the log use and the cloud on the log in the log use specification, not only can carry out local storage on the log, but also can pull the log off-line to the cloud end for checking, thereby solving the problems of log storage, management and checking channels.

Description

Log storage method and system, electronic equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a log storage method, system, electronic device, and storage medium.
Background
With the increasing computing power of vehicle-end chips and the increasing number of vehicle-end applications, it is very necessary to standardize the printing and storing of vehicle-end application logs. However, in the prior art, the vehicle-side log storage method has the problems of non-uniform and non-standard log storage and management, and the existing vehicle-side log storage method has the problems that the logs cannot be normalized and the log use specification is not detailed, for example, a uniform log storage, management and viewing channel cannot be provided, and a long-time log storage channel cannot be provided for a MCU (micro controller Unit, abbreviated as MCU) with few resources.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, the present application provides a log storage method, system, electronic device and storage medium to solve the above technical problems.
The application provides a log storage method, which comprises the following steps:
acquiring a log to be stored, and carrying out log specification on the log to be stored to obtain a specification log;
carrying out data source distinguishing on the normative log, and determining the log type of the normative log;
and matching a log transmission mode according to the log type, and uploading the standard log to a preset storage area based on the matched log transmission mode so as to perform standard storage on the log to be stored.
In an embodiment of the present application, the log normalization is performed on the log to be stored, and a process of obtaining a normalized log includes:
acquiring preset log specification modes, and selecting at least one log specification mode from the preset log specification modes;
performing corresponding log specification on the log to be stored by using the selected log specification mode to obtain a specified log, and recording the specified log as a specification log;
the preset log specification mode comprises at least one of the following modes: log format specification, log level specification, log storage specification, log use specification, and log cloud specification.
In an embodiment of the present application, the data source distinguishing of the normative log and the process of determining the log type of the normative log include:
comparing the data source of the specification log with the data source of the application log to determine whether the specification log is the application log;
if the data source of the specification log is the same as the data source of the application log, determining the log type of the specification log as the application log;
and if the data source of the specification log is different from the data source of the application log, determining that the log type of the specification log is a non-application log.
In an embodiment of the present application, the process of matching the log transfer manner according to the log type includes:
acquiring the log type of the normalized log, and matching a log transmission mode based on the log type of the normalized log;
if the log type of the normalized log is an application log, the log transmission mode matched with the normalized log comprises a data distribution service mode;
and if the log type of the specification log is a non-application log, the log transmission mode matched with the specification log comprises a socket mode.
In an embodiment of the present application, the process of uploading the normative log to a preset storage area based on a matched log transfer manner includes:
if the log transmission mode matched with the application log is a data distribution service mode, uploading the application log to a local storage area and/or a cloud storage area based on the data distribution service mode;
and if the log transmission mode matched with the non-application log is a socket mode, uploading the non-application log to a local storage area and/or a cloud storage area based on the socket mode.
In an embodiment of the application, the process of uploading the normative log to a preset storage area based on the matched log transfer manner further includes:
uploading the application log to a log module based on the data distribution service mode, and uploading the non-application log to the log module based on the socket mode;
uploading the log received by the log module to a local storage area for local storage; and the number of the first and second groups,
and responding to the log viewing request, acquiring a log to be viewed, and uploading the log to be viewed to a cloud storage area in a data distribution service mode.
In an embodiment of the present application, the method further includes:
acquiring log software development kit dependency and log software development kit environment;
copying a log software development kit folder to a root directory of an application program item based on the log software development kit dependency and a log software development kit environment;
modifying the configuration file in the log software development kit, and storing the modified configuration file to a root directory of an application program item;
compiling a software package compiling process input file under the root directory of the application program item, compiling and applying the compiling result, and integrating the log module.
The present application further provides a log storage system, the system comprising:
the acquisition module is used for acquiring logs to be stored;
the normalization module is used for normalizing the logs to be stored to obtain normalized logs;
the distinguishing module is used for distinguishing data sources of the normative logs and determining the log types of the normative logs;
and the storage module is used for matching a log transmission mode according to the log type and uploading the standard log to a preset storage area based on the matched log transmission mode so as to carry out standard storage on the log to be stored.
The present application further provides an electronic device, the electronic device including:
one or more processors;
storage means for storing one or more programs that, when executed by the one or more processors, cause the electronic device to implement the log storage method as recited in any of the above.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to perform the log storing method as defined in any one of the above.
As described above, the present application provides a log storage method, system, electronic device, and storage medium, which have the following beneficial effects:
firstly, acquiring a log to be stored, and then carrying out log specification on the log to be stored to obtain a specification log; carrying out data source distinguishing on the normalized logs, and determining the log types of the normalized logs; and finally, matching a log transmission mode according to the log type, and uploading the standard log to a preset storage area based on the matched log transmission mode, so as to carry out standard storage on the log to be stored. The log specification of the vehicle end application can be defined by performing specification processing on the log to be stored, so that the problem that the log is not normal when stored in the prior art can be solved. The method for locally storing the vehicle-side log based on the service oriented architecture service (SOA for short) can process the vehicle-side log module more uniformly and standardly, enables the vehicle-side log to monitor and manage the vehicle more carefully and in place, and accordingly can guarantee the safety of drivers. Meanwhile, the log format, the log grade, the log file storage, the log use and the cloud on the log in the log use specification can be more carefully, uniformly and specifically specified, so that the use specification of a vehicle-side log SDK (Software Development Kit, SDK for short) is provided, the log can be locally stored, the log can be offline pulled to the cloud for viewing, and the problem that unified log storage, management and viewing channels cannot be provided in the prior art is solved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:
FIG. 1 is a schematic diagram of an exemplary system architecture to which one or more embodiments of the present application may be applied;
fig. 2 is a schematic flowchart of a log storage method according to an embodiment of the present application;
fig. 3 is a schematic architecture diagram of a log storage method according to an embodiment of the present application;
FIG. 4 is a schematic flow chart of an integrated logging module according to an embodiment of the present application;
FIG. 5 is a diagram illustrating a hardware structure of a journal storage system according to an embodiment of the present disclosure;
fig. 6 is a hardware configuration diagram of an electronic device suitable for implementing one or more embodiments of the present application.
Detailed Description
Other advantages and effects of the present application will become apparent to those skilled in the art from the disclosure herein, wherein the embodiments of the present application will be described in detail with reference to the accompanying drawings and preferred embodiments. The application is capable of other and different embodiments and its several details are capable of modifications and various changes in detail without departing from the spirit of the application. It should be understood that the preferred embodiments are for purposes of illustration only and are not intended to limit the scope of the present disclosure.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present application, and the drawings only show the components related to the present application and are not drawn according to the number, shape and size of the components in actual implementation, and the type, number and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present application, however, it will be apparent to one skilled in the art that the embodiments of the present application may be practiced without these specific details, and in other embodiments, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring the embodiments of the present application.
SDA platform: an intelligent automobile super-digital platform for user full-scene service.
SOA: a service-oriented architecture.
DDS: data Distribution Service, referred to as DDS for short.
And OS: operating System, abbreviated as OS.
VIU: vehicle Identification Unit, VIU for short.
An ECU: electronic Control Unit, abbreviated as ECU.
APP: application, Application program, APP for short.
DLT: diagnostic Log and Trace, DLT for short.
Fig. 1 shows a schematic diagram of an exemplary system architecture to which technical solutions in one or more embodiments of the present application may be applied. As shown in fig. 1, system architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include various electronic devices such as a smart phone, a tablet computer, a notebook computer, and a desktop computer. The server 130 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud computing services. Network 120 may be a communication medium of various connection types capable of providing a communication link between terminal device 110 and server 130, such as a wired communication link or a wireless communication link.
The system architecture in the embodiments of the present application may have any number of terminal devices, networks, and servers, according to implementation needs. For example, the server 130 may be a server group composed of a plurality of server devices. In addition, the technical solution provided in the embodiment of the present application may be applied to the terminal device 110, may also be applied to the server 130, or may be implemented by both the terminal device 110 and the server 130, which is not particularly limited in this application.
In an embodiment of the present application, the terminal device 110 or the server 130 of the present application may obtain a log to be stored, and perform log specification on the log to be stored to obtain a specification log; carrying out data source distinguishing on the normative log, and determining the log type of the normative log; and matching a log transmission mode according to the log type, and uploading the standard log to a preset storage area based on the matched log transmission mode so as to perform standard storage on the log to be stored. By using the terminal device 110 or the server 130 to execute the log storage method, the vehicle-side log module can be processed more uniformly and normatively, so that the vehicle-side log can supervise and manage the vehicle more carefully, and the safety of the vehicle-driving personnel can be ensured. The log format, the log grade, the log file storage, the log use and the cloud on the log in the log use specification can be specified in detail, unified and specific. The service specification of the vehicle end log SDK is equivalently provided, the log can be locally stored, and the log can be offline pulled to the cloud end to be checked, so that the problem that unified log storage, management and checking channels cannot be provided in the prior art is solved.
The above section introduces the content of an exemplary system architecture to which the technical solution of the present application is applied, and then continues to introduce the log storage method of the present application.
Fig. 2 shows a schematic flowchart of a log storage method according to an embodiment of the present application. Specifically, in an exemplary embodiment, as shown in fig. 2, the present embodiment provides a log storage method, which includes the following steps:
s210, acquiring a log to be stored;
s220, carrying out log specification on the log to be stored to obtain a specification log. Specifically, the log specification is performed on the log to be stored, and the process of obtaining a specification log includes: acquiring preset log specification modes, and selecting at least one log specification mode from the preset log specification modes; performing corresponding log specification on the log to be stored by using the selected log specification mode to obtain a specified log, and recording the specified log as a specification log; the preset log specification mode comprises at least one of the following modes: log format specification, log level specification, log storage specification, log use specification, and log cloud specification.
The Log format is based on a dlt (diagnostic Log and Trace) open source protocol, and complies with the AUTOSAR (automotive open system architecture) standard. Specifically, the specification of the log format of the log to be stored includes: time, timestamp, message counter, echo (controller id), appid (application id), contextid (context id), message type, level, miscellaneous pattern, number of parameter segments, and log content (log content).
Wherein, the log level definition is based on a dlt source opening protocol and follows AUTOSAR standard, and the log level comprises: VERBOSE (detailed information), DEBUG (DEBUG information), INFO (notification information), WARN (alarm information), ERROR (ERROR information), far (system ERROR). Specifically, the process of performing log level specification on the log to be stored includes:
a. applying the APP default log storage level as WARN (alarm);
b. no special condition exists, the level of the log stored at the vehicle end by applying the APP is required to be at or above the INFO;
c. under special conditions, when the log level below the INFO is set, the log output quantity needs to be evaluated;
d. when the log stream of the vehicle-end application is read through the diagnosis middleware, the grade of the currently read log stream can be set, but the grade of the vehicle-end log storage is not influenced.
The log storage specification comprises a log storage mode specification and a log storage position specification. Specifically, the specification of the log storage mode refers to storage in a file form, and each vehicle-side application can correspond to a plurality of log files. File storage involves 4 parameters: logTraceFileNum (the maximum storable number of log files), logTraceFilePath (the storage location of log files), logTraceFileSize (the maximum storage byte of a single log file), and logTraceLogLevel (the level of storage of log files). The log storage position specification means that the log module is adopted for storing and managing the application logs at the vehicle end, the storage position of a log file is default/log/, and the file name is APP name + number, if: log/APP/DIAG _1, and in addition, the definition of each APP name at the vehicle end cannot be repeated.
Wherein the log usage specification comprises a log frequency specification and a log content specification. Specifically, the log frequency specification means that the printing frequency of the application log of the vehicle end is not higher than 1/10 ms in principle, and if the log frequency is higher than the required frequency, the influence on a vehicle end chip needs to be evaluated by the user, then the user is informed to a log module management team, and verification and confirmation are carried out. The log content specification means that the log content needs to contain 3 parts, which are respectively: each request corresponds to a 32-bit random character string, a function name and log contents, and each part is divided by using English character split numbers.
The cloud on log specification comprises a cloud on log stream specification and a cloud on log file specification in real time. Specifically, the cloud specification of the real-time log stream means that the cloud mode of the real-time log stream mainly includes a mode of a log module client tool-dlt viewer and a mode of a cloud diagnostic instrument-WEB interface through a diagnostic module. The cloud specification on the log file means that the cloud mode on the log file mainly comprises a mode of a client tool-dlt viewer through a log module and a mode of a cloud diagnostic instrument-WEB interface through a diagnostic module. Wherein, the C call in the definition of the log module API mainly comprises: eight ways of logsdk _ init _ logger, logsdk _ log, logsdk _ verbose, logsdk _ debug, logsdk _ info, logsdk _ war, logsdk _ error, and logsdk _ false.
S230, distinguishing data sources of the normative logs, and determining the log types of the normative logs;
s240, matching a log transmission mode according to the log type, and uploading the standard log to a preset storage area based on the matched log transmission mode so as to perform standard storage on the log to be stored.
In an exemplary embodiment, the data source distinguishing of the normative log, and the determining of the log type of the normative log includes: comparing the data source of the specification log with the data source of the application log to determine whether the specification log is the application log; if the data source of the specification log is the same as the data source of the application log, determining the log type of the specification log as the application log; and if the data source of the specification log is different from the data source of the application log, determining that the log type of the specification log is a non-application log. In addition, the process of matching the log transfer mode according to the log type comprises the following steps: acquiring the log type of the normalized log, and matching a log transmission mode based on the log type of the normalized log; if the log type of the normalized log is an application log, the log transmission mode matched with the normalized log comprises a data distribution service mode; and if the log type of the specification log is a non-application log, the log transmission mode matched with the specification log comprises a socket mode.
According to the above description, in an exemplary embodiment, the process of uploading the normative log to a preset storage area based on the matched log transfer manner may include: if the log transmission mode matched with the application log is a data distribution service mode, uploading the application log to a local storage area and/or a cloud storage area based on the data distribution service mode; and if the log transmission mode matched with the non-application log is a socket mode, uploading the non-application log to a local storage area and/or a cloud storage area based on the socket mode. Specifically, when the log type of the specification log is an application log, the application log is uploaded in a local Socket mode; and when the log type of the specification log is a non-application log, uploading the application log in a data distribution service DDS mode.
According to the above description, in an exemplary embodiment, the process of uploading the normative log to a preset storage area based on the matched log transfer manner may further include: uploading the application log to a log module based on the data distribution service mode, and uploading the non-application log to the log module based on the socket mode; uploading the log received by the log module to a local storage area for local storage; and responding to the log viewing request, acquiring the log to be viewed, and uploading the log to be viewed to a cloud storage area in a data distribution service mode. Specifically, as shown in fig. 3, fig. 3 provides a frame schematic diagram for implementing a log storage method, and in fig. 3, a log module is a software application for uniformly managing logs, and is mainly responsible for collecting logs in real time and reporting log information to a DDS service to a data acquisition module; wherein, the reported log comprises: OS log, middleware log, application log, VIU log, and ECU log. The data acquisition module is mainly responsible for collecting logs in DDS service and uploading the logs to the cloud. The APP module is mainly responsible for the real-time log reporting of the SDK. In the log module framework, the whole process of local storage and offline pulling to cloud checking including the vehicle-side log is mainly as follows: when an integrated log SDK in the APP module is transmitted to the log module in a local Socket mode, other log information is transmitted to the log module in a DDS mode to be stored locally, the log module reports the related log information to the data acquisition module in a DDS service mode, and the data acquisition module directly uploads the log information in the DDS service to the cloud after receiving the log information, so that the function of offline drawing to the cloud for checking is realized. The log module specification in this embodiment is not only for log module development to define log format, log level, log storage, and cloud specification on the log, but also for the APP that needs to integrate the log SDK. Wherein the various modules in fig. 3 are explained as follows:
and (4) marking: and other logs are transmitted to a log module for storage in a DDS mode.
The label is two: and the APP application log is transmitted to a log module for storage in a local Socket mode.
Symbol (c): when the cloud diagnostic instrument checks the log stream of the corresponding APP, the log module sends the log information of the corresponding APP to the data acquisition module in a DDS mode and uploads the log information to the cloud. The log file viewing modes mainly include the following two modes: (1) windows operating system: view files using the open source dlt viewer tool (the tool will be provided with the document); (2) the Linux operating system comprises: the file class content is viewed using the open source dlt viewer command.
Label iv: the log module stores the received log information locally.
In addition, the log module integration flow in the present embodiment is shown in fig. 4. As can be seen from the description of fig. 4, the integration process of the log module includes: acquiring log software development kit dependency and log software development kit environment; copying a log software development kit folder to a root directory of an application program item based on the log software development kit dependency and a log software development kit environment; modifying the configuration file in the log software development kit, and storing the modified configuration file to a root directory of an application program item; compiling a software package compiling process input file under the root directory of the application program item, compiling and applying the compiling result, and integrating the log module. The log module integration mode can be described as follows: adding log SDK dependency and environment; adding a log SDK folder to the project; modifying a configuration file in the log SDK; compiling a CMakeLists.txt file; compiling the application; and generating a result. Specifically, the method comprises the following steps: the log module integration process is explained as follows:
step 1: add log SDK dependencies and contexts. Placing the SDK file SDK _20220114_ NeuSAR _ aCore _1911_27075_ jetson _ agx _ linux.sh on an arm64 board to be executed directly: /sdk _20220114_ NeuSAR _ aacre _1911_27075_ jetson _ agx _ linux.sh, performing the generation: export toolchain/environment. config.
And 2, step: add the log SDK folder to the project. And copying the log SDK folder logs to the root directory of the application APP project.
And step 3: and modifying the configuration file in the log SDK. Modifying the content to be configured in the configuration file log/etc/processlogconfig.json of the log SDK, and automatically placing the configuration file into an etc folder under the root directory of the application APP after compiling.
And 4, step 4: create cmakelist. The logic itself is written in the cmakelist.
And 5: and compiling the application.
Step 6: and generating a result. And generating a bin file and an etc file. Wherein, an executable file of the current project is placed in the bin file, and a former configuration file processlogconfig.
In summary, the present application provides a log storage method, which includes obtaining a log to be stored, and then performing log specification on the log to be stored to obtain a specification log; carrying out data source distinguishing on the normalized logs, and determining the log types of the normalized logs; and finally, matching a log transmission mode according to the log type, and uploading the standard log to a preset storage area based on the matched log transmission mode, so that the standard storage is carried out on the log to be stored. According to the method, the log to be stored is subjected to standard processing, the log specification applied by the vehicle end can be defined, and therefore the problem that the log is not standard when being stored in the prior art can be solved. The method provides a local storage method of the vehicle-side log based on the service-oriented architecture service, and can process the vehicle-side log module more uniformly and normatively, so that the vehicle-side log can supervise and manage the vehicle more carefully and properly, and the safety of the vehicle-driving personnel can be ensured more. Meanwhile, the method can respectively carry out more detailed, unified and specific specifications on the log format, the log grade, the log file storage, the log use and the cloud on the log in the log use specification, which is equivalent to providing the use specification of a vehicle-end log software installation kit, not only can carry out local storage on the log, but also can pull the log offline to the cloud end for checking, thereby solving the problem that the prior art can not provide a unified log storage, management and checking channel.
As shown in fig. 5, the present application further provides a log storage system, which includes:
an acquisition module 510, configured to acquire a log to be stored;
and the normalizing module 520 is configured to perform log normalization on the log to be stored to obtain a normalized log. Specifically, the log specification is performed on the log to be stored, and the process of obtaining a specification log includes: acquiring preset log specification modes, and selecting at least one log specification mode from the preset log specification modes; performing corresponding log specification on the log to be stored by using the selected log specification mode to obtain a specified log, and recording the specified log as a specification log; the preset log specification mode comprises at least one of the following modes: the log format specification, the log level specification, the log storage specification, the log use specification and the cloud on log specification.
The Log format is based on a dlt (diagnostic Log and Trace) open source protocol, and complies with the AUTOSAR (automotive open system architecture) standard. Specifically, the specification of the log format of the log to be stored includes: time, timestamp, message counter, echo (controller id), appid (application id), contextid (context id), message type, level, miscellaneous pattern, number of parameter segments, and log content (log content).
Wherein, the log level definition is based on a dlt source opening protocol and follows AUTOSAR standard, and the log level comprises: VERBOSE (detailed information), DEBUG (DEBUG information), INFO (notification information), WARN (alarm information), ERROR (ERROR information), far (system ERROR). Specifically, the process of performing log level specification on the log to be stored includes:
a. applying the APP default log storage level as WARN (alarm);
b. no special condition exists, the level of the log stored at the vehicle end by applying the APP is required to be at or above the INFO;
c. under special conditions, when the log level below the INFO is set, the log output quantity needs to be evaluated;
d. when the log stream of the vehicle-end application is read through the diagnosis middleware, the grade of the currently read log stream can be set, but the grade of the vehicle-end log storage is not influenced.
The log storage specification comprises a log storage mode specification and a log storage position specification. Specifically, the specification of the log storage mode refers to storage in a file form, and each vehicle-side application can correspond to a plurality of log files. File storage involves 4 parameters: logTraceFileNum (the maximum storable number of log files), logTraceFilePath (the storage location of log files), logTraceFileSize (the maximum storage byte of a single log file), and logTraceLogLevel (the level of storage of log files). The log storage position specification means that the log module is adopted for storing and managing the application logs at the vehicle end, the storage position of a log file is default/log/, and the file name is APP name + number, if: log/APP/DIAG _1, and moreover, each APP name definition at the vehicle end cannot be repeated.
Wherein the log usage specification comprises a log frequency specification and a log content specification. Specifically, the log frequency specification means that the printing frequency of the application log of the vehicle end is not higher than 1/10 ms in principle, and if the log frequency is higher than the required frequency, the influence on a vehicle end chip needs to be evaluated by the user, then the user is informed to a log module management team, and verification and confirmation are carried out. The log content specification means that the log content needs to contain 3 parts, which are respectively: each request corresponds to a 32-bit random character string, a function name and log contents, and each part is divided by using English character split numbers.
The cloud on log specification comprises a cloud on log stream specification and a cloud on log file specification in real time. Specifically, the cloud specification of the real-time log stream means that the cloud mode of the real-time log stream mainly includes a mode of a log module client tool-dlt viewer and a mode of a cloud diagnostic instrument-WEB interface through a diagnostic module. The cloud specification on the log file means that the cloud mode on the log file mainly comprises a mode of a client tool-dlt viewer through a log module and a mode of a cloud diagnostic instrument-WEB interface through a diagnostic module. Wherein, the C call in the definition of the log module API mainly comprises: eight ways of logsdk _ init _ logger, logsdk _ log, logsdk _ verbose, logsdk _ debug, logsdk _ info, logsdk _ war, logsdk _ error, and logsdk _ false.
A distinguishing module 530, configured to distinguish data sources of the normative log and determine a log type of the normative log;
and the storage module 540 is configured to match a log transfer manner according to the log type, and upload the regular log to a preset storage area based on the matched log transfer manner, so as to perform regular storage on the log to be stored.
In an exemplary embodiment, the data source distinguishing of the normative log, and the determining of the log type of the normative log includes: comparing the data source of the specification log with the data source of the application log to determine whether the specification log is the application log; if the data source of the specification log is the same as the data source of the application log, determining the log type of the specification log as the application log; and if the data source of the specification log is different from the data source of the application log, determining that the log type of the specification log is a non-application log. In addition, the process of matching the log transfer mode according to the log type comprises the following steps: acquiring the log type of the normalized log, and matching a log transmission mode based on the log type of the normalized log; if the log type of the normalized log is an application log, the log transmission mode matched with the normalized log comprises a data distribution service mode; and if the log type of the normal log is a non-application log, the log transmission mode matched with the normal log comprises a socket mode.
According to the above description, in an exemplary embodiment, the process of uploading the normative log to a preset storage area based on the matched log transmission manner may include: if the log transmission mode matched with the application log is a data distribution service mode, uploading the application log to a local storage area and/or a cloud storage area based on the data distribution service mode; and if the log transmission mode matched with the non-application log is a socket mode, uploading the non-application log to a local storage area and/or a cloud storage area based on the socket mode. Specifically, when the log type of the specification log is an application log, the application log is uploaded in a local Socket mode; and when the log type of the specification log is a non-application log, uploading the application log in a data distribution service DDS mode.
According to the above description, in an exemplary embodiment, the process of uploading the normative log to a preset storage area based on the matched log transfer manner may further include: uploading the application log to a log module based on the data distribution service mode, and uploading the non-application log to the log module based on the socket mode; uploading the log received by the log module to a local storage area for local storage; and responding to the log viewing request, acquiring a log to be viewed, and uploading the log to be viewed to a cloud storage area in a data distribution service mode. Specifically, as shown in fig. 3, the log module is a software application for uniformly managing logs, and is mainly responsible for collecting logs in real time and reporting log information to the DDS service module; wherein, the reported log comprises: OS log, middleware log, application log, VIU log, and ECU log. The data acquisition module is mainly responsible for collecting logs in the DDS service and uploading the logs to the cloud. The APP module is mainly responsible for the real-time log reporting of the SDK. In the log module framework, the whole process of local storage and offline pulling to cloud checking including the vehicle-side log is mainly as follows: when an integrated log SDK in the APP module is transmitted to the log module in a local Socket mode, other log information is transmitted to the log module in a DDS mode to be stored locally, the log module reports the related log information to the data acquisition module in a DDS service mode, and the data acquisition module directly uploads the log information in the DDS service to the cloud after receiving the log information, so that the function of offline drawing to the cloud for checking is realized. The log module specification in the embodiment is not only used for developing the log module and defining the log format, the log level, the log storage and the cloud specification on the log, but also is used for providing requirements for an APP needing to integrate the log SDK. The respective blocks in fig. 3 are explained as follows:
and (4) marking: and other logs are transmitted to a log module for storage in a DDS mode.
The label is two: and the APP application log is transmitted to a log module for storage in a local Socket mode.
Symbol (c): when the cloud diagnostic instrument checks the log stream of the corresponding APP, the log module sends the log information of the corresponding APP to the data acquisition module in a DDS mode and uploads the log information to the cloud. The log file viewing modes mainly comprise the following two modes: (1) windows operating system: view files using the open source dlt viewer tool (the tool will be provided with the document); (2) the Linux operating system comprises: the file class container is viewed using the open source dlt viewer command.
Label iv: the log module stores the received log information locally.
In addition, the log module integration flow in the present embodiment is shown in fig. 4. As can be seen from the record of fig. 4, the integration process of the log module includes: acquiring the dependency of a log software development kit and the environment of the log software development kit; copying a log software development kit folder to a root directory of an application program item based on the log software development kit dependency and a log software development kit environment; modifying the configuration file in the log software development kit, and storing the modified configuration file to a root directory of an application program item; compiling a software package compiling process input file under the root directory of the application program item, compiling and applying the compiling result, and integrating the log module. The log module integration mode can be described as follows: adding log SDK dependency and environment; adding a log SDK folder to the project; modifying a configuration file in the log SDK; compiling a CMakeLists.txt file; compiling the application; and generating a result. Specifically, the method comprises the following steps: the log module integration process is explained as follows:
step 1: add log SDK dependencies and contexts. Placing the SDK file SDK _20220114_ NeuSAR _ aCore _1911_27075_ jetson _ agx _ linux.sh on an arm64 board to be executed directly: /sdk _20220114_ NeuSAR _ aacre _1911_27075_ jetson _ agx _ linux.sh, performing the generation: export toolchain/environment. config.
Step 2: add the log SDK folder to the project. And copying the log SDK folder logs to the root directory of the application APP project.
And step 3: and modifying the configuration file in the log SDK. Modifying the content to be configured in the configuration file log/etc/processlogconfig.json of the log SDK, and automatically putting the configuration file into the etc folder under the root directory of the application APP after compiling.
And 4, step 4: create cmakelist. The logic itself is written in the cmakelist.
And 5: and compiling the application.
Step 6: and generating a result. And generating a bin file and an etc file. Wherein, an executable file of the project is placed in the bin file, and a front configuration file processlogconfig.
In summary, the present application provides a log storage system, which first obtains a log to be stored, and then performs log specification on the log to be stored to obtain a specification log; carrying out data source distinguishing on the normalized logs, and determining the log types of the normalized logs; and finally, matching a log transmission mode according to the log type, and uploading the standard log to a preset storage area based on the matched log transmission mode, so that the standard storage is carried out on the log to be stored. The system can define the log specification of the vehicle end application by performing specification processing on the log to be stored, thereby solving the problem that the log is not normal when being stored in the prior art. The system provides a vehicle-end log local storage system based on service-oriented architecture service, and can perform more uniform and standard processing on a vehicle-end log module, so that vehicle-end log supervision and management on vehicles are more detailed and in place, and safety of driving personnel can be guaranteed. Meanwhile, the system can respectively carry out more detailed, unified and specific specifications on the log format, the log grade, the log file storage, the log use and the cloud on the log in the log use specification, which is equivalent to the use specification of a vehicle-end log software installation kit, not only can carry out local storage on the log, but also can pull the log offline to the cloud end for checking, thereby solving the problem that the prior art can not provide unified log storage, management and checking channels.
It should be noted that the log storage system provided in the foregoing embodiment and the log storage method provided in the foregoing embodiment belong to the same concept, and the specific manner in which each module executes operations has been described in detail in the method embodiment, and is not described herein again. In practical applications, the log storage system provided in the foregoing embodiment may allocate the above functions to different functional modules according to needs, that is, the internal structure of the system is divided into different functional modules to complete all or part of the above described functions, which is not limited herein.
An embodiment of the present application further provides an electronic device, including: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the electronic device to implement the log storage method provided in each of the above embodiments.
FIG. 6 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application. It should be noted that the computer system 1000 of the electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 6, the computer system 1000 includes a Central Processing Unit (CPU)1001 that can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 1002 or a program loaded from a storage portion 1008 into a Random Access Memory (RAM) 1003. In the RAM1003, various programs and data necessary for system operation are also stored. The CPU 1001, ROM 1002, and RAM1003 are connected to each other via a bus 1004. An Input/Output (I/O) interface 1005 is also connected to the bus 1004.
The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage portion 1008 including a hard disk and the like; and a communication section 1009 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.
In particular, according to embodiments of the present application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication part 1009 and/or installed from the removable medium 1011. When the computer program is executed by a Central Processing Unit (CPU)1001, various functions defined in the system of the present application are executed.
It should be noted that the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer-readable signal medium may comprise a propagated data signal with a computer-readable computer program embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
Another aspect of the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to execute the log storage method as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist alone without being assembled into the electronic device.
Another aspect of the application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the log storage method provided in the above embodiments.
The above-described embodiments are merely illustrative of the principles and utilities of the present application and are not intended to limit the application. Any person skilled in the art can modify or change the above-described embodiments without departing from the spirit and scope of the present application. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical concepts disclosed in the present application shall be covered by the claims of the present application.

Claims (10)

1. A method of log storage, the method comprising the steps of:
acquiring a log to be stored, and carrying out log specification on the log to be stored to obtain a specification log;
carrying out data source distinguishing on the normative log, and determining the log type of the normative log;
and matching a log transmission mode according to the log type, and uploading the standard log to a preset storage area based on the matched log transmission mode so as to perform standard storage on the log to be stored.
2. The log storage method according to claim 1, wherein the log to be stored is normalized, and the process of obtaining a normalized log comprises:
acquiring preset log specification modes, and selecting at least one log specification mode from the preset log specification modes;
performing corresponding log specification on the log to be stored by using the selected log specification mode to obtain a specified log, and recording the specified log as a specification log;
the preset log specification mode comprises at least one of the following modes: log format specification, log level specification, log storage specification, log use specification, and log cloud specification.
3. The log storage method according to claim 1 or 2, wherein the canonical log is subjected to data source differentiation, and the process of determining the log type of the canonical log comprises:
comparing the data source of the specification log with the data source of the application log to determine whether the specification log is the application log;
if the data source of the specification log is the same as the data source of the application log, determining the log type of the specification log as the application log;
and if the data source of the specification log is different from the data source of the application log, determining that the log type of the specification log is a non-application log.
4. The log storage method according to claim 3, wherein the process of matching the log transfer modes according to the log types comprises:
acquiring the log type of the normative log, and matching a log transmission mode based on the log type of the normative log;
if the log type of the normalized log is an application log, the log transmission mode matched with the normalized log comprises a data distribution service mode;
and if the log type of the normal log is a non-application log, the log transmission mode matched with the normal log comprises a socket mode.
5. The log storage method according to claim 4, wherein the process of uploading the normative log to a preset storage area based on the matched log transmission mode comprises:
if the log transmission mode matched with the application log is a data distribution service mode, uploading the application log to a local storage area and/or a cloud storage area based on the data distribution service mode;
and if the log transmission mode matched with the non-application log is a socket mode, uploading the non-application log to a local storage area and/or a cloud storage area based on the socket mode.
6. The log storage method according to claim 5, wherein the process of uploading the normative log to a preset storage area based on the matched log transfer manner further comprises:
uploading the application log to a log module based on the data distribution service mode, and uploading the non-application log to the log module based on the socket mode;
uploading the log received by the log module to a local storage area for local storage; and the number of the first and second groups,
and responding to the log viewing request, acquiring a log to be viewed, and uploading the log to be viewed to a cloud storage area in a data distribution service mode.
7. The log storage method of claim 6, wherein the method further comprises:
acquiring log software development kit dependency and log software development kit environment;
copying a log software development kit folder to a root directory of an application program item based on the log software development kit dependency and a log software development kit environment;
modifying the configuration file in the log software development kit, and storing the modified configuration file to a root directory of an application program item;
compiling a software package compiling process input file under the root directory of the application program item, compiling and applying the compiling result, and integrating the log module.
8. A journal storage system, comprising:
the acquisition module is used for acquiring logs to be stored;
the specification module is used for carrying out log specification on the log to be stored to obtain a specification log;
the distinguishing module is used for distinguishing data sources of the normative logs and determining the log types of the normative logs;
and the storage module is used for matching a log transmission mode according to the log type and uploading the standard log to a preset storage area based on the matched log transmission mode so as to perform standard storage on the log to be stored.
9. An electronic device, characterized in that the electronic device comprises:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the electronic device to implement the log storage method of any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to execute a log storage method according to any one of claims 1 to 7.
CN202210761297.8A 2022-06-29 2022-06-29 Log storage method and system, electronic equipment and storage medium Pending CN115114250A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210761297.8A CN115114250A (en) 2022-06-29 2022-06-29 Log storage method and system, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210761297.8A CN115114250A (en) 2022-06-29 2022-06-29 Log storage method and system, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115114250A true CN115114250A (en) 2022-09-27

Family

ID=83330260

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210761297.8A Pending CN115114250A (en) 2022-06-29 2022-06-29 Log storage method and system, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115114250A (en)

Similar Documents

Publication Publication Date Title
CN107608901B (en) Jmeter-based testing method and device, storage medium and electronic equipment
CN112015489A (en) Management method, device, storage medium and system for vehicle-mounted software
CN112948272A (en) Production environment-based data test system function method and device and related equipment
CN115578051A (en) Vehicle asset management method and device
US20230384750A1 (en) Efficient controller data generation and extraction
CN112306041A (en) Vehicle configuration information writing method and device and electronic equipment
CN111241048A (en) Web terminal log management method, device, medium and electronic equipment
CN115114250A (en) Log storage method and system, electronic equipment and storage medium
CN116521530A (en) Unit test method, device, apparatus, storage medium and product
CN116126710A (en) Vehicle-mounted software simulation method, device, equipment and medium
CN116302034A (en) Processing method, device, equipment and medium for software release
CN115982043A (en) Vehicle interface testing method, device, equipment and storage medium
CN116030548A (en) Remote log diagnosis method and device, electronic equipment and storage medium
CN115037587A (en) Method, device, equipment and storage medium for automatically issuing network equipment configuration
CN115877819A (en) Vehicle remote diagnosis management method, device, equipment and medium
CN112948195A (en) Interface testing method and device, electronic equipment and storage medium
CN115866073B (en) Data difference fusion method, device, equipment and medium
CN114564413B (en) Synchronous equipment testing method and device
CN116578060A (en) Automatic test method and device for vehicle diagnosis, electronic equipment and medium
CN115988020A (en) Data transmission method, device, equipment and storage medium
CN117411760A (en) Device management method, related device and computer readable medium
CN115720231A (en) Sensing data processing method and device, electronic equipment and storage medium
CN113778842A (en) Fault-tolerant test method and device, electronic equipment and storage medium
CN116302832A (en) Cabin software monitoring method, device, equipment and storage medium
CN116055485A (en) Whole vehicle health management method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination