CN116383164A - Data processing method and device - Google Patents

Data processing method and device Download PDF

Info

Publication number
CN116383164A
CN116383164A CN202310324232.1A CN202310324232A CN116383164A CN 116383164 A CN116383164 A CN 116383164A CN 202310324232 A CN202310324232 A CN 202310324232A CN 116383164 A CN116383164 A CN 116383164A
Authority
CN
China
Prior art keywords
memory mapping
memory
log data
log
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310324232.1A
Other languages
Chinese (zh)
Inventor
刘易
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lenovo Beijing Ltd
Original Assignee
Lenovo Beijing Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lenovo Beijing Ltd filed Critical Lenovo Beijing Ltd
Priority to CN202310324232.1A priority Critical patent/CN116383164A/en
Publication of CN116383164A publication Critical patent/CN116383164A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/1805Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815Journaling file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • G06F3/0611Improving I/O performance in relation to response time
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0679Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application provides a data processing method, which is applied to a log service process and comprises the following steps: establishing memory mapping information through a log service process; caching log data of each business process in a corresponding memory mapping area based on the memory mapping information; and if the log data in the memory mapping area meets the preset condition, writing the log data in the memory mapping area into a nonvolatile storage area. Meanwhile, the application also provides electronic equipment.

Description

Data processing method and device
Technical Field
The present disclosure relates to data processing technologies, and in particular, to a data processing method and apparatus.
Background
In 5G communication, the running state of the baseband processing unit (BBU, building Base band Unit) needs to be effectively monitored, and the log record is used as an important means for equipment running and fault analysis, and plays a vital role in the process of core service fault analysis. The current log recording method of the conventional log system cannot meet the performance requirement and the time delay requirement of the base station software log.
Disclosure of Invention
In view of this, it is desirable to provide a data processing method and apparatus.
In order to achieve the above purpose, the technical scheme of the application is realized as follows:
according to an aspect of the present application, there is provided a data processing method, the method being applied to a log service process, the method comprising:
establishing memory mapping information through a log service process;
caching log data of each business process in a corresponding memory mapping area based on the memory mapping information;
and if the log data in the memory mapping area meets the preset condition, writing the log data in the memory mapping area into a nonvolatile storage area.
In the above solution, the caching, based on the memory mapping information, log data of each service process in a corresponding memory mapping area includes:
determining a memory mapping area corresponding to each business process according to the memory configuration information in the memory mapping information; each memory mapping area corresponds to a service process;
and caching log data of the corresponding business process in each memory mapping area.
In the above solution, if the log data in the memory mapped area meets a preset condition, the method at least includes one of the following:
traversing state values of each thread segment in each memory mapping region;
if the state value represents that the log data to be read exists in at least one memory mapping area, determining that the log data of the memory mapping area meets a preset condition;
or if the state value represents that the log data to be read exists in at least one memory mapping area, and the log data to be read meets the capacity parameter, determining that the log data of the memory mapping area meets a preset condition.
In the above solution, writing the log data of the memory mapped area into a nonvolatile storage area includes:
reading the log data from the data segment of the memory mapped region;
and storing the log data in a nonvolatile storage area.
In the above scheme, the method further comprises:
receiving a configuration modification instruction for the memory mapped region;
modifying configuration parameters of the memory mapped region based on the configuration modification instruction; and the configuration parameters are used for caching log data of the target business process.
In the above aspect, after the writing of the log data of the memory mapped area into the nonvolatile storage area, the method further includes:
updating the state value of the data segment in the memory mapping area, wherein the state value of the data segment represents the read-write position of the log data in the memory mapping area.
According to another aspect of the present application, there is provided a data processing method, the method being applied to a business process, the method comprising:
loading memory mapping information created by a log service process to obtain configuration information in the memory mapping information;
and writing the log data into the corresponding memory mapping area in the memory mapping information based on the configuration information.
In the above solution, writing the log data into the corresponding memory mapping area in the memory mapping information based on the configuration information includes:
determining a state value of a data segment in the memory mapping area based on the configuration information;
determining an idle region in the memory mapping region according to the state value;
and writing log data into the free area.
In the above solution, after the writing of the log data into the free area, the method further includes:
updating the state value of the data segment in the memory mapping area, wherein the state value of the data segment represents the read-write position of the log data in the memory mapping area.
In the above scheme, the method further comprises:
receiving a configuration modification instruction for the memory mapped region;
modifying configuration parameters of the memory mapped region based on the configuration modification instruction;
and writing log data in the memory mapping area based on the modified configuration parameters.
According to a third aspect of the present application, there is provided an electronic device comprising:
the building unit is used for building memory mapping information through a log service process;
the first storage unit is used for caching log data of each business process in a corresponding memory mapping area based on the memory mapping information;
and the processing unit is used for writing the log data in the memory mapping area into the storage area of the nonvolatile second storage unit if the log data in the memory mapping area meets the preset condition.
According to a fourth aspect of the present application, there is provided an electronic device comprising:
the loading unit is used for loading the memory mapping information created by the log service process to obtain the configuration information in the memory mapping information;
and the processing unit is used for writing the log data into the corresponding memory mapping area in the memory mapping information based on the configuration information.
Drawings
FIG. 1 is a schematic diagram of a flow implementation of a data processing method in the present application;
FIG. 2 is a schematic diagram II of a flow implementation of a data processing method in the present application;
fig. 3 is a schematic diagram of log service structure of an electronic device in the present application;
FIG. 4 is a schematic diagram III of a process implementation of the processing method of the present application;
FIG. 5 is a schematic diagram of the structural components of the electronic device;
FIG. 6 is a schematic diagram II of the structural composition of the electronic device in the present application;
FIG. 7 is a schematic diagram of log service structure of an electronic device in the present application;
fig. 8 is a schematic diagram of the structural composition of the electronic device in the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure. Embodiments and features of embodiments in this application may be combined with each other arbitrarily without conflict. The steps illustrated in the flowchart of the figures may be performed in a computer system, such as a set of computer-executable instructions. Also, while a logical order is depicted in the flowchart, in some cases, the steps depicted or described may be performed in a different order than presented herein.
The technical scheme of the application is further elaborated below with reference to the drawings in the specification and the specific embodiments.
Fig. 1 is a schematic diagram of a flow implementation of a data processing method in the present application, where the method may be applied to log service process sides of various electronic devices, and the electronic devices may be, for example, a computer, a mobile phone, a tablet computer, and so on. As shown in fig. 1, the method includes:
step 101, establishing memory mapping information through a log service process;
in order to improve the data processing efficiency, the log service process and the service process are usually located on the same electronic device, and before executing the service process, the electronic device needs to first start the log service process, and perform memory mapping through the log service process to establish memory mapping information shared with each service process, where the memory mapping information may at least include relevant memory configuration information that is characterized by providing for each service process.
Step 102, caching log data of each business process in a corresponding memory mapping area based on the memory mapping information;
here, the electronic device may determine, according to the memory configuration information in the memory mapping information, a memory mapping area corresponding to each service process; wherein each memory mapping area corresponds to a service process; the electronic device may cache log data of the corresponding business process in each memory mapped region.
Step 103, if the log data in the memory mapping area meets the preset condition, writing the log data in the memory mapping area into a nonvolatile storage area.
Here, the electronic device may traverse the state values of the thread segments in the memory mapped regions; if the state value represents that the log data to be read exists in at least one memory mapping area, determining that the log data of the memory mapping area meets a preset condition; or if the state value represents that the log data to be read exists in at least one memory mapping area and the log data to be read meets the capacity parameter, determining that the log data of the memory mapping area meets a preset condition. Or if the state value represents that the number of the log data written in at least one memory mapping area is greater than a preset number threshold, determining that the log data of the memory mapping area meets a preset condition.
Here, the state value of each thread segment includes, but is not limited to, offset values of read and write logs, start addresses of read and write logs, thread ID, and the like.
Under the condition that the electronic equipment determines that the log data in the memory mapping area meets the preset condition, the log data can be read from the data segment of the memory mapping area; the log data is then stored to a non-volatile storage area.
According to the method and the system, the memory mapping space is built through the log service process, so that the business process only needs to write the log data into the internal storage buffer memory, the time delay requirement of the business process on writing the log can be met, and the log record requirement of short time and high frequency can be met. In addition, by separating the log service process from the business process, the log data is stored in a memory mapping mode, so that the efficiency of writing the log by the business process is greatly improved, and the influence of log printing on the performance and the business time sequence is reduced.
In the application, the electronic device may further receive a configuration modification instruction for the memory mapping region through a log service process; modifying configuration parameters of the memory mapped region based on the configuration modification instruction; and the configuration parameters are used for caching log data of the target business process. And the state value of the data segment in the memory mapping area can be updated after the log data of the memory mapping area is written into the nonvolatile storage area, wherein the state value of the data segment represents the read-write position of the log data in the memory mapping area.
Fig. 2 is a schematic diagram ii of a flow implementation of a data processing method in the present application, where the method may be applied to a service process side of various electronic devices, as shown in fig. 2, and the method includes:
step 201, loading memory mapping information created by a log service process to obtain configuration information in the memory mapping information;
here, in the process of starting the service process or under the condition of finishing starting operation, the memory mapping information created by the log service process can be loaded, the memory configuration information corresponding to the service process can be obtained based on the memory mapping information, and a storage condition is provided for the subsequent service process when the log is printed.
Step 202, writing log data into a corresponding memory mapping area in the memory mapping information based on the configuration information.
Here, the electronic device may further determine, based on the configuration information, a status value of the data segment in the memory mapping area when obtaining the memory configuration information corresponding to the service process; determining an idle region in the memory mapping region according to the state value; and writing the log data to be printed into the idle area. Here, after each log reading and writing, the service process may further update a state value of a data segment in the memory mapping area, where the state value of the data segment indicates a reading and writing position of log data in the memory mapping area.
In the application, the electronic device may further receive a configuration modification instruction for the memory mapping region through a service process; modifying configuration parameters of the memory mapped region based on the configuration modification instruction; and writing log data in the memory mapping area based on the modified configuration parameters.
According to the method and the system, the memory mapping space is built through the log service process, so that the business process only needs to write the log data into the internal storage buffer memory, the time delay requirement of the business process on writing the log can be met, and the log record requirement of short time and high frequency can be met. In addition, by separating the log service process from the business process, the log data is stored in a memory mapping mode, so that the efficiency of writing the log by the business process is greatly improved, and the influence of log printing on the performance and the business time sequence is reduced.
Fig. 3 is a schematic diagram of a log service structure of an electronic device in the present application, where, as shown in fig. 3, the electronic device includes a separate log service process and a service process, where the log service process establishes a shared memory mapping space in a memory mapping manner, each memory mapping space corresponds to a service process, the memory mapping space is used to store log data of a corresponding service process, and configuration information of the memory mapping space of each service process may be the same or different. When a business process exists, each business process can load a memory mapping space created by the log service process for the business process to obtain memory configuration information in the memory mapping space, and when the business process needs a printing log, the printing log can be written into a memory mapping area corresponding to the corresponding memory mapping space.
Here, the log service process may also receive an external command, which may be a configuration modification instruction for the memory mapped space; the log service process can modify the configuration parameters of the corresponding memory mapping space based on the configuration modification instruction; here, the configuration parameter is used to cache log data of the target business process.
For example, the configuration parameters may characterize log read-write permissions of the business process, log buffer area size of the business process, and so on.
Here, the log service process may also save the log data cached in the memory mapped space to the nonvolatile memory. Alternatively, the log data cached in the memory mapped space is saved to a file system of the nonvolatile memory. Thus, the log state can be conveniently read in real time and modified in real time.
In the present application, for a single memory mapping space, the data structure may include a data segment and a configuration segment, where the data segment is a cache area or a cache space for storing log data; the configuration section includes a common configuration section and a thread configuration section. The common configuration section is used for maintaining configuration parameters of the whole log module. Including but not limited to shared memory path, read-write thread pool size, log print level, log print permission, and bitmap of log print module. The thread configuration section is used for maintaining the current log printing state of a specific thread, including but not limited to thread ID, read and write log offset, read and write log start address and other information.
According to the method and the device, the memory space is divided into the configuration section and the data section through the memory mapping, and the log configuration information can be acquired and modified in real time in an external command mode, so that the flexibility of log printing is improved.
Fig. 4 is a schematic diagram three of a flow implementation of a processing method in the present application, as shown in fig. 4, where the method includes:
step 401, starting a log service process;
here, the log service process needs to be started first as a precondition of the service process, otherwise, the service process cannot verify the memory mapping space established by the log service process, which may cause log output failure. The service process starts, for example, creating a memory mapped file. Here, the memory map file characterizes memory map information of the business process. Including but not limited to memory configuration information for business processes.
Step 402, initializing a memory mapping file;
after the log service process is started, initializing a common section and a thread section of the memory mapping file, reading memory configuration information, and writing the memory configuration information into the configuration section of the memory mapping file. A thread pool is started that reads log data segments.
Step 403, the log service process creates a write log thread pool;
step 404, traversing the memory mapped spatial data segment;
by traversing the memory mapped space data segment, it can be checked whether the memory mapped space is cached with log files.
Step 405, determining whether a write log exists in the memory mapped space;
if yes, go to step 406; if not, go to step 404.
Step 406, modifying the status value of the log to be read;
here, when the cache log is written into the free storage area of the memory mapping space, the read log state value of the data segment may be updated, that is, the offset value of the log to be read may be modified.
Here, the log service process is further provided with an external command receiving inlet, and the configuration section data of the memory mapping file can be queried and modified in real time and can be validated in real time by receiving the external command.
Step 407, performing log caching and disk brushing operations;
when the log service process checks that the cached log data reaches the preset quantity, the log service process can write the log file into the nonvolatile memory, so that the log service process is convenient for engineering personnel to check at any time.
Step 408, starting a business process;
step 409, after the service process is started, checking whether the memory mapping file exists;
if present, execute step 410, if not present, execute step 414
Step 410, loading the memory mapping file created by the log service process, and reading the common header information configured in the memory mapping file;
step 411, judging whether an idle mapping area is found according to the common header information;
if found, go to step 412, if not found, go to step 414
Step 412, writing log data to the free area of the memory mapped file;
here, when the business process has a log print requirement, the output thread of the current print log is bound with the idle thread segment in the configuration parameter, so as to write the log data into the idle area of the memory mapping file.
Step 413, modifying the status value of the log to be written;
here, in the case of writing the print log into the free storage area of the memory mapped space, the write log state value of the thread segment, that is, the offset value of the modified thread write log may be updated.
Step 414, end.
According to the method and the device, the traditional log solidification storage operation is decoupled from the business process, and the efficiency of writing the log by the business process is greatly improved in a memory reading and writing mode, so that the influence of log printing on performance and business time sequence is reduced; by using the independent log service process, the flexibility of log service configuration is increased, and the log service configuration can be modified in real time under the condition of not affecting the service.
Fig. 5 is a schematic diagram of the structural composition of the electronic device in the present application, as shown in fig. 5, the electronic device includes:
a building unit 501, configured to build memory mapping information through a log service process;
a first storage unit 502, configured to cache log data of each service process in a corresponding memory mapping area based on the memory mapping information;
the processing unit 503 writes the log data in the memory mapped area into the storage area of the nonvolatile second storage unit 504 if the log data in the memory mapped area meets a preset condition.
The second storage unit 504 is at least used for storing the log data from the first storage unit 503.
In a preferred embodiment, the processing unit 503 is further configured to determine a memory mapping area corresponding to each service process according to memory configuration information in the memory mapping information; each memory mapping area corresponds to a service process;
the first storage unit 502 is configured to cache log data of a corresponding service process in each memory mapped area.
In a preferred embodiment, the log data in the memory mapping area meets a preset condition, and at least includes one of the following:
traversing state values of each thread segment in each memory mapping region;
if the state value represents that the log data to be read exists in at least one memory mapping area, determining that the log data of the memory mapping area meets a preset condition;
or if the state value represents that the log data to be read exists in at least one memory mapping area, and the log data to be read meets the capacity parameter, determining that the log data of the memory mapping area meets a preset condition.
In a preferred embodiment, the processing unit 503 is further configured to read the log data from the data segment of the memory mapped region; and stores the log data to a storage area of the nonvolatile second storage unit 504.
In a preferred embodiment, the processing unit 503 is further configured to receive a configuration modification instruction for the memory mapped region; modifying configuration parameters of the memory mapped region of the first storage unit 502 based on the configuration modification instruction; and the configuration parameters are used for caching log data of the target business process.
In a preferred embodiment, the processing unit 503 is further configured to update a status value of a data segment in the memory mapped area, where the status value of the data segment indicates a read-write location of log data in the memory mapped area.
The method and the system use independent log service processes, increase the flexibility of log service configuration, and can modify the log service configuration in real time under the condition of not affecting the service.
Fig. 6 is a schematic diagram ii of the structural composition of the electronic device in the present application, as shown in fig. 6, the electronic device includes:
the loading unit 601 is configured to load memory mapping information created by a log service process, so as to obtain configuration information in the memory mapping information;
an input unit 602, configured to write log data into a corresponding memory mapping area in the memory mapping information based on the configuration information.
In a preferred embodiment, the electronic device further includes:
a processing unit 603, configured to determine a status value of a data segment in the memory mapped area based on the configuration information; and determining an idle region in the memory mapped region according to the state value;
an input unit 602 for writing log data into the free area.
In a preferred embodiment, the electronic device further includes:
an updating unit 604, configured to update a status value of a data segment in the memory mapped area, where the status value of the data segment characterizes a read-write location of log data in the memory mapped area.
According to the method and the device, the traditional log solidification storage operation is decoupled from the business process, and the efficiency of writing the log by the business process is greatly improved in a memory reading and writing mode, so that the influence of log printing on performance and business time sequence is reduced.
Fig. 7 is a schematic diagram of a log service structure of an electronic device in the present application, as shown in fig. 7, including a service process module 701 and a service process module 702, where when the service process module 702 is started, memory mapping information may be established, common segment and thread segment configuration parameters in the memory mapping information are initialized, and a thread pool of read data segments is started. After the business process module 701 is started, the memory mapping information can be loaded, the configuration information in the memory mapping information can be read from the common header of the memory mapping information, the idle storage area in the memory mapping space can be found based on the configuration information, when the log printing requirement exists, the business thread can be bound to the data segment of the idle storage area, and the log business binding can be completed.
Here, the business process 701 may have a plurality of business threads, and each business thread may correspond to one memory storage area.
Here, the service process module 702 may create a thread pool, where service threads in each service process are managed. The service process module 702 may also receive configuration modification instructions for the memory mapping information through the external command interface 703; modifying configuration parameters of the memory mapping information based on the configuration modification instruction; and the configuration parameters are used for caching log data of the target business process. Therefore, engineering personnel can conveniently inquire and modify the configuration section data in real time and take effect in real time.
After the business thread module 701 prints the log, the data offset value of the thread segment can be modified, and the service process module 702 traverses the state value of the thread segment, and can read the log from the data segment and update the read data offset value. When the read log cache reaches a certain amount, a write log file operation may also be performed to write the cached log data to the nonvolatile memory (disk) 704.
It should be noted that: in the electronic device provided in the above embodiment, when performing data processing, only the division of each program module is used as an example, in practical application, the processing allocation may be performed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules, so as to complete all or part of the processing described above. In addition, the electronic device provided in the foregoing embodiment and the processing method embodiment provided in the foregoing embodiment belong to the same concept, and specific implementation processes of the electronic device are detailed in the method embodiment and are not repeated herein.
The embodiment of the application also provides electronic equipment, which comprises: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is configured to execute any of the method steps of the above-described processing method when the computer program is run.
Fig. 8 is a schematic diagram of the structural components of an electronic device 800 in the present application, which may be a mobile phone, a computer, a digital broadcast terminal, an information transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like. The electronic device 800 shown in fig. 8 includes: at least one processor 801, memory 802, at least one network interface 804, and a user interface 803. The various components in the electronic device 800 are coupled together by a bus system 805. It is appreciated that the bus system 805 is used to enable connected communications between these components. The bus system 805 includes a power bus, a control bus, and a status signal bus in addition to the data bus. But for clarity of illustration, the various buses are labeled as bus system 805 in fig. 8.
The user interface 803 may include, among other things, a display, keyboard, mouse, trackball, click wheel, keys, buttons, touch pad, or touch screen, etc.
It is to be appreciated that memory 802 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Wherein the nonvolatile Memory may be Read Only Memory (ROM), programmable Read Only Memory (PROM, programmable Read-Only Memory), erasable programmable Read Only Memory (EPROM, erasable Programmable Read-Only Memory), electrically erasable programmable Read Only Memory (EEPROM, electrically Erasable Programmable Read-Only Memory), magnetic random access Memory (FRAM, ferromagnetic random access Memory), flash Memory (Flash Memory), magnetic surface Memory, optical disk, or compact disk Read Only Memory (CD-ROM, compact Disc Read-Only Memory); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be random access memory (RAM, random Access Memory), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (SRAM, static Random Access Memory), synchronous static random access memory (SSRAM, synchronous Static Random Access Memory), dynamic random access memory (DRAM, dynamic Random Access Memory), synchronous dynamic random access memory (SDRAM, synchronous Dynamic Random Access Memory), double data rate synchronous dynamic random access memory (ddr SDRAM, double Data Rate Synchronous Dynamic Random Access Memory), enhanced synchronous dynamic random access memory (ESDRAM, enhanced Synchronous Dynamic Random Access Memory), synchronous link dynamic random access memory (SLDRAM, syncLink Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, direct Rambus Random Access Memory). The memory 802 described in embodiments of the present application is intended to comprise, without being limited to, these and any other suitable types of memory.
The memory 802 in the present embodiment is used to store various types of data to support the operation of the electronic device 800. Examples of such data include: any computer programs for operation on the electronic device 800, such as an operating system 8021 and application programs 8022; contact data; telephone book data; a message; a picture; audio, etc. The operating system 8021 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 8022 may contain various application programs, such as a Media Player (Media Player), a Browser (Browser), and the like, for implementing various application services. A program for implementing the method of the embodiment of the present application may be included in the application program 8022.
The method disclosed in the embodiments of the present application may be applied to the processor 801 or implemented by the processor 801. The processor 801 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry in hardware in the processor 801 or by instructions in software. The processor 801 may be a general purpose processor, a digital signal processor (DSP, digital Signal Processor), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 801 may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present application. The general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly embodied in a hardware decoding processor or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium in a memory 802. The processor 801 reads information from the memory 802 and in combination with its hardware performs the steps of the method described above.
In an exemplary embodiment, the electronic device 800 can be implemented by one or more application specific integrated circuits (ASIC, application Specific Integrated Circuit), DSPs, programmable logic devices (PLD, programmable Logic Device), complex programmable logic devices (CPLD, complex Programmable Logic Device), field-programmable gate arrays (FPGA, field-Programmable Gate Array), general purpose processors, controllers, microcontrollers (MCU, micro Controller Unit), microprocessors (Microprocessor), or other electronic components for performing the aforementioned methods.
In an exemplary embodiment, the present application also provides a computer readable storage medium, such as a memory 802, comprising a computer program executable by the processor 801 of the electronic device 800 to perform the steps described in the foregoing methods. The computer readable storage medium may be FRAM, ROM, PROM, EPROM, EEPROM, flash Memory, magnetic surface Memory, optical disk, or CD-ROM; but may be a variety of devices including one or any combination of the above-described memories, such as a mobile phone, computer, tablet device, personal digital assistant, or the like.
A computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs any of the method steps of the above-described processing method.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.
The units described as separate units may or may not be physically separate, and units displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
The methods disclosed in the several method embodiments provided in the present application may be arbitrarily combined without collision to obtain a new method embodiment.
The features disclosed in the several product embodiments provided in the present application may be combined arbitrarily without conflict to obtain new product embodiments.
The features disclosed in the several method or apparatus embodiments provided in the present application may be arbitrarily combined without conflict to obtain new method embodiments or apparatus embodiments.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A data processing method, the method being applied to a log service process, the method comprising:
establishing memory mapping information through a log service process;
caching log data of each business process in a corresponding memory mapping area based on the memory mapping information;
and if the log data in the memory mapping area meets the preset condition, writing the log data in the memory mapping area into a nonvolatile storage area.
2. The method of claim 1, wherein the caching log data of each business process in the corresponding memory mapped region based on the memory mapped information comprises:
determining a memory mapping area corresponding to each business process according to the memory configuration information in the memory mapping information; each memory mapping area corresponds to a service process;
and caching log data of the corresponding business process in each memory mapping area.
3. The method of claim 1, wherein if the log data in the memory mapped region satisfies a preset condition, at least one of the following is included:
traversing state values of each thread segment in each memory mapping region;
if the state value represents that the log data to be read exists in at least one memory mapping area, determining that the log data of the memory mapping area meets a preset condition;
or if the state value represents that the log data to be read exists in at least one memory mapping area, and the log data to be read meets the capacity parameter, determining that the log data of the memory mapping area meets a preset condition.
4. The method of claim 1, wherein the writing log data of the memory mapped region to a non-volatile storage region comprises:
reading the log data from the data segment of the memory mapped region;
and storing the log data in a nonvolatile storage area.
5. The method of claim 1, wherein the method further comprises:
receiving a configuration modification instruction for the memory mapped region;
modifying configuration parameters of the memory mapped region based on the configuration modification instruction; and the configuration parameters are used for caching log data of the target business process.
6. The method of claim 1, wherein after the writing of the log data of the memory mapped region to a non-volatile storage region, the method further comprises:
updating the state value of the data segment in the memory mapping area, wherein the state value of the data segment represents the read-write position of the log data in the memory mapping area.
7. A data processing method, the method being applied to a business process, the method comprising:
loading memory mapping information created by a log service process to obtain configuration information in the memory mapping information;
and writing the log data into the corresponding memory mapping area in the memory mapping information based on the configuration information.
8. The method of claim 7, the writing log data to a corresponding memory mapped region in the memory mapped information based on the configuration information, comprising:
determining a state value of a data segment in the memory mapping area based on the configuration information;
determining an idle region in the memory mapping region according to the state value;
and writing log data into the free area.
9. The method of claim 8, wherein after the writing log data to the free area, the method further comprises:
updating the state value of the data segment in the memory mapping area, wherein the state value of the data segment represents the read-write position of the log data in the memory mapping area.
10. An electronic device, comprising:
the building unit is used for building memory mapping information through a log service process;
the first storage unit is used for caching log data of each business process in a corresponding memory mapping area based on the memory mapping information;
the processing unit is used for writing the log data in the memory mapping area into a storage area of a nonvolatile second storage unit if the log data in the memory mapping area meets a preset condition;
and the second storage unit is at least used for storing the log data from the first storage unit.
CN202310324232.1A 2023-03-29 2023-03-29 Data processing method and device Pending CN116383164A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310324232.1A CN116383164A (en) 2023-03-29 2023-03-29 Data processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310324232.1A CN116383164A (en) 2023-03-29 2023-03-29 Data processing method and device

Publications (1)

Publication Number Publication Date
CN116383164A true CN116383164A (en) 2023-07-04

Family

ID=86976297

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310324232.1A Pending CN116383164A (en) 2023-03-29 2023-03-29 Data processing method and device

Country Status (1)

Country Link
CN (1) CN116383164A (en)

Similar Documents

Publication Publication Date Title
KR100393619B1 (en) Memory apparatus and therefor controling method for mobile station
US10789170B2 (en) Storage management method, electronic device and computer readable medium
CN110716845B (en) Log information reading method of Android system
CN115639971B (en) Data writing method, data writing device, electronic device, storage medium, and program product
CN112954244A (en) Method, device and equipment for realizing storage of monitoring video and storage medium
CN111857572B (en) Data writing method, device and equipment of TLC solid state disk and storage medium
CN111984204A (en) Data reading and writing method and device, electronic equipment and storage medium
CN114003181B (en) Data write mirror image system, method, device, electronic equipment and storage medium
KR20070063132A (en) Apparatus and method for manage a bad block
CN111857571B (en) Solid state disk physical block address distribution method, device, equipment and storage medium
CN116383164A (en) Data processing method and device
CN111143258A (en) Method, system, device and medium for accessing FPGA (field programmable Gate array) by system based on Opencl
CN115712394A (en) Data reading and writing method and device, computer equipment and readable storage medium
CN114005476A (en) Flash memory, flash memory erasing and writing counting method, electronic equipment and computer storage medium
US8812813B2 (en) Storage apparatus and data access method thereof for reducing utilized storage space
CN107870736B (en) Method and device for supporting non-linear flash memory larger than 4GB
CN114675995A (en) Data backup method and device and electronic equipment
CN117472295B (en) Memory, data processing method, device and medium
CN111435285B (en) Data writing method and device for storage volume, electronic equipment and machine-readable storage medium
CN113760195B (en) FATFS file system based on embedded type
US11947467B2 (en) Electronic device and operation method thereof
JP2003150458A (en) Fault detector, fault detecting method, program and program recording medium
TWI667571B (en) Data storage apparatus, method for programming system information and method for rebuilding system information
CN109542531B (en) File layout adjustment method and device
KR100695267B1 (en) Method for Saving Storage Space of Random Access Memory Used in Mobile Phone

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination