CN115688494B

CN115688494B - Data processing method, device, electronic equipment and computer readable storage medium

Info

Publication number: CN115688494B
Application number: CN202310005152.XA
Authority: CN
Inventors: 吴连奎; 王长欣; 刘韶鹏; 徐长安
Original assignee: Beijing Yunlu Technology Co Ltd
Current assignee: Beijing Yunlu Technology Co Ltd
Priority date: 2023-01-04
Filing date: 2023-01-04
Publication date: 2023-04-25
Anticipated expiration: 2043-01-04
Also published as: CN115688494A

Abstract

The application provides a data processing method, a data processing device, electronic equipment and a computer readable storage medium, and relates to the technical field of urban waterlogging simulation data management. The method comprises the following steps: determining multilevel index information in a storage database; positioning a target data file of target simulation data according to the multi-level index information and the target data of the target simulation data; and processing the target simulation data based on the target data file. By creating index information of multiple layers of different catalogs in the storage database, a large amount of simulation data generated in the urban waterlogging simulation early warning system can be processed respectively so as to be positioned to a data file corresponding to each simulation data for storage, and the simulation data are subjected to various processes such as corresponding storage, retrieval, backup, migration and the like based on the data file determined by positioning, so that the processing efficiency of the large amount of simulation data generated by the urban waterlogging simulation early warning system is effectively improved, and the early warning effect of the urban waterlogging simulation early warning system is optimized.

Description

Data processing method, device, electronic equipment and computer readable storage medium

Technical Field

The application relates to the technical field of urban waterlogging simulation data management, in particular to a data processing method, a device, electronic equipment and a computer readable storage medium.

Background

In recent years, the global climate change influence and the island effect caused by urbanization have a tendency to increase the intensity and frequency of partial urban storm. The ground subsides in a large area in the urban area to form a plurality of depressions with unsmooth drainage and even difficult drainage, and storm water accumulation is easy to cause. The increase of urban area and the increase of the watertight ratio directly lead to the increase of runoff coefficient and the shortening of confluence time, thereby leading to the aggravation of waterlogging. Therefore, in order to effectively prevent the waterlogging condition, urban waterlogging prevention and control measures are generally adopted, for example, an urban waterlogging simulation early warning system is built to early warn waterlogging, so that urban rainstorm waterlogging early warning level is improved, and waterlogging loss is reduced.

The existing urban waterlogging simulation early warning mode combines technologies such as weather forecast, physical model and monitoring data, hydrodynamics simulation, artificial intelligence, large-scale parallel computing and the like to carry out early warning, and can simulate the evolution process of space rainfall, ground water accumulation and pipe network drainage in the whole urban area range in a minute level, thereby realizing real-time data simulation of three-dimensional forecast prediction of urban rainfall and flood process and various data. However, the simulation data during early warning is more, the traditional database is mainly used for storing the current simulation data, along with the increase of the operation time of the early warning system, the history data storage amount is also increased continuously, so that the data storage and the search query are slower and slower, and the data backup or migration is difficult when the volume is larger, so that the processing efficiency of the simulation data is lower, and the result of urban waterlogging simulation early warning is adversely affected.

Disclosure of Invention

In view of the foregoing, an objective of the embodiments of the present application is to provide a data processing method, apparatus, electronic device, and computer readable storage medium, so as to solve the problem in the prior art that the processing efficiency of simulation data of urban waterlogging simulation early warning is low.

To solve the above problem, in a first aspect, an embodiment of the present application provides a data processing method, where the method includes:

determining multilevel index information in a storage database;

positioning a target data file of target simulation data according to the multi-level index information and the target data of the target simulation data;

and processing the target simulation data based on the target data file.

In the implementation process, when simulation early warning work of urban inland inundation is performed, because more data are generated when simulating videos or images, the efficiency of operations such as storing, searching, backing up and transferring simulation data in a storage database is low, and therefore, index information of multiple layers of different catalogues can be created in the storage database to respectively process a large amount of simulation data generated in the urban inland inundation simulation early warning system. The target data file corresponding to each simulation data is rapidly and accurately positioned through the multi-level index information, and the corresponding simulation data is subjected to various processes such as corresponding storage, retrieval, backup, migration and the like based on the target data file determined by positioning. The processing efficiency of a large amount of simulation data generated by the urban waterlogging simulation early-warning system is effectively improved, so that the early-warning effect of the urban waterlogging simulation early-warning system is optimized.

Optionally, the multi-level index information includes: first level index information, second level index information, and third level index information;

the determining to store the multi-level index information in the database includes:

determining the first-level index information corresponding to each sub-database in the storage database;

determining the second-level index information corresponding to each simulation frequency in each sub-database;

and determining the third-level index information corresponding to each data file in each simulation time.

In the implementation process, the multi-level index information includes three levels of index information corresponding to three different directories. Determining first-level index information under the corresponding sub-database directory according to each sub-database in the storage database, determining second-level index information under the corresponding simulation sub-number directory according to the simulation times performed by the urban waterlogging simulation early warning system based on each sub-database, and continuing classification of simulation data in each simulation time to determine third-level index information representing corresponding data files under different simulation data directories. According to the multi-level index information under different catalogues, the range corresponding to each catalogue can be determined from large to small according to different levels of the number of frames and the like from the sub-database to the simulation data, so that the accuracy of the sub-database, the simulation times or the data file corresponding to each level of index information is ensured, and the accuracy and the effectiveness of the multi-level index information are improved.

Optionally, the determining the first level index information corresponding to each sub-database in the storage database includes:

acquiring sub-database information of each sub-database in the storage database; wherein the sub database information includes: at least one of the number, name and user information of the sub database;

and creating corresponding first-level index information according to the sub-database information.

In the implementation process, when the first-level index information is determined, the relevant sub-database information of each sub-database can be obtained, and corresponding first-level index information is created according to the plurality of sub-database information with uniqueness, so that the first-level index information can comprise the uniqueness identification information corresponding to each sub-database, and the sub-databases corresponding to the simulation of the urban waterlogging simulation early warning system can be accurately and rapidly positioned in the catalogue of the sub-database during data positioning.

Optionally, the determining the second-level index information corresponding to each simulation number in each sub-database includes:

determining the simulation times based on each sub-database according to simulation requirements;

And creating corresponding second-level index information according to each simulation frequency.

In the implementation process, when the second-level index information is determined, corresponding simulation times can be determined according to simulation requirements of the urban waterlogging simulation early warning system when simulation is performed based on each sub-database, so that the corresponding second-level index information is created according to unique identification information corresponding to each simulation time, such as the number, the serial number and the like of each simulation time, and therefore, when data is positioned, the simulation times corresponding to the simulation time can be accurately and rapidly positioned in multiple simulations in the sub-database.

Optionally, the determining the third level index information corresponding to each data file in each simulation time includes:

determining simulation description information when simulation is carried out on each simulation frequency according to simulation requirements; wherein, the simulation description information comprises: at least one of grid information, index number, index data type, simulation group number, simulation time and time interval during simulation;

determining unique identification information of each data file in the simulation times;

and creating the third-level index information corresponding to each data file according to the simulation description information and the identification information.

In the implementation process, when the third-level index information is determined, simulation description information related to grids, index number, index data type, group number, time and time interval and the like during simulation in each simulation time can be determined according to the simulation requirement of the urban waterlogging simulation early warning system, and unique identification information of each data file under the catalog corresponding to the simulation time is determined. By combining the simulation description information and the identification information, third-level index information representing the simulation related information can be created, so that when data positioning is performed, data files corresponding to the simulation actual conditions can be accurately and rapidly positioned in the catalogue of the simulation times.

Optionally, the locating the target data file of the target simulation data according to the multi-level index information and the target data of the target simulation data includes:

determining the target data of the target simulation data according to target simulation requirements; wherein the target data includes: target sub-database information, target simulation times information and target simulation description information;

and searching in the multi-level index information based on the target data, and determining the corresponding target data file in the target simulation times in the corresponding target sub-database.

In the implementation process, the corresponding target data in the simulation is determined according to the target simulation requirement of the target simulation data, so that the target simulation times information and the target simulation description information in the target database are sequentially searched and matched in the multi-level index information according to the target sub-database information in the target data, and accordingly the corresponding target database, the corresponding target simulation times in the target database and the corresponding target data file in the target simulation times are sequentially determined. And the information of the actual condition of the simulation data is used for carrying out layer-by-layer retrieval in the multi-level index so as to realize the efficient positioning of the data file which finally stores the simulation data, thereby improving the efficiency and accuracy in positioning.

Optionally, the processing the target simulation data based on the target data file includes:

writing the target simulation data into the target data file; or (b)

Reading the stored target simulation data from the target data file; or (b)

Backing up the target data file to backup the target simulation data stored in the target data file; or (b)

And migrating the target data file to migrate the target simulation data stored in the target data file.

In the implementation process, the processing mode of the target simulation data can comprise various operations such as storage, reading, backup or migration after retrieval and positioning, so that the simulation data can be stored rapidly, accurately and reasonably, when related simulation data are required to be called, the corresponding simulation data can be retrieved and positioned rapidly and accurately, and when the corresponding simulation data are required to be backed up or migrated, substantial operations on the simulation data are not required, the backup or migration of the simulation data can be realized by directly backing up or migrating the target data file where the simulation data are located, and the efficiency and the accuracy of the various operations such as storage, retrieval, backup and migration of the simulation data are effectively improved, so that the working efficiency of the urban waterlogging simulation early warning system is improved.

In a second aspect, embodiments of the present application further provide a data processing apparatus, where the apparatus includes:

the index module is used for determining multi-level index information in the storage database;

the positioning module is used for positioning a target data file of the target simulation data according to the multi-level index information and the target data of the target simulation data;

And the processing module is used for processing the target simulation data based on the target data file.

In a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes a memory and a processor, where the memory stores program instructions, and when the processor reads and executes the program instructions, the processor executes steps in any implementation manner of the data processing method.

In a fourth aspect, embodiments of the present application further provide a computer readable storage medium having stored therein computer program instructions which, when read and executed by a processor, perform the steps of any implementation of the data processing method described above.

In summary, the embodiments of the present application provide a data processing method, apparatus, electronic device, and computer readable storage medium, by setting multi-level index information under different directories, to retrieve and locate a data file corresponding to simulation data generated by a city waterlogging simulation early warning system, so that the simulation data is correspondingly processed in the data file, and the processing efficiency of a large amount of simulation data generated by the city waterlogging simulation early warning system is effectively improved, thereby optimizing the early warning effect of the city waterlogging simulation early warning system.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic block diagram of an electronic device according to an embodiment of the present application;

fig. 2 is a schematic flow chart of a data processing method according to an embodiment of the present application;

fig. 3 is a detailed flowchart of step S200 provided in the embodiment of the present application;

fig. 4 is a detailed flowchart of step S210 provided in the embodiment of the present application;

fig. 5 is a detailed flowchart of step S220 provided in the embodiment of the present application;

fig. 6 is a detailed flowchart of step S230 provided in the embodiment of the present application;

fig. 7 is a detailed flowchart of step S300 according to an embodiment of the present application;

fig. 8 is a detailed flowchart of step S400 provided in the embodiment of the present application;

Fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.

Icon: 100-an electronic device; 111-memory; 112-a memory controller; 113-a processor; 114-a peripheral interface; 115-an input-output unit; 116-a display unit; 500-a data processing device; 510-an indexing module; 520-positioning module; 530-a processing module.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present application without making any inventive effort, are intended to be within the scope of the embodiments of the present application.

The existing urban waterlogging simulation early warning system can combine technologies such as weather forecast, physical model and monitoring data, hydrodynamics simulation, artificial intelligence, large-scale parallel calculation and the like, simulate the evolution process of space rainfall, ground water accumulation and pipe network drainage in the whole urban area range in a minute level, and realize real-time data simulation of three-dimensional forecast prediction of urban rainfall and flood process and various data. For example, when the city waterlogging simulation early warning system carries out simulation early warning on a medium-small scale level of 5 meters in a 100 km area, the number of grids obtained by dividing the area during simulation is 400 ten thousand, and rolling simulation is carried out every 5 minutes so as to early warn on water accumulation in the area within a future period of time, for example, 24 hours. The time interval during simulation is 5 minutes, 288 groups of simulation data can be included during one early warning, 400 ten thousand pieces of data can be included in each group of simulation data, namely, the total data quantity of the simulation data generated by one early warning work can reach 11.52 hundred million pieces.

Aiming at a large amount of simulation data generated when the urban waterlogging simulation early warning system is simulated, the conventional database is mainly used for storing the current simulation data. With the increase of the running time of the early warning system, the historical data storage amount is increased, so that the data storage and the search query are slower and slower, and the data backup or migration is difficult when the volume is larger, so that the processing efficiency of the simulation data is lower, and the simulation early warning result of urban waterlogging is adversely affected.

Therefore, in order to solve the above-mentioned problems, the embodiment of the present application provides a data processing method, which is applied to an electronic device, where the electronic device may be an electronic device with a logic computing function, such as a server, a personal computer (Personal Computer, PC), a tablet computer, a smart phone, a personal digital assistant (Personal Digital Assistant, PDA), etc., and is capable of efficiently and accurately processing simulation data generated when simulation is performed in an urban waterlogging simulation early warning system, so that the processing efficiency of the simulation data is effectively improved, and the effect when the waterlogging simulation is optimized.

Optionally, referring to fig. 1, fig. 1 is a schematic block diagram of an electronic device according to an embodiment of the present application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input output unit 115, and a display unit 116. Those of ordinary skill in the art will appreciate that the configuration shown in fig. 1 is merely illustrative and is not limiting of the configuration of the electronic device 100. For example, electronic device 100 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

The above-mentioned memory 111, memory controller 112, processor 113, peripheral interface 114, input/output unit 115 and display unit 116 are electrically connected directly or indirectly to each other to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The processor 113 is used to execute executable modules stored in the memory.

The Memory 111 may be, but is not limited to, a random access Memory (Random Access Memory, RAM), a Read Only Memory (ROM), a programmable Read Only Memory (Programmable Read-Only Memory, PROM), an erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), an electrically erasable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc. The memory 111 is configured to store a program, and the processor 113 executes the program after receiving an execution instruction, and a method executed by the electronic device 100 defined by the process disclosed in any embodiment of the present application may be applied to the processor 113 or implemented by the processor 113.

The processor 113 may be an integrated circuit chip having signal processing capabilities. The processor 113 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (digital signal processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field Programmable Gate Arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. The general purpose processor may be a microprocessor, but in the alternative, it may be any conventional processor or the like.

The peripheral interface 114 couples various input/output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 may be implemented in a single chip. In other examples, they may be implemented by separate chips.

The input-output unit 115 described above is used to provide input data to a user. The input/output unit 115 may be, but is not limited to, a mouse, a keyboard, and the like.

The display unit 116 described above provides an interactive interface (e.g., a user-operated interface) between the electronic device 100 and a user or is used to display image data to a user reference. In this embodiment, the display unit may be a liquid crystal display or a touch display. In the case of a touch display, the touch display may be a capacitive touch screen or a resistive touch screen, etc. supporting single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations simultaneously generated from one or more positions on the touch display, and the sensed touch operations are passed to the processor for calculation and processing. In the embodiment of the present application, the display unit 116 may display multiple data such as a multi-level directory in the storage database, an index corresponding to each level directory, and simulation data to be processed.

The electronic device in this embodiment may be used to execute each step in each data processing method provided in the embodiment of the present application. The implementation of the data processing method is described in detail below by means of several embodiments.

Referring to fig. 2, fig. 2 is a flowchart of a data processing method according to an embodiment of the present application, and the method may include steps S200-S400.

Step S200, determining multi-level index information in a storage database;

when the storage database is used for storing simulation data generated by simulating the urban waterlogging simulation early warning system into the storage database, multi-level catalogues with large-to-small range can be set in the storage database, and corresponding index information during retrieval in each level of catalogues is determined to obtain multi-level index information with different ranges.

Step S300, positioning a target data file of target simulation data according to the multi-level index information and the target data of the target simulation data.

The city waterlogging simulation early warning system can generate a large amount of simulation data, and simulation data needing to be processed in the simulation data can be used as corresponding target simulation data when being processed, so that corresponding positions under corresponding catalogues are determined in each level of index information in the multi-level index information according to the target simulation data, and accordingly corresponding catalogue positions are sequentially determined from large to small until corresponding target data files used for storing the simulation data in a final catalogue are determined, and the corresponding target data files are used as file positions of the simulation data positioned when being retrieved.

Alternatively, multiple data files may be included under the multi-level directory, and the data files may be data structures such as folders that are capable of storing and identifying data.

It should be noted that the target simulation data may be a set of simulation data formed by a plurality of associated simulation data obtained by simulating a certain frame of image in the video, and may process a plurality of different target simulation data at the same time, so as to further improve the processing efficiency of the simulation data.

Step S400, processing the target simulation data based on the target data file.

After locating the corresponding data file, the corresponding processing mode may be selected to perform various processes on the target data file according to the processing requirement of the target simulation data, for example: writing the target simulation data into the target data file, or reading the corresponding target simulation data in the target data file, and the like. And the method can correspondingly store, retrieve, backup, migrate and the like based on the position of the positioned data file.

Optionally, when the data processing method provided by the embodiment of the invention is used for processing the generated simulation data, the simulation data can be quickly and accurately stored to the corresponding position, or the simulation data required to be called can be read from the corresponding position, or the simulation data can be backed up or migrated to the corresponding position, so that the operation efficiency of the urban waterlogging simulation early warning system is improved, and the early warning effect of the urban waterlogging simulation early warning system is optimized.

In the embodiment shown in fig. 2, the processing efficiency of a large amount of simulation data generated by the urban waterlogging simulation early-warning system is effectively improved, so that the early-warning effect of the urban waterlogging simulation early-warning system is optimized.

It should be noted that, the sub-database, the number of simulations in the sub-database, and the three-level different directories of the data files in the number of simulations may exist in the storage database. The multi-level index information corresponding to the multi-level directory may include: first level index information, second level index information, and third level index information.

Optionally, referring to fig. 3, fig. 3 is a detailed flowchart of step S200 provided in the embodiment of the present application, and step S200 may include steps S210-S230.

Step S210, determining first-level index information corresponding to each sub-database in the storage database.

The storage database comprises a plurality of sub-databases, and the urban waterlogging simulation early warning system can simulate based on different sub-databases when simulating. Therefore, when processing simulation data generated by simulation, in order to distinguish different sub-databases during simulation, corresponding first-level index information for searching the sub-databases can be determined according to a plurality of sub-databases serving as catalogues.

Step S220, determining second-level index information corresponding to each simulation number in each sub-database.

When the urban waterlogging simulation early warning system simulates based on the corresponding sub-database, multiple simulation times may exist, and in order to distinguish different simulation times, corresponding second-level index information for searching the simulation times can be determined according to the multiple simulation times as a catalog.

Step S230, determining third-level index information corresponding to each data file in each simulation number.

When the urban waterlogging simulation early warning system is in the corresponding simulation times for simulation, the video can be simulated during simulation, so that multiple groups of different simulation data can be generated according to each frame of image in the video during single simulation. In order to distinguish the number of frames in the simulation times, the third-level index information for searching the data files for storing the simulation data with different frames can be determined according to the data files for storing the simulation data as a catalog.

In the embodiment shown in fig. 3, according to the multi-level index information under different directories, the range corresponding to each directory can be determined from large to small from the sub-database to the frame number of the simulation data, so as to ensure the accuracy of the sub-database, the simulation times or the data file corresponding to each level of index information, and improve the accuracy and the effectiveness of the multi-level index information.

Optionally, referring to fig. 4, fig. 4 is a detailed flowchart of step S210 provided in the embodiment of the present application, and step S210 may include steps S211-S212.

Step S211, obtaining sub-database information of each sub-database in the storage database.

The sub-database information may include: the number, name, user information and other information related to the sub-databases may be searched in the storage database for information related to each sub-database to determine sub-database information corresponding to each sub-database.

For example, the numbers of the sub-databases may be numbers of all sub-databases in the storage database, the number of all sub-databases used by the urban waterlogging simulation early warning system may be predetermined, and unique number information of each sub-database may be sequentially generated according to the number. The name of the sub-database may be name information corresponding to the sub-database, may be a full name in chinese, may be an english abbreviation, or the like. The user information of the sub-database may be account information, such as account number and corresponding password information, etc., when the sub-database is used.

Step S212, corresponding first-level index information is created according to the sub-database information.

The corresponding first-level index information can be created according to the sub-database information corresponding to each sub-database, so that when relevant information of the sub-databases is input, the first-level index information can be used for searching, and the corresponding sub-databases can be determined.

In the embodiment shown in fig. 4, during data positioning, the sub-database corresponding to the simulation of the urban waterlogging simulation early warning system can be accurately and rapidly positioned in the catalog of the sub-database according to the created first-level index information.

Optionally, referring to fig. 5, fig. 5 is a detailed flowchart of step S220 provided in the embodiment of the present application, and step S220 may include steps S221-S222.

Step S221, determining the simulation times based on each sub-database according to the simulation requirements.

When the urban waterlogging simulation early warning system simulates based on each sub-database, multiple simulations may be performed in order to improve accuracy of simulation early warning results. Therefore, the city waterlogging simulation early warning system has corresponding simulation requirements when in simulation work, the simulation requirements can comprise the times requirements for limiting the simulation times, and the simulation times of the city waterlogging simulation early warning system based on the sub-database can be determined according to the simulation requirements.

Step S222, corresponding second-level index information is created according to each simulation number.

In order to distinguish each simulation, each simulation may be identified sequentially according to a preset sequence, for example, different numbers, serial numbers, and the like may be set according to letters, numbers, and the like, for example, the first simulation may be set to be 001, the second simulation may be set to be 002, and the like. After the corresponding simulation times are obtained, corresponding second-level index information can be generated according to the unique identification information such as the serial numbers, the serial numbers and the like corresponding to each simulation time, so that when the related information of the simulation times is input, the second-level index information can be used for searching, and the simulation times in the corresponding sub-database catalogue can be determined.

In the embodiment shown in fig. 5, when data is located, the number of simulations corresponding to the time of the simulation can be accurately and quickly located in multiple simulations in the sub-database.

Optionally, referring to fig. 6, fig. 6 is a detailed flowchart of step S230 provided in the embodiment of the present application, and step S230 may include steps S231-S233.

Step S231, according to the simulation requirement, determining simulation description information when each simulation number is simulated.

In each simulation, if the simulation object is an object containing multiple frames of images or pictures, such as video, each frame of images or pictures has a corresponding set of simulation data. When each frame of image or picture is simulated, the simulation description information limiting the conditions during simulation may be different. Therefore, simulation description information of the urban waterlogging simulation early warning system when simulating each frame of image can be obtained, and the simulation description information can comprise: grid information, index number, index data type, simulation group number, simulation time, time interval and other various different simulation limiting conditions during simulation.

Optionally, when the urban waterlogging simulation early warning system simulates each frame of image, the image can be divided into a plurality of small grid information for improving the accuracy in simulation due to the fact that the image is larger, so that each grid information is simulated, and a group of corresponding simulation data is formed by grid data simulated by all grids. Therefore, the mesh information at the time of simulation may include data such as the total number of mesh information obtained by dividing the image and arrangement information. The number of indexes may be the total number of related information of each grid, and the type of index data may be specific information of related information of each grid, for example, the length and width of each grid, and whether the data of the length, the width, etc. are of integer type or decimal type, etc. The number of simulation groups may be the number of frames corresponding to the image to be simulated in the video, and the simulation time may be the time when the frame image is simulated, or the time when the first frame image is simulated, etc. The time interval may be an interval related to early warning, which is determined according to simulation requirements, for example, early warning results after 5 hours are obtained through simulation, or waterlogging early warning results after 24 hours are obtained through simulation.

Step S232, determining unique identification information of each data file in the simulation times.

Wherein, since each simulation can generate different groups of simulation data, different data files can be set to store different groups of simulation data in order to process each group of simulation data. In order to distinguish a plurality of data files respectively storing different groups of simulation data, each data file recorded by the simulation times can be identified to generate unique identification information.

Alternatively, when generating the identification information of the data file, the corresponding identification information may be generated according to the number of simulation groups of the simulation data, for example, when the number of simulation groups is that representing the simulation data is that of the 3 rd group of simulation data, the representation information may be 0003; each data file may be sequentially numbered according to the number of data files in the number of simulation times, for example, in the case of 128 data files, the identification information of each data file is set to 0001, 0002, 0003, etc. according to the serial numbers of each data file in all data files.

Step S233, third-level index information corresponding to each data file is created according to the simulation description information and the identification information.

After various simulation description information and identification information are acquired, the two information can be combined, so that third-level index information corresponding to each data file is created. The third-level index information generated by the multiple simulation description information and the identification information can occupy corresponding storage lengths, for example, the storage length of 32 bits is occupied, the multiple simulation description information and the identification information are written into the storage lengths in sequence, the identification information can be placed in the storage length of the head part during writing, so that the third-level index information occupies smaller storage positions, the third-level index information can be used as corresponding head data in each data file, and when information related to the data file is input, the third-level index information can be used for searching and corresponding data files placed in the frequency catalog are determined.

Optionally, the sub-database, the simulation times, the number of data files and related information can be set and adjusted according to actual conditions and storage requirements, and the generated first-level index information, second-level index information and third-level index information can be updated correspondingly according to the adjustment of the sub-database, the simulation times and the data files.

In the embodiment shown in fig. 6, third-level index information representing simulation-related information can be created, so that when data is located, a data file corresponding to the simulation actual situation can be accurately and quickly located in the directory of simulation times.

Optionally, referring to fig. 7, fig. 7 is a detailed flowchart of step S300 provided in the embodiment of the present application, and step S300 may include steps S310-S320.

Step S310, determining target data of target simulation data according to the target simulation demand.

Before the urban waterlogging simulation early warning system simulates, the target simulation requirement for generating corresponding target simulation data can be determined, and the target simulation requirement is synchronized into data processing. The target data may include: and the target sub-database information, the target simulation times information, the target simulation description information and the like are related to the sub-databases used for simulation, times and simulation conditions.

Optionally, the target sub-database information may include related information such as names, numbers, user information, etc. of sub-databases used when the urban waterlogging simulation early warning system performs simulation, and the target simulation number information may be the number of times of simulation, for example, first simulation, second simulation, etc., and the target simulation description information may include grid information, index number, index data type, simulation group number, simulation time, and limiting conditions during simulation such as time interval, etc.

Step S320, searching in the multi-level index information based on the target data, and determining the corresponding target data file in the target simulation times in the corresponding target sub-database.

In order to process the stored simulation data correspondingly, the file position during processing the simulation data can be positioned first, the positioning mode can be that the target data of the target simulation data are sequentially searched in multi-level index information, the corresponding target sub-database is determined in the first-level index information according to the target sub-database information in the target data, the corresponding target simulation times are determined in the second-level index information according to the target simulation times information in the target data in the target sub-database, and the data file with the corresponding simulation description information and the identification information is determined in the third-level index information according to the target simulation description information in the target simulation times, so that the data file with the corresponding simulation description information and the identification information is used as the target data file of the final file position obtained through positioning.

For example, when the acquired target data includes "a database, simulation 3, and third frame image", the sub database may be determined to be the a database in the first level index information in the multi-level index information, the number of simulations is determined to be the third simulation in the second level index information, the data file numbered 0003 is determined in the third level index information, and the grid information, the index number, the index data type, the simulation group number, the simulation time, the time interval, and the like in the data file are compared with the relevant data of the target simulation data, and if the data file corresponds to 0003, the data file is taken as the target data file.

In the embodiment shown in fig. 7, the layer-by-layer retrieval is performed in the multi-level index through the information of the actual situation of the simulation data, so that the efficient positioning of the data file which finally stores the simulation data is realized, and the efficiency and accuracy in positioning are improved.

Optionally, referring to fig. 8, fig. 8 is a detailed flowchart of step S400 provided in the embodiment of the present application, and step S400 may include steps S410-S440.

In step S410, the target simulation data is written into the target data file.

When the target simulation data is required to be stored, the target data file can be used as a storage position of the target simulation data, and a plurality of target simulation data are written into the corresponding target data file in batches by groups for storage.

Optionally, when writing, the pointer of the target data file can be directly operated, batch writing can be performed on the data of a plurality of grids in the target simulation data according to groups, tens of millions of grid data can be written at the same time, and the data writing speed is improved, so that the efficiency of storing the simulation data is improved.

Step S420, the stored target simulation data is read from the target data file.

When the city waterlogging simulation early warning system needs to call the stored simulation data, the target data file can be used as a search address obtained after the simulation data is searched, the target simulation data stored in the target data file is used as a search result, and the target simulation data in the target data file can be read according to the pointer position and the data length of the target data file. The method can quickly and accurately perform retrieval and positioning, and realize data calling through retrieval and reading.

Step S430, backing up the target data file to backup the target simulation data stored in the target data file.

When the city waterlogging simulation early warning system needs to backup the stored simulation data, the backup mode can be full backup or time incremental backup. The backup does not need to process the content stored in the target data file, but directly backs up the corresponding target data file, for example: the full-volume backup can directly copy the whole full volume of the target data file to the backup disc, and ensures that the directory structure is consistent with the original storage; the timed incremental backup can copy the target data file to the backup disc in an incremental manner according to the requirement. By carrying out backup processing on the target data file, the backup of the target simulation data stored in the target data file is realized, the difficulty in data backup is reduced, and the effectiveness in backup is improved.

In step S440, the target data file is migrated to migrate the target simulation data stored in the target data file.

When the city waterlogging simulation early warning system needs to migrate the stored simulation data, the stored content in the target data file is not required to be processed, and the corresponding target data file can be directly migrated so as to correspondingly migrate the target simulation data stored in the target data file, so that the difficulty in data migration is reduced, and the effectiveness in migration is improved.

In the embodiment shown in fig. 8, efficiency and accuracy in various operations such as storing, retrieving, backing up and transferring the simulation data are effectively improved, so that efficiency of the urban waterlogging simulation early warning system in working is improved.

Referring to fig. 9, fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application, and the data processing apparatus 500 may include:

an index module 510 for determining multi-level index information in a storage database;

the positioning module 520 is configured to position a target data file of the target simulation data according to the multi-level index information and the target data of the target simulation data;

a processing module 530, configured to process the target simulation data based on the target data file.

Wherein the multi-level index information includes: first level index information, second level index information, and third level index information; in an alternative embodiment, the indexing module 510 may further include: the device comprises a first-level index sub-module, a second-level index sub-module and a third-level index sub-module;

the first-level index sub-module is used for determining first-level index information corresponding to each sub-database in the storage database;

the second-level index sub-module is used for determining second-level index information corresponding to each simulation frequency in each sub-database;

And the third-level index sub-module is used for determining third-level index information corresponding to each data file in each simulation time.

In an alternative embodiment, the first level index sub-module may further include: an acquisition unit and a first construction unit;

the acquisition unit is used for acquiring the sub-database information of each sub-database in the storage database; wherein the sub-database information includes: at least one of the number, name and user information of the sub database;

the first construction unit is used for creating corresponding first-level index information according to the sub-database information.

In an alternative embodiment, the second level index submodule may further include: a number of times unit and a second construction unit;

the frequency unit is used for determining the simulation frequency of the simulation based on each sub-database according to the simulation requirement;

and the second construction unit is used for creating corresponding second-level index information according to each simulation frequency.

In an alternative embodiment, the third level index sub-module may further include: the device comprises a determining unit, an identifying unit and a third constructing unit;

the determining unit is used for determining simulation description information when each simulation frequency is simulated according to the simulation requirement; wherein, the simulation description information includes: at least one of grid information, index number, index data type, simulation group number, simulation time and time interval during simulation;

The identification unit is used for determining unique identification information of each data file in the simulation times;

and the third construction unit is used for creating third-level index information corresponding to each data file according to the simulation description information and the identification information.

In an alternative embodiment, the location module 520 may further include a data determination sub-module and a retrieval sub-module;

the data determining submodule is used for determining target data of target simulation data according to target simulation requirements; wherein the target data includes: target sub-database information, target simulation times information and target simulation description information;

and the retrieval sub-module is used for retrieving in the multi-level index information based on the target data and determining a corresponding target data file in the target simulation times in the corresponding target sub-database.

In an alternative embodiment, the processing module 530 may further include a storage sub-module, a reading sub-module, a backup sub-module, or a migration sub-module;

the storage sub-module is used for writing the target simulation data into the target data file;

the reading sub-module is used for reading the stored target simulation data in the target data file;

the backup sub-module is used for backing up the target data file so as to backup the target simulation data stored in the target data file;

And the migration submodule is used for migrating the target data file so as to migrate the target simulation data stored in the target data file.

Since the principle of the data processing apparatus 500 in the embodiment of the present application for solving the problem is similar to that of the foregoing embodiment of the data processing method, the implementation of the data processing apparatus 500 in the embodiment of the present application may refer to the description in the foregoing embodiment of the data processing method, and the repetition is omitted.

The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any one of the data processing methods provided in the embodiment are executed.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus may be implemented in other ways. The apparatus embodiments described above are merely illustrative, for example, block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of devices according to various embodiments of the present application. In this regard, each block in the block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams, and combinations of blocks in the block diagrams, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, the functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application, and various modifications and variations may be suggested to one skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

Claims

1. A method of data processing, the method comprising:

determining multilevel index information in a storage database;

processing the target simulation data based on the target data file;

wherein the multi-level index information includes: first level index information, second level index information, and third level index information; the determining to store the multi-level index information in the database includes: determining the first-level index information corresponding to each sub-database in the storage database; determining the second-level index information corresponding to each simulation frequency in each sub-database; and determining the third-level index information corresponding to each data file in each simulation time.

2. The method of claim 1, wherein said determining the first level index information corresponding to each sub-database in the storage database comprises:

3. The method of claim 1, wherein determining the second level index information corresponding to each simulation number in each sub-database comprises:

4. The method of claim 1, wherein said determining the third level index information corresponding to each data file in each of the simulation times comprises:

5. The method of claim 1, wherein locating the target data file of the target simulation data based on the multi-level index information and the target data of the target simulation data comprises:

6. The method of claim 1, wherein the processing the target simulation data based on the target data file comprises:

writing the target simulation data into the target data file; or (b)

Reading the stored target simulation data from the target data file; or (b)

7. A data processing apparatus, the apparatus comprising:

the processing module is used for processing the target simulation data based on the target data file;

wherein the multi-level index information includes: first level index information, second level index information, and third level index information; the index module comprises: the device comprises a first-level index sub-module, a second-level index sub-module and a third-level index sub-module; the first-level index sub-module is used for determining the first-level index information corresponding to each sub-database in the storage database; the second-level index sub-module is used for determining second-level index information corresponding to each simulation frequency in each sub-database; and the third-level index sub-module is used for determining the third-level index information corresponding to each data file in each simulation frequency.

8. An electronic device comprising a memory and a processor, the memory having stored therein program instructions which, when executed by the processor, perform the steps of the method of any of claims 1-6.

9. A computer readable storage medium, characterized in that the readable storage medium has stored therein computer program instructions which, when executed by a processor, perform the steps of the method of any of claims 1-6.