CN117272933B - Concrete pavement report data storage method - Google Patents
Concrete pavement report data storage method Download PDFInfo
- Publication number
- CN117272933B CN117272933B CN202311567196.8A CN202311567196A CN117272933B CN 117272933 B CN117272933 B CN 117272933B CN 202311567196 A CN202311567196 A CN 202311567196A CN 117272933 B CN117272933 B CN 117272933B
- Authority
- CN
- China
- Prior art keywords
- information
- region
- repetition rate
- detection item
- divided
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 115
- 238000013500 data storage Methods 0.000 title abstract description 11
- 238000001514 detection method Methods 0.000 claims abstract description 188
- 238000013519 translation Methods 0.000 claims abstract description 28
- 238000007906 compression Methods 0.000 claims abstract description 12
- 230000006835 compression Effects 0.000 claims abstract description 12
- 230000011218 segmentation Effects 0.000 claims abstract description 5
- 230000008569 process Effects 0.000 claims description 37
- 238000005192 partition Methods 0.000 claims description 25
- 238000010276 construction Methods 0.000 claims description 9
- 108091026890 Coding region Proteins 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000009825 accumulation Methods 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 7
- 239000013598 vector Substances 0.000 description 7
- 238000007689 inspection Methods 0.000 description 4
- 238000011084 recovery Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000013441 quality evaluation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention relates to the technical field of data processing, in particular to a concrete pavement report data storage method, which comprises the following steps: obtaining a segmentation area according to the scanning data; obtaining the region information repetition rate according to the information quantity of the divided regions; obtaining the region repetition rate according to the region information repetition rate; obtaining the importance degree of the regional information according to the regional information repetition rate; obtaining the region importance degree according to the region repetition rate; obtaining the importance degree of the information group according to the importance degree of the regional information and the importance degree of the regional information; obtaining the loss degree according to the region information repetition rate; obtaining a translation index according to the importance degree of the information group; obtaining priority according to the loss degree and the translation index; and performing compression storage according to the priority, and performing quality detection supervision. The invention reduces the possibility of losing important information in storage and ensures the effective storage of data.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a concrete pavement report data storage method.
Background
The quality inspection and supervision system for concrete pavement construction is a B/S (browser/Server) framework inspection information supervision system combined with an engineering quality inspection management system, a quality inspector can fill in concrete pavement reports after inspecting the concrete pavement of each construction, a large number of concrete pavement reports are generated due to huge engineering quantity, and in order to facilitate engineering supervision departments to supervise construction processes, the obtained concrete pavement reports are required to be converted into electronic reports, and the electronic reports are compressed and stored.
The traditional method for compressing the electronic report mainly comprises the steps of determining storage priority and encoding length according to the importance degree of the content, so as to compress and store, but because the text quantity required to be stored is large, and if some important information is lost in the storage process, huge workload is caused by information restoration, and the processing efficiency is reduced.
Disclosure of Invention
The invention provides a concrete pavement report data storage method, which aims to solve the existing problems: the amount of text to be stored is large, and if some important information is lost in the storage process, huge workload is caused by information restoration, and the processing efficiency is reduced.
The invention relates to a concrete pavement report data storage method which adopts the following technical scheme:
the method comprises the following steps:
collecting a plurality of pieces of information in each scanning data in a concrete pavement report of a plurality of detection items;
dividing the information in the scanning data into a plurality of divided areas; obtaining the region information repetition rate of the information in each divided region according to the information quantity of the same information in the same divided region in all the detection items; obtaining the region repetition rate of each divided region according to the region information repetition rate of the information in the same divided region in all the detection items; obtaining the importance degree of the regional information of the information in each divided region according to the regional information repetition rate and the proportion of the information quantity with the same information in the same divided region in all detection items; obtaining the region importance degree of the information in each divided region according to the region repetition rate and the ratio of the information quantity with the same information in the same divided region in all detection items; taking information in the same partition area of any two detection items as a reset information group; obtaining the importance degree of the information group of each reset information group according to the occupation ratio condition and the importance degree of the area information of the same segmentation area in all detection items;
Obtaining the loss degree of each reset information group according to the total amount of the regional information repetition rate of all the information in the reset information group, wherein the loss degree is used for describing the loss probability of the reset information group; obtaining a translation index of each reset information group according to the duty ratio of the importance degree of the information groups in all the reset information groups; obtaining the priority of each divided area according to the loss degree and the translation index;
and performing compression storage according to the priority of each partition area.
Preferably, the method for obtaining the region information repetition rate of the information in each divided region according to the information quantity with the same information in the same divided region in all the detection items includes the following specific steps:
recording any one detection item as a current detection item;
in the method, in the process of the invention,the region information repetition rate of the information in the f-th divided region in the current detection item is represented; />Representing the quantity of information in the f-th divided area in all the detection items and the same content as the information in the f-th divided area in the current detection item; />Representing preset super parameters; n represents the total number of detected items.
Preferably, the obtaining the region repetition rate of each divided region according to the region information repetition rate of the information in the same divided region in all the detection items includes the following specific methods:
And (3) recording the accumulated result of the region information repetition rate of the information in the f-th divided region in all detection items as the region repetition rate of the f-th divided region.
Preferably, the obtaining the importance degree of the area information of the information in each divided area according to the repetition rate of the area information and the occupation ratio of the information quantity with the same information in the same divided area in all the detection items includes the following specific methods:
recording any one detection item as a current detection item;
in the method, in the process of the invention,the regional information importance degree of the information in the f-th divided region in the current detection project is represented; />The region information repetition rate of the information in the f-th divided region in the current detection item is represented; />Representing the same quantity of information content in the f-th divided area in all detection items and the f-th divided area in the current detection item; n represents the total number of detection items; />An exponential function based on a natural number is represented.
Preferably, the obtaining the region importance degree of the information in each divided region according to the region repetition rate and the ratio of the information quantity with the same information in the same divided region in all the detection items includes the following specific methods:
Recording any one detection item as a current detection item;
in the method, in the process of the invention,region representing the f-th divided region in the current detection itemDomain importance degree; />The region repetition rate of the f-th divided region in the current detection item is represented; />Representing the same quantity of information content in the f-th divided area in all detection items and the f-th divided area in the current detection item; n represents the total number of detection items; />An exponential function based on a natural number is represented.
Preferably, the obtaining the importance degree of the information group of each reset information group according to the occupation ratio condition and the importance degree of the area information of the same partition area in all the detection items includes the following specific methods:
recording any one detection item as a current detection item;
in the method, in the process of the invention,an information group importance degree indicating information of the f-th divided region in the c-th detection item and information of the f-th divided region in the t-th detection item; />The region importance degree of the f-th divided region in the current detection item is represented; />The region information importance degree of the information in the f-th divided region in the t-th detection item is represented; />Representing preset super parameters; />The importance degree of the information of the f-th divided area in the c-th detection item is represented; / >A maximum value representing semantic correlation of information of the f-th divided region in the c-th detection item and information of the f-th divided region in the t-th detection item.
Preferably, the obtaining the loss degree of each reset information group according to the total amount of the region information repetition rate of all the information in the reset information group includes the following specific methods:
presetting a default loss rateThe method comprises the steps of carrying out a first treatment on the surface of the Referring to an acquisition method of the region information repetition rate of each piece of information in each partitioned region, obtaining the repetition rate of each piece of information in each reset information group, and recording the accumulation result of the repetition rate of each piece of information in the reset information group as the repetition rate of the reset information group; recording any one detection item as a current detection item; the loss degree of each reset information group is obtained according to the default loss rate, and the calculation method is as follows:
in the method, in the process of the invention,indicating the loss degree of the h reset information group in the current detection item; />Representing the repetition rate of the h-th reset information group; />An exponential function based on a natural number; />Indicating a default loss rate.
Preferably, the obtaining the translation index of each reset information group according to the duty ratio of the importance degree of the information groups in all the reset information groups includes the following specific methods:
Recording any one detection item as a current detection item;
in the method, in the process of the invention,representing a translation index of an h reset information group in a sigmoid function in a current detection item;a maximum value indicating the importance degree of the h reset information group in all the detection items; />Indicating the number of h reset information groups in all detection items.
Preferably, the obtaining the priority of each partition area according to the loss degree and the translation index includes the following specific methods:
recording any one detection item as a current detection item;
inputting the loss degree of the h reset information group in the current detection item and the translation index of the h reset information group in the current detection item in the sigmoid function as independent variables of the sigmoid function into the sigmoid function, and marking the output result of the sigmoid function as the priority of the h reset information group; for the partition area to which the h reset information group belongs, referring to a priority acquisition method of the h reset information group to obtain the priority of the rest reset information groups in each partition area; the result of the accumulation of the priorities of all the reset information groups in each divided area is then noted as the priority of each divided area.
Preferably, the compression storage is performed according to the priority of each partition area, which comprises the following specific methods:
the priority of each partition area is recorded as the priority of each data in each partition area;
in the method, in the process of the invention,representing an initial repetition rate at which each data in the f-th partitioned area is encoded; />Representing preset super parameters; />Representing the priority of each data in the f-th partitioned area; />An exponential function based on a natural number;
acquiring an initial repetition rate of each divided region; recording any detection item as a target detection item, performing Huffman coding and splicing on each data in an f-th division area in the target detection item to obtain a section of total coding sequence, recording the section of total coding sequence as an initial coding a of the f-th division area, and recording the spliced result of all initial total coding in all detection items as an initial total coding a1 of the f-th division area;
if the initial repetition rate of the f-th divided region is smaller than T1, the coding length of the initial total coding is required to be increased, the result of upward rounding of the initial repetition rate of the f-th divided region is recorded as the repetition number Q, the result after splicing a1 and Q a is recorded as the second total coding, the 8-bit binary conversion result of Q is recorded as the identification bit, and the result after splicing the second total coding and the identification bit is recorded as the final total coding;
If the initial repetition rate of the f-th divided region is greater than or equal to T1, the coding length of the initial total coding needs to be reduced, the result of upward rounding of the product of the coding length of the initial total coding and the initial repetition rate is recorded as a reduction number Q1, the 8-bit binary conversion result of Q1 is recorded as a second identification bit, Q1 a is deleted from the initial position of the initial total coding, the obtained result is recorded as a third total coding, and the result obtained by splicing the third total coding and the second identification bit is recorded as a final total coding;
and acquiring the final total code of each divided area, and then storing information into a concrete pavement construction quality detection and supervision system according to the order of the priority from large to small.
The technical scheme of the invention has the beneficial effects that: obtaining an area information repetition rate and an area repetition rate according to the scanning data, obtaining an area information importance degree and an area importance degree according to the area information repetition rate and the area repetition rate, obtaining an information group importance degree of a reset information group according to the area information importance degree and the area importance degree, obtaining a loss degree according to the information group importance degree, obtaining a priority according to the loss degree, and performing compression storage according to the priority, so that effective storage of data can be ensured even under the condition of information recording errors, the information storage capacity is not expanded, the possibility of losing important information is reduced, the compression degree of the information is as smaller as possible, and the compression rate is lower, thereby ensuring the integrity of the information in the information storage process.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of the steps of a method for storing concrete pavement report data according to the present invention.
Detailed Description
In order to further describe the technical means and effects adopted by the invention to achieve the preset aim, the following detailed description refers to the specific implementation, structure, characteristics and effects of a concrete pavement report data storage method according to the invention with reference to the attached drawings and the preferred embodiment. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following specifically describes a concrete pavement report data storage method provided by the invention with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of steps of a concrete pavement report data storage method according to an embodiment of the present invention is shown, the method includes the following steps:
step S001: and collecting scanning data of a plurality of concrete pavement reports.
It should be noted that, the method for compressing the electronic report in the conventional method mainly determines the storage priority and the encoding length according to the importance degree of the content, so as to perform compression storage, but because the text amount required to be stored is large, and if some important information is lost in the storage process, huge workload is caused by information recovery, and the processing efficiency is reduced. To this end, the present embodiment proposes a concrete pavement report data storage method.
Specifically, in order to implement the method for storing concrete pavement report data provided in this embodiment, firstly, scan data of a concrete pavement report needs to be collected, and the specific process is as follows: the concrete pavement reports of 20 detection items such as thickness, flatness and the like are scanned by using a scanner, and the concrete pavement reports are ordered according to the scanning time sequence. Each detection item corresponds to one concrete pavement report, the typesetting format of each concrete pavement report is completely consistent, and 30 forms such as a person filling a form, an engineering address and the like exist at the same time; and respectively marking the contents such as cities filled in the table phase of each concrete pavement report as information, wherein the semantic meaning of the corresponding content expression is the content of the corresponding information, for example, in the engineering address table phase, the filled cities are the information in the engineering address table phase, and the semantic meaning of the cities is the content of the information in the engineering address table phase. All information contents in the table phase can be obtained according to OCR character recognition technology, the technology is the prior art, the specific description is not carried out in the embodiment, the information contents are all printed fonts, the formats are Song Tixiao four formats, and the filling positions are fixed and centered.
So far, the scanning data of all concrete pavement reports are obtained through the method.
Step S002: dividing the information in the scanning data into a plurality of divided areas; obtaining the region information repetition rate of the information in each divided region according to the information quantity of the same information in the same divided region in all the detection items; obtaining the region repetition rate of each divided region according to the region information repetition rate of the information in the same divided region in all the detection items; obtaining the importance degree of the regional information of the information in each divided region according to the regional information repetition rate and the proportion of the information quantity with the same information in the same divided region in all detection items; obtaining the region importance degree of the information in each divided region according to the region repetition rate and the ratio of the information quantity with the same information in the same divided region in all detection items; acquiring a plurality of reset information groups; and obtaining the importance degree of the information group of each reset information group according to the occupation ratio condition and the importance degree of the area information of the same division area in all detection items.
In this embodiment, the a-th detection item is taken as an example of the current detection item, and there are a lot of information with high repetition rate in the concrete pavement report, for example: detecting the name and the engineering address of a person, etc.; the information with lower repetition rate is mainly evaluation opinion of the inspection person on the construction engineering quality, and the like. In the actual observation process, the information of main interest is often information with lower repetition rate, so that the information with higher repetition rate in the concrete pavement report can be compressed in order to improve the data processing efficiency. The scanning data can be subjected to region segmentation, the importance degree of each segmented region is obtained according to the repetition rate of each segmented region, and data compression of different degrees is performed according to the importance degree of each segmented region, so that the effective improvement of the data processing efficiency is realized.
Specifically, the area division is performed on the scanned data by using a fast R-CNN block detection algorithm to obtain a plurality of division areas, where the fast R-CNN block detection algorithm is in the prior art, and this embodiment is not described in detail. It should be noted that each partition area only contains one table phase, and each table phase has one piece of information in each detection item; since the typesetting formats of different detection reports are completely consistent, the sizes and positions of the segmentation areas obtained on different scanned images are the same.
1. And obtaining the repetition rate.
Specifically, the method for calculating the repetition rate of each piece of information in each divided area is as follows:
in the method, in the process of the invention,the repetition rate of the information in the f-th divided area in the current detection item is expressed and is recorded as the repetition rate of the area information; />Representing the quantity of information in the f-th divided area in all the detection items and the same content as the information in the f-th divided area in the current detection item; />Representing a preset hyper-parameter, preset +.>For preventing denominator from being 0; n represents the total number of detection items; />Representing the same total number of contents of all information in the f-th divided area in all the detection items.
The repetition rate of each piece of information in each divided area in each detection item is obtained through the repetition rate formula of the information.
In the method, in the process of the invention,the repetition rate of the f-th divided area in the current detection item is expressed and is recorded as the area repetition rate; />A region information repetition rate indicating information in the f-th divided region in the a-th detection item; n represents the total number of detected items.
The repetition rate of each divided region in each detection item is obtained by the above-described repetition rate formula of the divided region.
2. The importance level is obtained.
The lower the repetition rate of the f-th divided area, the smaller the amount of data contained in the f-th divided area, the greater the difficulty of restoration after information loss, and the higher the importance of the f-th divided area, and the higher the priority of the f-th divided area in compression storage. The importance degree of each divided region can be obtained according to the repetition rate of each divided region.
When the scan data is compressed and stored, due to the influence of factors such as the performance of the system and the stability of the cloud server, partial non-stored information of the stored scan data may be lost during storage. If the scan data is lost, the workload of recovering the scan data is huge, but in the actual process, the importance degree of the division area to which the information belongs is often larger because the information of which the quality evaluation is critical is carried out in the scan data, so that the evaluation of the engineering quality is not affected as long as the information in the division area with larger importance degree in the scan data is not lost. Therefore, the importance degree of the scanning data can be obtained and the priority of storing the divided areas can be obtained, and the storage safety of the scanning data can be improved.
In general, the lower the repetition rate, the greater the probability that a segmented region belongs to an important segmented region in the scan data, but other special cases may exist as well: the information in the non-important division area in the original scan data is changed due to the operator's mistake, so that the repetition rate of the non-important division area in the original scan data is low, and the important division area is mistakenly considered to be the wrong division area.
Further, the method of calculating the importance degree of each piece of information of each divided area is as follows:
in the method, in the process of the invention,the importance degree of the information in the f-th divided area in the current detection item is marked as the importance degree of the area information; />The repetition rate of the information in the f-th divided area in the current detection item is represented, namely the area information repetition rate; />Representing the same quantity of information content in the f-th divided area in all detection items and the f-th divided area in the current detection item; n represents the total number of detection items; />An exponential function based on a natural number is represented.
The importance degree of each piece of information in each divided area in each detection item is obtained through the information importance degree formula.
The method for calculating the importance degree of each divided region is as follows:
in the method, in the process of the invention,the importance degree of the f-th divided area in the current detection item is marked as the area importance degree; />Representing the repetition rate of the f-th divided area in the current detection item, namely the area repetition rate; />Representing the same quantity of information content in the f-th divided area in all detection items and the f-th divided area in the current detection item; n represents the total number of detection items; />An exponential function based on a natural number; />The degree of confusion of the f-th divided area is shown, and if the degree of confusion is larger, it is explained that the greater the data change of the f-th divided area is, the greater the restoration difficulty is, and therefore the greater the importance of the f-th divided area is.
So far, the importance degree of each divided area in each detection item is obtained through the importance degree formula.
3. The importance degree of the information group is acquired.
It should be noted that, when calculating the importance level of the same table phase information, the importance level of the same information in different detection items is different due to the error of part of information, and at this time, the importance level of the information is calculated by resetting through a semantic vector algorithm: for example, the actual height of the bridge is 10m, and the degree of importance of the calculation of the information is great due to the fact that the handwriting is 10mm, but the degree of importance of the information is the same as that of the corresponding information in other detection items. Since the semantic vector algorithm, an algorithm for generating text vectors, can well identify the error problem, the embodiment performs semantic information judgment according to the Word2Vec semantic vector algorithm. The Word2Vec semantic vector algorithm is the prior art, and this embodiment is not described by taking two information as an example, because the Word2Vec semantic vector algorithm needs to input at least two information to obtain a corresponding result.
Specifically, the method for calculating the importance degree of the information group of each pair of information according to the relation of different information in the same surface phase is as follows:
in the method, in the process of the invention,an information group importance degree indicating information of the f-th divided region in the c-th detection item and information of the f-th divided region in the t-th detection item; />The region importance degree of the f-th divided region in the current detection item is represented; />The region information importance degree of the information in the f-th divided region in the t-th detection item is represented; />Representing a preset hyper-parameter, preset +.>For preventing denominator from being 0; />The importance degree of the information of the f-th divided area in the c-th detection item is represented;/>a maximum value representing semantic correlation of information of the f-th divided region in the c-th detection item and information of the f-th divided region in the t-th detection item; />The ratio of the importance of the information of the f-th divided area in the c-th detection item to the importance of the information of the f-th divided area in the t-th detection item is represented.
It should be noted that in addition to this,the difference of importance levels of the two is shown, because when information under the same table is stored, the priority of the storage is the same, but the information under the table is abnormal due to the error of the information, and abnormal information exists, and the importance level of the abnormal information needs to be larger than the actual importance level, so the importance level after the information matching is added to the original importance level to show the importance level after the information is reset. / >Can be directly obtained by the Word2Vec semantic vector algorithm in the prior art.
Specifically, resetting the importance degree of all the information of each divided area in the current detection item according to the importance degree formula of the information group, wherein the specific process is as follows: taking two pieces of information of adjacent detection items in the current table phase as a group of reset information groups; inputting the reset information group into an information group importance degree formula, and taking the obtained result as the information group importance degree of the reset information group.
The information group importance degree of all the reset information groups in each partition area of the current detection item is obtained through the method, and the information group importance degree of all the reset information groups in each detection item is obtained.
Step S003: obtaining the loss degree of each reset information group according to the total amount of the regional information repetition rate of all the information in the reset information group; obtaining a translation index of each reset information group according to the duty ratio of the importance degree of the information groups in all the reset information groups; and obtaining the priority of each divided area according to the loss degree and the translation index.
It should be noted that, in the storage process, the split area in the scan data may have information lost, and the reset information set with a higher importance degree of the information set has a higher recovery difficulty after being lost in the storage process, and the reset information set with a lower importance degree of the information set has a lower recovery difficulty after being lost in the storage process, and in most cases, the recovery process is not required. The priority at which the set of reset information is stored may be determined based on the extent to which the set of reset information may be lost during storage.
Specifically, in the information storage, due to the influence of factors such as equipment, network and the like, a certain default loss rate exists in the information storageWherein the present embodiment is +.>The embodiment is not particularly limited, and is described by taking 0.02 as an example, wherein +_>Depending on the particular implementation. Taking information in the same partition area of any two detection items as a reset information group; and referring to an acquisition method of the repetition rate of each piece of information in each partition area, obtaining the repetition rate of each piece of information in each reset information group, and recording the accumulated result of the repetition rate of each piece of information in the reset information group as the repetition rate of the reset information group. The loss degree of each reset information group is obtained according to the default loss rate, and the calculation method is as follows:
in the method, in the process of the invention,indicating the loss degree of the h reset information group in the current detection item; />Representing the repetition rate of the h-th reset information group; />An exponential function based on a natural number; />Indicating a default loss rate.
So far, the loss degree of each reset information group in each detection item is obtained through the loss degree formula.
It should be further noted that, since the greater the loss degree of the reset information group, the description is made of the greater the loss degree of the other reset information groups, and the h reset information group is lost while waiting for storage, in order to reduce the possibility of the h reset information group being lost, the h reset information group needs to be stored preferentially, and the higher the priority of the h reset information group is. In order to better represent the priority, the embodiment uses an improved sigmoid function to represent the priority, because the loss degree and the repetition rate are in a direct proportion relation. Whereas the conventional sigmoid function is a decreasing function over the real set R, it is necessary to analyze the distribution of information to obtain a translation index. And because the translation index indicates the degree of loss at different levels The distribution of the lower function value range, therefore, requires that the translation index thereof be obtained according to the importance of the information. The sigmoid function is the prior art, and this embodiment is not described.
Further, a translation index is obtained according to the importance degree of the information group, and the calculation method is as follows:
in the method, in the process of the invention,representing a translation index of an h reset information group in a sigmoid function in a current detection item;a maximum value indicating the importance degree of the h reset information group in all the detection items; />Representing the number of h reset information groups in all detection items; since the reset information groups with higher importance degree have higher priority when compression is performed and the number of the encoding values is larger, and the reset information groups with smaller number have higher possibility of being completely lost, the priority of the reset information groups with smaller number when data storage encoding is performed can be ensured by multiplying the importance degree of the information groups by the corresponding number as a translation index.
Further, the expression of the conventional sigmoid function is as follows:
in the method, in the process of the invention,an exponential function based on a natural number; x represents an argument of a conventional sigmoid function;the dependent variables representing the conventional sigmoid function.
In this embodiment, the expression of the improved sigmoid function according to the translation index and the loss degree is as follows:
wherein x represents an argument that improves a sigmoid function;a dependent variable representing an improvement to the sigmoid function; />Representing the translation index of the reset information group corresponding to x in the current detection item in the improved sigmoid function; since the argument of the modified sigmoid function adds a translation exponent and does not take the opposite number from the argument of the conventional sigmoid function, the modified sigmoid function is an increasing function over the real set R.
The embodiment marks the loss degree of the h reset information group in the current detection item asTaking->The expression obtained is as follows:
in the method, in the process of the invention,indicating the loss degree of the h reset information group in the current detection item; />Representing the translation index of the h reset information group in the current detection item in the improved sigmoid function; />Indicating the priority of the h reset information group in the current detection item.
After the translation index of the h reset information group is obtained, taking the summation result of the translation index and the loss degree of the h reset information group as an independent variable input improved sigmoid function, and marking the output result as the priority of the h reset information group; for the partition area to which the h reset information group belongs, referring to a priority acquisition method of the h reset information group to obtain the priority of the rest reset information groups in each partition area; the result of the accumulation of the priorities of all the reset information groups in each divided area is then noted as the priority of each divided area.
Thus, the method takes the h reset information group in the f divided area in the a detection item as an example, and analyzes the priority of the f divided area in the a detection item. The priority of each divided area in each detection item is obtained according to the method described above in the present embodiment.
Step S004: and performing compression storage according to the priority of each partition area.
It should be noted that, the huffman coding algorithm is used to code according to the priority of each divided area in each detection item. As can be seen from the steps S002 and S003, if the current each divided area contains less information, the lower the repetition rate of the current each divided area, the greater the possibility that the current each divided area is lost when storing, so the longer the encoding length of the current each divided area and the higher the priority of storing when encoding information, thereby preventing the information from being lost in the storing process.
It should be further noted that, each piece of data of each divided area is encoded according to the order of priority of each divided area, and the encoding needs to be judged according to the priority, so as to determine whether to increase the encoding length, keep the encoding length unchanged, or decrease the encoding length.
Specifically, the priority of each partition area is recorded as the priority of each data in each partition area, wherein the calculation method of the initial repetition rate for performing repetition coding or deletion coding is as follows:
in the method, in the process of the invention,representing an initial repetition rate at which each data in the f-th partitioned area is encoded; />Representing a preset hyper-parameter, preset +.>For enlarging->；/>Representing the priority of each data in the f-th partitioned area; />An exponential function based on a natural number; the greater the priority, the greater the initial repetition rate of the code.
An initial repetition rate threshold T1 is preset, where the embodiment is described by taking t1=1 as an example, and the embodiment is not specifically limited, where T1 may be determined according to the specific implementation situation.
And (3) coding and splicing each piece of data in the f-th partitioned area in the current detection item to obtain a section of total coding sequence, marking the section of total coding sequence as an initial coding a of the f-th partitioned area, and marking the result of splicing all initial total codes in all detection items as an initial total coding a1 of the f-th partitioned area.
If the initial repetition rate of the f-th divided region is smaller than T1, the coding length of the initial total code needs to be increased, the result of rounding up the initial repetition rate of the f-th divided region is denoted as the repetition number Q, the result of splicing a1 and Q a is denoted as the second total code, the 8-bit binary conversion result of Q is denoted as the identification bit, and the result of splicing the second total code and the identification bit is denoted as the final total code, for example: the initial code of a certain divided area in a certain detection item is 10110101, the initial total code of a certain divided area in all detection items is 10110101 10110101, and the final total code in all detection items is 10110101 10110101 10110101 10110101 10110101 10110101 10110101 10110101 10110101 10110101 00001000, wherein 00001000 represents an identification bit, 00001000 represents 8-bit binary numbers of 8, and the divided area is repeated 8 times when coding;
If the initial repetition rate of the f-th partition area is greater than or equal to T1, the coding length of the initial total code needs to be reduced, the result of the product of the coding length of the initial total code and the initial repetition rate being rounded up is denoted as the number of times Q1, the 8-bit binary conversion result of Q1 is denoted as the second identification bit, Q1 a is deleted from the starting position of the initial total code, the obtained result is denoted as the third total code, and the result after the third total code and the second identification bit are spliced is denoted as the final total code, for example: if the initial code of a certain divided area in a certain detection item is "10111", the initial total code of a certain divided area in all detection items is "10111 10111 10111 10111 10111 10111 10111 10111 10111 10111", and the final total code is "10111 10111 10111 10111 10111 10111 00000110", wherein "00000110" is the second identification bit, 00000110 represents an 8-bit binary number of 4, and the number of times of reduction of the divided area in coding is 4.
Obtaining the final total code of each divided area, and then storing information to a concrete pavement construction quality detection and supervision system according to the order of the priority from large to small; and then decoding according to the 8-bit binary number after final total coding when the construction quality of the concrete pavement is detected, and finishing the restoration of the stored information in the data equipment for quality evaluation by a supervision department, thereby realizing the supervision of the construction engineering quality. The encoding method is huffman encoding, the decoding method is huffman tree, the huffman encoding and the huffman tree are the prior art, and this embodiment will not be described.
This embodiment is completed.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the invention, but any modifications, equivalent substitutions, improvements, etc. within the principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method for storing concrete pavement report data, the method comprising the steps of:
collecting a plurality of pieces of information in each piece of scanning data in a concrete pavement report of a plurality of detection projects;
dividing the information in the scanning data into a plurality of divided areas; obtaining the region information repetition rate of the information in each divided region according to the information quantity of the same information in the same divided region in all the detection items; obtaining the region repetition rate of each divided region according to the region information repetition rate of the information in the same divided region in all the detection items; obtaining the importance degree of the regional information of the information in each divided region according to the regional information repetition rate and the proportion of the information quantity with the same information in the same divided region in all detection items; obtaining the region importance degree of the information in each divided region according to the region repetition rate and the ratio of the information quantity with the same information in the same divided region in all detection items; taking information in the same partition area of any two detection items as a reset information group; obtaining the importance degree of the information group of each reset information group according to the occupation ratio condition and the importance degree of the area information of the same segmentation area in all detection items;
Obtaining the loss degree of each reset information group according to the total amount of the regional information repetition rate of all the information in the reset information group, wherein the loss degree is used for describing the loss probability of the reset information group; obtaining a translation index of each reset information group according to the duty ratio of the importance degree of the information groups in all the reset information groups; obtaining the priority of each divided area according to the loss degree and the translation index;
and performing compression storage according to the priority of each partition area.
2. The method for storing concrete pavement report data according to claim 1, wherein the obtaining the region information repetition rate of the information in each divided region according to the information quantity having the same information in the same divided region in all the detection items comprises the specific steps of:
recording any one detection item as a current detection item;
in the method, in the process of the invention,the region information repetition rate of the information in the f-th divided region in the current detection item is represented; />Representing the quantity of information in the f-th divided area in all the detection items and the same content as the information in the f-th divided area in the current detection item; />Representing preset super parameters; n represents the total number of detected items.
3. The method for storing concrete pavement report data according to claim 1, wherein the obtaining the region repetition rate of each divided region according to the region information repetition rate of the information in the same divided region in all the detection items comprises the following specific steps:
And (3) recording the accumulated result of the region information repetition rate of the information in the f-th divided region in all detection items as the region repetition rate of the f-th divided region.
4. The method for storing report data of concrete pavement according to claim 1, wherein the obtaining the importance degree of the regional information of the information in each divided region according to the repetition rate of the regional information and the ratio of the information quantity having the same information in the same divided region in all the detection items comprises the following specific steps:
recording any one detection item as a current detection item;
in the method, in the process of the invention,the regional information importance degree of the information in the f-th divided region in the current detection project is represented; />The region information repetition rate of the information in the f-th divided region in the current detection item is represented; />Representing the same quantity of information content in the f-th divided area in all detection items and the f-th divided area in the current detection item; n represents the total number of detection items;an exponential function based on a natural number is represented.
5. The method for storing concrete pavement report data according to claim 1, wherein the obtaining the regional importance of the information in each divided region according to the regional repetition rate and the ratio of the information quantity having the same information in the same divided region in all the detection items comprises the following specific steps:
Recording any one detection item as a current detection item;
in the method, in the process of the invention,the region importance degree of the f-th divided region in the current detection item is represented; />Representing the current inspectionMeasuring the region repetition rate of the f-th divided region in the project; />Representing the same quantity of information content in the f-th divided area in all detection items and the f-th divided area in the current detection item; n represents the total number of detection items; />An exponential function based on a natural number is represented.
6. The method for storing concrete pavement report data according to claim 1, wherein the obtaining the importance of each reset information group according to the area importance and the occupation ratio of the importance of the same divided area in all the detection items comprises the following specific steps:
recording any one detection item as a current detection item;
in the method, in the process of the invention,an information group importance degree indicating information of the f-th divided region in the c-th detection item and information of the f-th divided region in the t-th detection item; />The region importance degree of the f-th divided region in the current detection item is represented; />The region information importance degree of the information in the f-th divided region in the t-th detection item is represented; / >Representing preset super parameters;the importance degree of the information of the f-th divided area in the c-th detection item is represented; />A maximum value representing semantic correlation of information of the f-th divided region in the c-th detection item and information of the f-th divided region in the t-th detection item.
7. The method for storing concrete pavement report data according to claim 1, wherein the obtaining the loss degree of each reset information group according to the total amount of the regional information repetition rate of all the information in the reset information group comprises the following specific steps:
presetting a default loss rateThe method comprises the steps of carrying out a first treatment on the surface of the Referring to an acquisition method of the region information repetition rate of each piece of information in each partitioned region, obtaining the repetition rate of each piece of information in each reset information group, and recording the accumulation result of the repetition rate of each piece of information in the reset information group as the repetition rate of the reset information group; recording any one detection item as a current detection item; the loss degree of each reset information group is obtained according to the default loss rate, and the calculation method is as follows:
in the method, in the process of the invention,indicating the loss degree of the h reset information group in the current detection item; />Representing the repetition rate of the h-th reset information group; />An exponential function based on a natural number; / >Indicating a default loss rate.
8. The method for storing concrete pavement report data according to claim 1, wherein the obtaining the translation index of each reset information group according to the duty ratio of the importance of the information group in all the reset information groups comprises the following specific steps:
recording any one detection item as a current detection item;
in the method, in the process of the invention,representing a translation index of an h reset information group in a sigmoid function in a current detection item; />A maximum value indicating the importance degree of the h reset information group in all the detection items; />Indicating the number of h reset information groups in all detection items.
9. The method for storing concrete pavement report data according to claim 1, wherein the step of obtaining the priority of each divided area according to the loss degree and the translation index comprises the following specific steps:
recording any one detection item as a current detection item;
inputting the loss degree of the h reset information group in the current detection item and the translation index of the h reset information group in the current detection item in the sigmoid function as independent variables of the sigmoid function into the sigmoid function, and marking the output result of the sigmoid function as the priority of the h reset information group; for the partition area to which the h reset information group belongs, referring to a priority acquisition method of the h reset information group to obtain the priority of the rest reset information groups in each partition area; the result of the accumulation of the priorities of all the reset information groups in each divided area is then noted as the priority of each divided area.
10. The method for storing concrete pavement report data according to claim 1, wherein the compression storage is performed according to the priority of each divided area, comprising the specific steps of:
the priority of each partition area is recorded as the priority of each data in each partition area;
in the method, in the process of the invention,representing an initial repetition rate at which each data in the f-th partitioned area is encoded; />Representing preset super parameters;representing the priority of each data in the f-th partitioned area; />An exponential function based on a natural number;
acquiring an initial repetition rate of each divided region; recording any detection item as a target detection item, performing Huffman coding and splicing on each data in an f-th division area in the target detection item to obtain a section of total coding sequence, recording the section of total coding sequence as an initial coding a of the f-th division area, and recording the spliced result of all initial total coding in all detection items as an initial total coding a1 of the f-th division area;
if the initial repetition rate of the f-th divided region is smaller than T1, the coding length of the initial total coding is required to be increased, the result of upward rounding of the initial repetition rate of the f-th divided region is recorded as the repetition number Q, the result after splicing a1 and Q a is recorded as the second total coding, the 8-bit binary conversion result of Q is recorded as the identification bit, and the result after splicing the second total coding and the identification bit is recorded as the final total coding;
If the initial repetition rate of the f-th divided region is greater than or equal to T1, the coding length of the initial total coding needs to be reduced, the result of upward rounding of the product of the coding length of the initial total coding and the initial repetition rate is recorded as a reduction number Q1, the 8-bit binary conversion result of Q1 is recorded as a second identification bit, Q1 a is deleted from the initial position of the initial total coding, the obtained result is recorded as a third total coding, and the result obtained by splicing the third total coding and the second identification bit is recorded as a final total coding;
and acquiring the final total code of each divided area, and then storing information into a concrete pavement construction quality detection and supervision system according to the order of the priority from large to small.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311567196.8A CN117272933B (en) | 2023-11-23 | 2023-11-23 | Concrete pavement report data storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311567196.8A CN117272933B (en) | 2023-11-23 | 2023-11-23 | Concrete pavement report data storage method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117272933A CN117272933A (en) | 2023-12-22 |
| CN117272933B true CN117272933B (en) | 2024-02-02 |
Family
ID=89210952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311567196.8A Active CN117272933B (en) | 2023-11-23 | 2023-11-23 | Concrete pavement report data storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117272933B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006262294A (en) * | 2005-03-18 | 2006-09-28 | Ricoh Co Ltd | Image processing apparatus, image processing method, program, and recording medium |
| CN116011403A (en) * | 2023-03-27 | 2023-04-25 | 莱芜职业技术学院 | Repeated data identification method for computer data storage |
| CN116346289A (en) * | 2023-05-30 | 2023-06-27 | 泰山学院 | Data processing method for computer network center |
| CN116743181A (en) * | 2023-08-09 | 2023-09-12 | 北京杏林康云信息科技股份有限公司 | Classified storage method for medication safety data based on big data |
| CN116796354A (en) * | 2023-08-16 | 2023-09-22 | 莱芜职业技术学院 | Information security enhancement method for computer system |
| CN116996673A (en) * | 2023-09-28 | 2023-11-03 | 深圳市铁军智能科技有限公司 | Intelligent cloud management system based on passing in and out management and equipment running state |
-
2023
- 2023-11-23 CN CN202311567196.8A patent/CN117272933B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006262294A (en) * | 2005-03-18 | 2006-09-28 | Ricoh Co Ltd | Image processing apparatus, image processing method, program, and recording medium |
| CN116011403A (en) * | 2023-03-27 | 2023-04-25 | 莱芜职业技术学院 | Repeated data identification method for computer data storage |
| CN116346289A (en) * | 2023-05-30 | 2023-06-27 | 泰山学院 | Data processing method for computer network center |
| CN116743181A (en) * | 2023-08-09 | 2023-09-12 | 北京杏林康云信息科技股份有限公司 | Classified storage method for medication safety data based on big data |
| CN116796354A (en) * | 2023-08-16 | 2023-09-22 | 莱芜职业技术学院 | Information security enhancement method for computer system |
| CN116996673A (en) * | 2023-09-28 | 2023-11-03 | 深圳市铁军智能科技有限公司 | Intelligent cloud management system based on passing in and out management and equipment running state |
Non-Patent Citations (1)
| Title |
|---|
| 基于灰度值分割的感兴趣区域图像压缩算法;樊巧云;何博;;现代电子技术(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117272933A (en) | 2023-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110008809B (en) | Method and device for acquiring form data and server | |
| WO2021036550A1 (en) | Visual task-based method and system for evaluating compression quality of point cloud data | |
| CN104050247A (en) | Method for realizing quick retrieval of mass videos | |
| CN111382298B (en) | Image retrieval method and device based on picture content and electronic equipment | |
| US9275279B2 (en) | Image processing apparatus and image processing method | |
| KR20170004983A (en) | Line segmentation method | |
| CN106651972B (en) | Binary image coding and decoding methods and devices | |
| CN107423309A (en) | Magnanimity internet similar pictures detecting system and method based on fuzzy hash algorithm | |
| CN116110036B (en) | Electric power nameplate information defect level judging method and device based on machine vision | |
| KR102654003B1 (en) | Image recognition device and method | |
| CN111507260A (en) | Video similarity rapid detection method and detection device | |
| CN110879967A (en) | Video content repetition judgment method and device | |
| JP6070568B2 (en) | Feature coding apparatus, feature coding method, and program | |
| CN117376632A (en) | Data recovery method and system based on intelligent depth synthesis | |
| CN117272933B (en) | Concrete pavement report data storage method | |
| CN110490250A (en) | A kind of acquisition methods and device of artificial intelligence training set | |
| CN116861271B (en) | Data analysis processing method based on big data | |
| CN112580825A (en) | Unsupervised data binning method and unsupervised data binning device | |
| CN116975864A (en) | Malicious code detection method and device, electronic equipment and storage medium | |
| KR100717026B1 (en) | Apparatus and method for binary image compression | |
| KR20060112380A (en) | Apparatus and method for binary image compression | |
| CN109299260B (en) | Data classification method, device and computer readable storage medium | |
| CN113099269A (en) | String matching prediction method, encoding and decoding method, related equipment and device | |
| CN104063522A (en) | Image retrieval method based on reinforced microstructure and context similarity | |
| US9615111B2 (en) | Complexity-adaptive compression of color images using binary arithmetic coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |