CN109308336A - Data processing method, processing equipment and storage medium - Google Patents
Data processing method, processing equipment and storage medium Download PDFInfo
- Publication number
- CN109308336A CN109308336A CN201811039467.1A CN201811039467A CN109308336A CN 109308336 A CN109308336 A CN 109308336A CN 201811039467 A CN201811039467 A CN 201811039467A CN 109308336 A CN109308336 A CN 109308336A
- Authority
- CN
- China
- Prior art keywords
- data
- tables
- url
- file
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/14—Conversion to or from non-weighted codes
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention discloses a kind of data processing method, processing equipment and storage mediums, for improving Web system performance.The data processing method of the embodiment of the present invention, comprising: obtain tables of data from database；Read the primary data information in the tables of data；According to the primary data acquisition of information flow data；Set target URL；In the tables of data, corresponding URL or flow data in the primary data information are replaced using the target URL, to obtain target matrix；The target matrix is written back in the database；The flow data is sent to server, so that the server saves the flow data, the target URL is the address that the server saves the flow data.Flow data is read in this way, can facilitate from server, improves the performance of Web system.
Description
Technical field
The present invention relates to technical field of network information more particularly to a kind of data processing method, processing equipment and storage to be situated between
Matter.
Background technique
Currently, Web system has become one of the major way that people obtain information, wherein Web system includes data
The equipment such as library server, Web server, terminal.System concurrency amount of access is affected to Web system performance.
Existing some Web systems, on Web server other than deployment, run program file, many uploads it is various
Although its URL information of attachment files saves in the database, the corresponding actual file of URL is all completely put whether great or small
On Web server, lead to Web server overload.Alternatively, some Web systems are including text, picture, audio, video
Database is all stored in etc. almost all kinds of data.Because picture, audio or the video data volume are often larger, when transmission, compares
Occupied bandwidth, if database and Web server separation (i.e. not on same server), in order to obtain a picture
(or audio-video etc.), it is necessary to which evidence of fetching from client to the end Web, the end Web need to go to take from database side again, in this way can be same
When all form bottleneck on I/O at the database side and the end Web two.
As system concurrency amount of access is increasing, the bad Web system performance of above-mentioned design will sharply glide.
Summary of the invention
In view of the above technical problems, the embodiment of the invention provides a kind of data processing method, processing equipment and storages to be situated between
Matter, for improving Web system performance.
The first aspect of the embodiment of the present invention provides a kind of data processing method, comprising:
Tables of data is obtained from database；
Read the primary data information in the tables of data；
According to the primary data acquisition of information flow data；
Set target URL；
In the tables of data, the primary data information is replaced using the target URL, to obtain target matrix；
The target matrix is written back in the database；
The flow data is sent to server, so that the server saves the flow data, the target URL is described
Server saves the address of the flow data.
Optionally, the primary data information is text information and file link URL；
It is described according to the primary data acquisition of information flow data, comprising:
The flow data that the file link URL indicates is copied from Web server；
The flow data is deleted from the Web server.
Optionally, the primary data information read in the tables of data, comprising:
Judge the mark information for whether having file link URL in the tables of data；
If there is the mark information of file link URL in the tables of data, judge whether there is file chain in the tables of data
Connect the first position feature string of URL；
If there is the first position feature string of file link URL in the tables of data, it is based on the first position feature string,
File link URL is read from the tables of data；
It is described that the flow data that the file link URL indicates is copied from Web server, comprising:
When the file link URL is relative path, the mesh that the file link URL indicates is copied from Web server
Mark file.
Optionally, the primary data information read in the tables of data, comprising:
Judge whether there is flow data feature string in the tables of data；
If there is flow data feature string in the tables of data, whether have file link URL in the tables of data is judged
Two position feature strings；
If there is the second position feature string of file link URL in the tables of data, it is based on the second position feature string,
File link URL is read from the tables of data；
It is described that the flow data that the file link URL indicates is copied from Web server, comprising:
When the file link URL is relative path, the stream that the file link URL indicates is copied from Web server
Data.
Optionally, the primary data information is the coded data encoded containing base64；
It is described according to the primary data acquisition of information flow data, comprising:
According to base64 coding rule, the coded data is reduced into binary flow data.
Optionally, the primary data information read in the tables of data, comprising:
Judge whether there is flow data feature string in the tables of data；
If there is flow data feature string in the tables of data, judge whether there is base64 coding characteristic in the tables of data
String；
If there is base64 coding characteristic string in the tables of data, the coding of base64 coding is read from the tables of data
Data.
Optionally, described according to base64 coding rule, before the coded data is reduced into binary flow data,
The method also includes:
Reduction pretreatment is executed to the coded data；
The setting target URL, comprising:
It is read from the tables of data and judges streaming data types information；
Using the streaming data types information, filename is generated；
Target URL is set using the filename.
Optionally, the flow data is one of in picture file, video file and audio file；
The server is for saving a type of flow data.
Second aspect of the embodiment of the present invention provides a kind of processing equipment, comprising:
Acquiring unit, for obtaining tables of data from database；
Reading unit, for reading the primary data information in the tables of data；
The acquiring unit is also used to according to the primary data acquisition of information flow data；
Setup unit, for setting target URL；
Replacement unit, for being replaced using the target URL corresponding in the primary data information in the tables of data
URL or flow data, to obtain target matrix；
Write back unit, for the target matrix to be written back in the database；
Transmission unit, for sending the flow data to server, so that the server saves the flow data, it is described
Target URL is the address that the server saves the flow data.
The third aspect of the embodiment of the present invention provides a kind of storage medium, is stored with processor-executable instruction, institute
It states instruction to be loaded by one or more processors, to execute data processing method described in first aspect as above.
In technical solution provided in an embodiment of the present invention, after database acquisition tables of data, read initial in tables of data
Data information, according to primary data acquisition of information flow data.And setting target URL uses target URL in tables of data
Corresponding URL or flow data in primary data information are replaced, to obtain target matrix.Then, by target matrix write-back
Flow data is sent into database, and to server, so that server saves flow data, target URL is that server saves stream
The address of data.Therefore compared with the existing technology, after the embodiment of the present invention obtains flow data by the tables of data of database, use
Corresponding URL or flow data obtain target matrix in new target URL more new data table, and target matrix is written back to
In database, and to server transmission flow data, so that server saves flow data.And target URL is server preservation
The address of flow data.In this way, saving flow data in server, reduce flow data to the space hold of other network equipments, and
The flow data saved on server can be searched out by the target URL in target matrix, be read so as to convenient from server
Flow data improves the performance of Web system.
Detailed description of the invention
Fig. 1 is the network architecture diagram that a kind of data processing method that one embodiment of the invention provides is related to；
Fig. 2 is obtained network architecture diagram after the data processing method for executing the embodiment of the present invention；
Fig. 3 be another embodiment of the present invention provides a kind of data processing method flow chart；
Fig. 4 be another embodiment of the present invention provides a kind of data processing method flow chart；
Fig. 5 be another embodiment of the present invention provides a kind of processing equipment structural schematic diagram.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those skilled in the art's every other implementation obtained without creative efforts
Example, shall fall within the protection scope of the present invention.
In order to facilitate each embodiment of the accurate understanding present invention, some technologies that each embodiment of the present invention is related to below
Term is introduced.
1, base64 is encoded.Base64 coding be most commonly used on network transmission 8Bit bytecode coding mode it
One, base64 are a kind of methods for indicating binary data based on 64 kinds of printable characters.The figure uploaded by Web page
The binary files such as piece, audio-video are encoded with base64 store in the database, and in the Web system of part, these base64 are compiled
Code can individually be stored not as a field, but with the information mixed storage such as text in the same field.Therefore, it is necessary to
There is algorithm to separate text and with information such as picture, the audio-videos of the storage of base64 coding form.Base64 coding is to be used for
One of the coding mode of 8Bit syllabified code is transmitted, commonly used in being the number of visible character style binary data coding
According to this is a kind of reversible coding mode.Every 3 bytes are become 4 bytes by base64 coding, therefore length increases after coding
1/3.Data after coding are a character string, character wherein included are as follows: A-Z, a-z, 0-9 ,+,/totally 64 kinds of characters, it is specified that
"=" is used as and is filled character, therefore base64 coding is actually made of 65 kinds of characters.
2, uniform resource locator (Uniform Resource Locator, URL) is to can obtain from internet
Resource position and access method the succinct expression of one kind, be the address of standard resource on internet.It is every on internet
A file has a unique URL, and the information that it includes points out how the position of file and browser should handle it.
Fig. 1 is a kind of network architecture diagram that data processing method is related to provided in an embodiment of the present invention, the network architecture packet
Include database 101, Web server 102, user equipment 103, data server 104 and processing equipment 105, the following realities of the present invention
The data processing method for applying example offer is applied in the processing equipment 105.
Web server 102 refers to Website server, refers to the program for residing at certain type computer on internet, can be with
Document is provided to Web clients such as browsers, site file can also be placed, is browsed for user；Data file can also be placed,
It is downloaded for user.
Database (Database) 101 is to carry out the foundation of tissue, storage and management data according to data structure in computer
Store the warehouse in equipment.It can be considered the file cabinet of electronization --- the place of storage electronic document, user can be to file
In data the operation newly such as increased, intercepted, being updated, being deleted.In embodiments of the present invention, database 101 can store text envelope
The information such as breath, flow data, URL, Web server 102 can be interactive with database 101, thus 102 readable data of Web server
Information on library 101.
User equipment 103 is the equipment that uses of user, user equipment 103 include but is not limited to computer, smart phone and
Tablet computer etc..User equipment and Web server can be used to carry out information exchange for user, such as user is stepped on by user equipment
The information such as video, the picture that the website or reading Web service device 102 that record Web server 102 provides are obtained from database.?
In the embodiment of the present invention, user equipment 103 is also referred to as Web client.
Processing equipment 105 can be specially interactive device or embedded device, and database is housed in the processing equipment 105
Driver and website script handler etc., such as PHP script or python script etc., which can directly read,
With operating database 101, which be can run on Linux and/or Windows.In embodiments of the present invention, should
Processing equipment 105 is used to execute the data processing method that following embodiments provide, the processing equipment 105 can and Web server
102 and database 101 carry out data interaction.
Data server 104 is the server of storing data, and processing equipment 105 is in the data for executing following embodiments offers
When processing method, flow data is sent to data server 104, so that data server 104 saves these flow datas.In some realities
It applies in example, data server 104 can be used for storing the flow data of one or more types, such as data server 104 is to deposit
It stores up the server of picture file or stores the server of video file.
After executing the data processing method that following embodiments provide, the network architecture for users to use as shown in Fig. 2, this
When, the network architecture includes database 101, Web server 102, user equipment 103 and all kinds of servers for saving flow data, figure
These in 2 save all kinds of servers of flow data with file server 106, picture servers 107, audio server 108 and view
It is illustrated for frequency server 109.The data processing method of the embodiment of the present invention, by by data such as picture or audio-videos
Be stripped out and be individually stored on different servers from 101 end of database and Web server 102, take the files such as picture when
It waits, then user equipment 103 is only needed to go to obtain data by servers such as the agreements such as HTTP to picture servers, in this way,
Bandwidth occupancy can be reduced, I/O bottleneck at this time exists only in picture servers etc. and saves on the server of flow data, therefore,
Transmission speed will be accelerated, and Web system performance is improved.
It is appreciated that Fig. 1 and the network architecture shown in Fig. 2, only the data processing method of the embodiment of the present invention is related to
The example of subnetwork framework, the network architecture that the data processing method of the embodiment of the present invention is related to can also be other shapes
Formula.
Fig. 3 is a kind of flow chart of data processing method provided in an embodiment of the present invention, and this method can be applied to shown in Fig. 1
In the processing equipment of embodiment.
Refering to Fig. 3, the data processing method of the embodiment of the present invention includes:
Step 301: obtaining tables of data from database.
Tables of data is the table information of storage on the database.In tables of data, can be stored with text information, URL information or
Streaming data information etc., such as the coded data encoded with base64.For example, tables of data specific for one, table name are
Newsdata, the tables of data include this 3 fields of newsid, postdate and content.
Wherein, newsid indicates that the ID of certain news in the news table of entitled newsdata, postdate indicate entitled
The issuing time of certain news in the news table of newsdata.Content field is blob type, and the inside stores news content,
The base64 such as possible picture, audio, video coded data and URL link the file information.To which, processing equipment can be from
Base64 coded data or file link URL are obtained in content field.
Processing equipment connects database, tables of data is obtained from database, so as to obtain the letter of needs from tables of data
Breath.
The database can be the database of embodiment as shown in Figure 1 description.
Step 302: reading the primary data information in tables of data.
After processing equipment gets tables of data, the primary data information in tables of data is read.
The primary data information be text information and information relevant to flow data, such as primary data information be containing
The information of file link URL can obtain corresponding flow data or initial number by this document link URL from Web server
It is believed that breath is the coded data encoded containing base64, these two types of data informations and text information can be existed simultaneously in initial
In data information.
Step 303: according to primary data acquisition of information flow data.
Wherein, flow data is the data information that Web page is related to, including but not limited to text, picture, audio and video etc.
Deng.These flow datas can store in the database, or be stored on Web server.
Processing equipment is according to primary data acquisition of information flow data.It can be and convert to obtain fluxion by primary data information
According to, or according to the instruction of primary data information, flow data is obtained from other equipment.
For example, processing equipment is linked by this document when initial data information is the information containing file link URL
URL obtains corresponding flow data from Web server.When initial data information is the coded data encoded containing base64,
The coded data is decoded as binary flow data by processing equipment.
Step 304: setting target URL.
Target URL is the address for saving the flow data of step 303.
Processing equipment can set target URL according to default rule.Such as processing equipment is pre-configured with and completes server
The root path of storage file and the IP address of server and domain name.After obtaining flow data, server will be set according to preset rules
Then the title of stream data successively sets mesh using the title of the domain name of server, the root path of storage file and flow data
Mark URL.
Step 305: in tables of data, primary data information is replaced using target URL, to obtain target matrix.
Processing equipment is after getting target URL, in tables of data, uses phase in target URL replacement primary data information
The URL or flow data answered, to obtain target matrix.The target matrix does not include being replaced in primary data information
Part, corresponding URL or flow data are replaced by target URL in primary data information, and other numbers in primary data information
According to remaining unchanged.
In this way, target URL is recorded on target matrix, can be searched out pair according to the target URL on the target matrix
The flow data answered.
In addition, target matrix is lesser by space hold if primary data information includes the initial data of flow data
Target URL replaces the biggish primary data information of space hold, can reduce the information content of tables of data.
Step 306: target matrix is written back in database.
After processing equipment obtains target matrix, target matrix is written back in database, so that database is preserved
Target matrix, so that other equipment (such as user equipment) can read the target matrix from database.
Step 307: flow data is sent to server, so that server saves flow data.
Wherein, target URL is the address that server saves flow data.Server is the service dedicated for saving flow data
Device.
After processing equipment gets flow data, processing equipment can send flow data to server, and server can be reserved for the stream
Data.In this way, user directly can request flow data to the server by user equipment, without to other device request fluxions
According to.
The primary data information of the embodiment of the present invention there are many data type, the implementation of above-mentioned steps also there are many,
It is illustrated by taking two of them primary data information as an example below.
Example one: primary data information is the data containing file link URL.
In this example one, step 303 includes: the fluxion that processing equipment is indicated from Web server copied files link URL
According to then, from Web server deletion flow data.
When the primary data information of table data storage is the data containing file link URL, the instruction of this document link URL
Flow data be stored on Web server, thus the flow data that processing equipment is indicated from Web server copied files link URL,
In order to reduce the memory space occupancy situation on Web server, processing equipment deletes flow data from Web server.
Wherein, according to the different type of flow data, there are many implementations of this example one.It is realized below with two of them
It is illustrated for mode.
Implementation one:
In this implementation, flow data is indicated with file destination, and wherein file destination can be in addition to picture file, sound
File except frequency file, video file, such as WORD document, RAR condensed document, PDF document etc..
In this implementation, step 302 specifically includes step A1 to step A3.
Step A1: judge the mark information for whether having file link URL in tables of data.If there is file link in tables of data
The mark information of URL, thens follow the steps A2.
Wherein, the mark information of this document link URL indicates to include file link URL in tables of data.The mark information can
Some specific character strings are thought, for example, it is specified that " < a " is the URL of ordinary file in HTML (i.e. hypertext markup language)
(i.e. uniform resource locator) chained address start mark, and " > " terminates to mark for it.Arrived to which as long as processing equipment scans " <
A " and " > " feature string, that is, can determine that tables of data includes complete file link URL.
Processing equipment judges the mark information for whether having file link URL in tables of data after database acquisition tables of data.
If so, then locating file link URL, otherwise abandons executing subsequent step.
Step A2: judge the first position feature string for whether having file link URL in tables of data.If there is file in tables of data
The first position feature string of link URL, thens follow the steps A3.
First position feature string is for identifying position of the file link URL in tables of data, for example, " href=" feature
String.
After processing equipment judges the mark information for having file link URL in tables of data, it may be determined that tables of data includes complete
File link URL, then, processing equipment searched in tables of data indicate file link URL position where first position
Feature string.If there is the first position feature string of file link URL in tables of data, A3 is thened follow the steps, does not otherwise execute subsequent step
Suddenly.
Step A3: being based on first position feature string, and file link URL is read from tables of data；
Because first position feature string is for identifying position of the file link URL in tables of data, so that processing equipment can base
In first position feature string, position of the file link URL in tables of data is determined, to read file link from tables of data
URL。
Correspondingly, above-mentioned to be indicated from Web server copied files link URL after executing above-mentioned step A1 to A3
Flow data the step of specifically include: when file link URL be relative path when, refer to from Web server copied files link URL
The file destination shown.
If file link URL is not relative path, indicate that the file destination is not stored in Web server being processed
On, so there is no need to migrate to it from Web server.If file link URL is relative path, indicate that file destination is stored in just
On processed Web server, thus the file destination that processing equipment is indicated from Web server copied files link URL.
In some embodiments, in this implementation, in step 307, the server for saving file destination can be referred to as text
Part server, it is different from Web server.
In this way, the performance of Web server can be reduced if storing excessive file destination on Web server, because
File destination is separated from Web server, is stored on individual file server by this implementation by this, can
The burden of Web server is reduced, user can be from file server quick obtaining to file destination.
Implementation two:
In this implementation, flow data is stored on Web server, which can be picture file, video text
One of in part and audio file.The server of flow data is saved in step 307 for saving a type of fluxion
According to being specifically as follows picture servers, video server or audio server.Wherein, picture servers are for saving picture text
Part, video server is for saving video file, and audio server is for saving audio file.
In this implementation, step 302 specifically includes step B1 to step B3.
Step B1: judge whether there is flow data feature string in tables of data.If there is flow data feature string in tables of data, execute
Step B2.
Flow data feature string indicates to preserve the relevant information of flow data in tables of data, wherein different types of flow data
Can correspond to different flow data feature strings, for example, the flow data feature string of picture file be "<img"and ">" feature string；Video text
The flow data feature string of part be "<video " and ">" feature string；The flow data feature string of audio file be "<audio " and ">" spy
Sign string.
After processing equipment gets tables of data, judge whether there is flow data feature string in tables of data, if so, indicating can be from this
Information relevant to flow data is obtained in tables of data does not otherwise execute subsequent step thereby executing step B2.
Step B2: judge the second position feature string for whether having file link URL in tables of data.If there is file in tables of data
The second position feature string of link URL, thens follow the steps B3.
The second position feature string of file link URL for indicating position of the file link URL in tables of data, for example,
Second position feature string is " src=" feature string in content field.
Processing equipment is judged after having flow data feature string in tables of data, it may be determined that goes out to obtain and flow from the tables of data
The relevant information of data.Then processing equipment judges the second position feature string for whether having file link URL in tables of data, with true
Determine position of the file link URL in tables of data.
Step B3: being based on second position feature string, and file link URL is read from tables of data；
Because second position feature string is for identifying position of the file link URL in tables of data, so that processing equipment can base
In second position feature string, position of the file link URL in tables of data is determined, to read file link from tables of data
URL。
Correspondingly, above-mentioned to be indicated from Web server copied files link URL after executing above-mentioned step B1 to B3
Flow data specifically include: when file link URL be relative path when, from Web server copied files link URL indicate stream
Data.
If file link URL is not relative path, indicate that the flow data is not stored in Web server being processed
On, so there is no need to migrate to it from Web server.If file link URL is relative path, indicate that flow data is stored in just quilt
On the Web server of processing, thus the flow data that processing equipment is indicated from Web server copied files link URL.
In this way, Web can be reduced by dumping to the flow data being stored on Web server originally on other servers
The burden of server, and by the flow data acquisition request of other server quick response users, so that network system can be improved
System performance.
Example two: primary data information is the coding flow data encoded containing base64.
In this example, flow data is one of or a variety of in picture file, video file and audio file.
The server of flow data is saved in step 307 for saving a type of flow data.Wherein, picture servers are for saving figure
Piece file, video server is for saving video file, and audio server is for saving audio file.
Optionally, in this example two, step 302 specifically includes step C1 to C3.
Step C1: judge whether there is flow data feature string in tables of data.If there is flow data feature string in tables of data, execute
Step C2.
Flow data feature string indicates to preserve the relevant information of flow data in tables of data, wherein different types of flow data
Can correspond to different flow data feature strings, for example, the flow data feature string of picture file be "<img"and ">" feature string；Video text
The flow data feature string of part be "<video " and ">" feature string；The flow data feature string of audio file be "<audio " and ">" spy
Sign string.
After processing equipment gets tables of data, judge whether there is flow data feature string in tables of data, if so, indicating can be from this
Information relevant to flow data is obtained in tables of data does not otherwise execute subsequent step thereby executing step C2.
Step C2: judge whether there is base64 coding characteristic string in tables of data.If there is base64 coding characteristic in tables of data
String, thens follow the steps C3.
Base64 coding characteristic string indicates that the base64 in tables of data including relevant flow data is encoded.For example, for figure
Piece file, base64 coding characteristic string include " data:image/ " feature string, "；Base64, " feature string and ", " feature string；
For video file, base64 coding characteristic string include " data:video/ " feature string, "；Base64, " feature string and ", " it is special
Sign string；For audio file, base64 coding characteristic string include " data:audio/ " feature string, "；Base64, " feature string and
", " feature string.
Processing equipment judges whether there is base64 coding characteristic string in tables of data, if there is base64 coding characteristic in tables of data
String indicates that tables of data includes the base64 coding of flow data, thens follow the steps C3；Otherwise subsequent step is not executed.
Step C3: the coded data encoded containing base64 is read from tables of data.
After processing equipment determines the base64 coding that tables of data includes flow data, base64 can be read from tables of data
The coded data of coding.
In this example two, step 303 is specifically included: according to base64 coding rule, by coded data be reduced into two into
The flow data of system.
In this example two, according to base64 coding rule, coded data is reduced into the step of binary flow data
Before rapid, the method for the embodiment of the present invention further include: reduction pretreatment is executed to coded data.
The coded data for uploading to the base64 coding of database may cause its bas64 to encode due to various reasons
Tail portion has been attached some additional characters or certain code characters are replaced by forbidden character, leads to the coding obtained
Data cannot be reduced into corresponding binary file, so needing to be implemented reduction pretreatment, reduction pretreatment is so that coded number
According to the treatment process that can be reduced into binary file, for example, being carried out to the coded data of the base64 coding of picture file as follows
Processing: remove picture file base64 encoded tail single quotation marks and/or double quotation marks and later all data；Remove its tail portion
Space；The space of non-tail portion is all substituted for "+", then the software algorithm in 105 equipment will according to base64 coding rule
Picture file base64 coding is reduced into binary picture file.
In the embodiment having, it includes step D1 to step D3 that step 304, which sets target URL,.
Step D1: the reading flow data type information from tables of data.
In data containing base64 coding in tables of data, it is stored with data type information, such as belong to picture type
Png, jpg etc., perhaps belong to mp4, rmvb of video type etc. or belong to wav, amr etc. of audio types.
Processing equipment can from tables of data containing base64 encode data in reading flow data type information.
Step D2: using streaming data types information, generates filename.
After processing equipment gets streaming data types information, streaming data types information can be used, filename is generated, in conjunction with making
It can establish to obtain complete filename with temporal information, ordinal number and streaming data types information.
Step D3: target URL is set using filename.
After getting filename, filename can be used to set target URL for processing equipment, in other words, using filename as mesh
Mark a part of URL.For example, the target URL set is " http: // server domain name/path/filename ".
In this way, because the coded data of flow data is stored in the database, since stream data file is usually all bigger, and with
Capacity is bigger after base64 coding, therefore occupies very much database space, the execution of method through the embodiment of the present invention, will
The coded data of flow data is stripped out from tables of data, is replaced coded data with the information of target URL, is reduced tables of data
Size, in the database, only store the URL link address of flow data, database purchase can be mitigated significantly in this way
Pressure.
It is appreciated that in other embodiments of the invention, processing equipment can successively execute above-mentioned example one and show
Example two, the present invention is not especially limit this.
In conclusion the primary data information in tables of data is read, according to initial number after database acquisition tables of data
According to acquisition of information flow data.And setting target URL uses phase in target URL replacement primary data information in tables of data
The URL or flow data answered, to obtain target matrix.Then, target matrix is written back in database, and to service
Device sends flow data, so that server saves flow data, target URL is the address that server saves flow data.Therefore relative to
The prior art uses new target URL more new data table after the embodiment of the present invention obtains flow data by the tables of data of database
In corresponding URL or flow data obtain target matrix, and target matrix is written back in database, and to server
Flow data is sent, so that server saves flow data.And target URL is the address that server saves flow data.In this way, taking
Business device saves flow data, reduces flow data to the space hold of other network equipments, and can pass through the mesh in target matrix
Mark URL searches out the flow data saved on server, reads flow data from server so as to convenient, improves Web system
Performance.
It, hereafter will be with above-mentioned shown in FIG. 1 in order to which the data processing method to the embodiment of the present invention has more intuitive understanding
The network architecture is illustrated by taking a specific implementation scene as an example for the data processing method to the embodiment of the present invention, wherein this
The data processing method of inventive embodiments is applied in processing equipment shown in FIG. 1.
Execute the processing equipment of the data processing method of the embodiment of the present invention specifically, including initialization unit, fluxion
According to processing unit, write back data unit and file migration unit.
Refering to Fig. 4, the data processing method of the embodiment of the present invention includes:
Step 401: initialization data.
The initialization unit of processing equipment executes step 401, and step 401 specifically includes:
In embodiments of the present invention, the server of flow data is stored with file server, picture servers and audio-video clothes
It is illustrated for business device.At this point, file server, picture servers, audio server and view is respectively configured in initialization unit
The root path of frequency server storage file.For example, root path is respectively file, image, audio, video, and distribute each clothes
It is engaged in the corresponding IP address of device and domain name, for example, domain name is example.com, then the domain name of file, picture, audio/video server
It can be respectively set to file.example.com, image.example.com, video.example.com.In processing equipment
On, create directory file, image, audio, video, is respectively used to temporarily store file, picture, audio, the video isolated
File.
Step 402: stream data identification is extracted and is separated.
The flow data processing unit of processing equipment executes step 402, and step 402 specifically includes: to table each in database
Base64 coding and the scanning of URL link file data, identification, extracting and developing operation are executed, until entire database processing is complete
Finish.
Specifically, for one of tables of data, such as the table name of the tables of data is newsdata, is contained
This 3 fields of newsid, postdate, content, wherein content field is blob type, and the inside stores news content,
And the base64 such as possible picture, audio, video coded data and URL link the file information.To which the present invention is implemented
The stream data identification of example is extracted and is separated mainly for the execution of content field.
Scanning, identification, the main working process of extracting and developing data are as follows from content field:
Step E1: processing equipment connects database, and constructs query statement sql=" select newsid, postdate,
content from newsdata where content！=NULL " executes the sql sentence, by the table query result of return
Collection is stored in buffer variable rs.Wherein, table query results include several news records, and every records by newsid,
This 3 field compositions of postdate, content.
Step E2: processing equipment reads a record from result set rs, detects whether to read result set tail, if so, turning
Step E8；If it is not, postdate, content field contents assign global variable newsid by the newsid of current record,
Postdate, content, global variable bChanged assign initial value FALSE, then, successively execute step E3-E8.
Step E3: processing equipment identifies ordinary file URL, extracted and is separated.Specifically, step E3 can divide
Solution is step F1 to step F4.
Step F1: processing equipment initializes local variable iPosH=iPosT=iCount=0.
Step F2: using iPosT as initial position, whether there is or not " < a " feature string in scanning content string, the feature that will be scanned
String initial position is stored in iPosH, detection " iPosH≤- 1 " (being less than or equal to -1 indicates to arrive " < a " feature string without scanning, under
"≤" meaning stated is similar, is all to indicate " not scanning "), if so, indicating ordinary file URL chain all in this unit
It connects to have identified and handled and finish, to go to step F4；Otherwise, using iPosH as initial position, Infinite Cyclic in order to prevent, because
This needs motion scan position backward, therefore adjusting scanning starting position is using iPosH as initial position.Then, content is scanned
In whether there is or not " > " feature strings, the feature string initial position scanned is stored in iPosT, detect " iPosT≤- 1 ", if so, turn
Step F4；Otherwise, otherwise detection " iPosT > iPosH " goes to step F4 if so, going to step F3.Under normal circumstances, ordinary file
URL link address end feature string ">" occur position be centainly greater than its initiation feature string "<a " appearance position, if not
Meet this requirement, illustrate scanning recognition to be not a legal URL link address, therefore go to step F4 to terminate this unit
Processing.
Wherein, " iPosH≤- 1 " indicates to arrive " < a " feature string without scanning, if content string has been scanned to end
Just there is no " < a " feature string at all in tail or content string, the such case of iPosH≤- 1 can all occur.
This is because " < a " is that (i.e. unified resource is fixed by the URL of ordinary file as defined in HTML (i.e. hypertext markup language)
Position symbol) chained address start mark, and ">" terminates to mark for it, so, if "<a " feature string is arrived in scanning, meaning that may knowledge
It is clipped to the URL link of an ordinary file, if scanning " > " feature string again below, means that and recognizes one commonly
The complete URL link of file.
Step F3: using iPosH as initial position, whether there is or not " href=" feature strings in scanning content, if so, judgement
Whether the file link URL indicated in the quotation marks of the back " href=" is relative path, if it is not, illustrating that this document is not stored in just
On processed Web server, so there is no need to migrate, the initial point position scanned next time is updated are as follows: iPosT=iPosH+1 turns
Step F2；If so, extracting " days " according to postdate, created according under the file catalogue of " days " on a processing device
Path (path will include " days " information, if the path exists, without creation),
For example, the issuing time postdate of certain news is 2015-10-10 10:10:10, then, first at detection
Whether there is or not 201510 catalogues under file catalogue in reason equipment, if it is not, 201510 catalogues are created under file catalogue, with
Afterwards, all issuing times are that the news in October, 2015 is stripped out in the news if having ordinary file in the news
Ordinary file all store on a processing device /file/201510 this catalogue under, if the issuing time of certain news
Postdate is 2015-11-1111:11:11, then creates 201511 catalogues, and so on.
File name is extracted according to URL, and file is renamed as new file: file+iCount. suffix by following rule
Then name it is local to processing equipment to copy the file that original URL is directed toward from Web server: the road httpcopy (URL, " file/
Diameter/new filename ") (subsequent unit is processed similarly renaming and copying for file, repeats no more), and, it deletes in Web
The file that original URL on server is directed toward.Then modification content string, original URL is replaced with " http: //
The path file.example.com//new filename ", modified logo bChanged=TRUE, iCount add 1, and update is swept next time
The initial point position retouched are as follows: iPosT=iPosH+1 goes to step F2.
Step F2: being disposed, and goes to step E4.
Step E4: processing equipment encodes picture file base64 and URL is identified, extracting and developing.Specifically,
Step E4 can be analyzed to step G1 to step G5.
Step G1: processing equipment initializes local variable iPosH=iPosT=iCount=0, goes to step G2.
Step G2: processing equipment scans in content character string using iPosT as initial position whether there is or not " <img " feature string,
The feature string initial position scanned is stored in iPosH, is detected " iPosH≤- 1 ", if so, going to step G5；Otherwise, with
IPosH is initial position, and whether there is or not " > " feature strings in scanning content, and the feature string initial position scanned is stored to iPosT
In, it detects " iPosT≤- 1 ", if so, going to step G5；Otherwise, otherwise detection " iPosT > iPosH " turns if so, going to step G3
Step G5.
Step G3: processing equipment is using iPosH as initial position, and whether there is or not " data:image/ " features in scanning content
String, the feature string initial position scanned is stored in iPicPosH, is detected " iPicPosH≤- 1 ", if so, going to step G4；
Otherwise, using iPosH as initial position, scan content in whether there is or not "；Base64, " feature string, the feature string scanned is originated
Position is stored in iPicPosT, then according to the type of following sentence acquirement picture: iPicPosHOff=length (" data:
image/")
ImgType=substr (content, iPicPosH+iPicPosHOff, iPicPosT-iPicPosH-
iPicPosHOff)
Here iPicPosHOff is the length of picture base64 coding characteristic string (i.e. " data:image/ "), ImgType
For the picture type (such as png, jpg etc.) of acquirement, and substr is to take substring function, and first parameter content is to be taken
Character string, second parameter indicates that (offset is iPicPosH+ in upper example from the offset for being taken start of string to read
IPicPosHOff), third parameter is that (the substring length taken in upper example is iPicPosT-iPicPosH- for the substring length that takes
IPicPosHOff), the substring fetched is stored in variable ImgType and (similarly hereinafter, repeats no more).
Picture file for that will extract is named according to following rule: days+iCount.ImgType, then specifies text
The address URL of part is " path http://image.example.com//filename ".
Using iPicPosH as initial position, whether there is or not ", " feature strings in scanning content, and the feature string scanned is originated
Position is stored in iPosComma, is detected " iPosComma≤- 1 ", if so, update the initial point position scanned next time are as follows:
IPosT=iPosH+1 goes to step G2；Otherwise, it is intended to obtain picture base64 coded data: img=substr (content,
iPosComma+1,iPosT-iPosComma-1).The picture of upload may cause its bas64 code tail due to various reasons
Portion has been attached some additional characters or certain code characters are replaced by forbidden character, leads to the picture obtained text
Part base64 coding cannot be reduced into picture file, therefore need to encode picture file base64 and be handled as follows: remove img
Tail portion single quotation marks and/or double quotation marks and later all data；Remove the space of the tail portion img；The space of non-tail portion is all replaced
It changes "+" into, picture file base64 coding img is then reduced into binary picture file according to base64 coding rule,
Under the image catalogue of picture file storing path on a processing device, content string, the picture that will be scanned then are modified
Base64 coding (all substrings i.e. positioned at iPosH and iPosT between, including end to end) replace with it is similar as follows " http: //
The URL in the path img.example.com//filename ", iCount add 1, modified logo bChanged=TRUE, update next time
The initial point position of scanning are as follows: iPosT=iPosH+iPicPosHOff goes to step G2.
In this example, iPicPosH is used to store the initial position of " data:image/ " feature string that may be recognized,
It is located in after " data:image/ " feature string with the specific data content of the picture file of base64 coding and deviates several bytes
Position.If " iPicPosH≤- 1 ", the picture file that scanning recognition is stored in the form of URL to one is indicated, therefore turn
It goes to handle to G4；Otherwise indicating that a picture file with base64 coding storage may have been recognized, it is also necessary into one
Scanning recognition others mark string is walked, could finally be determined whether for a picture file with base64 coding storage.Hereafter
Video, audio file processing unit it is similar therewith, only specific distinguishing mark string is not exactly the same, repeats no more.
Step G4: using iPosH as initial position, whether there is or not " src=" feature strings in scanning content, if so, judgement " src
=" whether the file link URL that indicates in the quotation marks of back be relative path, just located if it is not, illustrating that this document is not stored in
On the Web server of reason, so there is no need to migrate, the initial point position scanned next time is updated are as follows: iPosT=iPosH+1 is (in order to anti-
Only endless loop, the position scanned next time must move backward, therefore this sentence of iPosT=iPosH+1 is arranged.), it goes to step
G2；If so, extracting " days " according to postdate, road is created according under the image catalogue of " days " on a processing device
Diameter extracts file name according to URL, and file is renamed as new file: file+iCount. suffix name by following rule, so
URL file is copied to processing equipment local from Web server afterwards, and, delete the original flow data text on Web server
Part.Then modification content string, replaces with " the path http://image.example.com//new file for original URL
Name ", modified logo bChanged=TRUE, iCount add 1, update the initial point position scanned next time are as follows: iPosT=
IPosH+1 goes to step G2.
Wherein, be described as follows to Partial Feature string: scanning recognition picture file arrives " <img " feature string firstly the need of scanning,
Then also need scanning recognition that could completely identify that picture file can be deposited in two ways to some other feature strings
It puts in the database, a kind of mode is the form storage with base64 coding, at this time except firstly the need of recognizing " <img " feature
Except string, it is also necessary to successively scanning recognition to " data:image/ " feature string, "；Base64, " feature string, ", " feature string, " > "
Feature string could completely identify the picture file encoded with base64 in this way；Another mode is deposited in the form of URL link
It puts, i.e., only saves the URL link of file in the database, and actual file is commonly stored on Web server, at this time except head
First need to recognize "<img"apart from the feature string ,also need to scan to identify "src="feature string ,">" feature string, in this way
It could completely identify the picture file stored in the form of URL link.
Step G5: being disposed, and goes to step E5.
Step E5: processing equipment encodes video file base64 and URL is identified, extracting and developing.Specifically,
Step E5 includes step H1 to step H5.
Step H1: initialization local variable iPosH=iPosT=iCount=0 goes to step H2.
Step H2: it using iPosT as initial position, scans in content character string whether there is or not " < video " feature string, will scan
To feature string initial position be stored in iPosH, detect " iPosH≤- 1 ", if so, going to step H5；Otherwise, it is with iPosH
Beginning position, scan content in whether there is or not " > " feature strings, feature string initial position scan is stored in iPosT, detection
" iPosT≤- 1 ", if so, going to step H5；Otherwise, otherwise detection " iPosT > iPosH " goes to step H5 if so, going to step H3.
Step H3: it using iPosH as initial position, scans in content whether there is or not " data:video/ " feature string, will scan
To feature string initial position be stored in iPicPosH, detect " iPicPosH≤- 1 ", if so, going to step H4；Otherwise, with
IPosH is initial position, scan content in whether there is or not "；Base64, " feature string, the feature string initial position scanned is deposited
Into iPicPosT, then according to the type of following sentence acquirement video: iPicPosHOff=length (" data:
video/")
ImgType=substr (content, iPicPosH+iPicPosHOff, iPicPosT-iPicPosH-
iPicPosHOff)
Here iPicPosHOff is the length of video base64 coding characteristic string (i.e. " data:video/ "), ImgType
For the video type (such as mp4, rmvb etc.) of acquirement.
Video file for that will extract is named according to following rule: days+iCount.ImgType, then specifies text
The address URL of part is " path http://video.example.com//filename ".
Using iPicPosH as initial position, whether there is or not ", " feature strings in scanning content, and the feature string scanned is originated
Position is stored in iPosComma, is detected " iPosComma≤- 1 ", if so, update the initial point position scanned next time are as follows:
IPosT=iPosH+1 goes to step H2；Otherwise, it is intended to obtain video base64 coded data: vid=substr (content,
iPosComma+1,iPosT-iPosComma-1).The video of upload may cause its bas64 code tail due to various reasons
Portion has been attached some additional characters or certain code characters are replaced by forbidden character, leads to the video obtained text
Part base64 coding cannot be reduced into video file, therefore need to encode video file base64 and be handled as follows: remove vid
Tail portion single quotation marks and/or double quotation marks and later all data；Remove the space of the tail portion vid；The space of the non-tail portion vid is whole
It is substituted for "+", vid is then reduced into binary video file, video file storing path according to base64 coding rule
Under video catalogue on a processing device, content string is then modified, the video base64 scanned coding (is located at
All substrings between iPosH and iPosT, including end to end) replace with similar following " http://video.example.com/
The URL in path/filename ", iCount add 1, modified logo bChanged=TRUE, update the initial point position scanned next time
Are as follows: iPosT=iPosH+iPicPosHOff goes to step H2.
Step H4: using iPosH as initial position, whether there is or not " src=" feature strings in scanning content, if so, judgement " src
=" whether the file link URL that indicates in the quotation marks of back be relative path, just located if it is not, illustrating that this document is not stored in
On the Web server of reason, so there is no need to migrate, the initial point position scanned next time is updated are as follows: iPosT=iPosH+1 is gone to step
H2；If so, extracting " days " according to postdate, road is created according under the video catalogue of " days " on a processing device
Diameter extracts file name according to URL, and file is renamed as new file: file+iCount. suffix name by following rule, so
URL file is copied to processing equipment local from Web server afterwards, and, delete the original flow data text on Web server
Part.Then, modification content string, replaces with " the path http://video.example.com//new file for original URL
Name ".Modified logo bChanged=TRUE, iCount add 1, update the initial point position scanned next time are as follows: iPosT=
IPosH+1 goes to step H2.
In this example, similar picture file, video or audio file can also be stored in database in two ways
In, i.e., the form encoded with base64, or in a manner of URL link address.When scanning recognition to " data:video/ " feature
When string, indicate that a video file with base64 coding storage may have been recognized, but this is not enough, it is also necessary to continue to sweep
Retouch, when again scanning recognition to "；Base64, " feature string, ", " feature string, etc., just indicate have identified one completely with
Base64 coding storage video file, thus can it is like above with base64 coding storage picture file it is the same,
It is first stripped out the specific data of video file from database, is then substituted for URL link address again.
Step H5: being disposed, and goes to step E6.
Step E6: processing equipment encodes audio file base64 and URL is identified, extracting and developing.Specifically,
Step E6 includes I1 to step I5.
Step I1: initialization local variable iPosH=iPosT=iCount=0 goes to step I2.
Step I2: it using iPosT as initial position, scans in content character string whether there is or not " < audio " feature string, will scan
To feature string initial position be stored in iPosH, detect " iPosH≤- 1 ", if so, going to step I5；Otherwise, it is with iPosH
Beginning position, scan content in whether there is or not " > " feature strings, feature string initial position scan is stored in iPosT, detection
" iPosT≤- 1 ", if so, going to step I5；Otherwise, otherwise detection " iPosT > iPosH " goes to step I5 if so, going to step I3.
Step I3: it using iPosH as initial position, scans in content whether there is or not " data:audio/ " feature string, will scan
To feature string initial position be stored in iPicPosH, detect " iPicPosH≤- 1 ", if so, going to step I4；Otherwise, with
IPosH is initial position, scan content in whether there is or not "；Base64, " feature string, the feature string initial position scanned is deposited
Into iPicPosT, then according to the type of following sentence acquirement audio: iPicPosHOff=length (" data:
audio/")
ImgType=substr (content, iPicPosH+iPicPosHOff, iPicPosT-iPicPosH-
iPicPosHOff)
Here iPicPosHOff is the length of audio base64 coding characteristic string (i.e. " data:audio/ "), ImgType
For the audio types (such as wav, amr etc.) of acquirement.
Audio file for that will extract is named according to following rule: days+iCount.ImgType, then specifies text
The address URL of part is " path http://video.example.com//filename ".
Using iPicPosH as initial position, whether there is or not ", " feature strings in scanning content, and the feature string scanned is originated
Position is stored in iPosComma, is detected " iPosComma≤- 1 ", if so, update the initial point position scanned next time are as follows:
IPosT=iPosH+1 goes to step I2；Otherwise, it is intended to obtain audio base64 coded data: aud=substr (content,
iPosComma+1,iPosT-iPosComma-1).The audio of upload may cause its bas64 code tail due to various reasons
Portion has been attached some additional characters or certain code characters are replaced by forbidden character, causes the aud obtained cannot
It is reduced into audio file, therefore needs that aud is handled as follows: removing the tail portion aud single quotation marks and/or double quotation marks and later institute
There are data；Remove the space of the tail portion aud；The space of the non-tail portion aud is all substituted for "+", then encodes and advises according to base64
Aud is then reduced into binary audio file, under the audio catalogue of audio file storing path on a processing device, then
Modify content string, by audio base64 scan encode (all substrings i.e. between iPosH and iPosT, including
The URL in similar following " path http://video.example.com//filename " is replaced with end to end), and iCount adds 1, modification
Indicate bChanged=TRUE, update the initial point position scanned next time are as follows: iPosT=iPosH+iPicPosHOff turns step
Rapid I2.
Step I4: using iPosH as initial position, whether there is or not " src=" feature strings in scanning content, if so, judgement " src
=" whether the file link URL that indicates in the quotation marks of back be relative path, just located if it is not, illustrating that this document is not stored in
On the Web server of reason, so there is no need to migrate, the initial point position scanned next time is updated are as follows: iPosT=iPosH+1 is gone to step
I2；If so, extracting " days " according to postdate, road is created according under the audio catalogue of " days " on a processing device
Diameter extracts file name according to URL, and file is renamed as new file: file+iCount. suffix name by following rule, so
URL file is copied to processing equipment local from Web server afterwards, and, delete the original flow data text on Web server
Part.Then modification content string, replaces with " the path http://video.example.com//new file for original URL
Name ", modified logo bChanged=TRUE, iCount add 1, update the initial point position scanned next time are as follows: iPosT=
IPosH+1 goes to step I2.
Step I5: being disposed, and goes to step 403.
Step 403: write-back is carried out to the data after separation.
Above-mentioned steps have carried out flow data identification to ordinary file, picture file, audio-video document, extracting and developing operates,
Certain news data inquired in the news table of entitled newsdata namely may be repeatedly modified, be substituted (to be temporarily stored in
In content variable) certain particular contents, therefore, it is necessary to the news data for having changed this be written back to again it is original
(the original content of the data is override) in database, is not every news data all write-backs, only detection modification mark
When whether will variable bChanged is TRUE, ability write-back.
The write back data unit of processing equipment executes step 403, specifically:
Processing equipment detects whether modified logo variable bChanged is TRUE, if so, illustrate that this record is modified,
It needs to be written back in database again, building query statement sql=" update news_dataset content=" "；
//str_replace is character string replacement function, by double quotation marks " escape replace with "
Content=str_replace (' " ', ' " ', content)；
Sql+=sql+content；
Sql+=" " where newsid="+newsid "；
execute(sql)；The sql sentence of the corresponding content field contents of // execution write-back table newsid
In embodiments of the present invention, because the news data inquired from the news table of entitled newsdata (inquires
Be temporarily stored in result set rs after coming) may more than one, and if step 402 execute it is primary only handle a record, need to follow
Ring executes the above-mentioned steps of the embodiment of the present invention, so that step 402 can be executed again after step 403, until result set whole
It is disposed.
Step 404: processing equipment moves to server to the binary file generated after separation.
Processing equipment will be temporarily stored in thereon after the All Files under file, image, video, audio catalogue are packaged respectively,
Upload to file server (domain name file.example.com1), picture servers (domain name image.example.com), sound
The migration for the All Files isolated is completed on video server (domain name video.example.com).
Step 405: processing equipment is closed result set and is connected with database.
After the completion of above-mentioned steps execute, processing equipment completes Data Migration work, so that processing equipment can close knot
Fruit collection, and close database connection.
In this way, the embodiment of the present invention is realized in terms of two to Web server and database server Reduction of Students' Study Load:
1) attachment files of various uploads (its URL link saves in the database) are stripped out from Web server；
2) information such as the various pictures, the audio-video that are saved in database with base64 coding are separated from database
Come.The above-mentioned file being stripped out all is dumped to individual file, picture, on audio/video server, reaches and not only lighten the burden but also divide
The purpose of flow data.
Fig. 5 is a kind of structural schematic diagram of processing equipment provided in an embodiment of the present invention, which can be used for executing
Above-mentioned Fig. 3 and data processing method shown in Fig. 4.The processing equipment can be integrated in processing equipment shown in FIG. 1.
Refering to Fig. 5, the processing equipment of the embodiment of the present invention, comprising: acquiring unit 501, reading unit 502, setup unit
503, replacement unit 504, write back unit 505 and transmission unit 506.
Acquiring unit 501 is used to obtain tables of data from database, and reading unit 502 is used to read the initial number in tables of data
It is believed that breath, the primary data information is related to flow data, thus, acquiring unit 501 can be used for according to primary data acquisition of information
Flow data.
Setup unit 503, so that replacement unit 504 is used in tables of data, uses target URL for setting target URL
Primary data information is replaced, to obtain target matrix.Then, write back unit 505 is used to target matrix being written back to data
In library.
Transmission unit 506 is used to send flow data to server, so that server saves flow data, target URL is service
The address of device preservation flow data.
In this way, updating number using new target URL after the embodiment of the present invention obtains flow data by the tables of data of database
Target matrix is obtained according to table, and target matrix is written back in database, and sends flow data to server, so that clothes
Business device saves flow data.And target URL is the address that server saves flow data.In this way, saving flow data in server, subtract
Flow data is lacked to the space hold of other network equipments, and server can have been searched out by the target URL in target matrix
The flow data of upper preservation reads flow data from server so as to convenient, improves the performance of Web system.
The embodiment of the invention also provides a kind of storage mediums, are stored with processor-executable instruction, described instruction
It is loaded by one or more processors, to execute above-mentioned Fig. 3 and data processing method shown in Fig. 4.
The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although with reference to the foregoing embodiments
Invention is explained in detail, those skilled in the art should understand that: it still can be to aforementioned each implementation
Technical solution documented by example is modified or equivalent replacement of some of the technical features；And these modification or
Replacement, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.
Claims (10)
1. a kind of data processing method characterized by comprising
Tables of data is obtained from database；
Read the primary data information in the tables of data；
According to the primary data acquisition of information flow data；
Set destination Uniform Resource finger URL URL；
In the tables of data, the primary data information is replaced using the target URL, to obtain target matrix；
The target matrix is written back in the database；
The flow data is sent to server, so that the server saves the flow data, the target URL is the service
Device saves the address of the flow data.
2. the method according to claim 1, wherein
The primary data information is text information and file link URL；
It is described according to the primary data acquisition of information flow data, comprising:
The flow data that the file link URL indicates is copied from Web server；
The flow data is deleted from the Web server.
3. according to the method described in claim 2, it is characterized in that,
The primary data information read in the tables of data, comprising:
Judge the mark information for whether having file link URL in the tables of data；
If there is the mark information of file link URL in the tables of data, judge whether there is file link URL in the tables of data
First position feature string；
If there is the first position feature string of file link URL in the tables of data, it is based on the first position feature string, from institute
State reading file link URL in tables of data；
It is described that the flow data that the file link URL indicates is copied from Web server, comprising:
When the file link URL is relative path, the target text that the file link URL indicates is copied from Web server
Part.
4. according to the method described in claim 2, it is characterized in that,
The primary data information read in the tables of data, comprising:
Judge whether there is flow data feature string in the tables of data；
If there is flow data feature string in the tables of data, the second for whether having file link URL in the tables of data is judged
Set feature string；
If there is the second position feature string of file link URL in the tables of data, it is based on the second position feature string, from institute
State reading file link URL in tables of data；
It is described that the flow data that the file link URL indicates is copied from Web server, comprising:
When the file link URL is relative path, the flow data that the file link URL indicates is copied from Web server.
5. the method according to claim 1, wherein
The primary data information is the coded data encoded containing base64；
It is described according to the primary data acquisition of information flow data, comprising:
According to base64 coding rule, the coded data is reduced into binary flow data.
6. according to the method described in claim 5, it is characterized in that,
The primary data information read in the tables of data, comprising:
Judge whether there is flow data feature string in the tables of data；
If there is flow data feature string in the tables of data, judge whether there is base64 coding characteristic string in the tables of data；
If there is base64 coding characteristic string in the tables of data, the coded number of base64 coding is read from the tables of data
According to.
7. according to the method described in claim 6, it is characterized in that,
It is described according to base64 coding rule, before the coded data is reduced into binary flow data, the method is also
Include:
Reduction pretreatment is executed to the coded data；
The setting target URL, comprising:
It is read from the tables of data and judges streaming data types information；
Using the streaming data types information, filename is generated；
Target URL is set using the filename.
8. according to the described in any item methods of claim 4-7, which is characterized in that
The flow data is one of in picture file, video file and audio file；
The server is for saving a type of flow data.
9. a kind of processing equipment characterized by comprising
Acquiring unit, for obtaining tables of data from database；
Reading unit, for reading the primary data information in the tables of data；
The acquiring unit is also used to according to the primary data acquisition of information text information and flow data；
Setup unit, for setting target URL；
Replacement unit, for being replaced using the target URL corresponding in the primary data information in the tables of data
URL or flow data, to obtain target matrix；
Write back unit, for the target matrix to be written back in the database；
Transmission unit, for sending the flow data to server, so that the server saves the flow data, the target
URL is the address that the server saves the flow data.
10. a kind of storage medium is stored with processor-executable instruction, described instruction is by one or more processors
Load, to execute such as data processing method any in claim 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811039467.1A CN109308336A (en) | 2018-09-06 | 2018-09-06 | Data processing method, processing equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811039467.1A CN109308336A (en) | 2018-09-06 | 2018-09-06 | Data processing method, processing equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109308336A true CN109308336A (en) | 2019-02-05 |
Family
ID=65224803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811039467.1A Pending CN109308336A (en) | 2018-09-06 | 2018-09-06 | Data processing method, processing equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109308336A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112019482A (en) * | 2019-05-30 | 2020-12-01 | 福建天泉教育科技有限公司 | Web-based communication method and computer-readable storage medium |
| JP2020194390A (en) * | 2019-05-29 | 2020-12-03 | ヤマハ発動機株式会社 | Circuit board production system, production line data management device, and production line data management method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528563A (en) * | 2015-09-11 | 2017-03-22 | 腾讯科技（深圳）有限公司 | Content access method and apparatus |
| CN106789601A (en) * | 2017-02-08 | 2017-05-31 | 奥秘智能科技（洛阳）有限公司 | Universal data collection and supervisor control and method based on wechat public platform |
| CN107403012A (en) * | 2017-08-01 | 2017-11-28 | 山东浪潮通软信息科技有限公司 | A kind of method for interchanging data and device |
-
2018
- 2018-09-06 CN CN201811039467.1A patent/CN109308336A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528563A (en) * | 2015-09-11 | 2017-03-22 | 腾讯科技（深圳）有限公司 | Content access method and apparatus |
| CN106789601A (en) * | 2017-02-08 | 2017-05-31 | 奥秘智能科技（洛阳）有限公司 | Universal data collection and supervisor control and method based on wechat public platform |
| CN107403012A (en) * | 2017-08-01 | 2017-11-28 | 山东浪潮通软信息科技有限公司 | A kind of method for interchanging data and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2020194390A (en) * | 2019-05-29 | 2020-12-03 | ヤマハ発動機株式会社 | Circuit board production system, production line data management device, and production line data management method |
| CN112019482A (en) * | 2019-05-30 | 2020-12-01 | 福建天泉教育科技有限公司 | Web-based communication method and computer-readable storage medium |
| CN112019482B (en) * | 2019-05-30 | 2023-02-28 | 福建天泉教育科技有限公司 | Web-based communication method and computer-readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7475341B2 (en) | Converting the format of a portion of an electronic document | |
| US10515142B2 (en) | Method and apparatus for extracting webpage information | |
| CN106354861B (en) | Film label automatic indexing method and automatic indexing system | |
| US6012068A (en) | Media manager for access to multiple media types | |
| CA2410747C (en) | System and method for saving browsed data | |
| US5892908A (en) | Method of extracting network information | |
| KR101409673B1 (en) | Persistent saving portal | |
| US8443346B2 (en) | Server evaluation of client-side script | |
| US7606840B2 (en) | Version control in a distributed computing environment | |
| CN105608134A (en) | Multithreading-based web crawler system and web crawling method thereof | |
| CN108021598B (en) | Page extraction template matching method and device and server | |
| CN109040252A (en) | Document transmission method, system, computer equipment and storage medium | |
| JP2006508436A (en) | Method and system for hyperlinking files | |
| US20090172520A1 (en) | Method of managing web services using integrated document | |
| CN103716394B (en) | Download the management method and device of file | |
| KR101340588B1 (en) | Method and apparatus for comprising webpage | |
| CN108509504A (en) | The online method for previewing of document, device, equipment, client and storage medium | |
| CN109308336A (en) | Data processing method, processing equipment and storage medium | |
| CN112637361A (en) | Page proxy method, device, electronic equipment and storage medium | |
| CN112612943A (en) | Asynchronous processing framework-based data crawling method with automatic testing function | |
| CN104978325B (en) | A kind of web page processing method, device and user terminal | |
| US20050278627A1 (en) | Editing an image representation of a text | |
| CN104978341A (en) | File processing method and equipment, and network system | |
| JP2006065467A (en) | Device for creating data extraction definition information and method for creating data extraction definition information | |
| JP2006065467A5 (en) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190205 |