CN105022699B - The preprocess method and system of buffer area data - Google Patents

The preprocess method and system of buffer area data Download PDF

Info

Publication number
CN105022699B
CN105022699B CN201510412138.7A CN201510412138A CN105022699B CN 105022699 B CN105022699 B CN 105022699B CN 201510412138 A CN201510412138 A CN 201510412138A CN 105022699 B CN105022699 B CN 105022699B
Authority
CN
China
Prior art keywords
mrow
msub
data
mover
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510412138.7A
Other languages
Chinese (zh)
Other versions
CN105022699A (en
Inventor
施文进
胡芳槐
阎九吉
吴青
王飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Huiyin Science & Technology Co Ltd
ZHENJIANG HUILONG YANGTSE RIVER PORT CO Ltd
WELLONG ETOWN INTERNATIONAL LOGISTICS Co Ltd
Original Assignee
Jiangsu Huiyin Science & Technology Co Ltd
ZHENJIANG HUILONG YANGTSE RIVER PORT CO Ltd
WELLONG ETOWN INTERNATIONAL LOGISTICS Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Huiyin Science & Technology Co Ltd, ZHENJIANG HUILONG YANGTSE RIVER PORT CO Ltd, WELLONG ETOWN INTERNATIONAL LOGISTICS Co Ltd filed Critical Jiangsu Huiyin Science & Technology Co Ltd
Priority to CN201510412138.7A priority Critical patent/CN105022699B/en
Publication of CN105022699A publication Critical patent/CN105022699A/en
Application granted granted Critical
Publication of CN105022699B publication Critical patent/CN105022699B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention proposes a kind of preprocess method and system of buffer area data, and this method can accurately estimate user's query time, user's residence time and user and inquire about content, specifically include：Record construction basic data, pre-processes basic data；LEAST SQUARES MODELS FITTING modelling customer behavior is established, the user job time is predicted and inquires about the data relationship between the parameters such as content；The data from caching input reception are stored to buffer area, are exported according to first in first out order from the buffer area.The system of the present invention is by way of machine learning, study the code of conduct of user, predict user's query time, each working time and inquiry content etc., system will set buffer area data in advance according to information of forecasting, so that the inquiry experience of user optimizes, solves the accurate technical problem estimated user job time, user's residence time and user and inquire about content in electronic commerce data processing system.

Description

The preprocess method and system of buffer area data
Technical field
The present invention relates to the preprocess method and system of a kind of data, more particularly to it is a kind of applied to the pre- of buffer area data Processing method and system.
Background technology
Data Preprocessing Technology mainly simply uses a kind of technology at present, and electronic commerce data is with sudden strong It is responsible for the features such as abnormal heavy with instantaneous data, very big data processing load can be caused using a kind for the treatment of technology merely, no Can meet the needs of e-commerce.
First Input First Output is a kind of traditional sequentially execution method, when buffer area data are full, at first into buffer area Data/commands first complete to perform and leave buffer area, and then just perform Article 2 data/commands.It is a kind of first in first out Data buffer, his difference with normal memory is that do not have exterior read-write address wire, so uses very simple, but is lacked Point is exactly that can only be sequentially written in data, the reading data of order, its data address reads and writes pointer automatically by inside plus 1 completes, no It can be determined to read by address wire as normal memory or write some address specified, it cannot be accurately estimated in electricity User's query time, residence time, inquiry content in sub- business data system；Statistical method, utilizes mathematical statistics method, system The system frequency counted, any active ues information preferably there are buffer area, by color register be buffered in buffer with In the corresponding buffer area of color of the region of memory of the physical address of current accessed, it is possible to increase the service efficiency of caching, is improved System performance, but the characteristics of this method still can not meet electronic commerce data.
The present invention provides a kind of preprocess method of buffer area data, this method passes through the method for machine learning, research The code of conduct of user, predicts user's query time, each working time and inquiry content etc., system will be carried according to information of forecasting Preceding setting buffer area data, so that the inquiry experience of user optimizes.
The content of the invention
An embodiment of the present invention provides a kind of preprocess method of buffer area data, the side that this method passes through machine learning Method, studies the code of conduct of user, predicts user's query time, each working time and inquiry content etc., system will be according to pre- Measurement information sets buffer area data in advance, so that the inquiry experience of user optimizes.
To reach above-mentioned purpose, the embodiment of the present invention adopts the following technical scheme that：
First aspect present invention provides a kind of buffer area data preprocessing method, including：
Record construction basic data, pre-processes basic data；
LEAST SQUARES MODELS FITTING modelling customer behavior is established, the user job time is predicted and inquires about between the parameters such as content Data relationship；
The data from caching input reception are stored to buffer area, are exported according to first in first out order from the buffer area.
Preferably, according in a first aspect, the record construction basic data, specifically includes：
Basic data refers to user's query time TimeUserQuery, user's residence time TimeUserStand and user Inquire about content ContentUserQuery.Construction TimeUserQuery, TimeUserStand and ContentUserQuery connect Mouth function obtains the query time of client user, residence time and inquiry content from initial server end；Described Preset timer Timer in TimeUserQuery and TimeUserStand functions, and cookie ActiveX Techniques are used, obtain and work as Query time and the residence time to move ahead as middle user；The data being collected into are sent to mesh by the asynchronous mode of GET, POST Mark server end；The basic data is shown to the destination server end by interface with JSON forms.
Preferably, the user inquires about content ContentUserQuery, specifically includes：
Have in the manipulable all inquiries of systemic presupposition user Loading, Unloading, Cargo, Carrier and One kind or its any combination (the predeterminable different inquiry contents of different industries and demand) in Route, The parameter of ContentUserQuery interface functions is Loading, Unloading, Cargo, Carrier and Route, according to The different operating behavior at family, returns and is set to 1 with the parameter value of displaying difference, the parameter return value for having carried out the inquiry content, The parameter return value for not carrying out the inquiry content is set to 0.
Preferably, according in a first aspect, it is described to basic data pre-process, specifically include：
After the destination server receives return value and returned content, system uses the Parse of JObject or JArray JSON character strings are converted to JSON objects by method, and the basic data is extracted by way of the JSON objects, analyze institute State basic data inquiry content and query time between association construct Loading, Unloading, Cargo, Carrier, The graph of a relation of Route and TimeUserQuery and TimeUserStand.
Preferably, according in a first aspect, it is described construction Loading, Unloading, Cargo, Carrier, Route and The graph of a relation of TimeUserQuery and TimeUserStand, a kind of possible implementation are：
Preferably, in the graph of a relation, TimeUserQuery and TimeUserStand respectively as dependent variable and As independent variable, observation figure finds linearly to return with certain by Loading, Unloading, Cargo, Carrier, Route Return trend, consideration is made prediction with least square method.
Preferably, least square method is a kind of mathematical optimization techniques, it finds data by minimizing the quadratic sum of error Optimal function matching, can easily try to achieve unknown data using least square method, and cause the data that these are tried to achieve with The quadratic sum of error is minimum between real data, can be in the hope of the optimal value of object function.
Step 1：The destination server receives the multiple inquiry operation of a user, and the user queried described look into The one or more of content is ask, if inquiry content is n, the time of each inquiry content of user's inquiry is denoted as respectively：
T=(t1,t2,t3,...ti...,tn) (1)
Wherein tiRepresent query time when content is inquired about in described i-th of user's inquiry.
Step 2：The query time of the m inquiry inquiry content of one user is expressed as：
y(t1,...,tn；x0,x1,…,xn)=x0+x1t1+…+xntn (2)
Wherein y represents the working time of user's inquiry inquiry content, x0,x1,…,xnRepresent model parameter, the parameter So that actual value and the quadratic sum of observed difference are minimum, x is usually taken0=1, it is expressed as with system of linear equations：
Wherein yiRepresent the query time used in user's ith inquiry inquiry content, tijRepresent the user Ith inquires about the query time used in the jth item inquiry content.
Usually by tijIt is denoted as data matrix A, the model parameter xiParameter vector X is denoted as, query time y described in useri Y is denoted as, then system of linear equations is represented by：
That is AX=Y (4)
Wherein,
Step 3：The query time of fitting real user behavior and the value of the model parameter matrix X of inquiry content are： By the observability estimate value of LEAST SQUARES MODELS FITTING definable user inquiry one inquiry contentWith the model parameter Estimate
Wherein i=1,2 ..., n, k=1,2 ..., m. (6)
Obtain：
Wherein
Then the estimate equation group with the model parameter is obtained：
According to (8) (9) obtain user inquiry it is described inquiry content used in time observation and estimate relation be：
According to the principle of least square, the value of the model parameter is：
Finally obtaining the estimate of the model parameter is：
Step 4：Predict the TimeUserQuery times of the user：
Wherein tiRepresent query time when content is inquired about in described i-th of user's inquiry.xiTable Show i-th corresponding model parameter of the inquiry content, wherein x0=1.If user only carries out Cargo operations, prediction Cargo query times are：
y3=x0+t3x3。 (13)
Wherein distinguish for described inquiry content Loading, Unloading, Cargo, Carrier, Route in tables of data One SessionId is set.Related parameter values are directly obtained by the SessionId in above-mentioned steps 4, and will be obtained Initial data of the data as buffer area input data.
Preferably, second aspect, there is provided a kind of buffer area data preprocessing method, further includes：
Master cache area is arranged to store the data received from caching input, and cache controller is for selective from institute State buffering area and the reception data are routed to spare buffer area so that the data received from caching input can be according to The reception data are output to the caching from the spare buffer area and exported by FIFO order.
Preferably, the spare caching is used for the reception number for storing the caching input or storage master cache receives According to, and with the master cache it is identical receive data order by it is described reception data be output to it is described caching export.
Preferably, the effect of the cache controller be when the master cache is empty data mode, the master cache from Caching input transmits data to the spare caching, or when the spare caching is full data mode, the spare caching Data are transmitted from caching input to the master cache, or when the master cache data mode is not empty, the reception data Data are transmitted from caching input to the master cache.
Preferably, the master cache and it is spare caching be can store different types of data independent fifo queue and The data space of master cache is more than the data space of spare caching.
Preferably, the third aspect, there is provided a kind of buffer area data pretreatment, including：
Transmission device：Send the data to buffer area；Buffer area：For receiving data from transmission device, and according to first entering The data of reception are sent to reception device by the order first gone out；Reception device：For receiving the data come from buffer area.
Wherein, the system is trained data and has been handled first, since data volume is larger, is filled first by transmitting Put and put it into buffer area.
Preferably, it is according to a kind of possible implementation of the third aspect：
Buffer area includes master cache and spare caching, and the master cache is configured to be mainly used for storage from caching input reception Data；The spare caching is mainly used for the reception data for storing the caching input or storage master cache receives, And with the master cache it is identical receive data order by it is described reception data be output to it is described caching export.
Preferably, the buffer area further includes cache controller, it is when the master cache is full data mode, the master Caching transmits data from caching input to the spare caching, or when the spare caching is full data mode, it is described standby Data are transmitted from caching input to the master cache with caching, or when the spare data cached state is discontented, the master Caching is from caching input to the spare caching transmission data.
Preferably, it is according to the third aspect, second of mode in the cards：
In order to improve the performance of the system, using least square method data are carried out with constantly training first and is pre-processed, Secondly it is the multiple buffer areas of system configuration, the data space of last master cache is more than the memory space of spare caching.
Brief description of the drawings
In order to illustrate more clearly of the embodiment of the present invention or technical solution of the prior art, below will to embodiment or Attached drawing needed to be used in the description of the prior art is briefly described, and drawings in the following description are only some of the present invention Embodiment.
Fig. 1 is a kind of partial function interface signal for buffer area data preprocessing method that the embodiment of the present invention provides Figure；
Fig. 2 provides the flow diagram of least square method modulus shape parameter for the embodiment of the present invention；
Fig. 3 is a kind of buffer area data preprocessing method flow diagram that the embodiment of the present invention provides；
Fig. 4 is a kind of structure diagram for buffer area data pretreatment that the embodiment of the present invention provides.
Embodiment
To make the technical problem to be solved in the present invention, technical solution and advantage clearer, below in conjunction with attached drawing and tool Body embodiment.
The embodiment of the present invention provides a kind of buffer area data preprocessing method and system.The present invention can be used for buffer area number Data preprocess, first against behavior record of certain user on platform, the inquiry parameter such as content and working time, based on number According to, recorded and pre-processed, according to the pretreatment basic data, established LEAST SQUARES MODELS FITTING and carry out modelling customer behavior, Predict user's query time and inquire about the data relationship between the parameters such as content, obtained data input what is received as from caching Data, distribute to buffer area, are exported according to first in first out order from the buffer area.
Specifically, the embodiment of the present invention provides a kind of buffer area data preprocessing method and system, existed according to certain user Behavior record, the inquiry parameter such as content and working time on platform, with reference to shown in Fig. 1, including herein below：
Record behavior record of the user on platform, the inquiry content of the user and working time, based on data, Specifically include：
Basic data refers to user's query time TimeUserQuery, user's residence time TimeUserStand and user Inquire about content ContentUserQuery.Construction TimeUserQuery, TimeUserStand and ContentUserQuery connect Mouth function obtains the query time of client user, residence time and inquiry content from initial server end；Described Preset timer Timer in TimeUserQuery and TimeUserStand functions, and cookie ActiveX Techniques are used, obtain and work as Query time and the residence time to move ahead as middle user；The data being collected into are sent to mesh by the asynchronous mode of GET, POST Mark server end；The basic data is shown to the destination server end by interface with JSON forms.
The user inquires about content ContentUserQuery, specifically includes：Systemic presupposition user is manipulable all Loading, Unloading, Cargo, Carrier and Route are had in inquiry, and (the different predeterminable differences of industry and demand are looked into Ask content), the parameters of ContentUserQuery interface functions for Loading, Unloading, Cargo, Carrier and Route, according to the different operating behavior of user, return is different with the parameter value of displaying, has carried out the parameter of the inquiry content Return value is set to 1, and the parameter return value for not carrying out the inquiry content is set to 0.
After recording the basic data, the basic data is pre-processed, is specifically included：The destination server connects After receiving return value and returned content, system is converted to JSON character strings using the Parse methods of JObject or JArray JSON objects, the basic data is extracted by way of the JSON objects, is analyzed the basic data inquiry content and is looked into Ask the time between association i.e. construct Loading, Unloading, Cargo, Carrier, Route and TimeUserQuery with And the graph of a relation of TimeUserStand.Construct Loading, Unloading, Cargo, Carrier, Route and The graph of a relation of TimeUserQuery and TimeUserStand, a kind of possible implementation are：
In the graph of a relation, TimeUserQuery and TimeUserStand respectively as dependent variable and Loading, Unloading, Cargo, Carrier, Route have found there is certain linear regression trend, examine as independent variable, observation figure Worry is modeled and predicted with least square method.
Modeling and pre- flow gauge the embodiment provides least square method, and try to achieve the optimal of model parameter Solution, with reference to shown in Fig. 2, comprises the following steps：
Least square method is a kind of mathematical optimization techniques, it finds the optimal letter of data by minimizing the quadratic sum of error Number matching, unknown data can be easily tried to achieve using least square method, and cause these data and real data for trying to achieve Between error quadratic sum for minimum, can be in the hope of the optimal value of object function.
Step 1：The destination server receives the multiple inquiry operation of a user, and the user queried described look into The one or more of content is ask, if inquiry content is n, the time of each inquiry content of user's inquiry is denoted as respectively：
T=(t1,t2,t3,...ti...,tn)(1)
Wherein tiRepresent query time when content is inquired about in described i-th of user's inquiry.
Step 2：The query time of the m inquiry inquiry content of one user is expressed as：
y(t1,...,tn；x0,x1,...,xn)=x0+x1t1+…+xntn(2)
Wherein y represents the working time of user's inquiry inquiry content, x0,x1,…,xnRepresent model parameter, the parameter So that actual value and the quadratic sum of observed difference are minimum, x is usually taken0=1, it is expressed as with system of linear equations：
Wherein yiRepresent the query time used in user's ith inquiry inquiry content, tijRepresent the user Ith inquires about the query time used in the jth item inquiry content.
Usually by tijIt is denoted as data matrix A, the model parameter xiParameter vector X is denoted as, query time y described in useri Y is denoted as, then system of linear equations is represented by：
That is AX=Y (4)
Wherein,
Step 3：The query time of fitting real user behavior and the value of the model parameter matrix X of inquiry content are： By the observability estimate value of LEAST SQUARES MODELS FITTING definable user inquiry one inquiry contentWith the model parameter Estimate
Wherein i=1,2 ..., n, k=1,2 ..., m. (6)
Obtain：
Wherein
Then the estimate equation group with the model parameter is obtained：
According to (8) (9) obtain user inquiry it is described inquiry content used in time observation and estimate relation be：
According to the principle of least square, the value of the model parameter is：
Finally obtaining the estimate of the model parameter is：
Step 4：Predict the TimeUserQuery times of the user：
Wherein tiRepresent query time when content is inquired about in described i-th of user's inquiry.xiRepresent to look into described in i-th Ask the corresponding model parameter of content, wherein x0=1.If user only carries out Cargo operations, prediction Cargo query times are：
y3=x0+t3x3。
Wherein distinguish for described inquiry content Loading, Unloading, Cargo, Carrier, Route in tables of data One SessionId is set.Related parameter values are directly obtained by the SessionId in above-mentioned steps 4, and will be obtained Initial data of the data as buffer area input data.
The embodiment provides a kind of buffer area data preprocessing method, the basic data conduct of the pretreatment The data that the caching input receives, the operational process in platform is with reference to shown in Fig. 3, including herein below：
Master cache area is arranged to store the data received from caching input, and cache controller is for selective from institute State buffering area and the reception data are routed to spare buffer area so that the data received from caching input can be according to The reception data are output to the caching from the spare buffer area and exported by FIFO order.
The spare caching is used to store the caching input or stores the reception data that master cache receives, and with With the master cache it is identical receive data order by it is described reception data be output to it is described caching export.
The effect of the cache controller is that the master cache is inputted from caching when the master cache is empty data mode To the spare caching transmission data；
Or；
When the spare caching is full data mode, the spare caching transmits number from caching input to the master cache According to；
Or；
When the master cache is not empty data mode, the reception data are transmitted from caching input to the master cache Data.
The master cache and spare caching are the independent fifo queue and master cache that can store different types of data Data space be more than spare caching data space.
The storage state of buffer area is updated, receives request of data；
Buffer area data are set to finish in advance.
The embodiment provides a kind of buffer area data pretreatment, with reference to shown in Fig. 4, including it is following interior Hold：
Transmission device：Send the data to buffer area；Buffer area：For receiving data from transmission device, and according to first entering The data of reception are sent to reception device by the order first gone out；Reception device：For receiving the data come from buffer area.
A kind of buffer area data preprocessing method is trained data and has been handled first, due to data volume compared with Greatly, buffer area is put it into first；The buffer area receives data from transmission device, and is received according to the order handle of first in, first out Data send reception device to.
Buffer area includes master cache and spare caching, and the master cache is configured to be mainly used for storage from caching input reception Data；The spare caching is mainly used for the reception data for storing the caching input or storage master cache receives, And with the master cache it is identical receive data order by it is described reception data be output to it is described caching export.
The buffer area further includes cache controller, its when the master cache is empty data mode, the master cache from Caching input transmits data to the spare caching, or when the spare caching is full data mode, the spare caching Data are transmitted from caching input to the master cache, or when the master cache data mode is not empty, the reception data Data are transmitted from caching input to the master cache.
In order to improve the performance of the system, using least square method data are carried out with constantly training first and is pre-processed, Secondly it is the multiple cachings of system configuration, the data space of last master cache is more than the memory space of spare caching.
The above is the preferred embodiment of the present invention, it is noted that is used for the middle-and-high-ranking technology of the art For family, without departing from the principles of the present invention, some improvements and modifications can also be made, these improvements and modifications It is the inevitable preceding exhibition in our inventions as a result, also should be regarded as protection scope of the present invention.

Claims (11)

  1. A kind of 1. preprocess method of buffer area data, it is characterised in that including：
    Record construction basic data, pre-processes basic data, specifically includes：Basic data refers to user's query time TimeUserQuery, user's residence time TimeUserStand and user inquire about content ContentUserQuery；Construction TimeUserQuery, TimeUserStand and ContentUserQuery interface function obtain client from initial server end The query time of user, residence time and inquiry content；It is pre- in TimeUserQuery the and TimeUserStand functions If timer Timer, and cookie ActiveX Techniques are used, obtain the query time of user and residence time in current behavior；Will The data being collected into are sent to destination server end by the asynchronous mode of GET, POST；The basic data by interface with JSON forms are shown to the destination server end；
    After destination server receives return value and returned content, system uses the Parse methods of JObject or JArray will JSON character strings are converted to JSON objects, extract the basic data by way of the JSON objects, analyze the basis Association between data query content and query time, that is, construct Loading, Unloading, Cargo, Carrier, Route With the graph of a relation of TimeUserQuery and TimeUserStand；
    In the graph of a relation, TimeUserQuery and TimeUserStand respectively as dependent variable and Loading, Unloading, Cargo, Carrier, Route, using least square method, establish LEAST SQUARES MODELS FITTING simulation as independent variable User behavior, predicts the user job time and inquires about the data relationship between content；
    The LEAST SQUARES MODELS FITTING modelling customer behavior of establishing includes prediction user query time TimeUserQuery, specifically Mode is：
    <mrow> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>=</mo> <msub> <mi>x</mi> <mn>0</mn> </msub> <mo>+</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msub> <mi>t</mi> <mi>i</mi> </msub> <msub> <mi>x</mi> <mi>i</mi> </msub> </mrow>
    Wherein tiRepresent query time when user inquires about i-th inquiry content；N is the number of inquiry content；xiRepresent the The i corresponding model parameters of the inquiry content, wherein x0=1；If user only carries out Cargo operations, when Cargo is inquired about Between can be predicted into：
    y3=x0+t3x3；
    One is set respectively in tables of data for described inquiry content Loading, Unloading, Cargo, Carrier, Route SessionId, directly related parameter values are obtained in above-mentioned LEAST SQUARES MODELS FITTING by the SessionId, and will be obtained Initial data of the data as buffer area input data；
    The data from caching input reception are stored to buffer area, are exported according to first in first out order from the buffer area.
  2. 2. according to the method described in claim 1, it is characterized in that, the user inquires about content ContentUserQuery, tool Body includes：
    Have in all inquiries of systemic presupposition user's operation in Loading, Unloading, Cargo, Carrier and Route One kind or its any combination, the parameters of ContentUserQuery interface functions is Loading, Unloading, Cargo, Carrier and Route, according to the different operating behavior of user, return is different with the parameter value of displaying, has carried out in the inquiry The parameter return value of appearance is set to 1, and the parameter return value for not carrying out the inquiry content is set to 0.
  3. 3. according to the method described in claim 1, it is characterized in that, described establish least-squares algorithm model, specifically further include：
    Step 1：The destination server receives the multiple inquiry operation of a user, and the user queried in the inquiry The one or more of appearance, if inquiry content is n, the time of each inquiry content of user's inquiry is denoted as respectively：
    T=(t1,t2,t3,...ti...,tn)(1)
    Wherein tiRepresent query time when user inquires about i-th inquiry content；
    Step 2：The query time of the m inquiry inquiry content of one user is expressed as：
    y(t1,...,tn；x0,x1,...,xn)=x0+x1t1+…+xntn(2)
    Wherein y represents the working time of user's inquiry inquiry content, x0,x1,…,xnRepresent model parameter, which causes Actual value and the quadratic sum of observed difference are minimum, are expressed as with system of linear equations：
    Wherein yiRepresent the query time used in user's ith inquiry inquiry content, tijRepresent user's ith Inquire about the query time used in jth item inquiry content；
    By tijIt is denoted as data matrix A, the model parameter xiParameter vector X is denoted as, query time y described in useriY is denoted as, then line Property equation group is represented by：
    That is AX=Y (4)
    Wherein,
    Step 3：The query time of fitting real user behavior and the value of the model parameter matrix X of inquiry content are：By minimum Square law model defines the observability estimate value that user inquires about an inquiry contentWith the estimate of the model parameter
    Wherein i=1,2 ..., n, k=1,2 ..., m (6)
    Obtain：
    <mrow> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <mfrac> <mo>&amp;part;</mo> <mrow> <mo>&amp;part;</mo> <msub> <mover> <mi>X</mi> <mo>^</mo> </mover> <mn>0</mn> </msub> </mrow> </mfrac> <mi>Q</mi> <mo>=</mo> <mn>0</mn> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mfrac> <mo>&amp;part;</mo> <mrow> <mo>&amp;part;</mo> <msub> <mover> <mi>X</mi> <mo>^</mo> </mover> <mn>1</mn> </msub> </mrow> </mfrac> <mi>Q</mi> <mo>=</mo> <mn>0</mn> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mfrac> <mo>&amp;part;</mo> <mrow> <mo>&amp;part;</mo> <msub> <mover> <mi>X</mi> <mo>^</mo> </mover> <mn>2</mn> </msub> </mrow> </mfrac> <mi>Q</mi> <mo>=</mo> <mn>0</mn> </mrow> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <mrow> <mfrac> <mo>&amp;part;</mo> <mrow> <mo>&amp;part;</mo> <msub> <mover> <mi>X</mi> <mo>^</mo> </mover> <mi>k</mi> </msub> </mrow> </mfrac> <mi>Q</mi> <mo>=</mo> <mn>0</mn> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>7</mn> <mo>)</mo> </mrow> </mrow>
    Wherein
    Then the estimate equation group with the model parameter is obtained：
    <mrow> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <mo>&amp;Sigma;</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>-</mo> <mo>&amp;Sigma;</mo> <mrow> <mo>(</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>0</mn> </msub> <mo>+</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>1</mn> </msub> <msub> <mi>t</mi> <mrow> <mn>1</mn> <mi>i</mi> </mrow> </msub> <mo>+</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>2</mn> </msub> <msub> <mi>t</mi> <mrow> <mn>2</mn> <mi>i</mi> </mrow> </msub> <mo>+</mo> <mn>...</mn> <mo>+</mo> <msub> <mi>x</mi> <mi>k</mi> </msub> <msub> <mi>t</mi> <mrow> <mi>k</mi> <mi>i</mi> </mrow> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mn>0</mn> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mo>&amp;Sigma;</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <msub> <mi>x</mi> <mrow> <mn>1</mn> <mi>i</mi> </mrow> </msub> <mo>-</mo> <mo>&amp;Sigma;</mo> <mrow> <mo>(</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>0</mn> </msub> <mo>+</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>1</mn> </msub> <msub> <mi>t</mi> <mrow> <mn>1</mn> <mi>i</mi> </mrow> </msub> <mo>+</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>2</mn> </msub> <msub> <mi>t</mi> <mrow> <mn>2</mn> <mi>i</mi> </mrow> </msub> <mo>+</mo> <mn>...</mn> <mo>+</mo> <msub> <mi>x</mi> <mi>k</mi> </msub> <msub> <mi>t</mi> <mrow> <mi>k</mi> <mi>i</mi> </mrow> </msub> <mo>)</mo> </mrow> <msub> <mi>x</mi> <mrow> <mn>1</mn> <mi>i</mi> </mrow> </msub> <mo>=</mo> <mn>0</mn> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mo>&amp;Sigma;</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <msub> <mi>x</mi> <mrow> <mn>2</mn> <mi>i</mi> </mrow> </msub> <mo>-</mo> <mo>&amp;Sigma;</mo> <mrow> <mo>(</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>0</mn> </msub> <mo>+</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>1</mn> </msub> <msub> <mi>t</mi> <mrow> <mn>1</mn> <mi>i</mi> </mrow> </msub> <mo>+</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>2</mn> </msub> <msub> <mi>t</mi> <mrow> <mn>2</mn> <mi>i</mi> </mrow> </msub> <mo>+</mo> <mn>...</mn> <mo>+</mo> <msub> <mi>x</mi> <mi>k</mi> </msub> <msub> <mi>t</mi> <mrow> <mi>k</mi> <mi>i</mi> </mrow> </msub> <mo>)</mo> </mrow> <msub> <mi>x</mi> <mrow> <mn>2</mn> <mi>i</mi> </mrow> </msub> <mo>=</mo> <mn>0</mn> </mrow> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <mo>.</mo> </mtd> </mtr> <mtr> <mtd> <mrow> <mo>&amp;Sigma;</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <msub> <mi>x</mi> <mrow> <mi>k</mi> <mi>i</mi> </mrow> </msub> <mo>-</mo> <mo>&amp;Sigma;</mo> <mrow> <mo>(</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>0</mn> </msub> <mo>+</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>1</mn> </msub> <msub> <mi>t</mi> <mrow> <mn>1</mn> <mi>i</mi> </mrow> </msub> <mo>+</mo> <msub> <mover> <mi>x</mi> <mo>^</mo> </mover> <mn>2</mn> </msub> <msub> <mi>t</mi> <mrow> <mn>2</mn> <mi>i</mi> </mrow> </msub> <mo>+</mo> <mn>...</mn> <mo>+</mo> <msub> <mi>x</mi> <mi>k</mi> </msub> <msub> <mi>t</mi> <mrow> <mi>k</mi> <mi>i</mi> </mrow> </msub> <mo>)</mo> </mrow> <msub> <mi>x</mi> <mrow> <mi>k</mi> <mi>i</mi> </mrow> </msub> <mo>=</mo> <mn>0</mn> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>9</mn> <mo>)</mo> </mrow> </mrow>
    According to (8) (9) obtain user inquiry it is described inquiry content used in time observation and estimate relation be：
    <mrow> <mi>Q</mi> <mo>=</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msubsup> <mi>e</mi> <mi>i</mi> <mn>2</mn> </msubsup> <mo>=</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msup> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mover> <mi>y</mi> <mo>^</mo> </mover> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mn>2</mn> </msup> </mrow>
    <mrow> <mo>=</mo> <msup> <mi>e</mi> <mi>t</mi> </msup> <mi>e</mi> <mo>=</mo> <msup> <mrow> <mo>(</mo> <mi>Y</mi> <mo>-</mo> <mi>A</mi> <mover> <mi>X</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mi>t</mi> </msup> <mrow> <mo>(</mo> <mi>Y</mi> <mo>-</mo> <mi>A</mi> <mover> <mi>X</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>10</mn> <mo>)</mo> </mrow> </mrow>
    Then the value of the model parameter is：
    <mrow> <mfrac> <mo>&amp;part;</mo> <mrow> <mo>&amp;part;</mo> <mover> <mi>X</mi> <mo>^</mo> </mover> </mrow> </mfrac> <msup> <mrow> <mo>(</mo> <mi>Y</mi> <mo>-</mo> <mi>A</mi> <mover> <mi>X</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mi>t</mi> </msup> <mrow> <mo>(</mo> <mi>Y</mi> <mo>-</mo> <mi>A</mi> <mover> <mi>X</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mo>=</mo> <mn>0</mn> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>11</mn> <mo>)</mo> </mrow> </mrow>
    Finally obtaining the estimate of the model parameter is：
    <mrow> <mfrac> <mo>&amp;part;</mo> <mrow> <mo>&amp;part;</mo> <mover> <mi>X</mi> <mo>^</mo> </mover> </mrow> </mfrac> <mrow> <mo>(</mo> <msup> <mi>Y</mi> <mi>t</mi> </msup> <mi>Y</mi> <mo>-</mo> <mn>2</mn> <mover> <mi>X</mi> <mo>^</mo> </mover> <msup> <mi>A</mi> <mi>t</mi> </msup> <mi>Y</mi> <mo>+</mo> <mover> <mi>X</mi> <mo>^</mo> </mover> <msup> <mi>A</mi> <mi>t</mi> </msup> <mi>A</mi> <mover> <mi>X</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mo>=</mo> <mn>0</mn> </mrow>
    <mrow> <mtable> <mtr> <mtd> <mrow> <msup> <mi>A</mi> <mi>t</mi> </msup> <mi>Y</mi> <mo>=</mo> <msup> <mi>A</mi> <mi>t</mi> </msup> <mi>A</mi> <mover> <mi>X</mi> <mo>^</mo> </mover> </mrow> </mtd> <mtd> <mrow> <mover> <mi>X</mi> <mo>^</mo> </mover> <mo>=</mo> <msup> <mrow> <mo>(</mo> <msup> <mi>A</mi> <mi>t</mi> </msup> <mi>A</mi> <mo>)</mo> </mrow> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </msup> <msup> <mi>A</mi> <mi>t</mi> </msup> <mi>Y</mi> </mrow> </mtd> </mtr> </mtable> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>12</mn> <mo>)</mo> </mrow> <mo>.</mo> </mrow>
  4. 4. according to the method described in claim 1, it is characterized in that, the storage is from the data that caching input receives to caching Area, exports from the buffer area according to first in first out order, specifically includes：
    Master cache area is arranged to the data that storage is received from caching input, and cache controller is used for the slave buffering area of selectivity The reception data are routed to spare buffer area so that the data received from caching input are according to FIFO order from described The reception data are output to the caching and exported by spare buffer area.
  5. 5. according to the method described in claim 4, it is characterized in that, the spare caching is used to store the caching input or deposits Store up the reception data that master cache receives, and with the identical order for receiving data in the master cache area by the reception Data are output to the caching output.
  6. 6. according to the method described in claim 4, it is characterized in that, the cache controller specifically includes：
    The effect of the cache controller is that the master cache is from caching input to institute when the master cache is empty data mode State spare caching transmission data；
    Or；
    When the spare caching is full data mode, the spare caching transmits data from caching input to the master cache；
    Or；
    When the master cache data mode is not empty, the reception data transmit data from caching input to the master cache.
  7. 7. according to the method described in claim 4, it is characterized in that, wherein described master cache and spare caching are can to store not The independent fifo queue of same type data and the data space of master cache are more than the data space of spare caching.
  8. 8. a kind of pretreatment system of buffer area data, it is characterised in that specifically include：
    Transmission device：Send the data to buffer area；
    Buffer area：For receiving data from transmission device, and the data of reception are sent to reception according to the order of first in, first out Device；
    Reception device：For receiving the data come from buffer area；
    Wherein, the system is trained data and has been handled first, first will by transmitting device since data volume is larger It is put into buffer area, specifically includes：Basic data refers to user's query time TimeUserQuery, user's residence time TimeUserStand and user inquire about content ContentUserQuery；Construct TimeUserQuery, TimeUserStand and ContentUserQuery interface functions obtain the query time of client user, residence time and inquiry from initial server end Content；The preset timer Timer in TimeUserQuery the and TimeUserStand functions, and use cookie controls Technology, obtains the query time of user and residence time in current behavior；The data being collected into are asynchronous by GET, POST Mode is sent to destination server end；The basic data is shown to the destination server end by interface with JSON forms；
    After destination server receives return value and returned content, system uses the Parse methods of JObject or JArray will JSON character strings are converted to JSON objects, extract the basic data by way of the JSON objects, analyze the basis Association between data query content and query time, that is, construct Loading, Unloading, Cargo, Carrier, Route With the graph of a relation of TimeUserQuery and TimeUserStand；
    In the graph of a relation, TimeUserQuery and TimeUserStand respectively as dependent variable and Loading, Unloading, Cargo, Carrier, Route, using least square method, establish LEAST SQUARES MODELS FITTING simulation as independent variable User behavior, predicts the user job time and inquires about the data relationship between content；
    The system is by least square method modelling customer behavior, including predicts user query time TimeUserQuery, specifically Mode is：
    <mrow> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>=</mo> <msub> <mi>x</mi> <mn>0</mn> </msub> <mo>+</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <msub> <mi>t</mi> <mi>i</mi> </msub> <msub> <mi>x</mi> <mi>i</mi> </msub> </mrow>
    Wherein tiRepresent query time when user inquires about i-th inquiry content；N is the number of inquiry content；xiRepresent the The i corresponding model parameters of the inquiry content, wherein x0=1；If user only carries out Cargo operations, when Cargo is inquired about Between can be predicted into：
    y3=x0+t3x3；
    One is set respectively in tables of data for described inquiry content Loading, Unloading, Cargo, Carrier, Route SessionId, directly related parameter values are obtained in above-mentioned LEAST SQUARES MODELS FITTING by the SessionId, and will be obtained Initial data of the data as buffer area input data.
  9. 9. system according to claim 8, it is characterised in that specifically include：
    Buffer area includes master cache and spare caching, and the master cache area is configured to be mainly used for what storage was received from caching input Data；The spare caching is mainly used for the reception data for storing the caching input or storage master cache receives, and And with the master cache it is identical receive data order by it is described reception data be output to it is described caching export.
  10. 10. system according to claim 9, it is characterised in that specifically include：
    The buffer area further includes cache controller, and when the master cache is empty data mode, the master cache is defeated from caching Enter to the spare caching transmission data, or when the spare caching is full data mode, the spare caching is from caching Input to the master cache and transmit data, or when the master cache data mode is not empty, the data that receive are from caching Input to the master cache and transmit data.
  11. 11. system according to claim 9, it is characterised in that specifically include：
    The multiple buffer areas of the system configuration, the data space of the master cache are more than the space of spare caching.
CN201510412138.7A 2015-07-14 2015-07-14 The preprocess method and system of buffer area data Active CN105022699B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510412138.7A CN105022699B (en) 2015-07-14 2015-07-14 The preprocess method and system of buffer area data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510412138.7A CN105022699B (en) 2015-07-14 2015-07-14 The preprocess method and system of buffer area data

Publications (2)

Publication Number Publication Date
CN105022699A CN105022699A (en) 2015-11-04
CN105022699B true CN105022699B (en) 2018-04-24

Family

ID=54412687

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510412138.7A Active CN105022699B (en) 2015-07-14 2015-07-14 The preprocess method and system of buffer area data

Country Status (1)

Country Link
CN (1) CN105022699B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107784364B (en) * 2016-08-25 2021-06-15 微软技术许可有限责任公司 Asynchronous training of machine learning models

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101369276A (en) * 2008-09-28 2009-02-18 杭州电子科技大学 Evidence obtaining method for Web browser caching data
WO2011019759A2 (en) * 2009-08-10 2011-02-17 Visa U.S.A. Inc. Systems and methods for targeting offers
CN103546394A (en) * 2013-10-25 2014-01-29 杭州华三通信技术有限公司 Communication device
CN103714130A (en) * 2013-12-12 2014-04-09 深圳先进技术研究院 Video recommendation system and method thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101369276A (en) * 2008-09-28 2009-02-18 杭州电子科技大学 Evidence obtaining method for Web browser caching data
WO2011019759A2 (en) * 2009-08-10 2011-02-17 Visa U.S.A. Inc. Systems and methods for targeting offers
CN103546394A (en) * 2013-10-25 2014-01-29 杭州华三通信技术有限公司 Communication device
CN103714130A (en) * 2013-12-12 2014-04-09 深圳先进技术研究院 Video recommendation system and method thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
个性化服务中基于用户行为分析的用户兴趣建模;付关友 等;《计算机工程与科学》;20051231;第27卷(第12期);第77页第5-8段、第12段 *

Also Published As

Publication number Publication date
CN105022699A (en) 2015-11-04

Similar Documents

Publication Publication Date Title
WO2021120775A1 (en) Method and device for detecting data abnormality
Zheng et al. Traffic state estimation using stochastic Lagrangian dynamics
JP5570008B2 (en) Kernel regression system, method and program
CN112639768A (en) Storage aggregator controller with metadata computation control
Woo et al. Real-time optimal power flow using twin delayed deep deterministic policy gradient algorithm
CN105590216B (en) Method and system for monitoring transaction risk in real time
CN113537850A (en) Storage optimization method and device, computer equipment and storage medium
CN109063935A (en) A kind of method, apparatus and storage medium of prediction task processing time
Sangal et al. A fuzzy environment inventory model with partial backlogging under learning effect
CN107271127A (en) Based on the operational modal parameter recognition methods and device extracted from iteration pivot
CN109241084A (en) Querying method, terminal device and the medium of data
CN105022699B (en) The preprocess method and system of buffer area data
CN106022631A (en) Index weight analysis method
KR20180058194A (en) Method for processing logistics information, logistics information processing server, and logistics managing serve using the method
CN110009155A (en) Estimation method, device and the electronic equipment of service area dispatching difficulty
CN109961328A (en) The method and apparatus for determining order cooling off period
CN107741834A (en) A kind of printer control method and terminal device
CN109544062A (en) A kind of Fast Packet vanning information processing system and method suitable for Air Logistics
Wang et al. A radio frequency identification-based quality evaluation system design for the wine industry
EP2899676A1 (en) Remaining useful life forecasting system
Nechifor et al. Autonomic monitoring approach based on cep and ml for logistic of sensitive goods
CN110400106A (en) Information acquisition method, device and electronic equipment
US20180211300A1 (en) System and method for assessing wait times in a facility
Ling et al. Frontiers in time series and financial econometrics: An overview
CN109685527A (en) Detect method, apparatus, system and the computer storage medium of trade company&#39;s wash sale

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
CB03 Change of inventor or designer information

Inventor after: Shi Wenjin

Inventor after: Hu Fanghuai

Inventor after: Yan Jiuji

Inventor after: Wu Qing

Inventor after: Wang Fei

Inventor before: Shi Wenjin

Inventor before: Yan Jiuji

Inventor before: Wu Qing

Inventor before: Wang Fei

CB03 Change of inventor or designer information
GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: Preprocessing methods and systems for cache data

Effective date of registration: 20230920

Granted publication date: 20180424

Pledgee: China Construction Bank Corporation Zhenjiang Runzhou Sub branch

Pledgor: ZHENJIANG HUILONG YANGTZE RIVER PORT CO.,LTD.

Registration number: Y2023980057474

PE01 Entry into force of the registration of the contract for pledge of patent right