CN1242848A - Multi-kernel neural network concurrent learning, monitoring and forecasting system - Google Patents
Multi-kernel neural network concurrent learning, monitoring and forecasting system Download PDFInfo
- Publication number
- CN1242848A CN1242848A CN 97181204 CN97181204A CN1242848A CN 1242848 A CN1242848 A CN 1242848A CN 97181204 CN97181204 CN 97181204 CN 97181204 A CN97181204 A CN 97181204A CN 1242848 A CN1242848 A CN 1242848A
- Authority
- CN
- China
- Prior art keywords
- value
- input
- weights
- feature vector
- output
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
A multi-kernel neural network computing architecture configured to learn correlations among feature values (34, 38) as the network monitors and imputes measured input values (30) and also predicts future output values (46). This computing architecture, referred to as a concurrent-learning information processor (CIP 10), includes a multi-kernel neural network array (14) with the capability to learn and predict in real time. The CIP (10) also includes a manager (16) and an input-output transducer (12) that may be used for input-output refinement. These components allow the computing capacity of the multi-kernel array (14) to be reassigned in response to measured performance or other factors. The output feature values (46) computed by the multi-kernel array (14) and processed by an output processor (44) of the transducer (12) are supplied to a response unit (18) that may be configured to perform a variety of monitoring, forecasting, and control operations in response to the computed output values. Important characteristics of the CIP (10), such as feature function specifications (35 and 49), connection specifications (42), learning weight schedules (55), and the like may be set by a technician through a graphical user interface (20). Refinement processes also allow the CIP (10) to be reconfigured in accordance with user commands for application to different physical applications.
Description
The present invention relates to the neural network information handling system, and be particularly related to multi-kernel neural network counting system structure, be disposed for when input value that network monitoring and estimation (impute) record and also predict the correlativity between the learning characteristic value in the following output valve.
Computer technology be in current flourish in, much concentrate on the order information disposal system, as the various systems in the wide field from the handheld personal computer to the mainframe computer.Say that generally most " flat file " order information disposal systems can be effectively when carrying out known in advance input, output and operation task.But they not too are fit to carry out input, output and the environmental factor of operation response change, the physical characteristics of variation etc. and time dependent adaptive task.In other words, typical " flat file " order information disposal system not too is fit to carry out the task of relating to study.
Neural network is a kind of computer technology that can be used for realizing learning system.Particularly the architecture of neural network computer is developed in order to simulate the information process that takes place in the thinking body.The realization of nerual network technique often utilizes special-purpose hardware processor, as the parallel processing logic array.Say that generally neural network is an interconnecting nodes system with input and output, the output of one of them given node is that the weighted sum by the input of this node drives.Neural network is well suited for input and output value and can uses in the corresponding supervision of physical parameter, forecast and control that a series of time duration of test are measured.More same values are monitored and forecast that the relation of allowing between the input and output value can obtain study by the input and output value that is applied to measure being carried out empirical analysis.Afterwards can be with the relational application of study in output valve from the input value prediction of measuring.
For a typical nerve network system is applied to physical application, neural network disposes the input and output that are suitable for given application.Neural network is in case formation just makes it be exposed to a series of times tests that comprise both measured values of input and output in the training stage.By empirical analysis in the training stage, the relation between the input and output value that e-learning is measured.After network is through training, just can be in forecast period calculate the output of predicting from the input of measuring at time duration of test thereafter.In other words, at forecast period, the output of predicting is calculated in the network based input that utilization is measured that concerns of learning in the training stage.In forecast was used, network is general to be received and the corresponding measured value of output valve at following time duration of test.Afterwards the output valve of these measurements and the output valve of prediction are compared to measure the performance or the precision of prediction of network.
Neural network also can often receive retraining, and the result forms training-predicted operation cycle.Though this common neural network input/output relation of Applied Learning is effectively carried out forecast analysis, network required a clear and definite training stage before carrying out forecast analysis.In other words, this network can not be learnt relation at forecast period.Because this same sign, network can not carry out forecast analysis in the training stage.This drawbacks limit common neural network purposes in some cases.
Particularly, common neural network can not be learnt simultaneously and predict that having limited these networks should investigate thoroughly the relation between the input and output as early as possible at those but be not appreciated that validity in the application how long study these relations need test.In this case, how long be difficult to this network of training of judgement need test.Similarly, common neural network not too is fit to the application that the relation between wherein the input and output can change with a kind of the unknown or uncertain mode.In this case, be difficult to judge when network is carried out retraining.
As a result, the common relation of neural network between the input and output that are applied to wherein must be investigated thoroughly and these relations wherein are subjected to serious restriction when being the supervision that changes with a kind of the unknown or uncertain mode, forecast and control task as early as possible.Certainly, much supervision, forecast and control task belong to these categories.Such as, the pass between the input and output that should investigate thoroughly is very soon fastened and is stood to change rapidly as guided missile controller and these machines of packet router.Essence may take place and lost efficacy in some other relating to, and is destroyed or the guided missile break-in is out of hand as structural member, the supervision and the control operation of machine often show the input and output relation that changes with a kind of the unknown or uncertain mode.
The response in time of common neural network input and output relation therein measure and be under many circumstances immesurable changing factor and in the application that changes its validity also be restricted.Such as, can expect that a kind of price index of commodity can respond as changing factors such as inventory level, demand for commodity, money supply, trader's psychology in uncertain mode in time and changes.Similarly, can expect that relation between power demands and the weather can respond as changing factors such as consensus data, heating and Refrigeration Technique, economic conditions in uncertain mode and changes.
The physical configuration of the network that has its source in of the another kind restriction that common neural network is run into normally customizes for one group of specific input and output.Though network is learnt the relation between these input and output easily, network is not that the performance for response measurement redefines its input and output and disposes.This is because the employed input/output relation weights of network can change when network stands retraining, but input and output remain unchanged.Do not have effective input and output development, invalid or redundant input and output can not be discerned and remove to network.As a result, network can not adapt to the situation of variation or improve prediction to a certain application continuously.
These two shortcomings (can not learn simultaneously and predict and lack effective input and output and improve (refinement) process) of interrelating with above-mentioned common neural network shortcoming of but being even in the most original thinking body, all obviously having overcome ironically.Really, predict simultaneously and the ability learnt is an importance of the awake or cognitive state in the thinking body.And can respond task repetition and on the amount of being distributed in ever-increasing input and output processing power be an importance of the study in the thinking body.Can be described as practice produces perfect.As a result, the common neural network that lacks this attribute just is subjected to serious restriction when the intelligent behavior of simulation thinking body.
So, generally need supervision, forecast and the control system that can learn simultaneously and predict technically now.In addition, now also need to comprise supervision, forecast and the control technology of effective input and output development technically.
The present invention is used for satisfying the above-mentioned needs in input value that is configured to be used for record in network monitoring and estimation and the multi-kernel neural network counting system structure of also forecasting the correlativity between the learning characteristic value in the following output valve.This counting system structure is called concurrent learning information processor (CIP), comprises a multi-kernel neural network array with study simultaneously and predictive ability.This concurrent learning information processor CIP also comprises a manager and one and can be used for the improved input/output converter of input and output.These parts are allowed the performance of response measurement or the computing power that other factors are given many kernels array again.Offer a response unit that can be configured to carry out various supervision, forecast and control operation by many kernels array computation and the eigenwert handled by the output processor of converter for the output valve of RESPONSE CALCULATION.The key property of this concurrent learning information processor can be set by graphical user interface by the technician as fundamental function standard (specification), connection standard, study weights figure etc.
Many kernels array is learnt simultaneously " in real time " and is predicted, checks each time test and carry out prediction study and circulate in each of array." in real time " of concurrent learning information processor learnt and the major progress of predictive ability representative aspect nerual network technique simultaneously.Manager is often represented another great progress to the improved ability of the input/output relation of many kernels array with input/output converter.Many kernels array can be organized and become various subarrays and make it adapt to various physical application to work out concurrent learning information processor.Many kernels array comprises at least one usually and monitors subarray and at least one forecast subarray.Monitor the eigenwert of the input and output of subarray estimation current time test.The eigenwert of these estimations is used for calculating the deviate of current time test and when needed the input feature vector value of estimation is provided for losing or exceeding the measurement input value of allowable error.The forecast subarray is to following time test forecast output characteristic value.
The operation cycle of each the time test in comprising the many kernels array that monitors subarray and forecast subarray all is estimation prediction (monitoring in the subarray and predicting), forecast study (learning in the forecast subarray), forecast prediction (predicting in the forecast subarray) and estimation study (monitoring in the subarray and learning).This sequence is allowed monitor that subarray carries out its estimation function and allowing the study from the current time test before prediction of forecast subarray thereafter the current time test before study.Study is deferred to and forecasts that subarray finishes its circulation afterwards to quicken the acquisition of prediction output valve in monitoring subarray.
Monitor and forecast in endorse to assemble to subarray and make it adapt to various physical application to customize concurrent learning information processor.Such as, the tissue of many kernels array can be corresponding to the spatial configuration of input and output, the time configuration of input and output or the room and time configuration of input and output.As special case, a customizable spatial configuration makes it adapt to a Flame Image Process and uses, and customizable time configuration makes its application that adapts to a kind of commodity price forecast, or customizable room and time combining and configuring is used its adaptation energy requirement forecast.
Say that generally the present invention is a kind of method that the calculating output valve according to the measurement input value that receives at the current time duration of test with at one or more historical time duration of test is responded of being used for.Measure input value and receive, and input feature vector value vector is according to the input value combination of measuring for the current time test.The input feature vector value offers multi-core processor.Each kernel of processor all can be used for receiving one or more input feature vector values and utilizes the input feature vector value to carry out a large amount of arithmetic operations.
Particularly, each kernel retrieval (retrieve) goes out to be used for to define the connection standard of mathematical relation, to calculate one or more output characteristic values according to the input feature vector value that receives.Kernel also retrieves one group of connection weights of representing the regression coefficient between input feature vector value that receives and the output characteristic value that calculates.Kernel also retrieves one group of study weights that are used for defining mathematical relation, connects weights to upgrade according to the input feature vector value that receives.Afterwards kernel according to the input feature vector value that receives, connect weights and be connected standard calculating output characteristic value.
Kernel also calculates the weights that are connected that upgrade according to the input feature vector value that receives, connection weights, connection standard and study weights.The element that connects the contrary covariance matrix of weights definable, and the step of the connection weights of calculating renewal can comprise the contrary covariance matrix of renewal.Other method is that the step of the connection weights that calculate to upgrade can comprise renewal corresponding to the covariance matrix of contrary covariance matrix and thereafter the covariance matrix that upgrades is being inverted.
Output characteristic value comprises the estimation output characteristic value that is used for the current time test usually and is used for the prediction output characteristic value of following time test.Each kernel all provides the visit to its output characteristic value.This allows that calculating the output valve vector makes up according to the output characteristic value that is calculated by each kernel.The vector of the output valve that goes out of RESPONSE CALCULATION and carry out an arithmetic operation afterwards is such as deviation arithmetic operation or control arithmetic operation.
According to an aspect of the present invention, input feature vector value vector can calculate according to the characteristics specify of input value of measuring and input.Such as, input feature vector value can according to the algebraic combination of the input value of measuring, with by the corresponding coefficient of approximation polynomial of the defined function of measuring of input value, with corresponding to by the pairing coefficient of the differential equation of the function of the input value definition of measuring or with corresponding to the corresponding coefficient of frequency domain by the function of the input value definition of measuring.Similarly, output valve can be calculated according to output characteristic value and output characteristic standard.Particularly, output valve can calculate by the mathematics inverse operation of carrying out the mathematical operation that corresponding measurement input value is carried out.
According to another aspect of the present invention, output characteristic value can comprise the estimation output characteristic value according to the input feature vector value that is used for one or more historical time tests.Output characteristic value also can comprise the output characteristic value according to the supervision of the input feature vector value that is used for the current time test.Deviate can be calculated by deduct the estimation output characteristic value from the output characteristic value that monitors.This deviate and threshold value can be compared afterwards and judge alarm condition or executivecontrol function.Output valve is normally according to the output characteristic value that monitors.
As there being one to surpass its dependent thresholds in the deviate, can carry out a biased operation, maybe the basis of the output valve calculated is determined on the estimation output characteristic value but not on the supervision output characteristic value of the output characteristic value that interrelates with the deviate that surpasses its dependent thresholds as the indication alarm condition.But also RESPONSE CALCULATION output valve and executivecontrol function compensates by the situation of calculating the output valve indication as showing the expression of the output valve of calculating or start a controlled variable on display device.
According to a further aspect of the invention, can carry out the improvement operation.These improve to operate to comprise deletes the invalid eigenwert that inputs or outputs, merge the redundant eigenwert that inputs or outputs, specify newly to input or output eigenwert, according to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the eigenwert standard, according to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the study weights, recomputate with the output valve that calculates according to measurement input value and to be connected standard the test of a plurality of times, according to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the output characteristic standard and redistribute functional between kernel.
Multi-core processor can comprise one or more supervision subarrays and one or more forecast subarray.For one comprise monitor and the configuration of forecast kernel at first according to the input feature vector value of one or more historical times tests, is connected weights and connects the estimation output characteristic value that standard calculates the supervision subarray.Calculate the renewal connection weights of forecast subarray afterwards according to input feature vector value, connection weights, connection standard, study weights and the estimation output characteristic value of the current time test that receives.Secondly, according to the input feature vector value of one or more historical time tests, the connection weights of renewal and the output characteristic value that the connection standard calculates the forecast subarray.Calculate the renewal of forecasting subarray according to the input feature vector value that receives, connection weights, connection standard and study weights afterwards and connect weights.
The configuration of customizable multi-core processor makes it adapt to specific physical application.Such as, multi-core processor can comprise several subarrays, and each subarray comprises one and monitors kernel and several forecast kernels.In the 1st kind of configuration, multi-core processor comprises one and corresponding space, continuum, space private core array, from wherein measuring input value and it being predicted output valve.In this configuration, each kernel all is configured as the measurement input value that is used for according to one group of adjacency and calculates one of output valve.Particularly, endorse corresponding in each of multi-core processor with a pixel of visual pattern, each measures input value can be corresponding with the measurement brightness of one of pixel of visual pattern, and each calculate output valve can be corresponding with the calculating brightness of one of pixel of visual pattern.
In the 2nd kind of configuration, multi-core processor comprises one and private core array of corresponding time of time-based index, from wherein measuring input value and it being predicted output valve.In this configuration, each kernel all is configured as and is used for predicting one of time-based mutual exclusion exponential quantity according to measuring input value.Such as, endorse in each of multi-core processor corresponding to specific forecast of mutual exclusion time, and each kernel can be configured to predict the price forecast that its corresponding mutual exclusion time is specific for being used for according to the input value of measuring for a kind of price index of commodity.In this occasion, the input value of measurement also comprises this commodity price index usually except the price index of currency and other commodity.
In the 3rd kind of configuration, multi-core processor comprise several each comprise core group with the corresponding a plurality of time private core of time-based index, from wherein measuring input value and to its prediction output valve.Each core group all comprise several each all be configured as and be used for predicting the independent kernel of a composition of time-based mutual exclusion exponential quantity according to measuring input value.These private core groups definition wherein each core group all is configured as the space private core group pattern of a composition that is used for calculating time-based index time.Such as, time-based index can comprise the electricity needs index, and each core group can be corresponding to several power provision points, and the input value of measuring can comprise electricity needs and short-range weather forecast.
The present invention also provides a kind of and comprises the iteration that is configured as the measurement input value that receives the current time test and according to the computer system of the input value combinatorial input eigenwert vector of measuring.This computer system also comprises one and is connected and is configured as the multi-core processor that is used for receiving input feature vector value vector with input processor.Each kernel of processor all can be used to receive one or more input feature vector values.
Kernel also can be used to be used to calculate according to the input feature vector value retrieval definition that receives the connection standard of the mathematical relation of one or more output characteristic values.Kernel also can be used for retrieving the connection weights of one group of regression coefficient between the output characteristic value of representing the input feature vector value that received and calculating.Kernel also can be used for retrieving the study weights of one group of definition mathematical relation to upgrade the connection weights according to the input feature vector value that receives.Kernel also can be used for according to the input feature vector value that receives, connect weights and connect standard and calculate output characteristic value.
Kernel also can be used for calculating according to the input feature vector value that receives, connect weights and connect standard study weights and upgrade the connection weights.Connect weights for upgrading, the renewable contrary covariance matrix of each kernel.Another way is the renewable covariance matrix of each kernel and afterwards the covariance matrix that upgrades is inverted.Kernel also can be used to the connection weights of storage update.Each kernel also all can be used to provide the visit to the output characteristic value that calculates.
This computer system also comprises an output processor that is connected to many kernels array and is configured as the output valve vector that is used for going out according to the output characteristic value combination calculation that calculated by each kernel.This computer system also comprises one and is connected to output processor and is configured as and be used for the response unit of the output valve vector that RESPONSE CALCULATION goes out.
This computer system also can comprise one and be configured as and be used for according to the measurement input value of a plurality of times test and calculate the manager that output valve recomputates the study weights.This manager also can be configured to recomputate the connection standard for being used for according to the measurement input value and the calculating output valve of a series of time tests.This manager also can be configured to being used for according to the measurement input value of a plurality of times test and calculating invalid the inputing or outputing eigenwert, merge and redundant input or output eigenwert, specify and new input or output eigenwert, recomputate and input or output characteristic criterion of output valve deletion, and gives functional between kernel again.
This computer system also can comprise one and be connected to many kernels array and can be used to and connect the parameter storage that weights, storage connect weights and the study that connects weights is provided for many kernels array from many kernels array received.The response unit of computer system can be used to show the expression of the output valve that calculates on display device, indication alarm condition and start-up control parameter are with the situation of compensation by the output valve indication that calculates.
About the present invention to the improvement of the shortcoming of common nerve network system and realize that above-mentioned advantage this point can be obtained understanding to embodiments of the invention description and accompanying drawing and claim by following.
Fig. 1 is the functional block diagram according to the concurrent learning information processor of one embodiment of the invention.
Fig. 2 illustrates the subarray structure of concurrent learning information processor.
Fig. 3 illustrates the analogy between the information process of concurrent learning information processor and thinking body.
Fig. 4 A illustrate core group be made into into the kernel of many kernels array of the corresponding concurrent learning information processor of the spatial configuration of input and output.
Fig. 4 B illustrate core group be made into into many kernels array of the corresponding concurrent learning information processor of the spatial configuration of input and output.
Fig. 5 illustrates core group and is made into to disposing many kernels array of corresponding concurrent learning information processor with the time of input and output.
Fig. 6 illustrate core group be made into into many kernels array of the corresponding concurrent learning information processor of room and time combining and configuring of input and output.
Fig. 7 A illustrates a typical study weights figure of concurrent learning information processor.
Fig. 7 B illustrates the study weights figure of the weights of nearer time test than the more positive concurrent learning information processor of the study weights among Fig. 7 A.
Fig. 7 C illustrates study is based on the concurrent learning information processor of some the time test that only occurs on round-robin basis a study weights figure.
Fig. 7 D illustrates the make amendment more positive study weights figure of study weights of the weights that make the test of nearer time to Fig. 7 C.
Fig. 8 A is illustrated in the typical input feature vector function of using in the converter of concurrent learning information processor.
Fig. 8 B illustrates and is configured as the concurrent learning information processor that is used for from electricity needs is measured and data of weather forecast calculating electricity needs is forecast.
Fig. 9 is the logical flow chart that the operation of concurrent learning information processor is shown.
Figure 10 is supervision, forecast that concurrent learning information processor is shown and the logical flow chart of learning routine.
Figure 11 is the logical flow chart that the input-output improvement routine of concurrent learning information processor is shown.
The present invention is one and can establishes by the specialized hardware computing equipment or by common sequential processes calculating The concurrent learning information processing device that the software of standby upper operation is realized. The specialized hardware embodiment is the suitableeest Close the application of the very fast processing speed of requirement, allow that processing speed is low and the implement software scheme is the most suitable Many application. Say that generally the cost that the implement software scheme realizes is much lower, because software can purchased Ready-made computer on move. On the other hand, the hardware implementation scheme requires special make special-purpose hard The part computing equipment. But, allow input and output because concurrent learning information processing device structure comprises Manager and converter that relation reconfigures are so a single hardware implementation scheme just can be joined Be set to carrying out the different task of broad range. Owing to this reason, hardware and software enforcement side Case both can be used as many purposes, and is general in a lot of schemes, processor.
The discrete parts that the implement software scheme preferably is configured as concurrent learning information processing device with The object-oriented architecture that discrete object is programmed. Each object comprises predefined Be used for defining with object and communicate interface with the agreement of swap data. By with discrete object Configuration software embodiment, each object can have corresponding discrete physical element or hardware classes seemingly is Element group in the system. Relation between this software and hardware embodiment is conducive to improve and test Module in the software, and in case be intact, then mass production hardware implementation scheme. But, Should understand, the present invention also realizes with the object-oriented architecture of other types and can adopt With the software engineering beyond the OOP.
Critical piece in the concurrent learning information processing device is input/output converter, many kernels god Through network array, manager, response unit and user interface. To concrete time test Processing sequence starts from converter and receives in the measurement input value. Converter comprises one and will measure Input value is the input of input feature vector value according to the input feature vector gauge transformation that is provided by manager The reason device. The connection standard that many kernels array provides according to the input feature vector value, by 1 manager and depositing Be stored in through the regression coefficient in the parameter storage of study and calculate output characteristic value. Output characteristic Value is returned to converter, wherein comprises a basis and will be failed by the output characteristic standard that manager provides Go out characteristic value and be transformed to the output processor that calculates output valve. Should calculate afterwards output valve transmits Carry out the response unit of various supervision, forecast and control operation to input value can be responded.
Check each time test in each of many kernels array and carry out predicted operation and study behaviour Do. This array comprises one or more be called the supervision kernel that monitors subarray and one usually Or a plurality of forecast kernels that forecast subarray that are called. Monitor subarray estimation input feature vector value and root The input feature vector value calculation deviation value of it is estimated. Estimation on this meaning means according to storage Historical data prediction input feature vector value in nearest tag memory. Afterwards according to the current time The measurement input value of test is calculated the difference conduct between estimation input feature vector value and the input feature vector value Deviate. This deviate and tolerance are compared to trigger biased operation, such as the indication warning feelings Condition or in calculating thereafter, utilize the estimation characteristic value but not measure characteristic value. In calculating thereafter The middle estimation characteristic value of utilizing is particularly useful when the measurement input value is lost or be destroyed.
The output characteristic value of forecast subarray predict future time test. Because the forecast subarray only Relevant with the future time test, it can be configured as in the output data to the future time test Learn from the input data of current time test before predicting. On the other hand, monitor son Array at first predicts by the estimated value of current time test, afterwards from the current time test The input data are learnt. Thus, the operation cycle of each time process of the test for estimation prediction ( Monitor in the subarray and predict), forecast study (at forecast subarray learning), forecast ( Predict in the forecast subarray) and estimation study (monitoring the subarray learning). This is in proper order fair Permitted to monitor that subarray carried out its forecast function to current time test before study, and allowed pre-Man who brings news of appointment's array is learnt from current test before carrying out its forecast function. To monitoring the study of subarray Be deferred to the forecast subarray and finish its circulation afterwards to accelerate the acquisition of output valve.
Concurrent learning information processing device is tested above the repetition a plurality of times under steady-state mode Operation cycle, until indicate operation improving. Usually repeat between the operation improving thousands of on Ten thousand operation cycle. During operation improving, manager can reconfigure concurrent aspect a lot Practise message handler, normally respond parameter and other instructions of receiving by user interface. Especially Be, the exportable descriptive statistic data of manager are deleted invalid characteristic value, merge the redundancy feature value, refer to Decide the new feature value, redefine many kernels array and connect standard, redefine the converter input processor The input feature vector standard, redefine the output characteristic standard of converter output processor, again fixed Justice is used for upgrading the study weights standard that connects weights, reconfigures the functional of converter and reaches again Give the computing capability of many kernels array. Operation improving makes concurrent learning information processing device obtain to ring The performance that should record, user indication and other factors and ability that itself is reconfigured.
Fig. 1 and below the purpose of discussion provide and a kind ofly can implement suitable meter of the present invention Calculate the general description of architecture. Description of the invention will be at the specialized hardware neural computer Or about functionally similar OO software program that the sequential processes computer moves Carry out in the literary composition. Description below in other words be applicable to the specialized hardware neutral net and similarly towards The software program of object both. Yet this professional personage will be appreciated that the present invention also can utilize The computing system of other kinds and software architecture are implemented. In addition, the present invention also can be logical in utilization Realize in the DCE that the teleprocessing device of news network connection is finished the work. DCE, such as the internet, among program module both can be positioned at the local memory storage device In, also can be arranged in remote memory storage device.
Below with reference to accompanying drawing embodiments of the invention are described.In this described, in these several accompanying drawings, same element adopted same label all along.
Fig. 1 is the functional block diagram of a concurrent learning information processor CIP 10.The critical piece of concurrent learning information processor 10 is input/output converter 12, multi-kernel neural network array 14, manager 16, response unit 18 and user interface 20.Many kernels array 14 comprises one or more supervision kernel and one or more forecast kernels that forecast subarray 24 that are called that monitor subarray 22 that are called.Monitor subarray 22 and forecast subarray 24 that each has relevant learning parameter storer 26a and 26b respectively.In fact, as shown in Figure 3, each kernel of each subarray preferably has relevant relevant learning parameter storer.
Concurrent learning information processor 10 all receives measurement input value 30 during each of a series of times test.A concrete time test processing order begins when converter 12 receives the iteration of measuring input value 30.The measurement input value that receives at each duration of test can think to form a n dimensional vector n of the two-dimensional array of a plurality of times tests time.Measure input value 30 and can represent value in any one of the multiple physical application that is configured from concurrent learning information processor 10.For example, measurement input value 30 can be represented the pixel intensity in the video image, the reading of strainmeter or groove position, the routing iinformation of packet, commodity price and other economic indexs, electricity needs and data of weather forecast or the like.
In more complicated occasion, input processor 32 on mathematics operational measure input value 30 to calculate input feature vector value 34.These calculating are to carry out according to the input feature vector standard 35 that is provided by manager 16.For example, the basis of input feature vector value 34 can be measure the algebraic combination of input value 30, with the approximate corresponding coefficient of polynomial expression by the function of measuring the input value definition, with corresponding to by the corresponding coefficient of the differential equation of the function of measuring the input value definition, with corresponding to by corresponding coefficient of frequency-domain function of the function of measuring the input value definition or the like.The input feature vector standard of other types can be clearly to those skilled in the art, is used for dissimilar special physical application because concurrent learning information processor is configurable.
The study regression coefficient that connects the covariance matrix that weights 40 expressions connect based on output characteristic value 38 that will be taken as dependent variable and the input feature vector value 34 that is taken as independent variable.In other words, many kernels array 14 calculates the weighted sum of input feature vector value 34 that each output characteristic value 38 is elements of covariance matrix as wherein connection weights 40.The study that connects weights 40 is by the input feature vector value 34 that is applied to the historical time test and the regretional analysis of output characteristic value 38.In order to be connected weights study carrying out simultaneously with the output characteristic value prediction, will connect weights 40 at each time duration of test and from learning parameter storer 26, retrieve, be used to predict that output characteristic value reaches at the new information update that retrieves at the time duration of test.The connection weights 40 that will upgrade are afterwards stored back the parameter storage of study.
More specifically say, each kernel of many kernels array 14 with contrary its corresponding each input feature vector value 34 that is applied to of covariance matrix to calculate its corresponding output characteristic value 38.In endorse by upgrade covariance matrix and after the covariance matrix of renewal is inverted carry out this operation.In this occasion, the connection weights 40 that are stored in learning parameter storer 26 are based on the inverting of element of covariance matrix.Yet, covariance matrix is inverted is the very big processing of calculating strength, preferably avoids.The way that replaces is that kernel can directly upgrade contrary covariance matrix, and the connection weights 40 that are stored in learning parameter storer 26 in this occasion are directly based on the element against covariance matrix.Because avoided the desired hardware of execution matrix inversion to connect, keep contrary covariance matrix but not covariance matrix can significantly improve the speed of kernel operations and simplify the physical configuration of the hardware embodiment of kernel.
Thus, each kernel of many kernels array 14 preferably utilizes contrary covariance matrix to calculate the study regression coefficient in the prediction-study circulation of each time test.This configuration of single kernel hardware embodiment be used to upgrade contrary covariance matrix, adopt contrary covariance matrix to calculate regression coefficient and support the mathematical derivation of this structure be described in common all U.S. Patent No.s _ _ _ in, its title is " Concurrent Learning And PerformanceInformation Processing System (concurrent study and an execution information handling system) ", the invention people is Robert J.Jannarone, date of application is on November 2nd, 1994, date of publication is _ _ _ _.This patent is unsettled U.S. Patent application now, and its application number is No.08/333,204, quote as a reference herein.
At each time duration of test, monitor the input and/or the output characteristic value 38 of this current time test of subarray 22 estimations.These estimation eigenwerts are used for calculating the deviate of current time test and the estimation eigenwert of losing or exceeding the eigenwert 38 of allowing being provided when needed.The output characteristic value of the following time test of forecast subarray 24 predictions.Because monitor the input and/or the output characteristic value 38 of subarray 22 estimation current time tests, if monitor subarray 22 before the input of its estimation current time test and/or output characteristic value 38 through study (promptly upgrading its corresponding weights that respectively connect), its purpose can not reach.Owing to this reason, supervision subarray 22 is operated the prediction study circulation of each time test.On the other hand, the output characteristic value 38 of 24 pairs of following time tests of forecast subarray is made a prediction.Therefore, forecast subarray 24 is operated the study-prediction loop of each time test, because it can be benefited from the study of current time test before prediction.
In addition, monitor preferably prediction before 24 study of forecast subarray of subarray 22.Estimation output characteristic value 38 is learnt although forecast subarray 24 preferably utilizes based on the output characteristic value 38 of effective measurement data of current time test, monitors that subarray 22 can detect to lose or ruined data.For corresponding to the losing or the input and/or the output characteristic value 38 of ruined data of current time test, forecast subarray 24 preferably utilizes by monitoring estimation input and/or the output characteristic value 38 that subarray 22 calculates rather than utilizing and learn based on the input and/or the output characteristic value 38 of invalid measurement data.Because estimation input and/or output characteristic value 38 are by monitoring that subarray 22 calculates, forecast subarray 24 is preferably learnt after monitoring subarray 22 (preferably) prediction.
For quickening the output characteristic value 24 of acquisition, can be deferred to the end of the study-prediction loop of forecast subarray to the prediction-study round-robin learning phase that monitors subarray 22 by 24 predictions of forecast subarray.Monitor that promptly subarray 22 is preferably forecasting that subarray 24 finishes to learn after its study-prediction loop to the current time test.The operation cycle of each time test that the result obtains is estimation prediction (monitoring the forecast period in the subarray 22), forecast study (learning phase in the forecast subarray 24), forecast prediction (prediction in the forecast subarray 24) and estimation study (monitoring the learning phase in the subarray 22).
Output characteristic value 38 turns back to the output processor 44 that output characteristic value is transformed to the converter 12 that calculates output valve 46 according to the output characteristic standard 49 that is provided by manager 16.Although output processor 44 can be carried out very diversified operation, the inverse operation of the mathematical operation normally carried out by input processor 32 of some operation at least.It is corresponding with some measurement input values 30 that this just allows that at least some calculate output valve 46.Predicted value and measured value compared make concurrent learning information processor 10 can measure the precision of its estimated performance.For example, in being configured as the concurrent learning information processor of prediction, measure electricity needs data and data of weather forecast that input value 30 can comprise instrumentation based on the electricity needs of data of weather forecast.In this occasion, calculate the electricity needs data that output valve 46 can comprise prediction.This just allows that the prediction electricity needs data that will the current time tester be calculated and the corresponding instrumentation electricity needs data that receive at following time duration of test compare.So just being configured as, concurrent learning information processor is used for measuring its electricity needs prediction precision.
To calculate output valve 46 afterwards is sent to and can responds the response unit 18 that output valve is carried out multiple supervision, forecast and control operation.For example, response unit 18 shows calculating output valve 46 usually on display device.Simple displaying is calculated output valve 46 may be enough for the further explanation of user before taking any concrete action according to output valve is suitable system.For being configured as the concurrent learning information processor that is used for forecasting commodity price index, such as, the trader can before making investment decision, not provide with concurrent learning information processor other for information about, as the amount of money that can be used for investing is together with reference to output valve.
In other cases, but response unit 18 RESPONSE CALCULATION output valves 46 and taking action automatically.Response unit 18 can automatically perform biased operation, indicates alarm condition such as RESPONSE CALCULATION output valve 46 exceeds predetermined permissible range.Concurrent learning information processor 10 can respond the situation that other exceed tolerance, as the measurement input value 30 and the input feature vector value 34 of instrumentation, and takes other biased operation automatically.Particularly, concurrent learning information processor 10 can respond the input value that exceeds tolerance and after calculating in adopt estimated value to replace based on invalid measurement data or obliterated data.
Concurrent learning information processor 10 repeats above-mentioned operation cycle to a plurality of time tests under steady-state mode, till indicating the improvement operation.Usually between the improvement operation, repeat operation cycle thousands of times.Improving operating period, manager 16 can reconfigure concurrent learning information processor 10 aspect a lot, and normally response is by parameter and other instructions of user interface 20 receptions.Be the help user, manager 16 can be the numerical value that previously selected parameter reception can be defined by the user with other information and by the utility routine that the option list that is used by user interface 20 drives to user interface 20 output descriptive statistic data.Manager 16 comprises one will be from the information of user interface 20 receptions and the integrated telegon 48 of other elements of manager 16.
The improvement operation of being carried out by manager 16 can be defined by the user or be calculated automatically by manager 16.Improving operation comprises the invalid eigenwert of deletion usually, merges the redundancy feature value and specifies the new feature value.These operate in to analyze in the connection controller 50 that connects weights 40 and carry out.Connection weights near zero indicate and connect controller 50 abolishable invalid eigenwerts.Can indicate near the connection weights of same value and to connect the redundancy feature value that controller 50 utilizes a single eigenwert with mean value of each redundancy value to replace usually.When the cancellation of invalid eigenwert and redundancy feature value merge, connect controller 50 make in the kernel array 14 the eigenwert capacity can with and converter 12 in the input and output capacity can use.Therefore connecting controller 50 can be configured to selecting new measurement input value, eigenwert and output valve for this release capacity (freed-up) automatically.
These new measurement input value, eigenwert and output valves are normally selected from the previously selected list that provides by user interface 20.Like this, concurrent learning information processor 10 just can be configured to can systematically estimating a large amount of input and output combination, eliminate to have hardly predictive value, and keeps and have that high predicted is worth.Therefore, through certain hour, concurrent learning information processor 10 just can the identification measurement input the most useful to given physical application from the candidate input of vastness.This is a kind ofly can obtain the input of a large amount of candidates but do not understand powerful improvement technology under the situation which input value has predictive value.For example, this improvement technology will be of great use when selecting the most useful statistical data of identification from a large amount of economic statistics and predict the price of a certain concrete commodity.In this case, concurrent learning information processor 10 can be identified in unconspicuous covariant relation directly perceived between economic statistics and this concrete commodity price.
Fig. 2 illustrates the typical sub-array structure 200 of concurrent learning information processor 10.This structure comprises a large amount of kernel of being represented to " N " 202n by kernel " A " 202a, and each all has the relevant learning parameter storer of being represented to " N " 204n by learning parameter storer " A " 204a.This subarray structure can provide in the hardware embodiment in same chip at each single kernel and relevant learning parameter memory bit thereof and connect the weights exchange.Because the software implementation scheme preferably is configured as and the similar OO software implementation scheme of relevant hardware embodiment, so software implementation scheme of the present invention preferably adopts subarray structure 200.Even so, also can use other subarray structures to other configurations.
Fig. 3 illustrates the analogy between the information process 300 of concurrent learning information processor 10 and thinking body.This analogy does not attempt to assert that the function of concurrent learning information processor 10 is accurately similar with the information process of thinking body.Not equal to being attempt, this analogy points out understanding more significant contrasts of structure possibility of concurrent learning information processor.With reference to figure 1 and 3, the measurement input value 30 of concurrent learning information processor 10 can be imported 302 analogies with the sensation of thinking body.The calculating output valve 46 of concurrent learning information processor 10 can respond 304 analogies with the cognition of thinking body.The nearest tag memory 36 of concurrent learning information processor 10 can with the short-term memory analogy of thinking body.The converter 12 of concurrent learning information processor 10 can with the sense process analogy of thinking body.Kernel array 14 can with the automated procedure analogy of thinking body.Learning parameter storer 26 can with the long-term memory analogy of thinking body.And manager 16 can with the brain process analogy of thinking body.
In this analogy, the stable state study-prediction loop of concurrent learning information processor 10 can with awake, the cognitive function analogy of thinking body.On the other hand, the improvement of concurrent learning information processor 10 operation can and between sleep period the function class ratio of thinking body.Similar with awake thinking body, concurrent learning information processor 10 is receiving and can learn simultaneously and predict importing when responding.And similar with sleep thinking body, the separate unit of concurrent learning information processor 10 response historical experiences is improved its information process periodically.Like this, similar with the thinking body, the life cycle of concurrent learning information processor 10 is to follow repetitive process that " sleep " improves cycle " awake " steady-state period afterwards.
Fig. 4 A illustrate core group be made into into the inner core 400 of the corresponding many kernels array of the spatial configuration of input and output.Such structure is used for estimating that to being configured as its monochrome information is lost or the image processing system of ruined pixel intensity can be of great use.Such as, in video image series, image processing system utilizes the valid data of the neighbor of a series of times test, is frame of video in this occasion, estimates and loses or the value of ruined pixel data.
Fig. 4 B illustrates inner core 400a-n tissue to be become and the corresponding many kernel processes array 408 of the spatial configuration of input and output.Each inner core 400a-n is configured as with similar to the described inner core of Fig. 4 A 400.So many kernels array 408 just comprises a grid with the corresponding inner core 400a-n of dot structure of beneath raster image.Should understand, for represent clear for the purpose of the half-pix of raster image just of the inner core 400 shown in Fig. 4 B.
Fig. 5 illustrates core group and is made into to disposing corresponding many kernel processes array 500 with the time of input and output.Such structure may be to the price of commodity price forecast system according to the historical price trend prediction commodity of commodity.Though the historical price trend of commodity is the unique measurement inputs shown in Fig. 5, also can comprise other input, as index, currency and other category informations of other commodity.
Many kernel processes array 500 comprises supervision kernel 502 and several forecast kernels 504a-n, and wherein each all is to be specifically designed to the commodity price index 506 that calculates the following time point of mutual exclusion.Such as during current time test " t ", monitor the commodity price index 506 of kernel 502 estimation current time tests, the commodity price index of the kernel 504a following time test of prediction " t+1 ", kernel 504b are predicted the commodity price index of another following time test " t+2 ", and the rest may be inferred.Measured value illustrates with hollow dots, and predicted value illustrates with solid dot.
Monitor that kernel 502 also calculates the tolerance 510a-b of the estimated value 512 of current time test.Normally one or two standard deviation of calculating and estimated value 512 is as tolerance 510a-b.Monitor that kernel 502 also calculates the estimated value 512 of commodity price index of current time test and the difference 516 between the measured value 514 as deviate.Deviate 516 can be used to judge that when measured value 514 most probables are lost or be destroyed.Lose or ruined occasion at measured value 514, can be in calculating thereafter, as the calculating of being undertaken by forecast kernel 504a-n at the current time duration of test, middle use estimated value 512.Estimated value 512 also can be by monitoring that kernel 502 and forecast kernel 504a-n use in time test thereafter.
Fig. 6 illustrates core group and is made into to combine the corresponding many kernel processes array 600 of configuration with the room and time of input and output.Such structure can be of great use to being configured as the electric power forecast system that is used for according to current power desired level, current weather data and data of weather forecast prediction electricity needs.Multinuclear is handled array 600 and is comprised one group of subarray 602a-n, and wherein each subarray all comprises a supervision kernel and several forecast kernels.Like this, each subarray is structurally all with similar with reference to the described many kernel processes array of figure 5 500.Each subarray 602a-n is according to generating at the geographic area at the electricity needs data of relevant geographic area instrumentation, the weather data and the data of weather forecast of instrumentation, as zone by common distributing electric power station or supply centre service, the electricity needs forecast.
In addition, subarray 602a-n organizes the spatial configuration that becomes similar with the kernel array of describing with reference to figure 4A-B 408.Yet, multinuclear is handled array 600, each subarray covers specific geographic area.Therefore this structure can be used to construct the electricity needs forecast system in whole electrical power services zone, gives a subarray to each distributing electric power substation or supply centre.But so output addition of subarray 602a-n and generate the overall electricity needs forecast 604 of whole service area.Should understand, multinuclear is handled array 600 not necessarily will be applied to the geographical continuous service area of, and also can be applied to comprise on a large amount of geography independently service provision point, as single city, single discrete commercial or industrial point, a plurality of island or the like.
Fig. 7 A-D illustrates the typical study weights figure of the connection weights 40 that are used for upgrading concurrent learning information processor 10.A shared study weights figure can be applicable to all connection weights, and a single study weights figure can be applicable to each single connection weights, perhaps the connection weights can be organized into to be a plurality of groups, and each group has a shared study weights figure.
Study weights Figure 70 2 shown in Fig. 7 A is positive weights figure, because it gives bigger weights to the measurement that is received at nearer time duration of test.Study weights Figure 70 4 shown in Fig. 7 B is more positive than study weights Figure 70 2 because study weights Figure 70 4 it the measurement that is received at nearer time duration of test given also want big weights.When calculate output valve study weights figure represented a plurality of times, the duration of test tendency changed quite rapidly the time, positive study weights figure is suitable.
Study is only according to some the time test that takes place on round-robin basis among study weights Figure 70 6 shown in Fig. 7 C.These type study weights figure may be suitable to corresponding peak load electric power forecast system during test of weighting time and the peak load.It is similar with Figure 70 6 that Fig. 7 D illustrates study weights Figure 70 8, makes the weights of nearer time test more positive except having made to change.The study weights figure of other types will be conspicuous to them when those skilled in the art design concurrent learning information processor for various physical application in the future.
Fig. 8 A is illustrated in the typical input feature vector function of using in the converter 12 of concurrent learning information processor 10.The input feature vector value is corresponding with the coefficient of approximate polynomial expression 804 by the defined function of measuring of input value in this input feature vector function.For example, power demand signal can be to measure input value 806, and with polynomial expression 804 corresponding coefficients can be the input feature vector value of calculating by converter 12 808.The input feature vector function of this type relies on the measurement input value through the time test of considerable quantity to make up the polynomial expression 804 with appropriate length.Therefore, the historical record of measurement input value is stored in the nearest tag memory 36 shown in Figure 1.
The input feature vector function shown in Fig. 8 A to definition can be reliably by a series of measurement input values of the repeat pattern of polynomial approximation, as the power demand signal of typical geographic area, be suitable.The measurement input value of other types can form repeat pattern in other domain of function.Owing to this reason, can be by the input feature vector value that converter 12 calculates corresponding to the concrete coefficient of measuring the function of input value that is fit to of the differential equation, frequency-domain function or other types.
Fig. 8 B illustrates and is configured as the concurrent learning information processor 820 that is used for from electricity needs is measured and data of weather forecast (the current weather data and the short-range weather forecast data that preferably comprise measurement) calculating electricity needs is forecast.So concurrent learning information processor 820 can generate electricity needs forecast 822 according to the input value 824 that comprises instrumentation electricity needs data, instrumentation weather data and data of weather forecast.Can believe now with reference to figure 6 described many kernels array structures and forecast concurrent learning information processor 820 with reference to figure 7A-D described study weights figure and with reference to the described input feature vector function 803 suitable electricity needs of figure 8A.
Fig. 9-11 is the logical flow chart that the operation of concurrent learning information processor 10 is shown.To the description of these figure also with reference to the functional block diagram that is shown in the concurrent learning information processor 10 of Fig. 1.Fig. 9 is the logical flow chart that the typical operation routine 900 of concurrent learning information processor 10 is shown.Routine 900 judges whether to have received the termination order in step 902 beginning at the concurrent learning information processor 10 of this step, such as passing through to check " end-of-file " record.Though concurrent learning information processor 10 begins check to stop order study-prediction loop, this inquiry can similarly come across the end of study-prediction loop or in other positions of routine 900.For example, concurrent learning information processor 10 can be checked and can make processing in steady state process after period and the termination order that stopped before concurrent learning information processor 10 improves operation.Stop order as receiving, then turn to " end " step through the shunt "Yes", and routine 900 stops from step 902.
As receive not stopping order, then turn to step 904 through the shunt "No" from step 902, concurrent learning information processor 10 is received in the measurement input value 30 of current time test of the input processor 32 of converter 12 in this step.Turn to routine 906 after the step 904, concurrent learning information processor 10 is carried out concurrent supervision, forecast and the learning manipulation of short-term time test in this routine 906.Below with reference to Figure 10 routine 906 is described.
Execution in step 908 in routine 906, and concurrent therein learning information processor 10 judges whether to carry out biased operation.Should carry out biased operation as concurrent learning information processor 10 judgements, just turn to step 910 from step 908 through the shunt "Yes", concurrent learning information processor 10 is carried out biased operation in this step.These biased operation comprise the indication alarm condition usually and/or utilize estimation or predicted value but not measured value in calculating thereafter.Should understand, biased operation (such as utilizing estimated value but not according in the calculating thereafter according to the eigenwert of measuring input value) also can come across after the estimating stage that monitors in the subarray 22 and before the learning phase in the forecast subarray 24.
Should not carry out biased operation as concurrent learning information processor 10 in step 908 judgement, then turn to step 912 through the shunt "No", concurrent learning information processor 10 judges whether that this carries out control operation in this step, normally by response unit 18.Step 912 also can be followed step 910.Should carry out control operation as concurrent learning information processor 10 judgements, just turn to step 914 from step 912, concurrent learning information processor 10 executivecontrol functions in this step through the shunt "Yes".These control operations comprise usually and show prediction output valve 46, and may also comprise and take from the dynamic response action, as Open valve, change the indication of packet route, starting switch or the like.
Should not carry out control operation as concurrent learning information processor 10 in step 912 judgement, just turn to step 916 along separate routes through "No", concurrent learning information processor 10 judges whether and improve operation by manager 16 in this step.Step 916 also can be followed step 914.Determine improve operation as concurrent learning information processor 10, then turn to step 918 from step 916, concurrent learning information processor executivecontrol function in this step through the shunt "Yes".Below with reference to Figure 11 routine 918 is described.After routine 918, routine 900 goes back to step 902, and routine 900 repeats under equilibrium mode another time test.
Figure 10 is the logical flow chart that supervision, forecast and study routine 906 in the concurrent learning information processor 10 are shown.Routine 906 then is shown in the step 904 of Fig. 9.In step 1002, concurrent learning information processor 10 reads in and checks the record of the measurement input value 30 that comprises the current time test.As the inspection indication protocol failure that carries out in step 1002, then routine 906 can be skipped this record, and Interrupt Process is indicated alarm condition or taked other predetermined actions.Step 1002 back is a step 1004, and the input processor 32 of converter 12 is according to the vector of the input feature vector standard 35 combinatorial input eigenwerts 34 of measuring input value 30 and being provided by fundamental function controller 52 in this step.Toply typical input feature vector standard had been described with reference to figure 8A.
Before the connection weights 40b that upgrades sent learning parameter storer 26b back to, connecting controller 50 can proofread and correct to avoid otherwise the digital issue that concurrent learning information processor 10 is stopped the connection weights 40b execution technique that upgrades.As previously mentioned, connecting controller 50 is to be configured as the connection weights that can discern the trend non-steady state and to limit its numerical value, normally by replace this calculating to connect weights with predetermined minimal value or maximum value.
Before the connection weights 40a that upgrades sent learning parameter storer 26a back to, connecting controller 50 can proofread and correct to avoid otherwise the digital issue that concurrent learning information processor 10 is stopped the connection weights 40a execution technique that upgrades.Step 1018 back is a step 1020, in this step the historical data in the nearest tag memory 36 is upgraded, normally according to first in first out.Other parameters can preserve in step 1020 or upgrade, and see how suitable how doing, so that concurrent learning information processor 10 is prepared for next time test.Step 1020 back is a step 1022, and it returns step 908 in Fig. 9.
Figure 11 is the logical flow chart that the input-output improvement routine 918 of concurrent learning information processor is shown.Routine 918 is after the step 916 that is shown in Fig. 9.Should understand, be not that the each in steps concurrent learning information processor 10 of institute of routine 918 is carried out when improving operation and all must be carried out.And not equal to be that routine 918 is lists of the executable when needed improvement operation of concurrent learning information processor 10.Whether concrete improvement operation needs and can be judged automatically by manager 16, or notifies concurrent learning information processor 10 by user's judgement and through user interface 20.Can handle with dirigibility in various degree by user interface 20 according to user's needs and improve operation.
In step 1102, the telegon 48 of manager 16 is to the descriptive statistical data of user interface 20 outputs.Step 1102 back is a step 1104, and telegon 48 receives user-defined instruction and other parameters of sending from user interface 20 in this step.Step 1104 back is a step 1106, connects controller 50 and eliminate invalid eigenwert in this step.Step 1106 back is a step 1108, connects controller 50 and merge redundant eigenwert in this step.Step 1108 back is a step 1110, connects controller 50 and specify new eigenwert in this step.
When invalid eigenwert is cancelled and redundancy feature value when being merged, connect controller 50 make in the kernel array 14 the eigenwert capacity can with and converter 12 in the input and output capacity can use.Therefore connecting controller 50 can be configured to being new measurement input value, eigenwert and the output valve of this release Capacity Selection automatically.These new measurement input value, eigenwert and output valves are normally selected from the previously selected list that provides by user interface 20.Like this, concurrent learning information processor 10 just can be configured to can systematically estimating a large amount of input and output combination, eliminate to have hardly predictive value, and keeps and have that high predicted is worth.Therefore, through certain hour, concurrent learning information processor 10 just can the identification measurement input the most useful to given physical application from the candidate input of vastness.
Therefore, step 1110 back is a step 1112, connects controller 50 and redefine connection standard 42 in this step.Step 1112 back is a step 1114, and fundamental function controller 52 redefines input feature vector standard 35 in this step.Fundamental function controller 52 is sent to new input feature vector standard 35 input processor 32 of converter 12.Step 1114 back is a step 1116, and fundamental function controller 52 redefines output characteristic standard 49 in this step.Fundamental function controller 52 is sent to new output characteristic standard 49 output processor 44 of converter 12.Step 1116 back is a step 1118, and study weights controller 54 redefines study weights Figure 55 in this step.
Consider above-mentioned diversified improvement operation, preferably manager 16 can reconfigure the functional of converter 12 very on a large scale and reconfigure the functional of many kernels array 14 very on a large scale.In other words, the manager 16 that all can make of above-mentioned improvement operation can fundamentally reconfigure concurrent learning information processor 10 at different physical application.This dirigibility is by step 1120 expression, and manager 16 reconfigures the functional of converter 12 in this step, and by step 1120, manager 16 is given the computing power of many kernels array 14 again in this step.Step 1122 back is a step 1124, returns the step 902 that is shown in Fig. 9 from this step.
The present invention just can provide a kind of supervision of multinuclear neural network, forecast and the control system that can learn simultaneously in real time and predict like this.This system comprises the input and output development efficiently of allowing that performance that system responses is measured and other factors reconfigure system itself.Development allows that also system reconfigures system according to user's order at different physical application.Should understand, foregoing only is the preferred embodiments of the present invention, and can carry out multiple change to embodiment described herein under the condition that does not break away from the spirit and scope of the present invention.
Claims (40)
1. method that is used to respond based on the calculating output valve of the measurement input value that receives at the current time duration of test with at one or more historical time duration of test, it step that comprises is:
(a) iteration of the measurement input value of reception current time test;
(b) according to measuring input value combinatorial input eigenwert vector;
(c) for multi-core processor provides input feature vector value vector, each kernel of processor is used for:
Receive one or more input feature vector values,
Retrieval definition is used for calculating the connection standard of the mathematical relation of one or more output characteristic values based on the input feature vector value that receives,
Retrieve one group and represent input feature vector value that receives and the connection weights that calculate the regression coefficient between the output characteristic value,
Retrieve one group and be used for defining the study weights that are used to upgrade based on the mathematical relation of the connection weights of the input feature vector value that receives,
According to the input feature vector value that receives, connect weights and be connected standard calculating output characteristic value,
According to the input feature vector value that receives, connection weights, connection standard and the calculating of study weights the weights that are connected through renewal,
The connection weights of storage through upgrading, and
Visit to the output characteristic value that calculates is provided;
(d) basis is by the vector of the output valve of the output characteristic value combination calculation of each kernel calculating;
(e) RESPONSE CALCULATION output valve vector; And
(f) judge whether that indication improves operation, and if indication improve operation, then be thereafter time to test repeating step (a) to (f).
2. method as claimed in claim 1, wherein:
The formation that connects weights comprises the element of contrary covariance matrix; And
The step of calculating the connection weights of process renewal comprises step of upgrading this contrary covariance matrix.
3. method as claimed in claim 1, wherein:
The formation that connects weights comprises the element of contrary covariance matrix; And
The step of calculating the connection weights of process renewal comprises:
Upgrade the covariance matrix of contrary covariance matrix correspondence therewith, and
This is inverted through the covariance matrix that upgrades.
4. method as claimed in claim 1, the step of wherein combinatorial input eigenwert vector comprise the step of calculating based on the input feature vector value of measuring input value and input feature vector standard.
5. method as claimed in claim 4, the step of wherein calculating the input feature vector value comprises from the mathematical operations as selecting next group:
Calculate the input feature vector value according to the algebraic combination of measuring input value;
According to the approximate corresponding coefficient calculations input feature vector of polynomial expression value by the function of measuring the input value definition;
According to corresponding to the corresponding coefficient calculations input feature vector of differential equation value by the function of measuring the input value definition; And
According to corresponding to the corresponding coefficient calculations input feature vector of frequency-domain function value by the function of measuring the input value definition.
6. method as claimed in claim 4, wherein the step of combination calculation output valve vector comprises the step of calculating based on the output valve of output characteristic value and output characteristic standard.
7. method as claimed in claim 5, wherein the step of combination calculation output valve vector comprises that execution is to the invert step of operation of the mathematics of one of mathematical operations of measuring input value and carrying out.
8. method as claimed in claim 6, wherein calculate the step that the step of output characteristic value comprises and be:
Estimate the output characteristic value of current time test according to the input feature vector value of one or more historical time tests;
Input feature vector value according to the current time test is calculated the output characteristic value that monitors;
According to the output characteristic value of estimation and the output characteristic value calculation deviation value of supervision; And
To calculate output valve based on the output characteristic value basis that monitors.
9. method as claimed in claim 1, the step of wherein calculating output characteristic value comprises the step of the output characteristic value of the following time test of prediction.
10. method as claimed in claim 1, wherein calculate the step that the step of output characteristic value comprises and be:
Estimate the output characteristic value of current time test according to the input feature vector value of one or more historical time tests;
And
The output characteristic value of the following time test of prediction.
11. method as claimed in claim 8, wherein the step that comprises of the step of RESPONSE CALCULATION output valve vector is:
Each deviate and a dependent thresholds are compared; And
Surpass its dependent thresholds as one in the deviate, then carry out one or more biased operation of from following one group, selecting,
The indication alarm condition, and
With the output valve calculated but not on the supervision output characteristic value basis of the output characteristic value relevant with the deviate that surpasses its dependent thresholds based on the estimation output valve.
12. method as claimed in claim 1, wherein the step of RESPONSE CALCULATION output valve vector comprises one or more control operations that execution is selected from following one group:
On display device, show the expression of calculating output valve, and
Starting a controlled variable compensates by the situation of calculating the output valve indication.
13. method as claimed in claim 6 if indication improves operation, then also comprises one or more improvement operations that execution is selected from following one group:
Delete the invalid eigenwert that inputs or outputs;
Merge the redundant eigenwert that inputs or outputs;
Specify the new eigenwert that inputs or outputs;
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the input feature vector standard;
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the study weights;
Recomputate with the output valve that calculates according to measurement input value and to be connected standard the test of a plurality of times;
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the output characteristic standard; And
Between kernel, redistribute functional.
14. method as claimed in claim 6 if indication improves operation, then also comprises the steps:
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the study weights;
Recomputate with the output valve that calculates according to measurement input value and to be connected standard the test of a plurality of times.
15. method as claimed in claim 1, wherein the formation of multi-core processor comprises:
One and corresponding space, continuum, space private core array are from wherein measuring input value and it being predicted output valve; And
Each kernel all is configured as one of output valve that is used for according to one group of adjacent measurement input value calculating.
16. as the method for claim 15, wherein:
Each kernel of multi-core processor is corresponding with a pixel of visual pattern;
The input value of each measurement is corresponding with the measurement brightness of one of pixel of visual pattern; And
The output valve of each calculating is corresponding with the calculating brightness of one of pixel of visual pattern.
17. method as claimed in claim 1, wherein the formation of multi-core processor comprises:
One and private core array of corresponding time of time-based index are from wherein measuring input value and it being predicted output valve; And
Each kernel all is configured as and is used for predicting one of time-based mutual exclusion exponential quantity according to measuring input value.
18. as the method for claim 15, wherein:
Each kernel of multi-core processor is corresponding to the specific price forecast of a kind of mutual exclusion time of price index of commodity;
Each kernel is configured as and is used for predicting the price forecast that its corresponding mutual exclusion time is specific according to the input value of measuring.
19. as the method for claim 18, the formation of wherein measuring input value comprises:
Commodity price index; And
Currency and other commodity price indexes.
20. method as claimed in claim 1, wherein the formation of multi-core processor comprises:
A plurality of core group, each all comprises the corresponding time private core of a plurality of and time-based index, from wherein measuring input value and it being predicted output valve;
Each core group all comprises a plurality of independent kernels, and each all is configured as and is used for predicting a composition of time-based mutual exclusion exponential quantity according to measuring input value;
A plurality of time private core groups of definition space private core group pattern;
Be configured as each core group of calculating a composition in the time-based index.
21. method as claimed in claim 1, wherein:
Time-based index comprises the electricity needs index;
Each core group is corresponding to several power provision points; And
The input value of measuring comprises electricity needs and weather data.
22. a method that is used to respond based on the calculating output valve of the measurement input value that receives at the current time duration of test with at one or more historical time duration of test, the step that its formation comprises is:
Receive the iteration of the measurement input value of current time test;
According to measuring input value combinatorial input eigenwert vector;
For comprising that the multi-core processor that monitors subarray and forecast subarray provides input feature vector value vector, each kernel of processor is used for:
Receive one or more input feature vector values,
Retrieval definition is used for calculating the connection standard of one or more mathematical relations based on the output characteristic value that receives the input feature vector value,
Retrieve one group of connection weights of representing the input feature vector value and calculating the regression coefficient between the output characteristic value,
Retrieve one group and define the study weights that are used to upgrade based on the mathematical relation of the connection weights that receive the input feature vector value,
Input feature vector value, connection weights and connection standard according to one or more historical time tests are estimated the output characteristic value that monitors subarray,
According to the reception input feature vector value of current time test, connect weights, connect the weights that are connected through upgrading of the output characteristic value CALCULATING PREDICTION subarray of standard, study weights and estimation,
According to the input feature vector value of one or more historical time tests, the connection weights and the output characteristic value that is connected standard CALCULATING PREDICTION subarray that process is upgraded,
Visit to output characteristic value is provided;
According to the input feature vector value that receives, connect weights, connect standard and the study weights calculate the weights that are connected through upgrading that monitor subarray, and
Storage monitors the weights that are connected through upgrading of subarray and forecast subarray,
According to the output characteristic value combination calculation output valve vector that calculates by each kernel;
RESPONSE CALCULATION output valve vector.
23. as the method for claim 22, wherein:
The connection weights of supervision subarray comprise the element of the contrary covariance matrix that monitors subarray; And
The step of calculating the connection weights of the process renewal that monitors subarray comprises upgrades the contrary covariance matrix that monitors subarray;
The connection weights of forecast subarray comprise the element of the contrary covariance matrix that forecasts subarray; And
The step of the connection weights that the process of CALCULATING PREDICTION subarray is upgraded comprises the step of the contrary covariance matrix that upgrades the forecast subarray.
24. as the method for claim 22, wherein:
The connection weights of supervision subarray comprise the element of the contrary covariance matrix that monitors subarray;
Calculating monitors that the step that the step of the connection weights that the process of subarray is upgraded comprises is:
Upgrade the covariance matrix of the supervision subarray corresponding with the contrary covariance matrix that monitors subarray, and
The covariance matrix through upgrading that monitors subarray is inverted;
The formation of the connection weights of forecast subarray comprises the element of the contrary covariance matrix that forecasts subarray; And
The step that the step of the connection weights that the process of CALCULATING PREDICTION subarray is upgraded comprises is:
Upgrade the covariance matrix of the forecast subarray corresponding with the contrary covariance matrix of forecast subarray, and
The covariance matrix through upgrading of forecast subarray is inverted.
25. one kind has and is used to carry out the computer-readable media of response based on the executable instruction of computing machine of the step of the calculating output valve of the measurement input value that receives at the current time duration of test with at one or more historical time duration of test, the step that comprises is:
(a) iteration of the measurement input value of reception current time test;
(b) according to measuring input value combinatorial input eigenwert vector;
(c) for multi-core processor provides input feature vector value vector, each kernel of processor can be used to:
Receive one or more input feature vector values,
Retrieval definition is used for calculating the connection standard of the mathematical relation of one or more output characteristic values based on the input feature vector value that receives,
Retrieve the connection weights of one group of regression coefficient between the output characteristic value of representing input feature vector value and calculating,
Retrieve one group of definition and be used to upgrade study weights based on the mathematical relation of the connection weights of the input feature vector value that receives,
According to the input feature vector value that receives, connect weights and be connected standard calculating output characteristic value,
According to the input feature vector value that receives, connection weights, connection standard and the calculating of study weights the weights that are connected through renewal,
The connection weights of storage through upgrading, and
Visit to the output characteristic value that calculates is provided;
(d) according to the output characteristic value combination calculation output valve vector that calculates by each kernel;
(e) RESPONSE CALCULATION output valve vector; And
(f) judge whether that indication improves operation, and if indication improve operation, then be thereafter time to test repeating step (a) to (e).
26. as the method for claim 25, wherein:
The formation that connects weights comprises the element of contrary covariance matrix; And
The step of calculating the connection weights of process renewal comprises step of upgrading this contrary covariance matrix.
27. as the method for claim 25, wherein:
The formation that connects weights comprises the element of contrary covariance matrix; And
The step of calculating the connection weights of process renewal comprises:
Upgrade the covariance matrix of contrary covariance matrix correspondence therewith, and
This inverts through the covariance matrix that upgrades.
28., wherein calculate the step that the step of output characteristic value comprises and be as the computer-readable media of claim 25:
Estimate the output characteristic value of current time test according to the input feature vector value of one or more historical time tests;
The output characteristic value of the following time test of prediction.
29., wherein calculate the step that the step of output characteristic value comprises and be as the computer-readable media of claim 25:
Estimate the output characteristic value of current time test according to the input feature vector value of one or more historical time tests;
Input feature vector value according to the current time test is calculated the output characteristic value that monitors;
According to the output characteristic value of estimation and the output characteristic value calculation deviation value of supervision; And
On the output characteristic value basis of output valve based on supervision of calculating.
30. as the computer-readable media of claim 28, wherein the step that comprises of the step of RESPONSE CALCULATION output valve vector is:
Each deviate and a dependent thresholds are compared; And
Surpass its dependent thresholds as one in the deviate, then carry out one or more biased operation of from following one group, selecting,
The indication alarm condition, and
With the output valve calculated but not on the supervision output characteristic value basis of the output characteristic value relevant with the deviate that surpasses its dependent thresholds based on the estimation output valve.
31. as the computer-readable media of claim 28, wherein the step of RESPONSE CALCULATION output valve vector comprises one or more control operations that execution is selected from following one group:
On display device, show the expression of calculating output valve, and
Starting a controlled variable compensates by the situation of calculating the output valve indication.
32., also comprise one or more improvement operations that execution is selected from following one group as the computer-readable media of claim 28:
Delete the invalid eigenwert that inputs or outputs;
Merge the redundant eigenwert that inputs or outputs;
Specify and newly input or output feature;
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the input feature vector standard;
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the study weights;
Recomputate with the output valve that calculates according to measurement input value and to be connected standard the test of a plurality of times;
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the output characteristic standard; And
Between kernel, redistribute functional.
33. the computer-readable media as claim 28 also comprises the steps:
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the study weights;
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the eigenwert standard.
34. a computer system comprises:
Input processor, it is configured to be used for
Receive the iteration of the measurement input value of current time test; And
According to measuring input value combinatorial input eigenwert vector;
Be connected and be configured as the multi-core processor of reception input feature vector value vector with input processor, each kernel of processor is used for:
Receive one or more input feature vector values,
Retrieval definition is used for calculating the connection standard of the mathematical relation of one or more output characteristic values based on the input feature vector value that receives,
Retrieve the connection weights of one group of regression coefficient between the output characteristic value of representing the input feature vector value that receives and calculating,
Retrieve one group of definition and be used to upgrade study weights based on the mathematical relation of the connection weights of the input feature vector value that receives,
According to the input feature vector value that receives, connect weights and be connected standard calculating output characteristic value,
According to the input feature vector value that receives, connection weights, connection standard and the calculating of study weights the weights that are connected through renewal,
The connection weights of storage through upgrading, and
Visit to the output characteristic value that calculates is provided;
Be connected with many kernels array and be configured as the output processor that is used for according to the output characteristic value combination calculation output valve vector that calculates by each kernel; And
Be connected with output processor and be configured as and be used for the response unit of the output valve vector that RESPONSE CALCULATION goes out.
35. as the computer system of claim 34, wherein:
Comprise the element of contrary covariance matrix with the formation that is connected weights that each kernel is associated; And
Each kernel can be used for calculating the connection weights that upgrade by upgrading contrary covariance matrix.
36. as the computer system of claim 34, wherein:
The formation of the connection weights relevant with each kernel comprises the element of contrary covariance matrix; And
Each kernel can be used for
Upgrade and the contrary corresponding covariance matrix of covariance matrix, and
This is inverted through the covariance matrix that upgrades.
37. as the computer system of claim 34, also comprise be connected with the multinuclear array and be used for following aspect manager:
According to the measurement input value of a plurality of time tests and the output valve that calculates are recomputated the study weights;
Recomputate with the output valve that calculates according to measurement input value and to be connected standard the test of a plurality of times.
38. as the computer system of claim 37, manager wherein is connected with input processor and output processor, and this manager also is used for:
Delete the invalid eigenwert that inputs or outputs;
Merge the redundant eigenwert that inputs or outputs;
Specify and newly input or output eigenwert;
According to being recomputated, the measurement input value of a plurality of times test and the output valve that calculates input or output characteristic criterion; And
Between kernel, redistribute functional.
39., also comprise being connected with the multinuclear array and being used for connecting weights, storing the parameter storage that connects weights and the connection weights are provided for the multinuclear array from the multinuclear array received as the computer system of claim 37.
40. as the computer system of claim 36, wherein response unit is configured to be used for: on display device, show to calculate the expression of output valve, the indication alarm condition, or start a controlled variable and compensate by the situation of calculating the output valve indication.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 97181204 CN1242848A (en) | 1996-11-20 | 1997-11-19 | Multi-kernel neural network concurrent learning, monitoring and forecasting system |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US60/031,195 | 1996-11-20 | ||
| CN 97181204 CN1242848A (en) | 1996-11-20 | 1997-11-19 | Multi-kernel neural network concurrent learning, monitoring and forecasting system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN1242848A true CN1242848A (en) | 2000-01-26 |
Family
ID=5178157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 97181204 Pending CN1242848A (en) | 1996-11-20 | 1997-11-19 | Multi-kernel neural network concurrent learning, monitoring and forecasting system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1242848A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1298458C (en) * | 2003-09-29 | 2007-02-07 | 宝山钢铁股份有限公司 | Method for real-time estimating temperature of liquid steel in RH fining furnace |
| CN101067742B (en) * | 2005-10-31 | 2010-11-10 | 台湾积体电路制造股份有限公司 | Method and system for virtual metrology |
| CN108307660A (en) * | 2016-11-09 | 2018-07-20 | 松下知识产权经营株式会社 | Information processing method, information processing unit and program |
| CN108694441A (en) * | 2017-04-07 | 2018-10-23 | 上海寒武纪信息科技有限公司 | A kind of network processing unit and network operations method |
| CN105934766B (en) * | 2014-01-23 | 2018-11-20 | 高通股份有限公司 | Neural network is monitored with shade network |
-
1997
- 1997-11-19 CN CN 97181204 patent/CN1242848A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1298458C (en) * | 2003-09-29 | 2007-02-07 | 宝山钢铁股份有限公司 | Method for real-time estimating temperature of liquid steel in RH fining furnace |
| CN101067742B (en) * | 2005-10-31 | 2010-11-10 | 台湾积体电路制造股份有限公司 | Method and system for virtual metrology |
| CN105934766B (en) * | 2014-01-23 | 2018-11-20 | 高通股份有限公司 | Neural network is monitored with shade network |
| CN108307660A (en) * | 2016-11-09 | 2018-07-20 | 松下知识产权经营株式会社 | Information processing method, information processing unit and program |
| CN108307660B (en) * | 2016-11-09 | 2023-08-22 | 松下知识产权经营株式会社 | Information processing method, information processing device, and program |
| CN108694441A (en) * | 2017-04-07 | 2018-10-23 | 上海寒武纪信息科技有限公司 | A kind of network processing unit and network operations method |
| CN108694441B (en) * | 2017-04-07 | 2022-08-09 | 上海寒武纪信息科技有限公司 | Network processor and network operation method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6647377B2 (en) | Multi-kernel neural network concurrent learning, monitoring, and forecasting system | |
| Bian et al. | Stochastic modeling and real-time prognostics for multi-component systems with degradation rate interactions | |
| Messac et al. | Multiobjective robust design using physical programming | |
| CN113486584B (en) | Method and device for predicting equipment failure, computer equipment and computer readable storage medium | |
| Bontempi et al. | Local learning for iterated time series prediction | |
| Branke et al. | Evolutionary search for difficult problem instances to support the design of job shop dispatching rules | |
| Gomez et al. | A neuro-evolution method for dynamic resource allocation on a chip multiprocessor | |
| CN114021784A (en) | Method and device for determining residual service life of equipment and electronic equipment | |
| JP2023514466A (en) | Inference computing device, model training device, and inference computing system | |
| Yarally et al. | Uncovering energy-efficient practices in deep learning training: Preliminary steps towards green ai | |
| CN111311014B (en) | Service data processing method, device, computer equipment and storage medium | |
| KR20220081872A (en) | Automated device for calculating optimal information for decision support based on digital twin | |
| Neto et al. | Building energy consumption models based on smartphone user’s usage patterns | |
| Wu et al. | Adaptive sequential predictive maintenance policy with nonperiodic inspection for hard failures | |
| CN1242848A (en) | Multi-kernel neural network concurrent learning, monitoring and forecasting system | |
| AU721842B2 (en) | Multi-kernel neural network concurrent learning, monitoring, and forecasting system | |
| Peters et al. | Rough ethograms: Study of intelligent system behavior | |
| JP2003223322A (en) | Device for analyzing combinatorial optimization problem | |
| Mori et al. | Leveraging state-based user preferences in context-aware reconfigurations for self-adaptive systems | |
| WO1998022885A9 (en) | Multi-kernel neural network concurrent learning, monitoring, and forecasting system | |
| CN115544803A (en) | Method, device, equipment and storage medium for predicting residual life of transformer | |
| CN115470403A (en) | Real-time updating method and device of vehicle service recommendation model, vehicle and medium | |
| Wall | Organizational dynamics in adaptive distributed search processes: effects on performance and the role of complexity | |
| JPH06161989A (en) | Prediction device | |
| CN118011280B (en) | Voltage quality on-line monitoring and analyzing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |