CN105867832B

CN105867832B - User and application oriented computer and intelligent equipment acceleration method and device

Info

Publication number: CN105867832B
Application number: CN201510022782.3A
Authority: CN
Inventors: 张维加
Original assignee: Individual
Current assignee: Individual
Priority date: 2015-01-19
Filing date: 2015-01-19
Publication date: 2020-07-24
Anticipated expiration: 2035-01-19
Also published as: WO2016115957A1; CN105867832A

Abstract

A computer and intelligent equipment accelerating method and device facing to users and applications, arranging cache and prefetch service control devices on a large number of computers, creating memory virtual disks on the devices by the devices, then carrying out different types of read-write tests on various hardware parts, modeling the devices into combinations of data devices with different performance parameters, connecting external solid hardware by the control devices and participating in modeling, preliminarily analyzing application information on the served devices by the control devices, analyzing user types through network operation and the like, submitting the user types to a cloud end together with hardware modeling, giving accelerating schemes aiming at different application and different user groups to different types of hardware according to prior files after cloud end analysis, returning the devices for preliminary processing, simultaneously counting read-write operation, I/O types, operation frequency and the like of each application by the devices, feeding back the cloud end again by combining effect feedback after a period of time, and the cloud records and gives a correction scheme. And repeatedly iterating to be basically perfect, and storing the final scheme and the history in a cloud.

Description

User and application oriented computer and intelligent equipment acceleration method and device

Technical Field

The product belongs to the technical field of computer equipment and information science. The method is a cross-device interaction computer and intelligent device acceleration method based on big data and cloud technology.

Background

It should be noted that the cache referred to in the present invention mainly refers to a disk cache of a computer and an intelligent computing device, that is, a cache for accelerating a computer or running and breaking through a disk performance bottleneck, rather than a video streaming media cache or a routing web cache.

Disk caching techniques have emerged to address disk speed bottlenecks. The improvement in disk performance lags behind processors and other electronic devices far enough that memory systems remain a performance bottleneck for the entire computer system. Caching (Caching) and Prefetching (Prefetching) are two very effective techniques that can improve memory system performance. The idea of the cache technology is to put frequently accessed data into a quick access device, so as to accelerate the access speed and reduce the waiting time. The prefetching technique is to prefetch data that is likely to be accessed soon in the future from a slow device to a fast device in advance. Since prefetching is also actually a kind of disk cache allocation, both are referred to as disk cache technologies herein.

Cache technology (Caching), as the name suggests, is a buffer layer between a higher-performance device at a previous level and a lower-performance device at a next level when the difference between the read and write performance of the devices at the previous level and the lower-performance device at the next level is large, the capacity of the buffer layer is lower than that of the lower-performance device at the next level, the performance of the buffer layer is often lower than that of the higher-performance device at the previous level, but the speed of the buffer layer is higher than that of the lower-performance device, and the performance is improved by transferring the read and write originally directed to the lower-performance device.

Prefetching (prefeching) is another important technique to improve memory system performance. The prefetching is to read the data that has not been accessed but is likely to be accessed in the future from a low-speed storage device such as a disk to a high-speed storage device such as a cache in a batch manner in advance, so as to increase the speed of data access and finally improve the performance of the whole storage system.

The effectiveness of prefetching techniques depends largely on two aspects: one is the precision of the prefetch and the cache hit rate affected by it, and the other is the sequential mining in the prefetch. Some studies have attempted to improve the accuracy of predictions by keeping more and longer historical access information. In another type of algorithm, access relations between files or between data blocks are mined through historical access information of the equipment, future access data are predicted based on the relations, and the cache hit rate is improved.

Whether caching or prefetching, there have been many problems that have affected their use.

For example, the old caching technology is targeted at devices, and aims to improve the performance of the devices, so that the performance of the devices can be improved when the devices do anything. The three disadvantages are that firstly, caches have to be designed by taking devices as objects, the caches have non-portability, hardware binding is formed, the caches cannot be used universally, secondly, performance improvement of one device does not help other devices, that is, marginal cost cannot be reduced, and marginal utility cannot be improved, for example, by setting a larger cache, a samsung 850EVO disk obtains better performance than an 840EVO, but the fact does not help the existing 840EVO, thirdly, the help to users is small, or taking samsung as an example, the cache of a solid state disk is generally designed to be lower, the reason is simple, as described by designers, users generally cannot feel performance improvement caused by the higher cache, performance indexes run out, and practical application satisfaction is lower.

For another example, the algorithm, optimization, and self-learning of the old cache prefetch are local, and the effect is expected to be immediate due to hardware for a specific device, and even if the later optimization is performed, the long-term correction is expected. This is because, in the past, any cache prefetch system cannot obtain cache information of other devices, nor affect the operation of other devices, and even in the case of not targeting applications and user types (user groups), but targeting devices, the large difference between devices seems meaningless even if the interaction between the systems of the respective devices is implemented:

reason 1. cache information of other devices cannot be obtained

In the past, disk caches have all formed a stand-alone system with the respective device, without any interaction with the caches of the other devices.

Reason 2. cannot affect the operation of other equipment

Since each is an isolated system, they cannot naturally affect each other.

Reason 3. the great difference between devices seems to be meaningless even if the interaction between the individual device systems is realized

Taking the establishment process of the cache as an example, the running data of the device needs to be accumulated to count the common files and cache the common files. Obviously, the common files are specific to specific equipment, and the concept of no common files exists after the specific equipment is separated. A computer program commonly used by computer engineers, such as Visual C or Dreamwaver, may not be installed at all on the computers of ordinary users, what is the cache system between them comparable? In any case, the caches of different devices are different greatly, most computers in the past are not provided with disk caches except for processor caches, a few servers are applied with cache technologies, some desktop computers are applied with solid state disk-based cache technologies (such as hybrid hard disks), and the devices are different greatly in no-cache, cache and completely different cache devices. Therefore, the device-oriented caching technology has no interaction possibility.

In a word, the disk cache or computer cache prefetching in the past is local with equipment as an object, which causes the problems of difficult transplantation, non-universality, low marginal utility, high marginal cost, low actual application satisfaction rate, slow optimization and time consumption and the like.

However, if such a mode is changed, redesign of existing various devices, software and hardware, and modification of the operation mode are required.

But this redesign is worthwhile. Although the present invention was only a preliminary search, unexpected results were also obtained.

And the mode of the disk cache is changed. In another patent (2014105350389), the inventor proposes a cross-device compute acceleration system, which is essentially performance delivery, and a short-distance but multi-channel network between the server and the served side in the cross-device cache system can deliver performance in a short distance, and the server side can interact with the server side by means of optical fibers and the like. Therefore, the cross-device cache system can form a network, obtain big data and apply cloud technology.

Disclosure of Invention

The invention provides a cross-device interaction computer and intelligent device acceleration method based on big data and cloud technology, aiming at solving the problems in the prior art.

The specific technical scheme of the invention is as follows: a computer and intelligent equipment accelerating method based on big data and cloud technology and using user and application as object, the method arranges control device on more than one equipment to be accelerated, these control devices identify or test the main hardware part of these equipment to be accelerated, and transfer part of internal memory of the equipment part to be served to virtualize it into magnetic disk as cache, and obtain the application program condition data and network operation user type characteristic data on the equipment to be accelerated by scanning the relevant program directory cache directory on the equipment to be accelerated, and submit them to the cloud remote server together with the identified hardware characteristic data, after receiving the above data, the cloud carries out calculation and analysis by combining the original existing database of the cloud, and by searching the best cache scheme and pre-fetching configuration mode of the corresponding application program under the similar hardware and similar user, giving an optimized cache or prefetch acceleration scheme aiming at specific hardware, specific application and specific user type of each different device to be accelerated, feeding back the optimized cache or prefetch acceleration scheme to each service control device in an active feedback or passive response mode, and carrying out corresponding cache acceleration, cache optimization or prefetch processing by the control device according to feedback information; the method also adopts cache shunting, the control device is combined with external solid-state hardware with a USB interface or Wigig connection, the control device is loaded together when being installed in the device, additional cache pre-taking hardware is provided when the control device works, the additional cache pre-taking hardware comprises 4K read-write cache to a virtualized memory disk, 512K and random read-write cache to the external solid-state hardware, cache shunting is realized, different shunting schemes are adopted for different applications, and the shunting scheme cloud server is determined by analyzing uploaded data, namely, a three-level structure of cooperative work is adopted on the framework: the method comprises the steps that a cache prefetching control device of a large number of terminals serves as a first-level acceleration device, is responsible for terminal service and contribution of big data, a cloud server serves as a second-level acceleration device, is responsible for optimization scheme calculation and database index iteration, an external solid-state hardware of a USB interface or a Wigig interface carried by the control device serves as a third-level acceleration device and is responsible for providing cache shunting for the control device, all uploaded and downloaded data between the control device and the cloud are transmitted in an encrypted form, the uploaded data of the control device further comprises characteristic data reflecting crowds to which equipment users belong, the characteristic data comprises user age ranges, occupation ranges and interest ranges, correspondingly, the optimization scheme of cloud feedback also comprises a scheme for optimizing or prejudging the use characteristics of different application objects according to different user types, and the uploaded data of the control device further comprises specific cache equipment hardware types or characteristic information, correspondingly, the optimization scheme given by cloud analysis is not that one application corresponds to one part, but is specific and classified, the control device obtains the application type through a scanning system registry, a scanning program installation directory obtains the number, size and read-write characteristics of the files of the application, a pre-fetching Prefetch directory, a system cache and a system log of the scanning system obtain the application use frequency, a TEMP folder, a favorite and a browser cache folder of the scanning system obtain a regular access website, deduces the type characteristics of a user to which the user belongs according to the website, the website and the cache file, deduces the habit of the user, and judges the occupation, interest and age of the user according to the type and age of equipment and the application distribution on the equipment; the method is characterized in that after a feedback scheme is received in a control state, the application is not directly finished but an iterative process is started, namely, cache prefetching service is configured according to preliminary indication of a server, a control device starts to count tracking state information, effect feedback or user satisfaction is combined to feed back a cloud again after a period of time, the cloud analyzes and gives a correction or a second preferred scheme again after receiving data and feedback, iteration is repeated until basic perfection is achieved, the final scheme is stored in the cloud, the cloud takes the application, hardware type, cache equipment type and user characteristic as indexes, the final optimized scheme is added to a database, and partial optimization process information or all optimization histories can be selectively recorded in the database, namely, all parts of equipment hardware are identified and modeled in the method flow, virtualization and cache creation, application data and user type data acquisition and data acquisition Application data, user type data and equipment model data are transmitted to a cloud end, initial optimization attempt of cloud computing is performed, feedback collection is transmitted to the cloud end, secondary correction feedback of the cloud end is performed, the process is repeated for multiple times until the feedback is close to perfect and recorded, and large data are formed, optimized and accumulated continuously through the cloud end.

As a further limitation of the present invention, the initially allocated memory virtualization cache is allocated according to a minimum size and is resized according to server instructions in a subsequent optimization process.

As a further limitation of the present invention, the control device itself serves as a new service node, so that the service capability of the accelerator network increases with the increase of the accelerated devices, including those control devices providing network or acceleration services such as cdn cache, short-range network cache, VPN service, SMTP service to other peripheral users according to the instruction of the cloud server.

As a further limitation of the present invention, the device itself also has an external solid-state hardware connected with USB or Wigig, and when the device works, an additional hardware for cache pre-fetching is provided as cache shunting, and when the service node mode is turned on, the external hardware provides required storage and network component support, that is, a three-level structure of cooperative work and self-expansion is adopted in the architecture: the cache pre-fetching control device of a large number of terminals is used as a first-level acceleration device and is responsible for terminal service and contribution of big data, the cloud server is used as a second-level acceleration device and is responsible for optimization scheme calculation and database index iteration, and external solid-state hardware of a USB interface or a Wigig interface carried by the control device is used as a third-level acceleration device and is responsible for providing cache shunting and hardware support for the control device and expanding accelerated equipment into a new service node in a node mode.

The invention achieves the following beneficial effects:

the invention creates a new working mode and a device manufacturing mode for cache prefetching.

Firstly, the disk cache or computer cache prefetching in the past is local with equipment as an object, which causes the problems of difficult transplantation, non-universality, low marginal utility, high marginal cost, low satisfaction rate of actual application, slow optimization and time consumption, and the like. The new cache prefetching provided by the invention is based on the network with the application and the user type (user group) as the object, has universality and portability, has the network scale effect, has high marginal utility and high application satisfaction rate, and can quickly complete the optimization configuration.

Secondly, different from the past cache prefetching technology, the cache prefetching technology provided by the invention adopts equipment modeling in the process flow, namely, the equipment modeling is transmitted to the cloud together with the user type data and the equipment model, the initial optimization attempt of cloud computing is performed, the feedback acquisition is transmitted to the cloud, the feedback is corrected and the data is recorded for the second time, and the process flow is repeated for multiple times until the feedback is completed and the history is recorded. Iteration exists in the work flow, big data reflecting the relation between the application characteristics and the user type characteristics and the hardware model and a new cloud computing mode of fuzzy analysis and iteration guidance are built from scratch in the work process.

Third, the conventional cache prefetching is often completed in one device, and the cache prefetching technology provided by the present invention adopts a three-level structure in architecture: the cache prefetching control device of a large number of terminals is used as a first-level control device to contribute large data, and a cloud server with small number but strong analysis, especially fuzzy operation capability is used as a second-level control device to perform cloud computing, and the external solid-state hardware of a USB interface carried by the control device is used for completing the cloud computing in three levels.

In terms of service scope, the three-level structure is no longer local, and they are connected with each other through a network to complete the cooperative work.

Fourth, past cache prefetching techniques have all ignored this essential difference in user types. There is a great difference in the demands of different users for the same device. The final object of the technical service should be a person, not a device. An elderly person using the same browser may be primarily watching video and news while a younger person may be playing a web game, this difference reflecting that the caching scheme of the application should be distinct. Of course, this is not entirely negligible or biased, and past solutions have no way to do so, and the device cannot predict its buyer before it is sold, and the program cannot predict its user before it is downloaded. By adopting the method and the device, the user types can be mined, the related big data can be created and applied to the cache prefetching technology.

The cache optimization and pre-fetching optimization mechanism of the computing equipment can be changed, and the cache acceleration capacity of the cache equipment to the first-time application, the newly-installed application, the newly-visited website and the application with low use frequency can be improved. For frequently used applications, the caching and prefetching effects can be further improved by aiming at device hardware characteristics, user type characteristics and the like.

Drawings

Fig. 1 is a basic principle diagram of the device.

FIG. 2 is a schematic diagram of a sample apparatus.

Detailed Description

The invention provides a computer acceleration method and device for users and applications.

The scheme of the invention changes the local characteristics of the prior caching and prefetching technology, excavates the data characteristics of the caching scheme and experience, changes the caching operation of the device object type into the caching operation of the application and user object type, changes the fixed device type into the cross-device networking cooperative operation, and changes the single caching and prefetching device into three levels.

The method comprises the following steps: the method comprises the steps of (control equipment installation and identification), equipment modeling, application and user type data are transmitted to a cloud together with an equipment model, initial optimization attempt of cloud computing, feedback collection and transmission to the cloud, secondary cloud correction feedback and data recording, and the steps of repeating for multiple times until the feedback is perfected and history is recorded.

The traditional cache prefetching is often completed in one device, and the cache prefetching technology provided by the invention adopts a three-level structure in the structure: the cache prefetching control device of a large number of terminals is used as a first-level control device to contribute large data, and a cloud server with small number but strong analysis, especially fuzzy operation capability is used as a second-level control device to perform cloud computing, and the external solid-state hardware of a USB interface carried by the control device is used for completing the cloud computing in three levels. In terms of service scope, the three-level structure is no longer local, and they are connected with each other through a network to complete the cooperative work.

The method requires a plurality of or a large number of cache prefetch terminal control devices and a cloud server with fuzzy analysis capability. The terminal control devices perform advanced processing on accelerated computing equipment, including loading acceleration hardware, detecting network equipment and partitioning memory and storage equipment, then the control devices perform different types of read-write tests (such as 4K read-write, sequential read-write and the like) on hardware parts of the equipment, the equipment is modeled into combinations of data devices with different performance parameters, and the devices also classify various types of cache equipment so as to apply an optimization scheme. For example, marking the parallel device and the serial device respectively, for the parallel I/O, adopting a fine-grained synchronous locking mechanism to increase the parallelism of the I/O process, thereby improving the I/O performance, and for example, marking and distinguishing the I/O type, judging the most adept random read operation I/O type of the cache device, and preferentially distributing the cache device for caching by judging the characteristics of the cache device in the I/O process.

And then, the control device preliminarily analyzes various application program information and user characteristic data of network operation on the served equipment, submits the information and the hardware modeling result to the cloud, performs statistics and fuzzy analysis after the cloud receives the data, provides an optimization acceleration scheme aiming at different user types (user groups) of different applications for different modeling hardware according to prior empirical data files, and returns the optimization acceleration scheme to the cache service device for primary processing.

After the first configuration guidance scheme returned by the cloud is preliminarily applied, after a period of time of self-learning and optimization, the control device counts the read operation and write operation proportion, the I/O request type, the number and the size of common files, the use frequency, the user type characteristics and the like of each application, and the self-test and the user feedback are combined to collect and feed back the cloud again after a period of time. And uploading the optimized cache mode configuration data in the respective systems to a processing server (cloud) in a ciphertext mode.

The cloud records data and feedback conditions and gives a correction or second preferred scheme. Repeating the steps for a plurality of times to achieve basic perfection and store the final result and the optimized history in a cloud.

The cloud end carries out statistics and analysis after receiving the optimized final data, and analyzes and summarizes the cache configuration or prefetching optimization schemes for different applications (or cache configuration or prefetching optimization schemes for different applications under specific conditions of different devices, users and the like) by taking application level objects such as application programs, games, websites and the like as statistical objects, so that the optimized cache scheme and prefetching scheme are returned to the cache service device for corresponding processing such as optimization, prejudgment and the like in modes of active feedback or passive response and the like. See figure 1.

Of course, all the data uploaded and downloaded between the cache service device and the cloud end are transmitted in a form of ciphertext.

Further, the data uploaded by the cache service device may also include cache hardware characteristics of respective devices, and may also be used in a scheme of applying cloud feedback. Thus, the cache optimization scheme given by the cloud analysis is not an application, but is specific and classified, for example, on what cache structure, what cache or prefetch scheme is adopted for the application. The different processing according to various cache devices is beneficial to applying an optimization scheme.

Further, the data uploaded by the cache service device may further include user group characteristic data, such as an age range, an occupation range, an interest range, and the like, and accordingly, the optimized cache scheme fed back by the cloud end also includes an optimized or predicted scheme for the usage characteristics of different application objects for different user types (user groups). For example, users in a specific industry and age group have respective obvious crowd characteristics, for example, the elderly do not use 3D games with a large amount of read random cache, but rather use browsers with more write cache. Knowing these features and applying them, the prefetching and caching functions can be better performed. Of course, these pieces of information are user group information, and the device neither needs nor never acquires any personal information of the user itself. These user group information are also encrypted.

The control means may also choose to turn on the service node mode (the user has the option). If the user allows the service node mode to be started, the control device provides cdn cache, short-range network cache, VPN service, smtp service and other services for other peripheral users according to the instruction of the cloud server. Meanwhile, the user can obtain a certain return on income.

According to the method, a plurality of sample devices are arranged, and the method is shown in the concrete embodiment part.

The effect is extensive, and on the user level, even if the device is just installed, the website and related website that the user is interested in can be accessed quickly, even if the user may only access the website for the first time or the second time (this proportion is large, and 60% of network accesses of general netizens are websites accessed less than three times), which was not possible in the past. The device can also dig more website relevance and acceleration technology by means of big data of the user group.

Similarly, even if the device has just been installed, the user's usual applications and favorite applications may be able to run smoothly. The more the users are, the wider the distribution is, the better the user experience is, and the network effect and the snowball effect are achieved.

For the application level, the effect scenario is as follows.

For example, one: if a certain folder of a certain game program on a large number of served devices shows a frequent reading characteristic, when the device newly loads the program, the device can directly perform the pre-judging work such as caching the folder which is frequently read and written on other devices to a high-speed device without accumulating the cached data again.

Example two: when a shopping browser is started, a large write cache can be allocated to the program in a pre-judging texture without accumulating cache data again.

In fact, many programs cannot learn the optimal cache on a single device because of low frequency of use by users, but the statistics and judgment of a large number of data samples can be performed across the acquisition of device data, so that many rarely used programs, even the first used program, can be accurately pre-optimized.

For the equipment level, the device is universal, portable and interconnected, and can complete continuous subsequent function upgrade by means of the upgrade cloud.

Based on the method of the invention, a device is designed and implemented. The device applying the method of the invention can be hardware, software or the combination of the hardware and the software. The sample apparatus shown here has two, the first is a software and hardware combination device, and the second sample omits an external cache device and a high-speed network component to be a piece of software.

The first example has a control device and an external solid state acceleration hardware with USB3.0 connection. The solid-state acceleration hardware has the sequential reading of 620MB per second, the sequential writing of 550MB per second, the 4K reading of 120MB per second and the 4K writing of 160MB per second, the above speeds are data parameters measured by Thunderbolt when the hardware is delivered from a factory, the performance can be approximately achieved under the USB3, and the work flow of the device (see the attached figure 2):

first, cache loading and virtualization work.

1. Loading solid acceleration hardware 2, calling a memory of a served device part, virtualizing the memory into a disk as a first-level cache, storing the content of the memory into a file data packet when the server is powered off, loading the data packet into the virtual memory disk when the server is powered on, calling the size of the memory to be a preliminary set minimum value, and then gradually modifying the memory after the cloud feedback process; 3. and detecting whether other available disk caches exist, for example, detecting whether a mobile device of a low-speed disk has a high-speed flash memory externally connected with a wigig, if the available caches can be detected, establishing the cache as a second-level cache (or establishing the cache by the consent of a user), so that caching and prefetching are carried out according to read-write operation and the like.

And secondly, measuring.

After the preparation operation is completed, the control device performs different types of read-write tests, such as 4K read-write, 512K random read-write, sequential read-write and the like, on the hardware of the device and various created cache components, judges the cache performance characteristics of each part of the device to be accelerated, and participates in the test on the external acceleration hardware, because the USB interface of the device brings great influence.

And thirdly, modeling work.

According to the measured data, other hardware information such as size, interface and the like is read through a system function such as a Windows function, then the equipment is modeled into a combination of data devices with different performance parameters, each read-write performance score and comprehensive score of each part are given, and classification is carried out, for example, whether the part belongs to a random read cache device or a 4K write cache device. The classification information can be encrypted and uploaded to the cloud together with cache optimization data of the local computer, and can also be used for a scheme fed back by the application cloud. The cache optimization scheme given by the cloud analysis is not applied to one part, but is specific and classified, such as what cache structure and what scheme is applied. The different processing according to various cache devices is beneficial to applying an optimization scheme. For example, the marks distinguish the types of I/O, judge the most adept type of random read operation I/O of the cache device, and preferentially allocate the cache device to cache by judging the characteristics of the device in the I/O process.

Fourth, the application state is scanned and the user type (user group) is roughly determined.

There are actually many implementations of this step. Our control device in the example does this: the method comprises the steps of scanning a program installation directory to obtain application types, scanning a Prefetch directory and a log to obtain application use frequency, scanning a TEMP folder of a system to obtain a frequently-accessed website, deducing user type characteristics according to the website and deducing user habits. The device roughly judges the group characteristics of the user according to the website and the cache file, and judges the occupation, interest, age and the like of the user according to the type and age of the equipment and the application distribution on the equipment. Of course, these pieces of information are characteristic information of the population of the user, and the device neither needs nor never acquires any personal information of the user itself. And all the user group information is transmitted to the cloud end in an encrypted form.

And fifthly, primarily uploading the data to the cloud.

The control device preliminarily analyzes and submits various application program information on the served equipment and user characteristic data of network operation to the cloud end, and hardware modeling results.

The uploaded data does not have any user privacy information, and is abstract model information and user group, for example, a typical example generally includes the following types of information, and the following information is only an example: the most commonly used applications (warrior, Taobao, Word), user features (20-30 years old, men, favorites shopping and browsing of automobile-related websites, and page games such as 4399), computer modeling features (test features: 32-bit system, 4GB physical DDR2 memory, system identification 3.2GB, connection of acceleration hardware components through USB3.0 common interface, 64GB total, and optimized acceleration using USB protocol, creation of memory virtual disk 512MB, single hard disk, Hiji hybrid hard disk 1TB, where memory virtual disk measurement is 2200MB per second read sequentially, 1020MB per second write sequentially, 500MB per second read 4K, 300MB per second write 4K, 480MB per second read sequentially with external acceleration hardware, 480MB per second write sequentially, 100MB per second read 4K, 160MB per second write 4K, 150MB per second read sequentially with hybrid hard disk, 120MB per second read sequentially, 1MB per second read 4K, other parameters such as 0.5MB per second are written in 4K, and the characteristics are modeled: a 4K cache a, a memory virtual write cache B, a sequential read cache C, a blend D-actual modeling is certainly more complicated than this and is illustrated here), and so on, which are uploaded to the cloud server in an encrypted format.

And sixthly, performing cloud preliminary fuzzy analysis.

First, as in the internet, if the cloud has no existing data at the beginning of the network building, the first files and data need to be input by a human engineer, including a large number of cache prefetching schemes for different applications of various user groups in various device environments. The imperfection of the scheme is irrelevant because the scheme can be continuously iterated and perfected in the later process.

We focus here next on the flow of course after the network has been initially established.

After receiving the uploaded data such as the application data, the user group characteristics, the hardware modeling result and the like, the cloud carries out statistics and fuzzy analysis after receiving the data, gives an optimized acceleration scheme aiming at different application user groups to different modeling hardware according to prior experience data files, and returns to the cache service device for carrying out first processing.

The processing manner such as the above example may be: according to the data of the server database, as a large number of folders contended by the magic beasts on the served equipment all present frequent reading characteristics, the cloud end return scheme requires caching the frequently read-written folder to C; according to the data of the server database, a large number of Taobao browsers on the served equipment show frequent writing work, so that a large writing cache is distributed to the Taobao browsers B; word on a large number of served devices relates to a large number of 4K read-write and is allocated to an area A; since users like shopping and browsing automobile-related websites, and page games such as 4399, the cloud return scheme requires cache prefetching of the main pages of the related websites, and arranging some caches in cdn technology for redirection to nearby nodes; and giving some other system and application cache prefetch configuration schemes, etc. for the uploaded data and models.

And after the analysis is finished, the server returns the scheme to the control device.

And seventhly, feeding back the depth data and the test effect.

After a period of time, the control device performs self-test and user feedback collection, uploads depth data obtained within a period of time, and the depth data comprises the proportion of read operation and write operation of each application, the type of I/O request, the number and size of common files, the use frequency as much as possible, and feeds back to a cloud end which gives a correction or a second preferred scheme according to the feedback condition.

And eighthly, repeatedly iterating the scheme.

After the above steps are repeated for a plurality of times, the basic perfection is achieved.

And step nine, updating the server database, and storing the final result and the optimization history in a cloud.

The cloud server receives the final optimized cache mode configuration data uploaded by the ciphertext in respective systems, processes statistical data of various application programs, games, network operations and related files cached by the server for a plurality of devices, and records the statistical data into a database by taking application, user and equipment models as classification units, such as optimal AutoCAD cache and prefetching schemes (the optimal cache prefetching schemes of the same application program on different types of users and different equipment are obviously different) on users in the building industry and Dell L satude 600 computers so as to coordinate new devices later.

Tenth, the node is re-served (user selectable mode).

The design of the apparatus of this example also includes: 1. providing intelligent compression and background automatic release for a system memory; 2. the device virtualizes the application program, so as to pre-store more or even all program files and system environment files required by the program in a cache (the virtualization principle can be redirection and environment virtualization technologies, etc., and the virtualized application program is self-contained and can be in the cache).

Workflow of the second example apparatus:

in the first step, cache creation and virtualization work.

1. Calling a memory of a served terminal device part, virtualizing the memory into a disk as a first-level cache, storing the content of the memory into a file data packet when the server is powered off, loading the data packet into a virtual memory disk when the server is powered on, calling the size of the memory to be a preliminarily set minimum value, and then gradually modifying the memory after the cloud feedback process; 2. and detecting whether an available disk cache exists, for example, detecting whether a mobile device of a low-speed disk has a high-speed flash memory externally connected with a wigig, if the available cache can be detected, establishing the cache as a second-level cache (or establishing the cache by the agreement of a user), so as to perform caching and prefetching according to read-write operation and the like.

And secondly, measuring.

After the preparation operation is completed, the control device performs different types of read-write tests, such as 4K read-write, 512K random read-write, sequential read-write and the like, on the device hardware and various created cache components, judges the cache performance characteristics of each part of the device to be accelerated, and when external hardware devices such as an external solid state disk and the like exist, the external devices also participate in the tests.

And thirdly, modeling work.

There are actually many implementations of this step. Our control device in sample two does this: the method comprises the steps of scanning a program installation directory to obtain application types, scanning a Prefetch directory and a log to obtain application use frequency, scanning a TEMP folder of a system to obtain a frequently-accessed website, deducing user group characteristics according to the website and deducing user habits. The device roughly judges the group characteristics of the user according to the website and the cache file, and judges the occupation, interest, age and the like of the user according to the type and age of the equipment and the application distribution on the equipment. Of course, these pieces of information are characteristic information of the population of the user, and the device neither needs nor never acquires any personal information of the user itself. And all the user group information is transmitted to the cloud end in an encrypted form.

And fifthly, primarily uploading the data to the cloud.

The uploaded data does not have any user privacy information, and is abstract model information and user group, for example, a typical example generally includes the following types of information, and the following information is only an example: the most commonly used applications (warrior, Taobao, Word), user features (20-30 years old, male, favorite shopping and browsing automobile related websites, and page games such as 4399), computer modeling features (test features: 32 bit system, 4GB physical DDR2 memory, system identification 3.2GB, creation of memory virtual disk 512MB, dual hard disk, where SSD is 32GB, HDD is 1TB, where memory virtual disk tests are 2200MB per second read sequentially, 1020MB per second write sequentially, 500MB per second read 4K, 300MB per second write 4K, 300MB per second read SSD sequentially, 300MB per second read sequentially, 120MB per second write sequentially, 280MB per second read 512K, 110MB per second write 512K, 10MB per second read 4K, 16MB per second write 4K, 80MB per second read sequentially for HDD, 60MB per second write sequentially, 0.1MB per second read 4K, 0.05MB per second write 4K, etc., other parameters are set as a cache area of 4A, a virtual write cache B, a sequential read cache C, a blend D-actual modeling is certainly more complex than this and is illustrated here), etc., which are uploaded to the cloud server in encrypted format.

And sixthly, performing cloud preliminary fuzzy analysis.

And seventhly, feeding back the depth data and the test effect.

And eighthly, repeatedly iterating the scheme.

The cloud server receives the final optimized cache mode configuration data uploaded by the ciphertext in respective systems, processes statistical data of various application programs, games, network operations and related files cached by the server for a plurality of devices, and records the statistical data into a database by taking application, user and equipment models as classification units, such as optimal AutoCAD cache and prefetching schemes on users in the building industry and Dell L satude 600 computers (because the optimal cache prefetching schemes of the same application program on different types of users and different equipment are obviously different) so as to coordinate new devices later.

The design of the second example device also includes: 1. providing intelligent compression and background automatic release for a system memory; 2. the device carries out virtualization processing on the application program, so that more or even all program files and system environment files required by the program are prestored in the cache.

In addition to cache load differences, see also FIG. 2.

While the invention has been described in connection with specific embodiments and implementations, many modifications and variations are possible in light of the above teaching or may be acquired from the teaching herein, and it is intended that all such modifications and variations be included within the scope of the invention as determined by the appended claims and their equivalents without departing from the true spirit and scope of the invention.

Claims

1. A computer and intelligent equipment accelerating method based on big data and cloud technology and using user and application as object, the method arranges control device on more than one equipment to be accelerated, these control devices identify or test the main hardware part of these equipment to be accelerated, and transfer part of internal memory of the equipment part to be served to virtualize it into magnetic disk as cache, and obtain the application program condition data and network operation user type characteristic data on the equipment to be accelerated by scanning the relevant program directory cache directory on the equipment to be accelerated, and submit them to the cloud remote server together with the identified hardware characteristic data, after receiving the above data, the cloud carries out calculation and analysis by combining the original existing database of the cloud, and by searching the best cache scheme and pre-fetching configuration mode of the corresponding application program under the similar hardware and similar user, giving an optimized cache or prefetch acceleration scheme aiming at specific hardware, specific application and specific user type of each different device to be accelerated, feeding back the optimized cache or prefetch acceleration scheme to each service control device in an active feedback or passive response mode, and carrying out corresponding cache acceleration, cache optimization or prefetch processing by the control device according to feedback information; the method also adopts cache shunting, the control device is combined with external solid-state hardware with a USB interface or Wigig connection, the control device is loaded together when being installed in the device, additional cache pre-taking hardware is provided when the control device works, the additional cache pre-taking hardware comprises 4K read-write cache to a virtualized memory disk, 512K and random read-write cache to the external solid-state hardware, cache shunting is realized, different shunting schemes are adopted for different applications, and the shunting scheme cloud server is determined by analyzing uploaded data, namely, a three-level structure of cooperative work is adopted on the framework: the method comprises the steps that a cache prefetching control device of a large number of terminals serves as a first-level acceleration device, is responsible for terminal service and contribution of big data, a cloud server serves as a second-level acceleration device, is responsible for optimization scheme calculation and database index iteration, an external solid-state hardware of a USB interface or a Wigig interface carried by the control device serves as a third-level acceleration device and is responsible for providing cache shunting for the control device, all uploaded and downloaded data between the control device and the cloud are transmitted in an encrypted form, the uploaded data of the control device further comprises characteristic data reflecting crowds to which equipment users belong, the characteristic data comprises user age ranges, occupation ranges and interest ranges, correspondingly, the optimization scheme of cloud feedback also comprises a scheme for optimizing or prejudging the use characteristics of different application objects according to different user types, and the uploaded data of the control device further comprises specific cache equipment hardware types or characteristic information, correspondingly, the optimization scheme given by cloud analysis is not that one application corresponds to one part, but is specific and classified, the control device obtains the application type through a scanning system registry, a scanning program installation directory obtains the number, size and read-write characteristics of the files of the application, a pre-fetching Prefetch directory, a system cache and a system log of the scanning system obtain the application use frequency, a TEMP folder, a favorite and a browser cache folder of the scanning system obtain a regular access website, deduces the type characteristics of a user to which the user belongs according to the website, the website and the cache file, deduces the habit of the user, and judges the occupation, interest and age of the user according to the type and age of equipment and the application distribution on the equipment; the method is characterized in that after a feedback scheme is received in a control state, the application is not directly finished but an iterative process is started, namely, cache prefetching service is configured according to preliminary indication of a server, a control device starts to count tracking state information, effect feedback or user satisfaction is combined to feed back a cloud again after a period of time, the cloud analyzes and gives a correction or a second preferred scheme again after receiving data and feedback, iteration is repeated until basic perfection is achieved, the final scheme is stored in the cloud, the cloud takes the application, hardware type, cache equipment type and user characteristic as indexes, the final optimized scheme is added to a database, and partial optimization process information or all optimization histories can be selectively recorded in the database, namely, all parts of equipment hardware are identified and modeled in the method flow, virtualization and cache creation, application data and user type data acquisition and data acquisition Application data, user type data and equipment model data are transmitted to a cloud end, initial optimization attempt of cloud computing is performed, feedback collection is transmitted to the cloud end, secondary correction feedback of the cloud end is performed, the process is repeated for multiple times until the feedback is close to perfect and recorded, and large data are formed, optimized and accumulated continuously through the cloud end.

2. A method as claimed in claim 1, wherein the initially allocated memory virtualisation cache is allocated according to a minimum size and is subsequently resized according to server instructions during the optimisation process.

3. A method according to claim 1, wherein the control devices themselves act as new service nodes, such that the service capacity of the network of acceleration devices increases with the number of accelerated devices, including the control devices providing network or acceleration services such as cdn cache, short-range network cache, VPN service, SMTP service to other peripheral users in accordance with the instructions of the cloud server.

4. A method as claimed in claim 1, wherein the device itself also has external solid state hardware with USB or Wigig connection, and provides additional cache pre-fetching hardware as cache offload when the device is operating, and the external hardware provides the required storage and network component support when the service node mode is turned on, i.e. adopts a three-level architecture of cooperative work and self-expansion in architecture: the cache pre-fetching control device of a large number of terminals is used as a first-level acceleration device and is responsible for terminal service and contribution of big data, the cloud server is used as a second-level acceleration device and is responsible for optimization scheme calculation and database index iteration, and external solid-state hardware of a USB interface or a Wigig interface carried by the control device is used as a third-level acceleration device and is responsible for providing cache shunting and hardware support for the control device and expanding accelerated equipment into a new service node in a node mode.