CN111782687A - Data retrieval system and method - Google Patents

Data retrieval system and method Download PDF

Info

Publication number
CN111782687A
CN111782687A CN202010431511.4A CN202010431511A CN111782687A CN 111782687 A CN111782687 A CN 111782687A CN 202010431511 A CN202010431511 A CN 202010431511A CN 111782687 A CN111782687 A CN 111782687A
Authority
CN
China
Prior art keywords
data
server
type
retrieval
request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010431511.4A
Other languages
Chinese (zh)
Inventor
曹磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Pierbulaini Software Co ltd
Original Assignee
Beijing Pierbulaini Software Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Pierbulaini Software Co ltd filed Critical Beijing Pierbulaini Software Co ltd
Priority to CN202010431511.4A priority Critical patent/CN111782687A/en
Publication of CN111782687A publication Critical patent/CN111782687A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2453Query optimisation
    • G06F16/24534Query rewriting; Transformation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/254Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • G06F16/285Clustering or classification

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data retrieval system and a method, wherein the data retrieval method is suitable for running in the data retrieval system and comprises the following steps: collecting and storing a first type of data at a data server; a data processing module in the control server selects first type data from the data server according to a preset rule to convert the first type data into second type data, and stores the second type data into a retrieval server; the interface module receives a data retrieval request from a client; the data query module analyzes the data retrieval request to determine whether to acquire corresponding first type data from the data server as request data or retrieve corresponding second type data from the retrieval server as request data, and sends the request data to the client through the interface module. The invention also discloses a computing device and a computer readable storage medium.

Description

Data retrieval system and method
Technical Field
The present invention relates to the field of data services, and in particular, to a data retrieval system and method.
Background
With the development of social life and computer technology, data information is becoming more massive, and it is becoming more important to store data information well and to find needed data from it quickly. When accessing data in the database, the data can be searched in a manner that an interface directly accesses the database, specifically, a query statement is written to implement a complex query logic to obtain required data, or the required data is screened out in a memory according to a self-defined logic algorithm, and the access amount of the database is reduced through a distributed cache service system.
However, such a way of accessing and querying data is not suitable for use in a system with a large amount of data or a complex service scenario, and in the case of the complex service scenario, the performance of the server is greatly affected. When data output by the interface can be obtained after a series of complex algorithms, the data required by the front end is calculated in real time every time the interface is called, so that a large amount of computer resources are consumed, the interface query is slower and slower, and server overload abnormity can occur if the data is not processed in time.
When the cache server is abnormal, the database server is most likely to be down directly. When the cache server or the network of the machine room where the cache server is located has a problem, the interface server is not connected with the cache server, so that a large amount of requests can return to the source to cause instantaneous index increase of the database pressure, and even more, the database server can be crushed.
And when the method is adopted, the data can be updated in time, and the synchronization of the newly added data can be delayed, so that the display delay on the page can be caused, and the problem of actual service can be caused in certain scenes with higher requirements on the real-time performance of the data.
To this end, a new data retrieval system and method is needed.
Disclosure of Invention
To this end, the present invention provides a data retrieval system and method in an attempt to solve, or at least alleviate, the problems presented above.
According to an aspect of the present invention, there is provided a data retrieval system, connected to a plurality of clients, and adapted to provide data retrieval services to the clients, the data retrieval system comprising: a data server adapted to store a first type of data; a retrieval server adapted to store a second type of data, the second type of data being generated based on the first type of data; and the control server is in communication connection with the database server and the retrieval server and comprises: the data processing module is suitable for selecting first type data from the database server according to a preset rule, converting the first type data into second type data and storing the second type data into the retrieval server; the interface module is suitable for receiving a data retrieval request from a client; and the data query module is suitable for analyzing the data retrieval request to determine whether corresponding first type data is acquired from the data server as request data or corresponding second type data is retrieved from the retrieval server as the request data, and the request data is sent to the client through the interface module.
Optionally, in the system according to the present invention, the data query module is further adapted to, after retrieving the corresponding second type of data from the retrieval server, obtain data corresponding to the second type from the data server, and combine the second type of data and the obtained data to form the request data.
Optionally, in the system according to the present invention, the data processing module is further adapted to classify the first type of data; and selecting data from the first type data of different classes as second data according to the user-defined service logic, and establishing an index for the second type data.
Optionally, in the system according to the present invention, the control server further includes: the data notification module is suitable for receiving a data update notification from the data server; and the data processing module is also suitable for converting the changed first type data into second type data according to the data updating notice and storing the second type data into the retrieval server.
Optionally, in the system according to the present invention, the data processing module is further adapted to convert all the first type data generated during the preset update interval time or the preset update interval number into the second type data according to the preset update interval time or the preset update interval number, and store the second type data in the retrieval server.
Optionally, in the system according to the present invention, a cache server is further included, and the cache server is connected in communication with the control server, and is adapted to store the request data; the data processing module is also suitable for storing the request data sent to the client in the cache server; the data query module is also suitable for analyzing the data retrieval request to determine whether the requested data is suitable for caching, and when the requested data is determined to be suitable for caching, retrieving the requested data corresponding to the retrieval request from the caching server and sending the requested data to the client.
Optionally, in the system according to the present invention, the control server further includes a local cache module, the local cache module being adapted to store the request data; the data processing module is also suitable for storing the request data sent to the client into the local cache module; the data query module is also suitable for retrieving the request data corresponding to the retrieval request from the local cache module and sending the request data to the client.
Optionally, in the system according to the present invention, a configuration server, communicatively connected to the control server, is further included, and is adapted to store the configuration information, and the data processing module is further adapted to determine whether to store the request data in the cache server or the local cache module according to the configuration information; the data query module is further suitable for retrieving data corresponding to the retrieval request from the cache server or the local cache module according to the configuration information and sending the data to the client.
Optionally, in the system according to the present invention, the control server further includes a monitoring module, the monitoring module is adapted to monitor the data processing module, when the monitoring module monitors that the data processing module cannot store the request data in the cache server, the data processing module caches the request data in the local cache module, and the configuration server modifies the configuration information.
Optionally, in the system according to the present invention, the monitoring module is further adapted to monitor that an alarm message is generated when the data processing module fails to store the second type of data in the retrieval server more than a predetermined number of times.
Optionally, in the system according to the present invention, the data query module is further adapted to, when it is determined that the requested data is suitable for caching and the requested data is not stored in the cache server, analyze the data retrieval request, obtain or retrieve the requested data from the data server, and send the requested data to the client through the interface module; and the data processing module is also suitable for determining to store the request data in the cache server or the local cache module according to the configuration information.
According to another aspect of the present invention, there is provided a data retrieval method adapted to operate in a data retrieval system, the system being connected to one or more clients, the method comprising the steps of: collecting and storing a first type of data at a data server; a data processing module in the control server selects first type data from the data server according to a preset rule to convert the first type data into second type data, and stores the second type data into the retrieval server; the interface module receives a data retrieval request from a client; the data query module analyzes the data retrieval request to determine whether corresponding first type data is acquired from the data server as request data or corresponding second type data is retrieved from the retrieval server as request data, and the request data is sent to the client through the interface module.
Optionally, in the method according to the present invention, the data processing module selecting the first type of data to be converted into the second type of data from the data server according to a predetermined rule includes: classifying the first type of data; and selecting data from the first type data of different types as second data according to the custom service logic, and establishing an index for the second type data.
Optionally, in the method according to the present invention, further comprising: and after the data query module retrieves the corresponding second type data from the retrieval server, acquiring the data corresponding to the second type data from the data server, and combining the second type data and the acquired data to form the request data.
Optionally, in the method according to the present invention, further comprising: the data notification module receives a notification of data update from the data server; and the data processing module converts the changed first type data into second type data according to the notification of data updating and stores the second type data into the retrieval server.
Optionally, in the method according to the present invention, further comprising: the data processing module converts all first type data generated during the preset updating interval time or the preset updating interval quantity into second type data according to the preset updating interval time or the preset updating interval quantity, and stores the second type data into the retrieval server.
Optionally, in the method according to the present invention, further comprising: the data processing module stores the request data sent to the client in a cache server; and the data query module analyzes the data retrieval request to determine whether the requested data is suitable for caching, and retrieves the request data corresponding to the retrieval request from the caching server and sends the request data to the client when the request data is determined to be suitable for caching.
Optionally, in the method according to the present invention, further comprising: the data processing module stores the request data sent to the client in a local cache module; and the data query module retrieves the request data corresponding to the retrieval request from the local cache module and sends the request data to the client.
Optionally, in the method according to the present invention, further comprising: the data processing module determines whether to store the request data in the cache server or the local cache module according to the configuration information; and the data query module retrieves the data corresponding to the retrieval request from the cache server or the local cache module according to the configuration information and sends the data to the client.
Optionally, in the method according to the present invention, further comprising: the monitoring module monitors the data processing module; when the monitoring module monitors that the data processing module cannot store the request data in the cache server, the data processing module caches the request data in the local cache module, and the configuration server modifies the configuration information.
Optionally, in the method according to the present invention, further comprising: and the monitoring module generates alarm information when the data processing module monitors that the second type data cannot be stored in the retrieval server for more than the preset times.
Optionally, in the method according to the present invention, further comprising: the data query module determines that the request data are suitable for caching, when the request data are not stored in the cache server, the data retrieval request is analyzed, the request data are obtained from the data server or retrieved from the retrieval server, and the request data are sent to the client through the interface module; and the data processing module determines to store the request data in the cache server or the local cache module according to the configuration information.
According to yet another aspect of the present invention, there is provided a computing device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods of a data retrieval method according to the present invention.
According to a further aspect of the present invention there is provided a computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform any of a data retrieval method according to the present invention.
The data processing module selects the first type data from the data server according to a preset rule to convert the first type data into the second type data and stores the second type data into the retrieval server by collecting and storing the first type data in the data server. The data query module analyzes the data retrieval request to determine whether to acquire corresponding first type data from the data server as request data or retrieve corresponding second type data from the retrieval server as request data, and sends the request data to the client through the interface module. Therefore, data can be retrieved in a plurality of servers, the data retrieval efficiency is improved, the pressure of a data server is shared, the stability of the whole system is improved, and the first type data is further classified when the first type data is selected from the data server to be converted into the second type data according to the preset rule; one data is selected from the first type data of different types as a second data according to the self-defined service logic, and an index is established for the second type data, so that the data retrieval speed in the retrieval server is higher than that in the data server, and the data retrieval efficiency is further improved.
Secondly, after the data query module retrieves the corresponding second type data from the retrieval server, the second type data is supplemented to serve as request data, and the request data is sent to the client through the interface module, so that the integrity of the request data is protected while the request data is rapidly retrieved.
And thirdly, when the data in the data server changes, the data server sends a data change notification to the data notification module in time, the data processing module converts the changed first type data into second type data according to the data update notification, stores the second type data into the retrieval server, and updates the changed data in the retrieval server in time, so that the accuracy and timeliness of the stored data are ensured, and the problem of actual service caused by data passing is avoided. Meanwhile, when the requirement of the changed data on the real-time performance is not high, frequent change is not needed, and under the condition that the actual service is not influenced, the updating interval time or the number of the preset updating intervals is preset, the data processing module converts all the first type data generated during the time period of the preset updating intervals or the number of the preset updating intervals into the second type data according to the preset updating interval time or the number of the preset updating intervals, and stores the second type data into the retrieval server, so that the changed data can be updated in batches, the pressure of the updated data of the system is reduced, and the consumption of computer resources is reduced.
Meanwhile, the data processing module converts the data of the first type of data into request data and stores the request data in the cache server; and the data query module analyzes the data retrieval request to determine whether the requested data is suitable for caching, and retrieves the request data corresponding to the retrieval request from the caching server and sends the request data to the client when the request data is determined to be suitable for caching. The data processing module caches the request data retrieved by each data retrieval request, and can directly return the cached data from the cache without acquiring the request data from the data server or the retrieval server when receiving the same data retrieval request for the request data at the next time. And the requested data is cached in the cache server or the local cache module, and when the data retrieval request hits the cache, the data is returned from the cache server or the local cache module, so that the retrieval and the reading of the cache are faster than those of the server, and the data retrieval speed is further improved.
And further, the monitoring module monitors the data processing module, when the monitoring module monitors that the data processing module cannot store the request data in the cache server, the data processing module caches the request data in the local cache module, and the configuration server modifies the configuration information, so that when the cache server fails and normally provides services, the local cache module is quickly started, seamless switching between the cache server and the services provided by the local cache service is realized, the request data can be stored or read from one cache, and the stability of the whole system is improved.
Drawings
To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which are indicative of various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to be within the scope of the claimed subject matter. The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description read in conjunction with the accompanying drawings. Throughout this disclosure, like reference numerals generally refer to like parts or elements.
FIG. 1 shows a schematic diagram 100 of a data retrieval system according to an exemplary embodiment of the present invention;
FIG. 2 illustrates a block diagram of a computing device 200, according to an exemplary embodiment of the invention;
FIG. 3 shows a flow diagram of a data retrieval method 300 according to one embodiment of the invention; and
FIG. 4 shows a flow diagram of a data retrieval process 400 according to one embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Like reference numerals generally refer to like parts or elements.
FIG. 1 shows a schematic diagram 100 of a data retrieval system according to an exemplary embodiment of the present invention. As shown in FIG. 1, the data retrieval system 100 includes a control server 110, a data server 120, a retrieval server 130, a cache server 140 and a configuration server 150, and is connected to clients 161-163. The manner in which data retrieval system 100 is connected to clients 161-163 is shown for exemplary purposes only, and in a particular practical situation, data retrieval system 100 may be connected to a plurality of clients 161-163, and the invention is not limited to the number of clients to which data retrieval system 100 is linked. And the data retrieval system 100 may connect different types of clients, as shown in fig. 1, the linked clients 161 and 163 of the data retrieval system 100 may be implemented as one type of computing device, and the linked client 162 may be implemented as one type of mobile terminal. The clients 161-163 linked to the data retrieval system 100 in fig. 1 are only exemplary, and in a specific implementation process, the types of the clients linked to the data retrieval system 100 may be different types of clients, and the types of the clients 161-163 linked to the data retrieval system 100 are not limited in the present invention.
In data retrieval system 100, control server 110 is communicatively coupled to data server 120, retrieval server 130, cache server 140, and configuration server 150. The control server 110 includes a data processing module 111, an interface module 112, a data query module 113, a data notification module 114, a local cache module 115, and a monitoring module 116. The data server 120 stores the first type of data. The data server 120 may be implemented as any database capable of storing the first type of data, and the present invention is not limited to the kind of the data server 120. According to one embodiment of the invention, the data Server 120 may be implemented as a Sql Server or MySql database. The first type data is data collected and stored according to business requirements, and can be divided into a plurality of categories according to business types and logics, and the category and the number of the categories of the first type data are not limited by the invention.
The retrieval server 130 stores the second type data. Search server 130 may be implemented as a search application server, and in a practical case, search server 130 may be implemented as any server capable of implementing a search function, and the present invention is not limited to the type of search server 130. The retrieval server 130 may be implemented as a search application server Solr by providing a search service interface for the data query module 113 to query the second type of data stored therein. The data query module 113 may provide a query request to the search application server Solr to obtain a returned result in an XML format returned by the Solr. The second type data is generated based on the first type data, and the second type data is selected from the first type data according to a preset rule and is converted.
According to an embodiment of the present invention, the first type data includes all vehicle models of the dealer, all vehicle systems of the dealer, all channel offer sets of a certain model of vehicle, and the vehicle models, vehicle systems, and offer sets do not belong to the same type according to the service type, and thus may be classified into data of the vehicle models, data of the vehicle systems, and data of prices by the data processing module 111. Taking the price data as an example, the price data includes quoted prices of various channels: promotional prices, dealer quotes, manufacturer-directed prices, etc. Because according to the specific business logic, when the vehicle is purchased, the promotion price of the vehicle is preferentially referred to, then the dealer quotes, and finally the manufacturer instruction price. Thus, the vehicle promotional price that has the most value to reference is taken as the second type of data and will be stored and indexed in search server 130 for subsequent queries.
The data processing module 111 selects a first type of data from the database server according to a predetermined rule and converts the first type of data into a second type of data, and stores the second type of data in the search server 130. The data processing module 111 is further arranged to convert the first type of data in the data server 120 into the second type of data and to store the frequency of global and incremental conversions in the retrieval server 130. The global transformation refers to transforming all the first type data in the data server 120 into the second type data, and the incremental transformation refers to transforming the newly added first type data in the data server 120 into the second type data, so as to perform global or incremental transformation and store on the first type data in the data server 120. The data processing module 111 also classifies the first type of data; and selecting data from the first type data of different classes as second data according to the custom service logic, and establishing an index for the second type data. Thereby realizing a faster speed in retrieving data in the retrieval server 130 than in the data server 120 and further improving the efficiency of retrieving data. Index server 130 also stores an index of the second type of data.
The interface module 112 receives data retrieval requests from the clients 161 to 163, and transmits request data retrieved according to the data retrieval requests to the clients 161 to 163.
The data query module 113 analyzes the data retrieval request received by the interface module 112 to determine whether to obtain corresponding first type data from the data server 120 as request data or retrieve corresponding second type data from the retrieval server 130 as request data, so as to obtain data from different servers according to different types of data retrieval requests, improve the efficiency of data retrieval, share the pressure of the data server 120, and improve the stability of the whole system. The data query module 113 also retrieves the corresponding second type data from the retrieval server 130, supplements the second type data as request data, and sends the request data to the clients 161-163 through the interface module 112, thereby ensuring that the integrity of the request data is protected while the request data is retrieved quickly.
According to an embodiment of the present invention, when the data retrieval request requests to retrieve the quote of the dealer of a certain vehicle, the data query module 113 analyzes the data retrieval request, and when the analyzed quote of the dealer of the vehicle is not suitable for being converted into the second type of data, the data query module 113 directly obtains the corresponding data from the data server 120 and returns the corresponding data to the client. When the data retrieval request requests to retrieve the promotional price of a certain vehicle, the data query module 113 analyzes the promotional price to obtain that the dealer quote price of the vehicle is suitable to be converted into the second type data, and retrieves the corresponding data from the retrieval server 130 and returns the data to the client. When the data retrieval request requests to retrieve all quotes of a certain vehicle, the data query module analyzes the quotes to obtain that the promotion and sale prices of the vehicles in the data requested by the data retrieval request are suitable for conversion and are used as second type data, corresponding data, namely the promotion and sale prices of the vehicles, are obtained from the retrieval server 130, the retrieved data are judged, and whether the retrieved data are used as request data are complete or not is judged. If the retrieved data is incomplete, the data server 120 continues to retrieve other data related to the vehicle promotional price through the vehicle promotional price, namely, the vehicle promotional price is used as an ID for retrieving other data, the dealer quote and the manufacturer guide price of the type of automobile are retrieved, and the dealer quote, the manufacturer guide price and the vehicle promotional price of the type of automobile are combined together to be sent to the client as all quotes of the vehicle.
The data notification module 114 sends a notification of data update to the control server 110 when the stored first type data changes. A data notification module 114 that receives a data update notification from the data server 120; and the data processing module 111 also converts the changed first type data into second type data according to the data update notification, and stores the second type data in the search server 130. The changed data can be timely updated in the retrieval server 130, the accuracy and timeliness of the stored data are guaranteed, and the problem of actual business caused by data passing is avoided. Furthermore, when the changed data has low requirement on real-time performance and does not need to be frequently changed, and under the condition that actual services are not affected, the update interval time or the preset update interval number is preset for the data processing module 111, and the data processing module 111 converts all the first type data generated during the preset update interval time or the preset update interval number into the second type data according to the preset update interval time or the preset update interval number, and stores the second type data in the retrieval server 130, so that the changed data can be updated in batches, the pressure of the updated data of the system is reduced, and the consumption of computer resources is reduced.
According to an embodiment of the present invention, when the vehicle type sales promotion price, the dealer quote and the manufacturer guide price stored in the data server 120 change, the vehicle type sales promotion price changes, and the change of the vehicle type sales promotion price has a great influence on the decision of whether the user purchases the vehicle, so the data server 120 timely sends the data processing module 111 a notification of the vehicle type sales promotion price data change, and the data processing module timely processes the notification of the data change, and correspondingly changes the second type of data stored in the search server, that is, the sales promotion price of the vehicle type.
When the prizes of a future holiday activity in the data server 120 change in batch, corresponding data in the retrieval server 130 need to be updated, update interval time or update interval data can be set for the type of data, when a preset update interval event is reached, first type data of the type change in the preset update time is converted into second type data after being updated in batch and stored to finish updating the data, or according to the preset interval data, when the changed number of the type data reaches 50 or 100 of a preset value, the 50 or 100 changed first type data are converted into the second type data and stored.
Cache server 140 stores the requested data; the data processing module 111 also converts the data of the first type data into request data, and stores the request data in the cache server 140; the data query module 113 further analyzes the data retrieval request to determine whether the requested data is suitable for caching, and when it is determined that the requested data is suitable for caching, retrieves the requested data corresponding to the retrieval request from the caching server 140 and sends the retrieved requested data to the clients 161-163.
The control server 110 further includes a local cache module 115, where the local cache module 115 stores the request data; the data processing module 111 also converts the data of the first type data into request data, and stores the request data in the local cache module 115; the data query module 113 further retrieves the request data corresponding to the retrieval request from the local cache module 115 and sends the request data to the clients 161-163. The local cache module 115 may provide the same function as the cache server 140 when the cache server 140 fails, so as to improve the fault tolerance of the data retrieval system 100 during the service process. The data processing module 111 caches the requested data retrieved for each data retrieval request, and when receiving the same data retrieval request for the requested data next time, can return the cached data from the cache without acquiring the requested data from the data server 120 or the retrieval server 130. Moreover, by caching the requested data in the cache server 140 or the local cache module 115, when the data retrieval request hits the cache, the data is returned from the cache server 140 or the local cache module 115, which is faster than retrieving and reading the cache from the server, thereby further improving the speed of data retrieval.
The configuration server 150 stores configuration information, and the data processing module 111 further determines whether to store the requested data in the cache server 140 or the local cache module 115 according to the configuration information; the data query module 113 further retrieves data corresponding to the retrieval request from the cache server 140 or the local cache module 115 according to the configuration information and sends the data to the clients 161-163. The configuration server 150 controls the locations where the data processing module 111 stores the requested data and the data query module 113 retrieves the requested data by modifying the configuration information, so as to modify the default locations for caching and retrieving the requested data, thereby providing uninterrupted requested data caching and retrieving service.
The monitoring module 116 monitors the data processing module 111; when the monitoring module 116 monitors that the data processing module 111 cannot store the request data in the cache server 140, the data processing module 111 caches the request data in the local cache module 115, and the configuration server 150 modifies the configuration information, so that when the cache server 140 fails and normally provides services, the local cache module 115 is quickly enabled, seamless switching between services provided by the cache server 140 and the local cache service is realized, it is ensured that the request data can be stored or read from one cache, and the stability of the whole system is improved.
The monitoring module 116 also monitors that the data processing module 111 cannot store the second type data in the search server 130 more than a predetermined number of times, and generates alarm information. The manager can find the alarm information in time, search and remove the fault of the retrieval server 130, and recover the corresponding function of the retrieval server 130 as soon as possible, so as to provide the service of storing and retrieving the second type of data.
The data query module 113 further analyzes the data retrieval request when it is determined that the requested data is suitable for caching and the requested data is not stored in the cache server 140, retrieves the requested data from the data server 120 or the retrieval server 130, and sends the requested data to the clients 161-163 through the interface module 112; and the data processing module 111 also determines to store the requested data in the cache server 140 or the local cache module 115 according to the configuration information. Therefore, when the cached request data is not cached in the cache server 140, the cached data is returned in time, and the request data is stored correspondingly, so that the influence on providing normal data retrieval service due to the problem of cache efficiency is avoided, and the stability and fault tolerance of the service are improved.
According to an embodiment of the present invention, after the second type of data stored in the data retrieval server 130 and the promotional price of a certain vehicle model are retrieved in advance, the data processing module determines that it is suitable for caching, and then caches it to the cache server 140. When the interface module 112 receives the data retrieval request of the sales promotion price of the vehicle type and judges that the data retrieval request is suitable for caching, the data piece is retrieved from the cache server 140 directly. If the cache server 140 fails, corresponding data is obtained from the local cache module 115 in the control server 110 according to the configuration information stored in the configuration server 150 and returned to the client.
If the cache server 140 is good and can provide the corresponding data retrieval service, the data is retrieved from the cache server 140, and if the cache server 140 is judged that the promotion price of the vehicle type is not yet cached, the corresponding data is still retrieved from the data server 120 or the retrieval server 130 and returned to the client.
The control server 110, the data server 120, the search server 130, the cache server 140, and the configuration server 150 in the data search system 100 may each be embodied as a computing device. FIG. 2 illustrates a block diagram of a computing device 200, according to an exemplary embodiment of the invention. As shown in FIG. 2, in a basic configuration 202, a computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 may be used for communicating between the processor 204 and the system memory 206.
Depending on the desired configuration, the processor 204 may be any type of processing, including but not limited to: a micro-processor (μ P), a micro-controller (μ C), a Digital Signal Processor (DSP), or any combination thereof. The processor 204 may include one or more levels of cache, such as a level one cache 210 and a level two cache 212, a processor core 214, and registers 216. Example processor cores 214 may include Arithmetic Logic Units (ALUs), Floating Point Units (FPUs), digital signal processing cores (DSP cores), or any combination thereof. The example memory controller 218 may be used with the processor 204, or in some implementations the memory controller 218 may be an internal part of the processor 204.
Depending on the desired configuration, system memory 206 may be any type of memory including, but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 206 may include an operating system 220, one or more programs 222, and program data 224. In some implementations, the program 222 can be arranged to execute instructions on the operating system with the program data 224 by the one or more processors 204.
Computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to the basic configuration 202 via the bus/interface controller 230. The example output device 242 includes a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 252. Example peripheral interfaces 244 can include a serial interface controller 254 and a parallel interface controller 256, which can be configured to facilitate communications with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I/O ports 258. The example communication device 246 may include a network controller 260, which may be arranged to facilitate communications with one or more other computing devices 262 over a network communication link via one or more communication ports 264.
A network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media, in a modulated data signal, such as a carrier wave or other transport mechanism. A "modulated data signal" may be a signal that has one or more of its data set or its changes made in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or private-wired network, and various wireless media such as acoustic, Radio Frequency (RF), microwave, Infrared (IR), or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
In computing device 200 according to the present invention, application 222 comprises program instructions that perform a data retrieval method 300 that can direct processor 204 to perform some of the steps of a data retrieval method 300 of the present invention that operates in data retrieval system 100, such that the various parts of data retrieval system 100 enable the provision of data retrieval services to clients 161-163 by performing a data retrieval method 300 of the present invention.
Computing device 200 may be implemented as a server, e.g., a file server, a database server, an application server, etc., which may be, for example, a Personal Digital Assistant (PDA), a wireless web-browsing device, an application-specific device, or a hybrid device that include any of the above functions. May be implemented as a personal computer including both desktop and notebook computer configurations, and in some embodiments, computing device 200 is configured to perform a data retrieval method 300.
FIG. 3 shows a flow diagram of a data retrieval method 300 according to one embodiment of the invention. As shown in fig. 3, a data retrieval method 300 begins at step S310 by collecting and storing a first type of data at the data server 120. The collection and storage of the first type of data in the data server 120 may be performed by computer or by importing from other databases, servers, or by manual entry and modification, and the present invention is not limited to the manner of collecting and storing the first type of data.
Subsequently, step S320 is performed, and the data processing module 111 in the control server 110 selects the first type data from the data server 120 to convert the first type data into the second type data according to a predetermined rule, and stores the second type data in the search server 130. The step of converting the first type of data into the second type of data comprises: classifying the first type of data; selecting one data from the first type data of different types as a second data according to the user-defined service logic, and establishing an index for the second type data. The first type data can be classified into a plurality of types of data according to the service types, the output processing module processes each type of data, selects data from the first type data of each type as second type data, and establishes an index for the selected second type data so as to be retrieved subsequently. When the first type data is converted into the second type data, whether the data type of the first type data is suitable for being converted into the second type data is judged, and if not, the data type of the first type data is not converted.
According to an embodiment of the present invention, the first type data includes all vehicle models of the dealer, all vehicle systems of the dealer, all channel offer sets of a certain model of vehicle, and the vehicle models, vehicle systems, and offer sets do not belong to the same type according to the service type, and thus may be classified into data of the vehicle models, data of the vehicle systems, and data of prices by the data processing module 111. Taking the price data as an example, the price data includes quoted prices of various channels: promotional prices, dealer quotes, manufacturer-directed prices, etc. Because according to the specific business logic, when the vehicle is purchased, the promotion price of the vehicle is preferentially referred to, then the dealer quotes, and finally the manufacturer instruction price. Therefore, the vehicle promotion price having the most reference value is taken as the second type data, and is stored in the search server 130 and indexed for subsequent query.
According to one embodiment of the invention, the frequency of the global translation and the incremental translation is set in the data server 120, translated into the second type of data, and stored in the search server 130. The global transformation refers to transforming all the first type data in the data server 120 into the second type data, and the incremental transformation refers to transforming the newly added first type data in the data server 120 into the second type data, so as to perform global or incremental transformation and storage on the first type data in the data server 120. The first type data is converted into the second type data in batches, so that the data conversion is convenient to manage, and the data retrieval efficiency is improved finally.
Subsequently, step S330 is executed, and the interface module 112 receives the data retrieval request from the client 161-163. In the following, the client 161 is taken as an example, and sends a data retrieval request to the data retrieval system 100, and receives the returned request data, and a data retrieval method of the present invention is specifically described. The client 161 sends a data retrieval request for a certain item of data to the data retrieval system 100, and the interface module 112 in the control server 110 receives the data retrieval request, so that other modules of the data retrieval system 100 cooperate to perform corresponding processing.
Subsequently, step S340 is executed, the data query module 113 analyzes the data retrieval request to determine whether to obtain the corresponding first type data from the data server 120 as the request data or retrieve the corresponding second type data from the retrieval server 130 as the request data, and sends the request data to the client 161 through the interface module 112. The data requested by the data retrieval request can be retrieved from the retrieval server 130 or can only be obtained from the data server 120, so that the data query module 113 analyzes the data to obtain the data from the corresponding position of the data and returns the data.
Furthermore, when the data query module 113 analyzes the data retrieval request, it follows the service logic of the data and the storage rule of the data retrieval system 100 of the present invention. First, it is determined whether the requested data type is suitable for conversion into the second type of data to be stored in the search server 130, and if the requested data type is not suitable for conversion, the search is directly performed in the data server 120. Next, if the data type is suitable for conversion, it is determined whether the data is suitable as the second type data in the type data, and if the data is suitable as the second type data, the search is performed in the search server 130, and if the data is not suitable as the second type data, the data is acquired from the data server 120. The retrieved request data is eventually returned to client 161 through interface module 112.
According to another embodiment of the present invention, the data retrieving method further includes the data query module 113 retrieving the corresponding second type data from the retrieval server 130, supplementing the second type data as request data, and sending the request data to the client 161 through the interface module 112. When the second type data queried from the search server 130 is not yet complete as the request data, the data query module 113 obtains the rest of the corresponding data from the data server 120, completes the request data, and combines the complete request data with the second type data to be sent to the client 161 as the request data. For example, when the client 161 sends a data search request to acquire all information of a certain vehicle type, the information of the certain vehicle type is stored as the first type data in the data server 120, and the id of the certain vehicle type is stored as the second type data in the search server 130. After the interface module 112 receives the data retrieval request, the data query module 113 retrieves the ID of the vehicle type from the retrieval server 130, and then retrieves and acquires all information of the vehicle type from the data server 120 according to the ID of the vehicle type and returns the information as request data to the client 161.
According to yet another embodiment of the present invention, the data retrieval method further comprises the data notification module 114 receiving a notification of data update from the data server 120; the data processing module 111 converts the changed first type data into the second type data according to the notification of the data update, and stores the second type data in the retrieval server 130. Since the data stored in the data server 120 will change with time according to market, specific service and other reasons in a specific practical situation, the second type of data obtained according to the first type of data needs to be updated accordingly, so as to meet the requirements of data query on data accuracy and timeliness. After the first type data stored in the data server 120 is changed, a data update notification of the corresponding data is sent to the control server 110, so that the changed data can be updated in the search server 130 in time, the accuracy and timeliness of the stored data are ensured, and the problem of actual service caused by outdated data is avoided.
Further, when the changed data has low requirement on real-time performance, does not need to be frequently changed, and does not affect the actual service, the data retrieval method further includes presetting an update interval time or a preset update interval number for the data processing module 111, and the data processing module 111 converts all the first type data generated during the preset update interval time or the preset update interval number into the second type data according to the preset update interval time or the preset update interval number, and stores the second type data in the retrieval server 130, so that the changed data can be updated in batches, the pressure of the updated data of the system is reduced, and the consumption of computer resources is reduced. For example, when a vehicle is purchased, if the real-time sale promotion price of the vehicle in the retrieved data changes, the sale promotion price of the vehicle as the second type data is updated in time. Under other conditions, for example, when the specific rules of the scheduled promotion activities at a certain festival are changed in the future, the updating interval time can be set, so that the business requirements can be met by updating within a certain time under the condition that the actual business is not influenced.
According to another embodiment of the present invention, the data retrieving method further comprises the data processing module 111 storing the request data sent to the client 161 in the cache server 140; the data query module 113 analyzes the data retrieval request to determine whether the requested data is suitable for caching, and when it is determined that the requested data is suitable for caching, retrieves the requested data corresponding to the retrieval request from the caching server 140 and sends the retrieved requested data to the client 161.
According to another embodiment of the present invention, the data retrieving method further includes the data processing module 111 storing the request data sent to the client 161 in the local cache module 115; the data query module 113 retrieves the request data corresponding to the retrieval request from the local cache module 115 and sends the request data to the client 161.
After the data retrieval system 100 obtains the corresponding request data for the data retrieval request, the data processing module 111 may determine whether the obtained request data is suitable for storage, so that when the same data retrieval request for the request data is received next time, the data processing module can directly return the cache data from the cache without obtaining the request data from the data server 120 or the retrieval server 130. And, by caching the requested data in the cache server 140 or the local cache module 115, when the data retrieval request hits the cache, the data is returned from the cache server 140 or the local cache module 115, which is faster than retrieving and reading the cache from the server, thereby further increasing the speed of data retrieval. When the data processing module 111 determines that the request is not suitable for storage, for example, the request data changes frequently, the request data is not cached.
According to another embodiment of the present invention, the data retrieving method further comprises the data processing module 111 determining whether to store the requested data in the cache server 140 or the local cache module 115 according to the configuration information; the data query module 113 retrieves data corresponding to the retrieval request from the cache server 140 or the local cache module 115 according to the configuration information and sends the data to the client 161. The configuration server 150 stores the configuration information, and the configuration server 150 controls the data processing module 111 to store the request data and the data query module 113 to retrieve the request data by modifying the configuration information, so as to modify the default cache and retrieve the location of the request data, so as to provide uninterrupted request data cache and retrieval service. The configuration information comprises a switch for controlling the positions of the cache and the retrieval request data, and the modification of the corresponding configuration is completed by modifying the parameter value.
According to another embodiment of the present invention, the data retrieving method further comprises the monitoring module 116 monitoring the data processing module 111; when the monitoring module 116 monitors that the data processing module 111 cannot store the request data in the cache server 140, the data processing module 111 caches the request data in the local cache module 115, and the configuration server 150 modifies the configuration information, so that when the cache server 140 fails and normally provides services, the local cache module 115 is quickly enabled, seamless switching between services provided by the cache server 140 and the local cache service is realized, it is ensured that the request data can be stored or read from one cache, and the stability of the whole system is improved.
According to another embodiment of the present invention, the data retrieving method further includes the step of generating an alarm message by the monitoring module 116 when the data processing module 111 cannot store the second type data in the retrieval server 130 more than a predetermined number of times. The invention does not limit the size of the preset times, and managers can modify the preset times according to specific practical conditions. The manager can find the alarm information in time, search and remove the fault of the retrieval server 130, and recover the corresponding function of the retrieval server 130 as soon as possible, so as to provide the service of storing and retrieving the second type of data.
According to another embodiment of the present invention, the data retrieving method further includes the data querying module 113 determining that the requested data is cached, when the requested data is not stored in the cache server 140, analyzing the data retrieving request, retrieving the requested data from the data server 120 or the retrieval server 130, and sending the requested data to the client 161 through the interface module 112; and the data processing module 111 determines to store the requested data in the cache server 140 or the local cache module 115 according to the configuration information. Therefore, when the cached request data is not cached in the cache server 140, the cached data is returned in time, and the request data is stored correspondingly, so that the influence on providing normal data retrieval service due to the problem of cache efficiency is avoided, and the stability and fault tolerance of the service are improved.
FIG. 4 shows a flow diagram of a data retrieval process 400 according to one embodiment of the invention. As shown in fig. 4, the data retrieval process 400 is suitable for step S410, taking the client 161 as an example, the client 161 sends a data retrieval request to the data retrieval system 100, and the interface module 112 receives the data retrieval request. Subsequently, in step S420, the data retrieving module 113 analyzes the data retrieving request, and analyzes whether the requested data is suitable for caching, if the requested data is suitable for caching, step S421 is executed, otherwise, step S430 is executed. When step S421 is executed, the data query module first retrieves corresponding data from the cache server 140, and executes step S422 to determine whether the cache server fails, and if the cache server fails, executes step S423; if the cache server is normal and data can be normally obtained from the cache server, step S424 is executed. When step S423 is executed, corresponding data is acquired from the local cache module 115, and step S470 is executed after the data is acquired.
If the cache server is normal, go to step S424, continue to determine whether the cache server 140 stores the corresponding request data, and if the cache is not reached, go back to step S430; if the data processing module 111 caches the corresponding data in the cache server, step S425 is executed, the requested data is retrieved from the cache server 140, and step S470 is executed after the data is retrieved.
When the corresponding requested data is not cached in the cache server 140 or the data retrieved by the data retrieval request is not suitable for caching, step S430 is executed, whether the data of the data retrieval request is suitable for conversion is continuously analyzed, if the requested data is obtained through analysis and is not suitable for conversion, step S425 is executed, the corresponding data is directly obtained from the data server 120, and step S470 is executed after the corresponding data is obtained.
If the requested data is suitable for conversion after analysis, the data processing module 111 converts the first type data stored in the data server 120 into the second type data in advance and stores the second type data in the search server 130. The data query module executes step S440 to retrieve corresponding data from the retrieval server 130. Then, step S450 is executed to check whether the data retrieved from the retrieval server 130 is complete, the data requirement to be retrieved is requested by the composite data retrieval request, and if the retrieved data is complete, step S470 is directly executed.
If the retrieved data is not complete, step S460 is executed to obtain the corresponding data from the data server, complete the requested data, and combine the second type data retrieved from the retrieval server 130 with the requested data as the requested data. Finally, step S470 is executed to send the request data corresponding to the retrieved data retrieval request to the client 161 requesting the data through the interface module 112, thereby completing the data retrieval process.
The data processing module selects the first type data from the data server according to a preset rule to convert the first type data into the second type data and stores the second type data into the retrieval server by collecting and storing the first type data in the data server. The data query module analyzes the data retrieval request to determine whether to acquire corresponding first type data from the data server as request data or retrieve corresponding second type data from the retrieval server as request data, and sends the request data to the client through the interface module. Therefore, data can be retrieved in a plurality of servers, the data retrieval efficiency is improved, the pressure of a data server is shared, the stability of the whole system is improved, and the first type data is further classified when the first type data is selected from the data server to be converted into the second type data according to the preset rule; one data is selected from the first type data of different types as a second data according to the self-defined service logic, and an index is established for the second type data, so that the data retrieval speed in the retrieval server is higher than that in the data server, and the data retrieval efficiency is further improved.
Secondly, after the data query module retrieves the corresponding second type data from the retrieval server, the second type data is supplemented to serve as request data, and the request data is sent to the client through the interface module, so that the integrity of the request data is protected while the request data is rapidly retrieved.
And thirdly, when the data in the data server changes, the data server sends a data change notification to the data notification module in time, the data processing module converts the changed first type data into second type data according to the data update notification, stores the second type data into the retrieval server, and updates the changed data in the retrieval server in time, so that the accuracy and timeliness of the stored data are ensured, and the problem of actual service caused by data passing is avoided. Meanwhile, when the requirement of the changed data on the real-time performance is not high, frequent change is not needed, and under the condition that the actual service is not influenced, the updating interval time or the number of the preset updating intervals is preset, the data processing module converts all the first type data generated during the time period of the preset updating intervals or the number of the preset updating intervals into the second type data according to the preset updating interval time or the number of the preset updating intervals, and stores the second type data into the retrieval server, so that the changed data can be updated in batches, the pressure of the updated data of the system is reduced, and the consumption of computer resources is reduced.
Meanwhile, the data processing module converts the data of the first type of data into request data and stores the request data in the cache server; and the data query module analyzes the data retrieval request to determine whether the requested data is suitable for caching, and retrieves the request data corresponding to the retrieval request from the caching server and sends the request data to the client when the request data is determined to be suitable for caching. The data processing module caches the request data retrieved by each data retrieval request, and can directly return the cached data from the cache without acquiring the request data from the data server or the retrieval server when receiving the same data retrieval request for the request data at the next time. And the requested data is cached in the cache server or the local cache module, and when the data retrieval request hits the cache, the data is returned from the cache server or the local cache module, so that the retrieval and the reading of the cache are faster than those of the server, and the data retrieval speed is further improved.
And further, the monitoring module monitors the data processing module, when the monitoring module monitors that the data processing module cannot store the request data in the cache server, the data processing module caches the request data in the local cache module, and the configuration server modifies the configuration information, so that when the cache server fails and normally provides services, the local cache module is quickly started, seamless switching between the cache server and the services provided by the local cache service is realized, the request data can be stored or read from one cache, and the stability of the whole system is improved.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
A8, the system of A6 or A7, further comprising a configuration server communicatively connected to the control server and adapted to store configuration information, the data processing module further adapted to determine whether to store the requested data in the cache server or the local cache module according to the configuration information; the data query module is further adapted to retrieve data corresponding to the retrieval request from the cache server or the local cache module according to the configuration information and send the data to the client.
A9, the system according to a6-A8, wherein the control server further comprises a monitoring module, the monitoring module is adapted to monitor the data processing module, when the monitoring module monitors that the data processing module cannot store the requested data in the cache server, the data processing module caches the requested data in the local cache module, and the configuration server modifies the configuration information.
A10, the system of A9, wherein the monitoring module is further adapted to generate an alert message when the data processing module fails to store the second type of data in the retrieval server more than a predetermined number of times.
A11, the system according to a6-a10, wherein the data query module is further adapted to, when it is determined that the requested data is suitable for caching and the requested data is not stored in the cache server, analyze the data retrieval request, obtain or retrieve the requested data from the data server, and send the requested data to the client through the interface module; and the data processing module is further adapted to determine to store the requested data in the cache server or the local cache module according to the configuration information.
B13, the method according to B12, wherein the selecting, by the data processing module, the first type data to be converted into the second type data according to the predetermined rule from the data server includes: classifying the first type of data; selecting data from different types of first type data as second data according to the user-defined service logic, and establishing an index for the second type data.
B14, the method of B12 or B13, further comprising: and after the data query module retrieves the corresponding second type data from the retrieval server, acquiring the data corresponding to the second type data from the data server, and combining the second type data and the acquired data to form the request data.
B15, the method of B14, further comprising: the data notification module receives a notification of data update from the data server; and the data processing module converts the changed first type data into second type data according to the notification of the data updating and stores the second type data into the retrieval server.
B16, the method of B15, further comprising: the data processing module converts all first type data generated during the preset updating interval time or the preset updating interval quantity into second type data according to the preset updating interval time or the preset updating interval quantity, and stores the second type data into the retrieval server.
The method of any one of B17, B12-B16, further comprising: the data processing module stores the request data sent to the client in the cache server; and the data query module analyzes the data retrieval request to determine whether the requested data is suitable for caching, and retrieves the requested data corresponding to the retrieval request from the cache server and sends the requested data to the client when the requested data is determined to be suitable for caching.
B18, the method of B17, further comprising: the data processing module stores the request data sent to the client in the local cache module; and the data query module retrieves the request data corresponding to the retrieval request from the local cache module and sends the request data to the client.
B19, the method of B18, further comprising: the data processing module determines whether the request data is stored in the cache server or the local cache module according to the configuration information; and the data query module retrieves the data corresponding to the retrieval request from the cache server or the local cache module according to the configuration information and sends the data to the client.
B20, the method of B17-B18, further comprising: the monitoring module monitors the data processing module; when the monitoring module monitors that the data processing module cannot store the request data in the cache server, the data processing module caches the request data in the local cache module, and the configuration server modifies the configuration information.
B21, the method of B20, further comprising: and the monitoring module generates alarm information when the data processing module cannot store the second type data into the retrieval server and exceeds a preset number of times.
B22, the method of B21, further comprising: the data query module determines that the request data are suitable for caching, analyzes the data retrieval request when the request data are not stored in the cache server, obtains the request data from the data server or retrieves the request data from the retrieval server, and sends the request data to the client through the interface module; and the data processing module determines to store the request data in the cache server or the local cache module according to the configuration information.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules or units or groups of devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into multiple sub-modules.
Those skilled in the art will appreciate that the modules in the device in the embodiments may be adaptively changed and disposed in one or more devices different from the embodiments. Modules or units or groups in embodiments may be combined into one module or unit or group and may further be divided into sub-modules or sub-units or sub-groups. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than others, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
Furthermore, some of the described embodiments are described herein as a method or combination of method elements that can be performed by a processor of a computer system or by other means of performing the described functions. A processor having the necessary instructions for carrying out the method or method element forms a means for carrying out the method or method element. Further, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is used to implement the functions performed by the elements for the purpose of carrying out the invention.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the computing device will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to execute the method for determining a device down state of the present invention according to instructions in the program code stored in the memory.
By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer-readable media includes both computer storage media and communication media. Computer storage media store information such as computer readable instructions, data structures, program modules or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of computer readable media.
As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this description, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The present invention has been disclosed in an illustrative rather than a restrictive sense, and the scope of the present invention is defined by the appended claims.

Claims (10)

1. A data retrieval system, said data retrieval system being connected to a plurality of clients and adapted to provide data retrieval services to said clients, said data retrieval system comprising:
a data server adapted to store a first type of data;
a retrieval server adapted to store a second type of data, the second type of data being generated based on the first type of data; and
a control server, communicatively coupled to the database server and the search server, comprising:
the data processing module is suitable for selecting first type data from the database server according to a preset rule, converting the first type data into second type data and storing the second type data into the retrieval server;
an interface module adapted to receive a data retrieval request from the client; and
and the data query module is suitable for analyzing the data retrieval request to determine whether corresponding first type data is acquired from the data server as request data or corresponding second type data is retrieved from the retrieval server as request data, and the request data is sent to the client through the interface module.
2. The system of claim 1, wherein the data query module is further adapted to retrieve data corresponding to a second type of data from the data server after retrieving the corresponding second type of data from the retrieval server, and combine the second type of data with the retrieved data to form the requested data.
3. The system of any of claims 1-2, wherein the data processing module is further adapted to classify the first type of data;
selecting data from different types of first type data as second data according to the user-defined service logic, and establishing an index for the second type data.
4. The system of any one of claims 1-3, wherein the control server further comprises:
a data notification module adapted to receive a data update notification from the data server; and
the data processing module is further adapted to convert the changed first type data into second type data according to the data update notification, and store the second type data in the retrieval server.
5. The system of claim 4, wherein the data processing module is further adapted to convert all of the first type data generated during the preset update interval time or the preset update interval number into the second type data according to the preset update interval time or the preset update interval number, and store the second type data into the search server.
6. The system of any one of claims 1-5, further comprising a cache server communicatively coupled to the control server and adapted to store requested data;
the data processing module is also suitable for storing the request data sent to the client in the cache server;
the data query module is further adapted to analyze the data retrieval request to determine whether the requested data is suitable for caching, and when it is determined that the requested data is suitable for caching, retrieve the requested data corresponding to the retrieval request from the cache server and send the requested data to the client.
7. The system of claim 6, wherein the control server further comprises a local cache module adapted to store the request data;
the data processing module is also suitable for storing the request data sent to the client in the local cache module;
the data query module is also suitable for retrieving the request data corresponding to the retrieval request from the local cache module and sending the request data to the client.
8. A data retrieval method adapted to operate in a data retrieval system as claimed in any one of claims 1 to 7, the system being connected to one or more clients, the method comprising the steps of:
collecting and storing the first type of data at the data server;
the data processing module in the control server selects first type data from the data server according to a preset rule, converts the first type data into second type data and stores the second type data into the retrieval server;
the interface module receives a data retrieval request from the client;
the data query module analyzes the data retrieval request to determine whether to acquire corresponding first type data from the data server as request data or retrieve corresponding second type data from the retrieval server as request data, and sends the request data to the client through the interface module.
9. A computing device, comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the method of claim 8.
10. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the method of claim 8.
CN202010431511.4A 2020-05-20 2020-05-20 Data retrieval system and method Pending CN111782687A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010431511.4A CN111782687A (en) 2020-05-20 2020-05-20 Data retrieval system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010431511.4A CN111782687A (en) 2020-05-20 2020-05-20 Data retrieval system and method

Publications (1)

Publication Number Publication Date
CN111782687A true CN111782687A (en) 2020-10-16

Family

ID=72754264

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010431511.4A Pending CN111782687A (en) 2020-05-20 2020-05-20 Data retrieval system and method

Country Status (1)

Country Link
CN (1) CN111782687A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102915380A (en) * 2012-11-19 2013-02-06 北京奇虎科技有限公司 Method and system for carrying out searching on data
CN103324713A (en) * 2013-06-19 2013-09-25 北京奇虎科技有限公司 Data processing method and device in multistage server and data processing system
CN106095911A (en) * 2016-06-07 2016-11-09 腾讯科技(深圳)有限公司 Search system and method for data synchronization
CN106210117A (en) * 2016-08-01 2016-12-07 浪潮软件股份有限公司 A kind of high performance service framework realized by high in the clouds caching
CN108595523A (en) * 2018-03-27 2018-09-28 广州供电局有限公司 device data retrieval model construction method, device and computer equipment
CN108696374A (en) * 2017-04-10 2018-10-23 北京京东尚科信息技术有限公司 Update the method and apparatus of client configuration

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102915380A (en) * 2012-11-19 2013-02-06 北京奇虎科技有限公司 Method and system for carrying out searching on data
CN103324713A (en) * 2013-06-19 2013-09-25 北京奇虎科技有限公司 Data processing method and device in multistage server and data processing system
CN106095911A (en) * 2016-06-07 2016-11-09 腾讯科技(深圳)有限公司 Search system and method for data synchronization
CN106210117A (en) * 2016-08-01 2016-12-07 浪潮软件股份有限公司 A kind of high performance service framework realized by high in the clouds caching
CN108696374A (en) * 2017-04-10 2018-10-23 北京京东尚科信息技术有限公司 Update the method and apparatus of client configuration
CN108595523A (en) * 2018-03-27 2018-09-28 广州供电局有限公司 device data retrieval model construction method, device and computer equipment

Similar Documents

Publication Publication Date Title
US9015238B1 (en) Implementation of a web scale data fabric
US10853847B2 (en) Methods and systems for near real-time lookalike audience expansion in ads targeting
CN111651464B (en) Data processing method, system and computing device
US9930113B2 (en) Data retrieval via a telecommunication network
CN109388657B (en) Data processing method, device, computer equipment and storage medium
CN111553560A (en) Service index monitoring method, monitoring server and system
CN108984574B (en) Data processing method and device
US11093496B1 (en) Performance-based query plan caching
CN110912757A (en) Service monitoring method and server
US9047328B2 (en) Searching entity-key associations using in-memory objects
CN113377289A (en) Cache management method, system, computing device and readable storage medium
CN111752945A (en) Time sequence database data interaction method and system based on container and hierarchical model
CN111917573B (en) Monitoring method, monitoring system and computing equipment
CN116048817B (en) Data processing control method, device, computer equipment and storage medium
US20100332501A1 (en) System and method for on-demand indexing
CN111782687A (en) Data retrieval system and method
JP2023015275A (en) Observation information processing method, apparatus, electronic device, storage medium, and computer program
CN116450890A (en) Graph data processing method, device and system, electronic equipment and storage medium
CN116185578A (en) Scheduling method of computing task and executing method of computing task
JP6869313B2 (en) Service dispatch system and method based on user behavior
CN110716911A (en) Data processing method and device, electronic equipment and storage medium
EP2990960A1 (en) Data retrieval via a telecommunication network
CN108629003B (en) Content loading method and device
CN113590699A (en) Interface request processing method and system and computing equipment
CN111814029A (en) Data query method, system and computing device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination