CN108519984B - Weather data processing method, server and computer readable storage medium - Google Patents
Weather data processing method, server and computer readable storage medium Download PDFInfo
- Publication number
- CN108519984B CN108519984B CN201810124487.2A CN201810124487A CN108519984B CN 108519984 B CN108519984 B CN 108519984B CN 201810124487 A CN201810124487 A CN 201810124487A CN 108519984 B CN108519984 B CN 108519984B
- Authority
- CN
- China
- Prior art keywords
- weather
- weather data
- dimension
- data
- dimensions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The invention discloses a weather data processing method, which comprises the following steps: setting a mapping relation between a corresponding weather data service website link and a city name aiming at a target city; acquiring weather dimensions of each weather data service website; setting a union set of all weather dimensions of the corresponding website as an output dimension; acquiring current weather data of each website according to the output dimension; and outputting the acquired weather data of each dimension. The invention also provides a server and a computer readable storage medium. The weather data processing method, the server and the computer readable storage medium provided by the invention can obtain weather data with higher quality, and establish a weather database with perfect information so as to solve the problems of inconsistent weather data and incomplete information of different websites.
Description
Technical Field
The present invention relates to the field of data analysis technologies, and in particular, to a weather data processing method, a server, and a computer-readable storage medium.
Background
The weather data plays a very important role in the field of health care research, and the accuracy and integrity of data sources are important in data services. Currently, some mainstream weather data service websites respectively provide data with different dimensions, and the dimensions of a single weather website are insufficient to show all aspects of weather. And due to some reasons, such as data import error, data may be different between different websites or data is partially lost, which is difficult to be used for establishing a weather database with perfect information.
Disclosure of Invention
In view of the above, the present invention provides a weather data processing method, a server and a computer readable storage medium, so as to solve the problem how to deal with the inconsistent weather data and incomplete information of different websites.
Firstly, in order to achieve the above object, the present invention provides a method for processing weather data, including the steps of:
setting a mapping relation between a corresponding weather data service website link and a city name aiming at a target city;
acquiring weather dimensions of each weather data service website;
setting a union set of all weather dimensions of corresponding websites as an output dimension;
acquiring current weather data of each website according to the output dimension; and
and outputting the acquired weather data of each dimension.
Optionally, the method further includes, before the step of outputting the acquired weather data of each dimension, the steps of:
and screening the output data of the dimension according to a preset rule for the dimension data shared by a plurality of websites.
Optionally, the preset rule is: selecting a mode as output data for a classification variable aiming at dimension data shared by a plurality of websites; for numerical variables, the average value is selected as the output data.
Optionally, the step of obtaining the weather dimension of each weather data service website includes:
constructing request addresses of the target city under different weather data service websites according to the mapping relation;
and entering different websites corresponding to the target city through the request address to acquire all weather dimensions of the weather data of the target city contained in the websites.
Optionally, the step of obtaining the weather dimension of each weather data service website further includes:
and unifying the naming of the same weather dimension for each weather data service website according to the preset standard naming aiming at each weather dimension and the corresponding word bank.
In addition, to achieve the above object, the present invention further provides a server, including a memory and a processor, where the memory stores a weather data processing system operable on the processor, and the weather data processing system, when executed by the processor, implements the following steps:
setting a mapping relation between a corresponding weather data service website link and a city name aiming at a target city;
acquiring weather dimensions of each weather data service website;
setting a union set of all weather dimensions of corresponding websites as an output dimension;
acquiring current weather data of each website according to the output dimension; and
and outputting the acquired weather data of each dimension.
Optionally, the weather data processing system further implements, when executed by the processor, the steps of:
and screening the output data of the dimension according to a preset rule for the dimension data shared by a plurality of websites.
Optionally, the preset rule is: selecting a mode as output data for a classification variable aiming at dimension data shared by a plurality of websites; for numerical variables, the average value is selected as the output data.
Optionally, the step of obtaining the weather dimension of each weather data service website includes:
constructing request addresses of the target city under different weather data service websites according to the mapping relation;
entering different websites corresponding to the target city through the request address to acquire all weather dimensions of the weather data of the target city contained in the websites;
and unifying the naming of the same weather dimension for each weather data service website according to the preset standard naming aiming at each weather dimension and the corresponding word bank.
Further, to achieve the above object, the present invention also provides a computer readable storage medium storing a weather data processing system, which is executable by at least one processor to cause the at least one processor to execute the steps of the weather data processing method as described above.
Compared with the prior art, the weather data processing method, the server and the computer readable storage medium provided by the invention can set mapping relations of corresponding different website links and city names aiming at a target city, enter a corresponding weather data service website according to the mapping relations, and acquire the weather data of the target city, wherein a union set of all weather dimensions of corresponding websites is used as an output dimension, and a mode or an average value of the weather data is screened as the output data, so that a weather database with perfect information is established, and the problems of inconsistency and incomplete information of the weather data of different websites are solved.
Drawings
FIG. 1 is a schematic diagram of an alternative hardware architecture for a server according to the present invention;
FIG. 2 is a schematic diagram of program modules of a weather data processing system according to a first embodiment of the invention;
FIG. 3 is a schematic diagram of program modules of a weather data processing system according to a second embodiment of the invention;
FIG. 4 is a schematic flow chart diagram of a first embodiment of a weather data processing method of the invention;
fig. 5 is a flowchart illustrating a weather data processing method according to a second embodiment of the present invention.
Reference numerals:
| server | 2 |
| Memory device | 11 |
| Processor with a memory for storing a plurality of data | 12 |
| Network interface | 13 |
| Weather |
200 |
| |
201 |
| |
202 |
| |
203 |
| |
204 |
The implementation, functional features and advantages of the present invention will be further described with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
Fig. 1 is a schematic diagram of an alternative hardware architecture of the server 2 according to the present invention.
In this embodiment, the server 2 may include, but is not limited to, a memory 11, a processor 12, and a network interface 13, which may be communicatively connected to each other through a system bus. It is noted that fig. 1 only shows the server 2 with components 11-13, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
The server 2 may be a rack server, a blade server, a tower server, or a rack server, and the server 2 may be an independent server or a server cluster formed by a plurality of servers.
The memory 11 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 11 may be an internal storage unit of the server 2, such as a hard disk or a memory of the server 2. In other embodiments, the memory 11 may also be an external storage device of the server 2, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, provided on the server 2. Of course, the memory 11 may also comprise both an internal storage unit of the server 2 and an external storage device thereof. In this embodiment, the memory 11 is generally used for storing an operating system installed in the server 2 and various types of application software, such as program codes of the weather data processing system 200. Further, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 12 is typically used to control the overall operation of the server 2. In this embodiment, the processor 12 is configured to run the program code stored in the memory 11 or process data, for example, run the weather data processing system 200.
The network interface 13 may comprise a wireless network interface or a wired network interface, and the network interface 13 is generally used for establishing communication connection between the server 2 and other electronic devices.
The hardware structure and functions of the apparatus according to the present invention have been described in detail. Various embodiments of the present invention will be presented based on the above description.
First, the present invention provides a weather data processing system 200.
Referring to fig. 2, a block diagram of a first embodiment of a weather data processing system 200 according to the invention is shown.
In this embodiment, the weather data processing system 200 includes a series of computer program instructions stored on the memory 11, which when executed by the processor 12, can implement the weather data processing operations of the embodiments of the present invention. In some embodiments, weather data processing system 200 may be divided into one or more modules based on the particular operations implemented by the portions of the computer program instructions. For example, in fig. 2, the weather data processing system 200 may be divided into a setting module 201, an obtaining module 202, and an outputting module 203. Wherein:
the setting module 201 is configured to set a mapping relationship between a corresponding weather data service website link (URL) and a city name for a target city.
Specifically, the target city is a city for which weather data needs to be integrated and displayed, and may be one or more selected cities. For the target city, relevant weather data can be obtained from multiple weather data service websites, and names of the target city by each website may be different, for example, a common name or a short name may be used. Therefore, the website link and city name corresponding to the target city need to be preset, and the mapping relationship between the three is recorded.
For example, for this target city of Shenzhen, the mapping relationship can be recorded in the following manner:
and according to the mapping relation, the request address of the city under different weather data service websites can be constructed. For example, according to the record named shenzhen with the URL www.tianqiyuubao.com, the request address for constructing weather data for acquiring Shenzhen under the website is: http:// www.tianquiubao.com/lishi/shenzhen.html.
The obtaining module 202 is configured to obtain weather dimensions of each weather data service website.
Specifically, different weather data service websites corresponding to the target city are accessed according to the mapping relation, and all weather dimensions of the weather data of the target city contained in the websites are obtained. The weather dimension is a data dimension commonly used in urban weather forecast, such as region, date, weather condition, precipitation, average temperature, highest temperature, lowest temperature, average humidity, highest humidity, lowest humidity, air quality level, wind power, wind direction and the like. In this embodiment, after the request address of the target city under different websites is constructed according to the mapping relationship, the corresponding different websites can be accessed through the request address to obtain the weather dimension.
It is noted that since various websites may have different designations for the same weather dimension, such as average temperature and average air temperature, the designations for each weather dimension need to be unified. In this embodiment, a standard name and a corresponding thesaurus may be set for all the common weather dimensions in advance, and when the name of a certain weather dimension in a certain website or in certain websites belongs to a name in the thesaurus of a certain standard name, the names are unified as the standard name. For example, for the weather dimension of the average temperature, a standard named "average temperature" is set, and the corresponding synonym library is { average temperature, average temperature value }. When the name in the thesaurus is encountered, the name is unified into the standard name of "average temperature".
In this embodiment, all the acquired weather dimensions of each website corresponding to the target city are stored in the data table. For example, for Shenzhen as a target city, the weather dimension of the A website includes [ region, date, weather condition, highest temperature, lowest temperature ], and the weather dimension of the B website includes [ region, date, weather condition, highest temperature, lowest temperature, highest humidity, lowest humidity ], and the like.
The setting module 201 is further configured to set, for the target city, a union of all weather dimensions of the corresponding websites as an output dimension.
Specifically, after all weather dimensions contained in different weather data service websites corresponding to the target city are acquired, a union set of all weather dimensions of all websites corresponding to the target city is obtained, and the weather dimensions contained in the union set are subsequently used as output dimensions of the weather data of the target city.
For example, for Shenzhen as a target city, the weather dimension of the A website includes [ region, date, weather condition, highest temperature, lowest temperature ], the weather dimension of the B website includes [ region, date, weather condition, highest temperature, lowest temperature, highest humidity, lowest humidity ], the weather dimension of the C website includes [ region, date, weather condition, precipitation, average temperature, highest temperature, lowest temperature, average humidity, highest humidity, lowest humidity, sea level air pressure, AQI index, pollutant, air quality level, wind power, wind direction ], the weather dimension of the D website includes [ region, date, weather condition, precipitation, highest temperature, lowest temperature, wind power, wind direction, sunrise time, sunset time, visibility, dew point ], and the like. Through solving the union, the output dimensionality of the Shenzhen is obtained as [ region, date, weather condition, precipitation, average temperature, highest temperature, lowest temperature, average humidity, highest humidity, lowest humidity, sea level air pressure, AQI index, pollutant, air quality level, wind power, wind direction, sunrise time, sunset time, visibility, dew point ].
The obtaining module 202 is further configured to obtain, for the target city, current weather data of each website according to the output dimension.
Specifically, for the target city, a sub-crawler program corresponding to each website is set according to the mapping relation between the website link and the city name.
For example, the method includes the steps of inputting shenzhen, obtaining all website links and city names corresponding to the shenzhen, entering a sub-crawler program corresponding to each website, and crawling weather data of a plurality of websites. Such as def tianqiyubao (city name, start date, end date) and def wunderground (city name, start date, end date).
And each sub-crawler program crawls weather data corresponding to all the output dimensions of the target city in parallel without reserved null values of the corresponding data.
The output module 203 is configured to output the acquired weather data of each dimension.
Specifically, for the target city, the acquired weather data corresponding to all output dimensions are integrated and then stored locally (for example, in a weather database) for subsequent use. When the weather data of the target city needs to be displayed, the integrated weather data of all output dimensions can be output to display equipment to be displayed for a user to check. Therefore, weather data with higher quality can be obtained, and a weather database with perfect information is established to solve the problems of inconsistent weather data and incomplete information of different websites.
Referring to fig. 3, a block diagram of a second embodiment of a weather data processing system 200 according to the invention is shown. In this embodiment, the weather data processing system 200 further includes a screening module 204 in addition to the setting module 201, the obtaining module 202, and the output module 203 in the first embodiment.
The screening module 204 is configured to screen, according to a predetermined rule, output data of a dimension common to the multiple websites.
Specifically, the dimension data common to the multiple websites refers to that, for the same weather dimension, weather data corresponding to the dimension exists in the multiple weather data service websites. For example, if the weather data corresponding to the weather dimension [ the highest temperature ] exists in both the website a and the website B, the [ highest temperature ] is dimension data common to the websites.
The preset rule is as follows: for dimension data common to a plurality of websites, selecting a mode as output data for classification variables (variable values are qualitative and show as mutually incompatible categories or attributes, such as wind directions); for numerical variables (the type of variable is a numerical value, e.g. temperature), the average value is selected as output data. For example, for dimension data common to a plurality of websites, which is the [ wind direction ], belonging to the classification variables, the [ wind direction ] data of each website (the a website is south wind, the B website is southwest wind, the C website is south wind, and the D website is south wind) is acquired, and the mode "south wind" is selected as output data of the output dimension of [ wind direction ]. For example, the dimension data common to a plurality of sites [ maximum temperature ] belongs to a numerical variable, and an average value of [ maximum temperature ] of each site is calculated and used as output data of an output dimension [ maximum temperature ].
In addition, the invention also provides a weather data processing method.
Fig. 4 is a schematic flow chart of a weather data processing method according to a first embodiment of the present invention. In this embodiment, the execution order of the steps in the flowchart shown in fig. 4 may be changed and some steps may be omitted according to different requirements.
Step S400, aiming at a target city, setting a mapping relation between a corresponding weather data service website link (URL) and a city name.
Specifically, the target city is a city for which weather data needs to be integrated and displayed, and may be one or more selected cities. For the target city, relevant weather data can be obtained from a plurality of weather data service websites, and names of the target city by each website may be different, for example, some names are full names, some names are short names, and the like. Therefore, the website link and city name corresponding to the target city need to be preset, and the mapping relationship between the three is recorded.
For example, for this target city of Shenzhen, the mapping relationship can be recorded in the following manner:
and according to the mapping relation, the request address of the city under different weather data service websites can be constructed. For example, according to the record named shenzhen with the URL www.tianqiyuubao.com, the request address for constructing the weather data for acquiring Shenzhen on the website is as follows: http:// www.tianqiiyuba.com/lishi/shenzhen.html.
Step S402, acquiring the weather dimension of each weather data service website.
Specifically, different weather data service websites corresponding to the target city are accessed according to the mapping relationship, and all weather dimensions of the weather data of the target city contained in the websites are obtained. The weather dimension is a data dimension commonly used in urban weather forecast, such as region, date, weather condition, precipitation, average temperature, highest temperature, lowest temperature, average humidity, highest humidity, lowest humidity, air quality level, wind power, wind direction and the like. In this embodiment, after the request address of the target city under different websites is constructed according to the mapping relationship, the corresponding different websites can be accessed through the request address to obtain the weather dimension.
It is noted that since various websites may have different designations for the same weather dimension, such as average temperature and average air temperature, the designations for each weather dimension need to be unified. In this embodiment, a standard name and a corresponding thesaurus may be set for all common weather dimensions in advance, and when the name of a certain weather dimension in a certain website or in certain websites belongs to a name in the thesaurus of a certain standard name, the name is unified as the standard name. For example, for the weather dimension of the average temperature, a standard named "average temperature" is set, and the corresponding synonym library is { average temperature, average temperature value }. When a name in the thesaurus is encountered, it is unified as the standard name of "average air temperature".
In this embodiment, all the acquired weather dimensions of each website corresponding to the target city are stored in the data table. For example, for Shenzhen as a target city, the weather dimension of the A website includes [ region, date, weather condition, highest temperature, lowest temperature ], and the weather dimension of the B website includes [ region, date, weather condition, highest temperature, lowest temperature, highest humidity, lowest humidity ], and the like.
Step S404, aiming at the target city, setting a union set of all weather dimensions of the corresponding website as an output dimension.
Specifically, after all weather dimensions contained in different weather data service websites corresponding to the target city are acquired, a union set of all weather dimensions of all websites corresponding to the target city is obtained, and the weather dimensions contained in the union set are subsequently used as output dimensions of the weather data of the target city.
For example, for Shenzhen as a target city, the weather dimension of the A website includes [ region, date, weather condition, highest temperature, lowest temperature ], the weather dimension of the B website includes [ region, date, weather condition, highest temperature, lowest temperature, highest humidity, lowest humidity ], the weather dimension of the C website includes [ region, date, weather condition, precipitation, average temperature, highest temperature, lowest temperature, average humidity, highest humidity, lowest humidity, sea level air pressure, AQI index, pollutant, air quality level, wind power, wind direction ], the weather dimension of the D website includes [ region, date, weather condition, precipitation, highest temperature, lowest temperature, wind power, wind direction, sunrise time, sunset time, visibility, dew point ], and the like. Through solving the union set, the output dimensionality of the Shenzhen is [ region, date, weather condition, precipitation, average temperature, maximum temperature, minimum temperature, average humidity, maximum humidity, minimum humidity, sea level air pressure, AQI index, pollutant, air quality level, wind power, wind direction, sunrise time, sunset time, visibility, dew point ].
And step S406, acquiring current weather data of each website according to the output dimension aiming at the target city.
Specifically, for the target city, a sub-crawler program corresponding to each website is set according to the mapping relation between the website link and the city name.
For example, the method includes the steps of inputting the shenzhen, obtaining all website links and city names corresponding to the shenzhen, entering a sub-crawler program corresponding to each website, and crawling weather data of a plurality of websites. Such as def tianqiyubao (city name, start date, end date) and def wunderground (city name, start date, end date).
And each sub-crawler program crawls weather data corresponding to all the output dimensions of the target city in parallel without reserved null values of the corresponding data.
And step S408, outputting the acquired weather data of each dimension.
Specifically, for the target city, the acquired weather data corresponding to all output dimensions are integrated and then stored locally (for example, in a weather database) for subsequent use. When the weather data of the target city needs to be displayed, the integrated weather data of all output dimensions can be output to display equipment to be displayed for a user to check. Therefore, weather data with higher quality can be obtained, and a weather database with perfect information is established to solve the problems of inconsistent weather data and incomplete information of different websites.
The weather data processing method provided by this embodiment may set mapping relationships between corresponding different website links and city names for a target city, enter a corresponding weather data service website according to the mapping relationships, and acquire weather data of the target city, where a union of all weather dimensions of the corresponding websites is used as an output dimension, so as to establish a weather database with perfect information and address the problem of incomplete weather data information of different websites.
Fig. 5 is a schematic flow chart of a weather data processing method according to a second embodiment of the invention. In this embodiment, steps S500 to S506 and S510 of the weather data processing method are similar to steps S400 to S408 of the first embodiment, except that the method further includes step S508.
The method comprises the following steps:
step S500, aiming at the target city, setting a mapping relation between a corresponding weather data service website link and a city name.
Specifically, the target city is a city for which weather data needs to be integrated and displayed, and may be one or more selected cities. For the target city, relevant weather data can be obtained from a plurality of weather data service websites, and names of the target city by each website may be different, for example, some names are full names, some names are short names, and the like. Therefore, the website link and city name corresponding to the target city need to be preset, and the mapping relationship between the three is recorded.
For example, for this target city of Shenzhen, the mapping relationship can be recorded in the following manner:
and according to the mapping relation, the request address of the city under different weather data service websites can be constructed. For example, according to the record named shenzhen with the URL www.tianqiyuubao.com, the request address for constructing the weather data for acquiring Shenzhen on the website is as follows: http:// www.tianqiiyuba.com/lishi/shenzhen.html.
Step S502, weather dimensions of each weather data service website are obtained.
Specifically, different weather data service websites corresponding to the target city are accessed according to the mapping relation, and all weather dimensions of the weather data of the target city contained in the websites are obtained. The weather dimension is a data dimension commonly used in urban weather forecast, such as region, date, weather condition, precipitation, average temperature, highest temperature, lowest temperature, average humidity, highest humidity, lowest humidity, air quality level, wind power, wind direction and the like. In this embodiment, after the request address of the target city under different websites is constructed according to the mapping relationship, the corresponding different websites can be accessed through the request address to obtain the weather dimension.
It is noted that since various websites may have different names for the same weather dimension, such as average temperature and average temperature, there is a need to unify the names for each weather dimension. In this embodiment, a standard name and a corresponding thesaurus may be set for all the common weather dimensions in advance, and when the name of a certain weather dimension in a certain website or in certain websites belongs to a name in the thesaurus of a certain standard name, the names are unified as the standard name. For example, a standard named as "average temperature" is set for the weather dimension of the average temperature, and the corresponding synonym library is { average temperature, average temperature value }. When a name in the thesaurus is encountered, it is unified as the standard name of "average air temperature".
In this embodiment, all the acquired weather dimensions of each website corresponding to the target city are stored in the data table. For example, for Shenzhen as a target city, the weather dimension of the A website includes [ region, date, weather condition, highest temperature, lowest temperature ], and the weather dimension of the B website includes [ region, date, weather condition, highest temperature, lowest temperature, highest humidity, lowest humidity ], and the like.
Step S504, aiming at the target city, setting a union set of all weather dimensions of the corresponding websites as an output dimension.
Specifically, after all weather dimensions contained in different weather data service websites corresponding to the target city are acquired, a union set of all weather dimensions of all websites corresponding to the target city is obtained, and then the weather dimensions contained in the union set are used as output dimensions of the weather data of the target city.
For example, for the target city of shenzhen, the weather dimension of the a website includes [ region, date, weather condition, highest temperature, lowest temperature ], the weather dimension of the B website includes [ region, date, weather condition, highest temperature, lowest temperature, highest humidity, lowest humidity ], the weather dimension of the C website includes [ region, date, weather condition, precipitation, average temperature, highest temperature, lowest temperature, average humidity, highest humidity, lowest humidity, sea level air pressure, AQI index, pollutant, air quality level, wind power, wind direction ], the weather dimension of the D website includes [ region, date, weather condition, precipitation, highest temperature, lowest temperature, wind power, wind direction, sunrise time, sunset time, visibility, dew point ], and the like. Through solving the union, the output dimensionality of the Shenzhen is obtained as [ region, date, weather condition, precipitation, average temperature, highest temperature, lowest temperature, average humidity, highest humidity, lowest humidity, sea level air pressure, AQI index, pollutant, air quality level, wind power, wind direction, sunrise time, sunset time, visibility, dew point ].
And S506, acquiring the current weather data of each website according to the output dimension aiming at the target city.
Specifically, for the target city, a sub-crawler program corresponding to each website is set according to the mapping relation between the website link and the city name.
For example, the method includes the steps of inputting the shenzhen, obtaining all website links and city names corresponding to the shenzhen, entering a sub-crawler program corresponding to each website, and crawling weather data of a plurality of websites. Such as def tianqiyubao (city name, start date, end date) and def wunderground (city name, start date, end date).
And each sub-crawler program crawls weather data corresponding to all the output dimensions of the target city in parallel without reserved null values of the corresponding data.
Step S508, for dimension data common to multiple websites, output data of the dimension is filtered according to a predetermined rule.
Specifically, the dimension data common to the multiple websites refers to that, for the same weather dimension, weather data corresponding to the dimension exists in the multiple weather data service websites. For example, the website a and the website B both have weather data corresponding to the weather dimension [ maximum temperature ], and then [ maximum temperature ] is dimension data common to a plurality of websites.
The preset rule is as follows: for dimension data common to a plurality of websites, selecting a mode as output data for a classification variable (variable value is qualitative and shows as mutually incompatible categories or attributes such as wind direction); for numerical variables (the type of variable is a numerical value, e.g. temperature), the average value is selected as output data. For example, for dimension data common to a plurality of websites, which is the [ wind direction ], belonging to the classification variables, the [ wind direction ] data of each website (the a website is south wind, the B website is southwest wind, the C website is south wind, and the D website is south wind) is acquired, and the mode "south wind" is selected as output data of the output dimension of [ wind direction ]. For example, the dimension data common to a plurality of sites [ maximum temperature ] belongs to a numerical variable, and the average value of [ maximum temperature ] of each site is calculated and used as the output data of the output dimension [ maximum temperature ].
And step S510, outputting the screened weather data of each dimension.
Specifically, for the target city, output data corresponding to all output dimensions are sorted out and then stored locally (for example, in a weather database) for subsequent use. When the weather data of the target city needs to be displayed, the integrated weather data of all output dimensions can be output to display equipment to be displayed for a user to view. Therefore, weather data with higher quality can be obtained, and a weather database with perfect information is established to solve the problems of inconsistent weather data and incomplete information of different websites.
The weather data processing method provided by this embodiment may set a mapping relationship between corresponding different website links and city names for a target city, enter a corresponding weather data service website according to the mapping relationship, acquire weather data of the target city, use a union of all weather dimensions of the corresponding websites as an output dimension, and filter a mode or an average of the weather data as output data, thereby establishing a weather database with perfect information to deal with the problem that weather data information of different websites is incomplete. In addition, for dimension data shared by a plurality of websites, a mode can be selected as output data for a classification variable, and an average value can be selected as output data for a numerical variable, so that the condition that the plurality of websites are inconsistent in the dimension data can be effectively processed.
The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are also included in the scope of the present invention.
Claims (7)
1. A weather data processing method is applied to a server, and is characterized by comprising the following steps:
setting a mapping relation between a corresponding weather data service website link and a city name aiming at a target city;
constructing a request address of the target city on each weather data service website according to the mapping relation, and acquiring all weather dimensions of each weather data service website corresponding to the target city through the request address;
carrying out uniform standard naming on all weather dimensions according to a preset word bank, and carrying out union on all weather dimensions to be used as output dimensions of weather data of the target city;
acquiring current weather data of each website according to the output dimension;
screening the output data of the common dimensionality according to a preset rule for the dimensionality data common to a plurality of websites;
outputting the screened weather data of each dimension and establishing a weather database.
2. A weather data processing method as claimed in claim 1, wherein the predetermined rule is: selecting a mode as output data for a classification variable aiming at dimension data shared by a plurality of websites; for numerical variables, the average value is selected as the output data.
3. A weather data processing method as claimed in claim 1, wherein the step of obtaining all weather dimensions of each weather data service website corresponding to the target city further comprises:
and unifying the naming of the same weather dimension for each weather data service website according to the preset standard naming aiming at each weather dimension and the corresponding word bank.
4. A server, comprising a memory, a processor, the memory having stored thereon a weather data processing system operable on the processor, the weather data processing system when executed by the processor performing the steps of:
setting a mapping relation between a corresponding weather data service website link and a city name aiming at a target city;
constructing a request address of the target city at each weather data service website according to the mapping relation, and acquiring all weather dimensions of each weather data service website corresponding to the target city through the request address;
carrying out uniform standard naming on all weather dimensions according to a preset synonym library, and carrying out union on all weather dimensions to be used as output dimensions of weather data of the target city;
acquiring current weather data of each website according to the output dimension;
screening the output data of the common dimensionality according to a preset rule for the dimensionality data common to the websites;
outputting the screened weather data of each dimension and establishing a weather database.
5. The server according to claim 4, wherein the predetermined rule is: selecting a mode as output data for a classification variable aiming at dimension data shared by a plurality of websites; for numerical variables, the average value is selected as the output data.
6. The server of claim 4, wherein the step of obtaining all weather dimensions of each weather data service website corresponding to the target city comprises:
and unifying the naming of the same weather dimension for each weather data service website according to the preset standard naming aiming at each weather dimension and the corresponding word stock.
7. A computer readable storage medium storing a weather data processing system executable by at least one processor to cause the at least one processor to perform the steps of the weather data processing method as claimed in any one of claims 1 to 3.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810124487.2A CN108519984B (en) | 2018-02-07 | 2018-02-07 | Weather data processing method, server and computer readable storage medium |
| PCT/CN2018/089417 WO2019153599A1 (en) | 2018-02-07 | 2018-05-31 | Weather data processing method, server and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810124487.2A CN108519984B (en) | 2018-02-07 | 2018-02-07 | Weather data processing method, server and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108519984A CN108519984A (en) | 2018-09-11 |
| CN108519984B true CN108519984B (en) | 2022-11-04 |
Family
ID=63433035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810124487.2A Active CN108519984B (en) | 2018-02-07 | 2018-02-07 | Weather data processing method, server and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108519984B (en) |
| WO (1) | WO2019153599A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109639880B (en) * | 2018-11-08 | 2021-02-02 | 维沃移动通信有限公司 | Weather information display method and terminal equipment |
| CN112417073A (en) * | 2020-11-18 | 2021-02-26 | 中科三清科技有限公司 | Automatic air quality condition broadcasting method and device, electronic equipment and storage medium |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006108871A (en) * | 2004-10-01 | 2006-04-20 | Nec Commun Syst Ltd | Weather information notification system, management server and weather information notification program |
| CN101557268A (en) * | 2008-04-11 | 2009-10-14 | 北京闻言科技有限公司 | Method for intelligently providing weather forecast data |
| US8698640B1 (en) * | 2010-03-04 | 2014-04-15 | Daniel R. Gropper | Monitored weather and emergency alert system |
| TW201216194A (en) * | 2010-10-08 | 2012-04-16 | Weatherrisk Co Ltd | The method of integrating user's feedback and variety weather information into the new meteorological platform by using network |
| CN102521716B (en) * | 2011-12-31 | 2016-01-06 | 成都信息工程学院 | Integrative specialized weather service integrated system |
| CN102750351A (en) * | 2012-06-11 | 2012-10-24 | 迪尔码国际营销服务(北京)有限公司 | Matching method of address information based on rules |
| CN103246726B (en) * | 2013-05-09 | 2017-04-12 | 北京奇付通科技有限公司 | Method, device and system for searching network information |
| CN104021170B (en) * | 2014-05-30 | 2018-01-16 | 华为技术有限公司 | A kind of information acquisition method and cloud server |
| CN105338030A (en) * | 2014-08-06 | 2016-02-17 | 神州数码信息系统有限公司 | Method for intelligently providing weather forecast data |
| CN105676312A (en) * | 2016-02-26 | 2016-06-15 | 广东欧珀移动通信有限公司 | Display method and system for weather forecast |
| CN106776756B (en) * | 2016-11-22 | 2020-12-01 | 杭州联络互动信息科技股份有限公司 | Method and device for processing weather resource data by server side and client side |
| CN106372765B (en) * | 2016-11-29 | 2019-11-12 | 东南大学 | A kind of expressway weather station data recovery method based on fused data |
| CN107301253B (en) * | 2017-08-23 | 2020-02-04 | 杭州安恒信息技术股份有限公司 | Method and device for improving accuracy of multi-site search keywords |
-
2018
- 2018-02-07 CN CN201810124487.2A patent/CN108519984B/en active Active
- 2018-05-31 WO PCT/CN2018/089417 patent/WO2019153599A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| CN108519984A (en) | 2018-09-11 |
| WO2019153599A1 (en) | 2019-08-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108519984B (en) | Weather data processing method, server and computer readable storage medium | |
| CN111444455A (en) | Browser compatible method, system, computer device and storage medium | |
| CN112214561B (en) | Map data processing method, map data processing device, computer equipment and storage medium | |
| CN113220657B (en) | Data processing method and device and computer equipment | |
| CN109299205B (en) | Method and device for warehousing spatial data used by planning industry | |
| CN110162540B (en) | Block chain account book data query method, electronic device and storage medium | |
| CN111159183A (en) | Report generation method, electronic device and computer readable storage medium | |
| CN108427701B (en) | Method for identifying help information based on operation page and application server | |
| CN108763341B (en) | Electronic device, automatic table building method and storage medium | |
| CN107844515B (en) | Data compliance checking method and device | |
| CN112000692B (en) | Page query feedback method and device, computer equipment and readable storage medium | |
| CN113806653A (en) | Page preloading method and device, computer equipment and storage medium | |
| CN112579608A (en) | Case data query method, system, device and computer readable storage medium | |
| CN110866007B (en) | Information management method, system and computer equipment for big data application and table | |
| CN112035551A (en) | Time series data conversion method, system, computer device and storage medium | |
| CN113127125A (en) | Page automatic adaptation method, device, equipment and storage medium | |
| CN111414395A (en) | Data processing method, system and computer equipment | |
| CN111737330A (en) | Spatial data standardization method and device, computer equipment and storage medium | |
| CN111221517A (en) | Model creating method and device, computer equipment and readable storage medium | |
| CN114817152A (en) | Method and system for querying slice file | |
| CN113436725B (en) | Data processing method, system, computer device and computer readable storage medium | |
| CN109885710B (en) | User image depicting method based on differential evolution algorithm and server | |
| CN108829574B (en) | Test data laying method, test server and computer readable storage medium | |
| CN114169306A (en) | Method, device and equipment for generating electronic receipt and readable storage medium | |
| CN112417324A (en) | Chrome-based URL (Uniform resource locator) interception method and device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |