CN111723117A - Big data product development screening method and device and computer readable storage medium - Google Patents
Big data product development screening method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN111723117A CN111723117A CN202010571851.7A CN202010571851A CN111723117A CN 111723117 A CN111723117 A CN 111723117A CN 202010571851 A CN202010571851 A CN 202010571851A CN 111723117 A CN111723117 A CN 111723117A
- Authority
- CN
- China
- Prior art keywords
- data
- product
- platform
- information
- screening
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- General Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Operations Research (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Marketing (AREA)
- Economics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a big data product development and screening method, which comprises the following steps: confirming a target data platform for product data screening, and capturing category data of the target data platform; extracting product information of the target data platform according to the category data and storing the product information into a preset database, wherein the product information comprises product summary data, product detail data, product sales detail data, product comment detail data and product listing time detail data; and screening data information meeting the user data requirements from the extracted product information and pushing the data information to a corresponding ERP platform. The invention also discloses a device and a computer readable storage medium. According to the invention, product information is captured on the data platform according to the data level according to the requirements of user application data, and data meeting conditions are screened based on the captured product information and pushed to the ERP for calling, so that the technical problems of product data screening and screening of the data platform are realized.
Description
Technical Field
The invention relates to the technical field of data screening, in particular to a method and a device for developing and screening big data products and a computer readable storage medium.
Background
The grabbing operation of product development data developed by the current massive SKU foreign trade e-commerce products basically stays at the platform of manually inquiring products of various large sales platforms or inquiring 1688 product suppliers, or participates in product exchange to know the hot sales dynamics of industry products, the method has the defects of untimely understanding of the dynamic behaviors of the industry products, low product development efficiency, limitation on collected product sales data and inaccurate judgment on hot sales or new products. The other method is to crawl a product with large total sales of the product through software to be developed as a hot sales reference, and in this case, only the product with large sales can be found, but the latest hot sales cannot be identified, and meanwhile, whether the product exists or not cannot be identified with a seller system. Two ways of screening sources based on the existing product data are product sales data of suppliers and product sales data of foreign trade sales platforms. How to acquire data, how to acquire mass data, what data to acquire, and how to screen data are technical problems which are urgently needed to be solved in the development of current e-commerce products.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a method and a device for developing and screening big data products and a computer readable storage medium, and aims to solve the technical problem that data screening is difficult due to excessive data sources in the existing e-commerce product development process.
In order to achieve the above object, the present invention provides a method for developing and screening big data products, which comprises the following steps:
confirming a target data platform for product data screening, and capturing category data of the target data platform;
extracting product information of the target data platform according to the category data and storing the product information into a preset database, wherein the product information comprises product summary data, product detail data, product sales detail data, product comment detail data and product listing time detail data;
and screening data information meeting the user data requirements from the extracted product information and pushing the data information to a corresponding ERP platform.
Optionally, before the step of capturing category data of the target data platform, the step of confirming the product data screening further includes:
acquiring user data requirements, and confirming a target data platform according to the user data requirements;
and creating a data connection relation with the target data platform.
Optionally, the step of creating a data connection relationship with the target data platform includes:
confirming a data application interface of the target data platform;
and creating a data connection relation with the data application interface.
Optionally, the step of capturing category data of the target data platform, which is used for confirming the product data screening, includes:
confirming a classification page of the target data platform;
and capturing the category data according to the confirmed classification page.
Optionally, before the step of extracting the product information of the target data platform according to the category data and storing the product information in a preset database, the method further includes:
determining a data hierarchy of the target data platform according to the category data;
and establishing a preset database by the data hierarchy.
Optionally, the step of extracting the product information of the target data platform according to the category data and storing the product information in a preset database includes:
and creating data extraction conditions according to the category data, and extracting product information in the target data platform based on the data extraction conditions.
Optionally, the step of screening the extracted product information for data information meeting the user data requirement and pushing the data information to a corresponding ERP platform includes:
generating a data screening condition according to the user data requirement;
and screening the data information meeting the data requirements of the user from the product information according to the generated data screening conditions.
Optionally, the step of screening the extracted product information for data information meeting the user data requirement and pushing the data information to a corresponding ERP platform includes:
generating a data screening condition according to the user data requirement;
and screening the data information meeting the data requirements of the user from the product information according to the generated data screening conditions.
In addition, in order to achieve the above object, the present invention further provides a big data product development screening apparatus, including: the device comprises a memory and a processor, wherein the memory stores a computer program capable of being called by the processor, and the computer program realizes the steps of the big data product development screening method when being executed by the processor.
The invention also provides a computer readable storage medium, wherein the computer readable storage medium is stored with a big data product development screening program, and the big data product development screening program realizes the steps of the big data product development screening method when being executed by a processor.
The embodiment of the invention provides a big data product development and screening method. Confirming a target data platform for product data screening, and capturing category data of the target data platform; extracting product information of the target data platform according to the category data and storing the product information into a preset database, wherein the product information comprises product summary data, product detail data, product sales detail data, product comment detail data and product listing time detail data; and screening data information meeting the user data requirements from the extracted product information and pushing the data information to a corresponding ERP platform. According to the invention, product information is captured on the data platform according to the data level according to the requirements of user application data, and data meeting conditions are screened based on the captured product information and pushed to the ERP for calling, so that the technical problems of product data screening and screening of the data platform are realized.
Drawings
FIG. 1 is a schematic diagram of a terminal \ device structure of a hardware operating environment according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a big data product development screening method according to a first embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The main solution of the embodiment of the invention is as follows: confirming a target data platform for product data screening, and capturing category data of the target data platform; extracting product information of the target data platform according to the category data and storing the product information into a preset database, wherein the product information comprises product summary data, product detail data, product sales detail data, product comment detail data and product listing time detail data; and screening data information meeting the user data requirements from the extracted product information and pushing the data information to a corresponding ERP platform.
The method has the technical problem of difficulty caused by excessive data sources in the development process of E-commerce products.
The invention provides a solution, product information is captured on a data platform according to data levels according to the requirements of user application data, the data meeting conditions are screened based on the captured product information and pushed to ERP for calling, and the technical problem of data screening based on the product data platform is achieved.
As shown in fig. 1, fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present invention.
The terminal of the embodiment of the invention can be PC terminal equipment.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the terminal structure shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a big data product development filter.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be configured to invoke the big data product development filter stored in the memory 1005 and perform the following operations:
confirming a target data platform for product data screening, and capturing category data of the target data platform;
extracting product information of the target data platform according to the category data and storing the product information into a preset database, wherein the product information comprises product summary data, product detail data, product sales detail data, product comment detail data and product listing time detail data;
and screening data information meeting the user data requirements from the extracted product information and pushing the data information to a corresponding ERP platform.
Further, the processor 1001 may call the big data product development filter stored in the memory 1005, and also perform the following operations:
acquiring user data requirements, and confirming a target data platform according to the user data requirements;
and creating a data connection relation with the target data platform.
Further, the processor 1001 may call the big data product development filter stored in the memory 1005, and also perform the following operations:
confirming a data application interface of the target data platform;
and creating a data connection relation with the data application interface.
Further, the processor 1001 may call the big data product development filter stored in the memory 1005, and also perform the following operations:
confirming a classification page of the target data platform;
and capturing the category data according to the confirmed classification page.
Further, the processor 1001 may call the big data product development filter stored in the memory 1005, and also perform the following operations:
determining a data hierarchy of the target data platform according to the category data;
and establishing a preset database by the data hierarchy.
Further, the processor 1001 may call the big data product development filter stored in the memory 1005, and also perform the following operations:
and creating data extraction conditions according to the category data, and extracting product information in the target data platform based on the data extraction conditions.
Further, the processor 1001 may call the big data product development filter stored in the memory 1005, and also perform the following operations:
generating a data screening condition according to the user data requirement;
and screening the data information meeting the data requirements of the user from the product information according to the generated data screening conditions.
Further, the processor 1001 may call the big data product development filter stored in the memory 1005, and also perform the following operations:
extracting a product identifier in the data information, and confirming whether the ERP platform stores repeated data of the product identifier, wherein the product identifier comprises a product name, a product picture and a product description;
and if the ERP platform does not store repeated data, pushing the data information to the ERP platform.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of a big data product development screening method applied to a dioptric topographer, the big data product development screening method including:
step S10, confirming a target data platform for product data screening, and capturing category data of the target data platform;
and confirming a corresponding target data platform according to the current user data requirement, wherein the target data platform is defined as a data source platform for screening the user data requirement, namely product data, and comprises an online shopping platform, a data storage platform and other system platforms with data application. Analyzing the data hierarchy of the target data platform according to the confirmed target data platform, wherein the data hierarchy can be defined as a data classification structure of the target data platform, and the data classification structure of the data application platform is generally embodied in a classification structure of product classification, so that after the target data platform screened by the current product data is confirmed, the data hierarchy of the target data platform is determined by using a classification page of a product in the target data platform, and category data of the target data platform, namely the target data platform screened by the confirmed product data, is extracted based on the classification page, and the step of capturing the category data of the target data platform comprises the following steps:
confirming a classification page of the target data platform;
and capturing the category data according to the confirmed classification page.
According to the currently confirmed target data platform, confirming a classification page of the target data platform, wherein the classification page is defined as a product classification catalog of the current target data platform, the product classification catalog of the classification page is related to the application type of the current target data platform, and therefore, according to the confirmed classification page, the category data of the classification page is captured, and the category data is the product category data and comprises product identification. Product classification, product name, sales volume, product picture and other specific product information, and the sales condition of the product, such as sales volume, time on shelf, historical sales record and other information, can be analyzed from the product information extracted by the product classification target.
Further, before the step of capturing category data of the target data platform, the step of confirming the target data platform for product data screening further includes:
acquiring user data requirements, and confirming a target data platform according to the user data requirements;
and creating a data connection relation with the target data platform.
The method comprises the steps of obtaining user data requirements of a current user, wherein the user data requirements are data characteristic information based on application, and accordingly, determining a target data platform based on the data characteristic information, namely a data source platform based on the data characteristic information of the current application, including but not limited to a public/non-public system platform with data application, such as a shopping platform. The method comprises the following steps of confirming a corresponding target data platform based on current user data requirements, confirming a connection mode of the target data platform, and establishing a data connection relation with the target data platform in the connection mode, wherein in addition, in system platform application, data transmission work is generally carried out through a data interface, so that when target data of the target data platform are captured, a data connection relation is established according to interface contents, namely the step of establishing the data connection relation with the target data platform comprises the following steps:
confirming a data application interface of the target data platform;
and creating a data connection relation with the data application interface.
And confirming a data application interface of the target data platform according to the currently confirmed target data platform, establishing a data connection relation with the data application interface based on the connection mode of the data application interface, and testing the connection condition of the established data connection relation under the condition that the data application interface can be normally used so as to capture target data after confirming that the connection is normal.
Step S20, extracting product information of the target data platform according to the category data and storing the product information into a preset database, wherein the product information comprises product summary data, product detail data, product sales detail data, product comment detail data and product listing time detail data;
after category data is extracted from a currently confirmed target data platform, product information is captured on the target data platform by taking the category data as a condition, wherein the product information is product data information of detailed products under the category data, and comprises product summary data (attributes such as category codes, category names, product codes and product delivery places), product detail data (attributes such as codes, titles, sales volumes, colors, sizes, prices and comments), product sales volume detail data (attributes such as product codes, product names, total product sales volumes and recent product sales volumes), product comment detail data and product listing time detail data. Further, the product information may have other product data different from the specific information content of the product information according to the data types of different target data platforms, and the specific information content of the product information is related to the product information setting content of the target data platform. Therefore, based on the data classification condition, when capturing product information based on the category data, the step of performing data extraction operation on the condition of the category data, that is, extracting the product information of the target data platform according to the category data and storing the product information in a preset database includes:
and creating data extraction conditions according to the category data, and extracting product information in the target data platform based on the data extraction conditions.
According to the currently captured category data, creating a data extraction condition by taking the category data as a condition, wherein the data extraction condition is created based on category information of the current category data, the category data is data hierarchy information of a current target data platform, so that the category data comprises a plurality of hierarchies and a plurality of classification information under the plurality of hierarchies, and different category data can have different data extraction conditions when the data extraction condition is created by the category data. Furthermore, when extracting product information according to different data extraction conditions, certain data screening operation can be performed according to category data of the data extraction conditions so as to avoid capturing invalid data, for example, the category data is a jacket, and when generating data extraction conditions of the jacket, the creation time can be defined within 30 days, so that the accuracy of data extraction is improved.
As described above, before the step of extracting the product information of the target data platform according to the category data and storing the product information into the preset database, the method further includes:
determining a data hierarchy of the target data platform according to the category data;
and establishing a preset database by the data hierarchy.
The method comprises the steps of confirming a data hierarchy of product information according to the product information captured from a target data platform at present, wherein the data hierarchy is classification information of category data of the product information, and thus, creating a preset database based on the data hierarchy to store the product information, wherein the generated preset database comprises but is not limited to data sub-tables, data connection relations among the data tables, table headers and other contents. Therefore, when the extracted product information is stored in a preset database, the product information needs to be analyzed, the analyzed product information is stored in different data tables in a classified manner according to the analyzed data of the product information, the information classification of the product information is analyzed according to the captured product information, the product information is stored in a preset database classification data table in a classified manner according to the analysis result, the preset database classification data table comprises but is not limited to a sales detail table, a product classification table, a product category data table and the like, and the specific type table is related to the data classification of the product information.
And step S30, screening the extracted product information for data information meeting the data requirements of the user and pushing the data information to a corresponding ERP platform.
According to the product information extracted from the target data platform, screening data information meeting the user data requirement from the product information, wherein the screened data information is defined as data information with data characteristics in the user data requirement, such as data characteristics related to highest sales volume, latest marketing and the like, further, future product sales trend can be calculated based on the obtained data characteristics, and the data screening work related to sales volume can be defined as data screening and analyzing work of product explosion money, and the specific operation steps of the data screening and analyzing work can include but are not limited to sales volume difference of the same product in a certain time period, the data screening and analyzing work can be analyzed based on the product information grabbed by the current target data platform, namely when the product information is extracted or not extracted, the data screening condition can be limited to carry out screening work of historical product information or real-time product information of the product, and performing data analysis based on the data screening result to obtain product information. Therefore, when the user application data is screened, a data screening condition based on the target data needs to be created, and data information is screened from the target data according to the data screening condition, that is, the data information meeting the user data requirement is screened from the extracted product information and pushed to a corresponding ERP platform, which includes the steps of:
generating a data screening condition according to the user data requirement;
and screening the data information meeting the data requirements of the user from the product information according to the generated data screening conditions.
And generating corresponding data screening conditions according to the current user data requirements, wherein the data screening conditions are based on data characteristics, such as sales, new products, growth speed and the like, of data information meeting the user data application requirements in the currently captured target data. And screening corresponding data information from the captured product information according to the generated data screening conditions, outputting the screened data information to an ERP platform, and providing the data information to a product development place for reference development operation.
In addition, when the captured data information based on the product information is pushed to the ERP platform for storage, the repeated product data needs to be eliminated in consideration of the data redundancy of the ERP platform, that is, the step of screening the data information meeting the user data requirement from the extracted product information and pushing the data information to the corresponding ERP platform includes:
extracting a product identifier in the data information, and confirming whether the ERP platform stores repeated data of the product identifier, wherein the product identifier comprises a product name, a product picture and a product description;
and if the ERP platform does not store repeated data, pushing the data information to the ERP platform.
According to the product information extracted from a target data platform at present, when the product information is pushed to a corresponding ERP platform for storage, information based on a product identifier in the product information is extracted for data duplication checking operation, the product identifier is defined as a unique identifier of the product information and includes but is not limited to a product name, a product picture, product description and the like, the specific requirement is determined based on data characteristics of the current data information, further, when the extracted product identifier is used for data duplication checking, duplication checking work is required to be carried out based on data stored in the current ERP, namely whether the ERP platform has the same product information as the product identifier or not is searched, and if the ERP platform does not store the same data information as the product identifier, the data information is pushed to the corresponding ERP platform for storage.
In addition, when the product information is captured based on the target data platform, since the target data platform includes the types of sales platform, supplier platform and the like in the application type, when the data platform is applied, based on the product information of the data platform, the product data has considerable data throughput, in the case of data throughput, which may be in billions, the data screening method provided in this embodiment, the server distributed technology executing the method can automatically capture platform product data from the target data platform through the crawler technology, and when capturing data, different data extraction conditions can be set according to the data application requirements for data screening, for example, platform sales data, platform product data, etc., and a specific lesson performs 24-hour uninterrupted data screening work on a plurality of servers by setting relevant screening conditions by relevant technicians.
In this embodiment, product information is captured on a data platform according to a data hierarchy according to the requirement of user application data, and data meeting conditions is screened based on the captured product information and pushed to an ERP for calling, so that the technical problem of product data screening based on the data platform is solved.
Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, where a big data product development filter is stored on the computer-readable storage medium, and when executed by a processor, the big data product development filter implements the following operations:
confirming a target data platform for product data screening, and capturing category data of the target data platform;
extracting product information of the target data platform according to the category data and storing the product information into a preset database, wherein the product information comprises product summary data, product detail data, product sales detail data, product comment detail data and product listing time detail data;
and screening data information meeting the user data requirements from the extracted product information and pushing the data information to a corresponding ERP platform.
Further, the big data product development filter, when executed by the processor, further performs the following operations:
acquiring user data requirements, and confirming a target data platform according to the user data requirements;
and creating a data connection relation with the target data platform.
Further, the big data product development filter, when executed by the processor, further performs the following operations:
confirming a data application interface of the target data platform;
and creating a data connection relation with the data application interface.
Further, the big data product development filter, when executed by the processor, further performs the following operations:
confirming a classification page of the target data platform;
and capturing the category data according to the confirmed classification page.
Further, the big data product development filter, when executed by the processor, further performs the following operations:
determining a data hierarchy of the target data platform according to the category data;
and establishing a preset database by the data hierarchy.
Further, the big data product development filter, when executed by the processor, further performs the following operations:
and creating data extraction conditions according to the category data, and extracting product information in the target data platform based on the data extraction conditions.
Further, the big data product development filter, when executed by the processor, further performs the following operations:
generating a data screening condition according to the user data requirement;
and screening the data information meeting the data requirements of the user from the product information according to the generated data screening conditions.
Further, the big data product development filter, when executed by the processor, further performs the following operations:
extracting a product identifier in the data information, and confirming whether the ERP platform stores repeated data of the product identifier, wherein the product identifier comprises a product name, a product picture and a product description;
and if the ERP platform does not store repeated data, pushing the data information to the ERP platform.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A big data product development screening method is characterized by comprising the following steps:
confirming a target data platform for product data screening, and capturing category data of the target data platform;
extracting product information of the target data platform according to the category data and storing the product information into a preset database, wherein the product information comprises product summary data, product detail data, product sales detail data, product comment detail data and product listing time detail data;
and screening data information meeting the user data requirements from the extracted product information and pushing the data information to a corresponding ERP platform.
2. The big data product development screening method of claim 1, wherein before the step of capturing the category data of the target data platform, the step of confirming the target data platform for product data screening further comprises:
acquiring user data requirements, and confirming a target data platform according to the user data requirements;
and creating a data connection relation with the target data platform.
3. The big data product development screening method of claim 2, wherein the step of creating a data connection relationship with the target data platform comprises:
confirming a data application interface of the target data platform;
and creating a data connection relation with the data application interface.
4. The big data product development screening method of claim 1, wherein the step of confirming the target data platform for product data screening and capturing the category data of the target data platform comprises:
confirming a classification page of the target data platform;
and capturing the category data according to the confirmed classification page.
5. The big data product development screening method of claim 1, wherein before the step of extracting the product information of the target data platform according to the category data and storing the product information in a preset database, the method further comprises:
determining a data hierarchy of the target data platform according to the category data;
and establishing a preset database by the data hierarchy.
6. The big data product development screening method of claim 1, wherein the step of extracting the product information of the target data platform according to the category data and storing the product information in a preset database comprises:
and creating data extraction conditions according to the category data, and extracting product information in the target data platform based on the data extraction conditions.
7. The big data product development screening method of claim 1, wherein the step of screening the extracted product information for data information meeting the user data requirements and pushing the data information to a corresponding ERP platform comprises:
generating a data screening condition according to the user data requirement;
and screening the data information meeting the data requirements of the user from the product information according to the generated data screening conditions.
8. The big data product development screening method of claim 1, wherein the step of screening the extracted product information for data information meeting the user data requirements and pushing the data information to a corresponding ERP platform comprises:
extracting a product identifier in the data information, and confirming whether the ERP platform stores repeated data of the product identifier, wherein the product identifier comprises a product name, a product picture and a product description;
and if the ERP platform does not store repeated data, pushing the data information to the ERP platform.
9. The big data product development screening device is characterized by comprising: a memory, a processor, and a big data product development filter stored on the memory and executable on the processor, the big data product development filter when executed by the processor implementing the steps of the big data product development filter method of any of claims 1 to 8.
10. A computer-readable storage medium, comprising a big data product development filter stored on the computer-readable storage medium, the big data product development filter, when executed, implementing the steps of the big data product development filter method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010571851.7A CN111723117A (en) | 2020-06-19 | 2020-06-19 | Big data product development screening method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010571851.7A CN111723117A (en) | 2020-06-19 | 2020-06-19 | Big data product development screening method and device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111723117A true CN111723117A (en) | 2020-09-29 |
Family
ID=72569853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010571851.7A Pending CN111723117A (en) | 2020-06-19 | 2020-06-19 | Big data product development screening method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111723117A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015055094A1 (en) * | 2013-10-15 | 2015-04-23 | 北京百度网讯科技有限公司 | Method and device for providing screening conditions and method and device for searching |
| CN104965904A (en) * | 2015-06-30 | 2015-10-07 | 北京奇虎科技有限公司 | Multi-platform data grabbing method and apparatus |
| CN110334185A (en) * | 2019-07-05 | 2019-10-15 | 政采云有限公司 | The treating method and apparatus of data in a kind of platform |
-
2020
- 2020-06-19 CN CN202010571851.7A patent/CN111723117A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015055094A1 (en) * | 2013-10-15 | 2015-04-23 | 北京百度网讯科技有限公司 | Method and device for providing screening conditions and method and device for searching |
| CN104965904A (en) * | 2015-06-30 | 2015-10-07 | 北京奇虎科技有限公司 | Multi-platform data grabbing method and apparatus |
| CN110334185A (en) * | 2019-07-05 | 2019-10-15 | 政采云有限公司 | The treating method and apparatus of data in a kind of platform |
Non-Patent Citations (1)
| Title |
|---|
| 周亮: ""分类-产品"结构的网页数据精确抽取方法探寻" * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110019486B (en) | Data acquisition method, device, equipment and storage medium | |
| CN106844372B (en) | Logistics information query method and device | |
| CN110956269A (en) | Data model generation method, device, equipment and computer storage medium | |
| CN110569159A (en) | Baffle generation method, device, equipment and computer storage medium | |
| CN111553137B (en) | Report generation method and device, storage medium and computer equipment | |
| CN111475694A (en) | Data processing method, device, terminal and storage medium | |
| CN109784738B (en) | Approval method and approval device | |
| CN111061733B (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| US9665574B1 (en) | Automatically scraping and adding contact information | |
| CN115617780A (en) | Data import method, device, equipment and storage medium | |
| CN114265737A (en) | System migration integrity detection method and device and electronic equipment | |
| CN111723117A (en) | Big data product development screening method and device and computer readable storage medium | |
| CN108345600B (en) | Management of search application, data search method and device thereof | |
| CN113672497A (en) | Method, device and equipment for generating non-buried point event and storage medium | |
| CN114880239A (en) | Interface automation testing framework and method based on data driving | |
| CN111597519B (en) | Customer data storage method and device based on customer management system, electronic equipment and storage medium | |
| JP7108566B2 (en) | Digital evidence management method and digital evidence management system | |
| CN114356396A (en) | Service publishing method and device adaptive to multiple services | |
| CN113435830A (en) | Mail information summarizing method, system, electronic device and storage medium | |
| CN113487053A (en) | Maintenance service method, device, equipment and computer readable storage medium | |
| CN112257450A (en) | Data processing method, device, readable storage medium and equipment | |
| CN111831683A (en) | Automatic auditing method and system based on dynamic extended scene matching | |
| CN112540820A (en) | User interface updating method and device and electronic equipment | |
| CN116010467B (en) | Risk discovery method, device, equipment and storage medium based on communication map | |
| US11995584B2 (en) | Training assignment tool |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |