CN112347336A - System for realizing data collection and distribution based on web page and processing method thereof - Google Patents
System for realizing data collection and distribution based on web page and processing method thereof Download PDFInfo
- Publication number
- CN112347336A CN112347336A CN202011252319.5A CN202011252319A CN112347336A CN 112347336 A CN112347336 A CN 112347336A CN 202011252319 A CN202011252319 A CN 202011252319A CN 112347336 A CN112347336 A CN 112347336A
- Authority
- CN
- China
- Prior art keywords
- page
- data
- processing
- distribution
- component
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009826 distribution Methods 0.000 title claims abstract description 82
- 238000013480 data collection Methods 0.000 title claims abstract description 65
- 238000003672 processing method Methods 0.000 title abstract description 7
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000012545 processing Methods 0.000 claims description 83
- 238000007726 management method Methods 0.000 claims description 26
- 238000013461 design Methods 0.000 claims description 20
- 238000007405 data analysis Methods 0.000 claims description 15
- 238000013523 data management Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 5
- 230000002688 persistence Effects 0.000 claims description 4
- 238000007619 statistical method Methods 0.000 claims description 4
- 230000006978 adaptation Effects 0.000 claims description 3
- 238000012938 design process Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 3
- 230000002452 interceptive effect Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 230000006386 memory function Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a system for realizing data collection and distribution based on a web page and a processing method thereof. The method can quickly create a published data collection page, collect data in real time, analyze and process the data and distribute the data through different client web pages, and therefore the office can quickly respond to the business needs. Data are collected based on visiting web pages, compatibility is strong, and the application range is wide. The collection page can be rapidly produced, the collection fields of the page can be changed at will, and the operation is simple. The page can be adapted to various clients such as a pc end, a mobile phone end and the like.
Description
Technical Field
The invention relates to a data collecting and distributing system and a processing method thereof, in particular to a system for realizing data collecting and distributing based on web pages and a processing method thereof.
Background
The data collection and distribution based on the web page is realized by inputting the value of a corresponding data item in the web page at the web page end, clicking to submit the data to achieve the purpose of data collection, processing and packaging the data at the server end, and providing the data for an external system to call in a service mode to realize data distribution.
Currently, in the government affair field, the government offices need to collect some business data externally or obtain some business data among departments to perform statistical analysis to meet the business targets. If the channel of data collection is relatively single, it can be realized by a single system and a certain amount of manpower. However, with the development of information technology and the increase of cooperation among departments, a single system and certain manpower cannot meet the requirements of each business office, certain data must be collected from the cooperation departments by manpower, then summary analysis is carried out, and if data circulation among the departments is involved, the collected data is tidied and transmitted to the cooperation departments, so that the self goal and achievement are achieved.
The processing flow of collecting and transferring data for the office above often results in poor accuracy, poor timeliness, poor data quality, and some other uncertainties. Mainly aims at the following aspects:
1. firstly, the validity and the accuracy of the data can be greatly reduced by a mode of manually inputting the data.
2. Data inaccuracy exists through manual data collection, if data fields are adjusted repeatedly, the collected data are inconsistent, workload of manual intervention arrangement is large, and error data are likely to be generated.
3. The accuracy of data analysis by manual processing is poor and the efficiency is low.
4. The collaboration data is transmitted in the form of electronic documents, and for collaboration departments, the collaboration departments process the data again after receiving the data documents, and unnecessary labor cost is wasted.
In view of the above-mentioned drawbacks, the present designer is actively making research and innovation to create a system for realizing data collection and distribution based on web pages and a processing method thereof, so that the system has industrial utility value.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a system for implementing data collection and distribution based on web pages and a processing method thereof.
The invention relates to a data collection and distribution system based on web pages, which comprises a data collection management terminal, a client filling component and a data processing core engine,
the data collection management terminal is provided with a page management module and a data management module, the page management module is used for designing pages with different service collection requirements, and the data management module is used for managing data collected by the client terminal;
the client-side filling assembly is provided with a page filling module and a filling recording module, the page filling module is used for collecting data of a user, and the filling recording module is used for displaying the filled record of the user;
the data processing core engine comprises a data collection and analysis module and a data distribution and processing module, wherein the data collection and analysis module is used for processing, analyzing and collecting page data, and the data distribution and processing module is used for distributing and processing the collected data.
Further, in the above system for implementing data collection and distribution based on web pages, the elements in the industry include one or more of a page title, a page layout configuration, a page component configuration, a page filing policy configuration, and a page data distribution configuration;
the data collected by the management client side comprises one or more of data list display, data statistical analysis and data export.
Furthermore, the data collection and distribution system implemented on the basis of the web page is described above, wherein the data of the users includes one or more of page adaptation display, page data item verification processing, and page data submission verification processing; the record which is shown and filled by the user comprises one or more of record list display, record multi-dimensional query function and detailed data display of filling records.
Furthermore, the data collection and distribution system is implemented based on a web page, wherein the processing, analysis and collection of the page data includes one or more of data rule checking, data processing, data dimension statistics, data persistence processing and data exception processing; the distribution processing of the collected data comprises one or more of data assembling processing, data sending logic processing and data sending exception processing.
5. The method for realizing data collection and distribution based on the web page comprises the following steps:
firstly, quickly designing a collection page;
secondly, the page is adapted to a multi-end browser;
dragging an interactive mode design page;
step four, data collection is realized based on the page;
step five, realizing real-time data distribution based on the page;
in the first step: configuring basic service metadata of a page, configuring layout metadata of the page, configuring page components, issuing after configuration is completed, producing a page address in a page list page, and accessing and filling a copy address in a browser;
in the second step, whether the type of the page is a PC end page or a mobile phone end page is configured in advance in the page design process, and after the selection is completed, different page templates and styles are loaded when the page is rendered in a browser according to the type of the page;
in the third step, the selection of the components is completed through the dragging of a mouse;
in the fourth step, a data collection page is designed and issued by the page management module, after the page is issued, metadata information of the page is produced, the metadata of the page can be persisted into the database, a user accesses the database at the browser end, and related data is updated into an example data group object of the metadata information of the page;
and in the fifth step, distributing configuration metadata is configured, a user accesses the browser end, field data contained in the page is input, and the page program judges and distributes the data.
Further, in the above method for implementing data collection and distribution based on a web page, the parameters of the layout metadata of the page in the first step include one or more of width, height, font, background, and page type;
the configuration process of the page components in the first step is that the needed components are selected from a component library and dragged to a page design panel by a mouse; clicking the component by using a mouse, popping up a page configured by basic metadata of the component, and configuring the basic metadata, wherein parameters corresponding to the basic metadata configuration comprise one or more of component name, display name, whether to be filled, default value, input length limit and custom regular expression configuration; configuring form filling strategy metadata; and after the filling strategy is configured, previewing or storing by one key and releasing a button.
Furthermore, in the above method for realizing data collection and distribution based on web pages, in the second step, the PC-side page and the mobile phone-side page are processed independently,
the PC side page is provided with at least one visual area with the minimum width, and the page is rendered in the visual area based on the style; the components in the page of the mobile phone terminal page are displayed in a hundred-percent width mode and are arranged from top to bottom in a block-shaped area, and the width of each component is changed along with the width of the mobile phone.
Furthermore, in the third step, when the page selects a component, the mouse is moved to the icon of the component to be selected, then the left button is clicked and does not need to be released, the mouse is dragged to the page design container, then the left button of the mouse is released, the effect of the component can be displayed in the page design container, and the addition of one component is completed;
when the component moves, the component which needs to move in the page design container is clicked, a moving icon appears, the mouse is moved to the position right above the icon, the left button is clicked and not loosened, and then the mouse is dragged up and down to move the position of the component.
Furthermore, in the fourth step, the metadata information in the data collection page includes one or more of a page unique identifier, a total filling amount, page configuration metadata, component configuration metadata, distribution configuration metadata, filling policy metadata, and an instance data array;
a user accesses field data contained in an input page at a browser end to submit, a page program can assemble the data input by the user, then obtains a unique identifier of a current page, sends the unique identifier and a component data set to an engine of a back-end server for processing data collection and analysis in a JSON format, and after the engine receives the data, the component data set is updated to an example data set object of metadata information of the page according to the unique identifier of the page;
the assembly rule is that the field name of each component is used as a key value, object data of a data value is filled, and a plurality of components form a data set.
Still further, in the method for realizing data collection and distribution based on the web page, in the fifth step, distribution configuration metadata including one or more of a received system name, an interface system interface address and a failure attempt number is configured and distributed during page design, and the page is released after configuration is completed;
the user accesses the field data contained in the input page at the browser end, clicks a submit button of the page, the page program starts to assemble the related data, after the assembly is completed, the page program judges, if distribution configuration exists, the data is sent to a data distribution processing service interface provided by a data distribution processing engine,
the processing mode of the data distribution processing service interface comprises,
the method comprises the steps of actively pushing, after receiving data, a distribution processing service in an engine sends the post-filled data of a user bar to an interface address of a receiving system according to a distribution configuration strategy, and if the sending fails, executing retry operation according to the strategy;
the external assistance system actively requests the mode, the external system regularly requests a data synchronization service interface provided by the core engine, and receives and processes the data in the system, and the parameters of the data synchronization service interface comprise parameters such as page unique identification, data filling time, data filling end type, page filling data and the like.
By the scheme, the invention at least has the following advantages:
1. the method can quickly create a published data collection page, collect data in real time, analyze and process the data and distribute the data through different client web pages, and therefore the office can quickly respond to the business needs.
2. Data are collected based on visiting web pages, compatibility is strong, and the application range is wide.
3. The collection page can be rapidly produced, the collection fields of the page can be changed at will, and the operation is simple.
4. The page can be adapted to various clients such as a pc end, a mobile phone end and the like.
5. The data analysis, processing and distribution do not need manual intervention.
6. The collection of pages is light in weight and easy to operate, and the operation experience of a user is improved.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood and to implement them in accordance with the contents of the description, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings.
Drawings
FIG. 1 is a schematic structural diagram of a data collection and distribution system implemented based on web pages.
Fig. 2 is a schematic diagram of a data active distribution flow.
Fig. 3 is a schematic diagram of a data passive distribution flow.
FIG. 4 is a schematic view of a page lifecycle state.
FIG. 5 is a data page creation publication flow diagram.
Fig. 6 is a flow chart of data collection analysis.
Detailed Description
The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
The data collection and distribution system based on web pages in fig. 1 is characterized in that: the system comprises a data collection management terminal, a client filling component and a data processing core engine which are communicated with each other. Specifically, the data collection management terminal is configured with a page management module and a data management module, the page management module is used for designing pages with different service collection requirements, and the data management module is used for managing data collected by the client terminal. Meanwhile, the adopted client-side filling component is provided with a page filling module and a filling recording module, the page filling module is used for collecting data of a user, and the filling recording module is used for displaying the filled record of the user. And the data processing core engine comprises a data collection and analysis module and a data distribution and processing module, wherein the data collection and analysis module is used for processing, analyzing and collecting the page data, and the data distribution and processing module is used for distributing and processing the collected data.
In a preferred embodiment of the present invention, the elements in the industry include one or more of a page title, a page layout configuration, a page component configuration, a page filing policy configuration, and a page data distribution configuration; the data collected by the management client side comprises one or more of data list display, data statistical analysis and data export. Meanwhile, the data of the collection user comprises one or more of page adaptation display, page data item verification processing and page data submission verification processing; the record which is shown and filled by the user comprises one or more of record list display, record multi-dimensional query function and detailed data display of filling records. And the processing, analyzing and collecting page data comprises one or more of data rule checking processing, data dimension statistics, data persistence processing and data exception processing, and the distribution processing of the collected data comprises one or more of data assembling processing, data sending logic processing and data sending exception processing.
As shown in fig. 2 to 6, in order to better implement the present invention, a method for implementing data collection and distribution based on web pages is now provided, which is characterized by comprising the following steps:
step one, a collection page is designed rapidly. Specifically, basic business metadata (which may include a page header) of the configuration page, and layout metadata of the configuration page. And then, configuring the page component, and issuing after the configuration is finished. Thus, the page address is generated in the page list page, and the user can access and fill in the copied address in the browser.
And step two, the page is adapted to the multi-end browser. During the period, the type of the page is configured in advance in the page design process, namely the PC end page or the mobile phone end page. And after the user finishes the selection, loading different page templates and styles according to the page types when the page is rendered in the browser.
And step three, dragging the interactive mode design page by the user to perform corresponding operation, and finishing the selection of the components through mouse dragging.
And step four, realizing data collection based on the page. During implementation, a data collection page is designed and published by the page management module, after the page is published, metadata information of the page is produced, the metadata of the page is persisted in a database, a user accesses the database at a browser end, and related data is updated into an example data group object of the metadata information of the page.
And step five, realizing real-time data distribution based on the page. By configuring and distributing configuration metadata, a user accesses at a browser end, inputs field data contained in a page, and distributes the data after the page program judges.
In terms of practical implementation, the parameters of the layout metadata of the page in the step one include one or more of width, height, font, background, and page type. Also, these parameters have default values, which may or may not be filled in if not required. Meanwhile, the configuration process of the page component in the first step is as follows: selecting a needed component from the component library, and dragging the component to a page design panel by using a mouse; clicking the component by a mouse, popping up a page configured by basic metadata of the component, and configuring the basic metadata, wherein parameters corresponding to the basic metadata configuration comprise one or more of component name, display name, whether to be filled, default value, input length limit and custom regular expression configuration. These parameters vary from component to component, and are substantially different from component to component. The form is then configured to fill in the policy metadata (such parameters may include whether to fill in repeatedly). And after the filling strategy is configured, previewing or storing by one key and releasing a button.
In the second step, the PC side page and the mobile phone side page can be independently processed. Meanwhile, the size of the PC-side browser is larger, so that at least one visual area with the minimum width is arranged on a PC-side page, and the page is rendered in the area based on the style, so that the PC-side page is adaptive to browsers with different sizes. For the mobile phone end page, the components in the page are displayed in a hundred-percent width mode and are arranged from top to bottom in a block area, the width of the components is changed along with the width of the mobile phone, and the mobile phone end page can be compatible with mobile phone end browsers with different sizes. The reason for this is that the size of the mobile phone end is small, and the typesetting up and down arrangement satisfies the habit of using the mobile phone.
In the third step, when the page selects the component, the mouse is moved to the component icon (such as the upper middle part) to be selected, then the left button is clicked and is not loosened, the mouse is dragged to the page design container, then the left button of the mouse is loosened, the effect of the component can be displayed in the page design container, and the addition of one component is completed. Meanwhile, when the component moves, the component which needs to move in the page design container is clicked, a moving icon appears, the mouse is moved to the position right above the icon, the left button is clicked and not loosened, and then the mouse is dragged up and down to move the position of the component.
In step four, the metadata information in the data collection page includes one or more of page unique identifier, total filling amount, page configuration metadata, component configuration metadata, distribution configuration metadata, filling policy metadata, and example data array. In actual implementation, the instance data array may be empty.
A user accesses field data contained in an input page at a browser end to submit, a page program can assemble the data input by the user, then obtains a unique identifier of a current page, and sends the unique identifier and a component data set to a back-end server in a JSON format to process a data collection and analysis engine. After the engine receives the data, the component data set is updated to the instance data set object of the metadata information of the page according to the unique identification of the page. During implementation, the assembly rule is that the field name of each component is used as a key value, the object data of the data value is filled, and a plurality of components form a data set.
In step five, the distribution configuration metadata may be configured at page design time. Specifically, multiple configurations may be added, each configuration data may include one or more of a receiving system name, an interface system interface address, a number of failed attempts. The page is published after configuration is complete.
And then, the user accesses the field data contained in the input page at the browser end, clicks a submission button of the page, the page program starts to assemble the related data, after the assembly is finished, the page program judges, and if distribution configuration exists, the data are sent to a data distribution processing service interface provided by a data distribution processing engine.
Meanwhile, the processing mode of the data distribution processing service interface comprises an active pushing mode, after the distribution processing service in the engine receives the data, the data filled in the user bar is sent to the interface address of the receiving system in a post mode according to the strategy of distribution configuration, and if the sending fails, retry operation is executed according to the strategy.
According to practical implementation, the external system can also request the data synchronization service interface provided by the core engine regularly to receive and process the data into the system, and the parameters of the data synchronization service interface include the unique page identifier, the data filling time, the type of the data filling end, the page filling data and other parameters.
The working principle of the invention is as follows:
data collection and distribution system based on web pages as shown in FIG. 1
A data collection management end is formed by a data management and control center 1, a data core processing engine 2 is formed by a data processing core engine, and a client side report filling component is formed by a data report filling end 3. The data management and control center 1 is used for design and life cycle management of collected pages and management of page collected data. The data core processing engine is used for processing data processing, statistics and distribution of page user report. The data filling terminal 3 is mainly used for providing a user with a filling entry and a data submission collecting function.
Meanwhile, the data management and control center 1 includes a page management module 11 and a data management module 12. The page management module manages the page needing to be collected by a user, and comprises a page design function and a life cycle management function of the page. The data management module 12 mainly provides management functions of page data, including multi-dimensional statistical result display of data page filling, list display of page filling data, viewing of detailed information, and exporting of page filling data.
Meanwhile, the data core processing engine 2 includes a data collection and analysis module 21 and a data distribution processing module 22. The data collection and analysis module 21 is used for rule checking of page data and data analysis processing persistence in the database. The data distribution processing module 22 is configured to actively push data to a receiver interface according to distribution configuration of a page, and also start an exception timer to monitor an exception, and once transmission fails, repeatedly transmit data according to retry times of the distribution configuration. The data filling and reporting end 3 mainly comprises a page state rendering and adaptive display function, a data filling and reporting function, a non-submitted data memory function and a filling and reporting record checking function.
In view of the data active distribution flow process shown in fig. 2: after receiving the data sent by the page, the data distribution processing module inquires the distribution configuration metadata information of the page in a database according to the unique page identification parameter in the data, and generates a corresponding data sending task and a data sending abnormal timer according to the configuration information, wherein the data sending task pushes the data to a destination interface in a POST mode. If the pushing is abnormal, the sending task informs an abnormal timer, and the timer can repeatedly push data to a destination interface at regular time according to distribution configuration.
The passive data distribution flow shown in fig. 3 shows the process as follows: the collection, collection and analysis module receives data filled in real time, through collection and processing, the filled records are persisted to a database, and an assistance system calls a collected data query interface in a timing or long connection mode to obtain the data records filled in by the latest user.
In view of practical implementation, the lifecycle of the data collection page during the implementation of the present invention is shown in fig. 4, and includes five states of creation a, preview B, release C, off-shelf D, and deletion E. Where it can be changed from the create a state directly to the publish state C. After the shelf-off operation is performed, the data collection page becomes the most primitive creation state A.
As shown in fig. 5 and 6, during the implementation of the present invention:
firstly, entering a page management module to configure page service metadata, page layout information and a filling strategy.
And then adding page components, basic information of the configuration components and extended information metadata. And clicking to store and enter a page list page, clicking to release, and releasing the page to a data filling end web server. When a user accesses the collection page, the data filling end carries out multi-end adaptive rendering display according to the metadata configuration of the access page. When a user accesses a page to perform filling, if data is not submitted, a filling end program stores the filled data based on the browser client, and the next time the user accesses the page again, whether the data exists in the local cache of the browser is inquired, and if so, the data is automatically filled into a component of the corresponding page.
And then, after the data is filled in without errors, the client program assembles the related data into a data packet by clicking a submit button, calls a data providing service interface and sends the data to a data processing engine. After data packets sent from the front end are collected and processed, data are analyzed, verified, processed, distributed and analyzed, the processed data are persisted into a database, and data support is provided for a data management module and an external system in butt joint and pushing.
And ending the whole data collection processing flow.
The invention has the following advantages by the aid of the character expression and the accompanying drawings:
1. the method can quickly create a published data collection page, collect data in real time, analyze and process the data and distribute the data through different client web pages, and therefore the office can quickly respond to the business needs.
2. Data are collected based on visiting web pages, compatibility is strong, and the application range is wide.
3. The collection page can be rapidly produced, the collection fields of the page can be changed at will, and the operation is simple.
4. The page can be adapted to various clients such as a pc end, a mobile phone end and the like.
5. The data analysis, processing and distribution do not need manual intervention.
6. The collection of pages is light in weight and easy to operate, and the operation experience of a user is improved.
Furthermore, the indication of the orientation or the positional relationship described in the present invention is based on the orientation or the positional relationship shown in the drawings, and is only for convenience of describing the present invention and simplifying the description, but does not indicate or imply that the indicated device or configuration must have a specific orientation or be operated in a specific orientation configuration, and thus, should not be construed as limiting the present invention.
The terms "primary" and "secondary" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "primary" or "secondary" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically limited otherwise.
Also, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "connected" and "disposed" are to be construed broadly, e.g., as meaning fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; the two components can be directly connected or indirectly connected through an intermediate medium, and the two components can be communicated with each other or mutually interacted. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations. And it may be directly on the other component or indirectly on the other component. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or be indirectly connected to the other element.
It will be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like, refer to an orientation or positional relationship illustrated in the drawings, which are used for convenience in describing the invention and to simplify the description, and do not indicate or imply that the device or component being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the invention.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, it should be noted that, for those skilled in the art, many modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (10)
1. The system for realizing data collection and distribution based on the web pages is characterized in that: comprises a data collection management terminal, a client filling component and a data processing core engine,
the data collection management terminal is provided with a page management module and a data management module, the page management module is used for designing pages with different service collection requirements, and the data management module is used for managing data collected by the client terminal;
the client-side filling assembly is provided with a page filling module and a filling recording module, the page filling module is used for collecting data of a user, and the filling recording module is used for displaying the filled record of the user;
the data processing core engine comprises a data collection and analysis module and a data distribution and processing module, wherein the data collection and analysis module is used for processing, analyzing and collecting page data, and the data distribution and processing module is used for distributing and processing the collected data.
2. The web-based web page implementation data collection and distribution system according to claim 1, wherein: the elements in the industry comprise one or more of page titles, page layout configuration, page component configuration, page filling strategy configuration and page data distribution configuration;
the data collected by the management client side comprises one or more of data list display, data statistical analysis and data export.
3. The web-based web page implementation data collection and distribution system according to claim 1, wherein: the data of the set user comprises one or more of page adaptation display, page data item verification processing and page data submission verification processing; the record which is shown and filled by the user comprises one or more of record list display, record multi-dimensional query function and detailed data display of filling records.
4. The web-based web page implementation data collection and distribution system according to claim 1, wherein: the page data collected by the processing analysis comprises one or more of data rule checking processing, data dimension statistics, data persistence processing and data exception processing; the distribution processing of the collected data comprises one or more of data assembling processing, data sending logic processing and data sending exception processing.
5. The method for realizing data collection and distribution based on the web page is characterized by comprising the following steps:
firstly, quickly designing a collection page;
secondly, the page is adapted to a multi-end browser;
dragging an interactive mode design page;
step four, data collection is realized based on the page;
step five, realizing real-time data distribution based on the page;
in the first step: configuring basic service metadata of a page, configuring layout metadata of the page, configuring page components, issuing after configuration is completed, producing a page address in a page list page, and accessing and filling a copy address in a browser;
in the second step, whether the type of the page is a PC end page or a mobile phone end page is configured in advance in the page design process, and after the selection is completed, different page templates and styles are loaded when the page is rendered in a browser according to the type of the page;
in the third step, the selection of the components is completed through the dragging of a mouse;
in the fourth step, a data collection page is designed and issued by the page management module, after the page is issued, metadata information of the page is produced, the metadata of the page can be persisted into the database, a user accesses the database at the browser end, and related data is updated into an example data group object of the metadata information of the page;
and in the fifth step, distributing configuration metadata is configured, a user accesses the browser end, field data contained in the page is input, and the page program judges and distributes the data.
6. The web-based web page-based data collection and distribution method according to claim 5, wherein: the parameters of the layout metadata of the page in the first step comprise one or more of width, height, font, background and page type;
the configuration process of the page components in the first step is that the needed components are selected from a component library and dragged to a page design panel by a mouse; clicking the component by using a mouse, popping up a page configured by basic metadata of the component, and configuring the basic metadata, wherein parameters corresponding to the basic metadata configuration comprise one or more of component name, display name, whether to be filled, default value, input length limit and custom regular expression configuration; configuring form filling strategy metadata; and after the filling strategy is configured, previewing or storing by one key and releasing a button.
7. The web-based web page-based data collection and distribution method according to claim 5, wherein: in the second step, the PC end page and the mobile phone end page are independently processed,
the PC side page is provided with at least one visual area with the minimum width, and the page is rendered in the visual area based on the style; the components in the page of the mobile phone terminal page are displayed in a hundred-percent width mode and are arranged from top to bottom in a block-shaped area, and the width of each component is changed along with the width of the mobile phone.
8. The web-based web page-based data collection and distribution method according to claim 5, wherein: in the third step, when the page selects the component, the mouse is moved to the component icon needing to be selected, then the left button is clicked without being loosened, the mouse is dragged to the page design container, then the left button of the mouse is loosened, the effect of the component can be displayed in the page design container, and the addition of one component is completed;
when the component moves, the component which needs to move in the page design container is clicked, a moving icon appears, the mouse is moved to the position right above the icon, the left button is clicked and not loosened, and then the mouse is dragged up and down to move the position of the component.
9. The web-based web page-based data collection and distribution method according to claim 5, wherein: in the fourth step, the metadata information in the data collection page includes one or more of a page unique identifier, a total filling amount, page configuration metadata, component configuration metadata, distribution configuration metadata, filling policy metadata, and an example data array;
a user accesses field data contained in an input page at a browser end to submit, a page program can assemble the data input by the user, then obtains a unique identifier of a current page, sends the unique identifier and a component data set to an engine of a back-end server for processing data collection and analysis in a JSON format, and after the engine receives the data, the component data set is updated to an example data set object of metadata information of the page according to the unique identifier of the page;
the assembly rule is that the field name of each component is used as a key value, object data of a data value is filled, and a plurality of components form a data set.
10. The web-based web page-based data collection and distribution method according to claim 5, wherein: in the fifth step, distributing configuration metadata is configured during page design, wherein the distribution configuration metadata comprises one or more of a receiving system name, an interface system interface address and failure attempt times, and the page is issued after configuration is completed;
the user accesses the field data contained in the input page at the browser end, clicks a submit button of the page, the page program starts to assemble the related data, after the assembly is completed, the page program judges, if distribution configuration exists, the data is sent to a data distribution processing service interface provided by a data distribution processing engine,
the processing mode of the data distribution processing service interface comprises,
the method comprises the steps of actively pushing, after receiving data, a distribution processing service in an engine sends the post-filled data of a user bar to an interface address of a receiving system according to a distribution configuration strategy, and if the sending fails, executing retry operation according to the strategy;
the external assistance system actively requests the mode, the external system regularly requests a data synchronization service interface provided by the core engine, and receives and processes the data in the system, and the parameters of the data synchronization service interface comprise parameters such as page unique identification, data filling time, data filling end type, page filling data and the like.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011252319.5A CN112347336A (en) | 2020-11-11 | 2020-11-11 | System for realizing data collection and distribution based on web page and processing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011252319.5A CN112347336A (en) | 2020-11-11 | 2020-11-11 | System for realizing data collection and distribution based on web page and processing method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112347336A true CN112347336A (en) | 2021-02-09 |
Family
ID=74363261
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011252319.5A Pending CN112347336A (en) | 2020-11-11 | 2020-11-11 | System for realizing data collection and distribution based on web page and processing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112347336A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103631877A (en) * | 2013-11-08 | 2014-03-12 | 四川长虹电器股份有限公司 | Webpage table processing method |
| WO2017143498A1 (en) * | 2016-02-22 | 2017-08-31 | 美通云动（北京）科技有限公司 | Double-render-engine browser, and pc end web page application page conversion method and apparatus |
| CN108776702A (en) * | 2018-06-11 | 2018-11-09 | 浪潮软件股份有限公司 | A kind of data make a report on page user-defined visual configuration method |
| CN110795671A (en) * | 2019-10-29 | 2020-02-14 | 北京齐尔布莱特科技有限公司 | Data filling method, computing device and storage medium |
-
2020
- 2020-11-11 CN CN202011252319.5A patent/CN112347336A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103631877A (en) * | 2013-11-08 | 2014-03-12 | 四川长虹电器股份有限公司 | Webpage table processing method |
| WO2017143498A1 (en) * | 2016-02-22 | 2017-08-31 | 美通云动（北京）科技有限公司 | Double-render-engine browser, and pc end web page application page conversion method and apparatus |
| CN108776702A (en) * | 2018-06-11 | 2018-11-09 | 浪潮软件股份有限公司 | A kind of data make a report on page user-defined visual configuration method |
| CN110795671A (en) * | 2019-10-29 | 2020-02-14 | 北京齐尔布莱特科技有限公司 | Data filling method, computing device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110325961B (en) | Spreadsheet-based software application development | |
| US7051031B2 (en) | Method, system, and program for managing accesses to data objects by multiple user programs over a network | |
| US9015651B2 (en) | Gateway data distribution engine | |
| CN108197237B (en) | Visual data acquisition and display system | |
| US20020070953A1 (en) | Systems and methods for visualizing and analyzing conditioned data | |
| US9448860B2 (en) | Method and architecture for providing data-change alerts to external applications via a push service | |
| US20020122063A1 (en) | System and method for storing and displaying information | |
| US7783724B2 (en) | System and method for delivering business intelligence data in a client/server architecture | |
| US20070136268A1 (en) | Just in time loading list | |
| WO2007047300A1 (en) | Managing relationships between resources stored within a repository | |
| WO2008033415A2 (en) | Dynamic accessible reporting tool (dart) | |
| US11755531B1 (en) | System and method for storage of data utilizing a persistent queue | |
| JP2003091633A (en) | Information collecting and managing system and device, terminal device for system, recording medium recording collected information, method of browsing collected information, program and recording medium storing program | |
| US8059301B2 (en) | Systems and methods for mass printing | |
| CN105163142A (en) | User preference determination method, video recommendation method, user preference determination system and video recommendation system | |
| WO2014089230A2 (en) | Method and system of storing data files and retrieving data in a data file and requesting information from a computer | |
| US20230102947A1 (en) | Providing operations in accordance with worksheet relationships and data object relationships | |
| US6934721B2 (en) | Method, system, and program for managing information for an application program using a file management system | |
| US10795907B2 (en) | Management of data sources in database system | |
| CN115543282A (en) | Page code generation method and device, storage medium and computer equipment | |
| EP2698753A1 (en) | Data management system for generating a report document by linking technical data to intellectual property rights data | |
| US20060085492A1 (en) | System and method for modifying process navigation | |
| CN112347336A (en) | System for realizing data collection and distribution based on web page and processing method thereof | |
| CN100412858C (en) | Report generating system and method | |
| CN111309932B (en) | Comment data query method, comment data query device, comment data query equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |