CN111752688A - Data acquisition method and device, electronic equipment and storage medium - Google Patents
Data acquisition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111752688A CN111752688A CN202010496532.4A CN202010496532A CN111752688A CN 111752688 A CN111752688 A CN 111752688A CN 202010496532 A CN202010496532 A CN 202010496532A CN 111752688 A CN111752688 A CN 111752688A
- Authority
- CN
- China
- Prior art keywords
- function
- data acquisition
- service
- data
- configuration file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000006870 function Effects 0.000 claims abstract description 357
- 238000013480 data collection Methods 0.000 claims abstract description 22
- 238000005538 encapsulation Methods 0.000 claims abstract description 21
- 238000005516 engineering process Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 abstract description 10
- 230000008569 process Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
Abstract
The invention provides a data acquisition method, a data acquisition device, electronic equipment and a storage medium. The method comprises the following steps: receiving a data collection instruction, wherein the data collection instruction comprises parameters and a configuration file for acquiring data, the configuration file comprises at least one of address information and interface information of at least one function, the function is obtained by performing functional encapsulation on any data acquisition service, and the data acquisition services corresponding to the functions are different from each other; calling each function according to the configuration file, and executing each function based on the parameters; and acquiring an execution result of each function and returning. Therefore, decoupling is carried out through the relation of each data acquisition service, each function can be called independently or in a combined mode, flexibility can be improved, and time cost and communication cost are reduced.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data acquisition method and apparatus, an electronic device, and a storage medium.
Background
The data collection service is to collect desired data by calling other services or customized functions through some data in the sample data. Moreover, since the service requirements of different services are different, the data required by each service is correspondingly different, that is, the called service is not fixed, and there is a possibility that the service a needs to call two services, and the service b needs to call the other three existing services, etc., these services all need to be hard-coded in advance, the whole data collection program is completely issued to be used, and if the service that needs to be called is not in the accessed service, the data collection program needs to be regenerated.
Therefore, different services need to be called and different services are required, each service corresponds to which service, hard coding is required to be issued in advance, and the workload is large; moreover, if a user cannot adjust the service logic at will when accessing a new service, communication cost and time cost are increased, resulting in inflexible operation.
Disclosure of Invention
Embodiments of the present invention provide a data acquisition method, an apparatus, an electronic device, and a storage medium, so as to solve the problems of low flexibility, and high communication cost and time cost in the existing scheme.
In order to solve the technical problem, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a data acquisition method, including:
receiving a data collection instruction, wherein the data collection instruction comprises parameters and a configuration file for acquiring data, the configuration file comprises at least one of address information and interface information of at least one function, the function is obtained by performing functional encapsulation on any data acquisition service, and the data acquisition services corresponding to the functions are different from each other;
calling each function according to the configuration file, and executing each function based on the parameters;
and acquiring an execution result of each function and returning.
Optionally, before the receiving the data collection instruction, the method further includes:
receiving a function selection instruction, and displaying a preset function issuing page, wherein the function issuing page comprises a function identifier of each currently available function;
and generating the configuration file according to the function identifier selected by the user through the function publishing page.
Optionally, the step of receiving a function selection instruction and displaying a preset function publishing page includes:
receiving a function selection instruction, and acquiring a service to which the function instruction belongs;
and displaying a preset function publishing page, and displaying the function identifier of each function available under the service in the function publishing page.
Optionally, the method further comprises:
receiving a function issuing request, wherein the function issuing request carries at least one function to be issued;
and under the condition that a sender of the function issuing request has function issuing authority, acquiring each function to be issued, and recording at least one of address information and interface information of each function to be issued.
Optionally, the functional encapsulation manner of the data acquisition service includes at least one of a functional encapsulation by using a FaaS technology and a functional encapsulation by using a Groovy technology.
Optionally, the step of calling each function according to the configuration file and executing each function based on the parameters includes:
responding to a plurality of functions corresponding to the configuration file, calling each function through multiple threads, and executing each function based on the parameters;
each thread is used for calling and executing a function, and the functions called by the threads are different.
In a second aspect, an embodiment of the present invention provides a data acquisition apparatus, including:
the data acquisition system comprises an instruction receiving module, a data acquisition module and a data processing module, wherein the instruction receiving module is used for receiving a data acquisition instruction, the data acquisition instruction comprises parameters and a configuration file for acquiring data, the configuration file comprises at least one of address information and interface information of at least one function, the function is obtained by carrying out functional encapsulation on any data acquisition service, and the data acquisition services corresponding to the functions are different from each other;
the function calling module is used for calling each function according to the configuration file and executing each function based on the parameters;
and the data return module is used for acquiring the execution result of each function and returning the execution result.
Optionally, the apparatus further comprises:
the function display module is used for receiving a function selection instruction and displaying a preset function release page, wherein the function release page comprises a function identifier of each currently available function;
and the configuration file generation module is used for generating the configuration file according to the function identifier selected by the user through the function publishing page.
Optionally, the function presentation module includes:
the service determining submodule is used for receiving a function selection instruction and acquiring a service to which the function instruction belongs;
and the function display sub-module is used for displaying a preset function release page and displaying the function identification of each function available under the service in the function release page.
Optionally, the apparatus further comprises:
the request receiving module is used for receiving a function publishing request, wherein the function publishing request carries at least one function to be published;
and the function issuing module is used for acquiring each function to be issued and recording at least one of address information and interface information of each function to be issued under the condition that a sender of the function issuing request has a function issuing authority.
Optionally, the functional encapsulation manner of the data acquisition service includes at least one of a functional encapsulation by using a FaaS technology and a functional encapsulation by using a Groovy technology.
Optionally, the function call module is further configured to, in response to a plurality of functions corresponding to the configuration file, call each of the functions through multiple threads, and execute each of the functions based on the parameters;
each thread is used for calling and executing a function, and the functions called by the threads are different.
In a third aspect, an embodiment of the present invention additionally provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the data acquisition method according to the first aspect.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements the steps of the data acquisition method according to the first aspect.
In the embodiment of the invention, the relation of each data acquisition service is decoupled, so that each function can be independently or combinatively called, the flexibility can be improved, and the time cost and the communication cost can be reduced.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without inventive labor.
FIG. 1 is a flow chart of the steps of a method of data acquisition in an embodiment of the invention;
FIG. 2 is a flow chart of steps of another data acquisition method in an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a data acquisition apparatus according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of another data acquisition device in an embodiment of the present invention;
fig. 5 is a schematic diagram of a hardware structure of an electronic device in the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flowchart illustrating steps of a data acquisition method according to an embodiment of the present invention is shown.
In the embodiment of the invention, in order to facilitate the multiplexing of each service, a user can customize the service required by the user and call the corresponding service, and the data acquisition flexibility is improved, each data acquisition service can be functionalized in advance to obtain the mutually decoupled function corresponding to each data acquisition service, so that the flexible calling of each data acquisition service can be realized in the form of calling the function, and when the function is updated each time, other functions cannot be influenced, and code files corresponding to all functions do not need to be reloaded.
In addition, in practical application, when data is acquired, a range of data to be acquired is generally defined based on a given parameter, and then data corresponding to the parameter, that is, the data required currently, may be acquired by referring to the given parameter through a data acquisition service corresponding to the data required currently to be collected. The given parameters may define to some extent the data that needs to be acquired. For example, in the model training process, in order to filter suitable sample data, a certain condition may be set on the sample data to be acquired in a manner of presetting parameters to limit the range of the sample data to be acquired, and then when acquiring the sample data, a currently required service may be invoked to acquire the required sample data based on the preset parameters.
For example, if a given UserID (user identification) is used as a parameter for acquiring data to invoke a cloud authentication service to acquire authenticated information, to invoke a representation service to acquire representation information, and the like, which are material data, then authentication information, representation information, and the like of the UserID defined by the parameter can be acquired by setting the parameter. The cloud authentication service, the portrait service and the like are all data acquisition services, and the data acquisition services may include any relevant services that can be used for acquiring data, such as a feature snapshot service for acquiring portrait data, a content algorithm service for performing content analysis, a custom service for satisfying personalized requirements, and the like.
In the embodiment of the present invention, the data obtaining service may be functionalized in any available manner, for example, the code segment corresponding to each service is stripped from the original whole code segment and encapsulated as a function that can be called separately, and the embodiment of the present invention is not limited thereto. The specific content of the parameter can also be set by self-definition according to specific application scenarios, requirements and the like, and the embodiment of the present invention is not limited.
Moreover, after each data acquisition service is functionalized, in order to call a currently required function subsequently to acquire data, a configuration file meeting the current data acquisition requirement may be further carried in the data acquisition instruction based on the current data acquisition requirement, where the configuration file may include, but is not limited to, at least one of address information and interface information of at least one function, and is used to call each currently required function to perform an operation of data acquisition. For example, after the data acquisition service is encapsulated as a function, an interface provided by the corresponding function to the outside may be set, that is, interface information of the function; or acquiring the address information of the corresponding function, namely the address information for calling the corresponding function; and so on.
In addition, in the embodiment of the present invention, the data collection instruction may be triggered in any available manner, and the embodiment of the present invention is not limited thereto. For example, after the user selects the currently desired function, a corresponding profile may be generated and data collection instructions triggered, and so on.
Next, in the embodiment of the present invention, the reference information may be stored in any available manner, and the embodiment of the present invention is not limited thereto. For example, it may be set to store the reference information data and the like through a hive data table.
After receiving the data collection instruction, each currently required function may be called according to a configuration file included in the corresponding data collection instruction, and each function may be executed based on the parameters. Specifically, each currently required function may be found according to address information, interface information, and the like included in the configuration file, and then the corresponding function is called, and the corresponding function may be executed based on the parameter included in the corresponding data collection instruction, so as to obtain an execution result of the corresponding function, that is, the material data corresponding to the corresponding parameter information. For example, assuming that the parameters are user identification UserID1 and the function includes a function functionally packaged for a portrait service used to obtain user portrait data, user portrait data of UserID1 may be obtained by executing the function.
And step 130, obtaining the execution result of each function and returning.
After the execution of each currently called function is completed, the execution result of each function may be obtained and returned to the corresponding requester.
Referring to fig. 2, in the embodiment of the present invention, the method may further include:
And 150, generating the configuration file according to the function identifier selected by the user through the function publishing page.
In the embodiment of the present invention, in order to facilitate the user to select the currently required data acquisition service, currently available functions may be displayed in the form of a page. Then, when a function selection instruction is received, a preset function issue page may be presented, and a function identifier of each currently available function may be presented in the function issue page. Further, the user can select the required function identifier by clicking the function identifier in the function publishing page in any available manner. Moreover, after the user selects the required function identifier, the corresponding configuration file can be generated for the selected function identifier. At this time, at least one of address information and interface information of each function selected by the user and corresponding to the function identification may be included in the configuration file.
The function selection instruction may be triggered in any available manner, and the embodiment of the present invention is not limited thereto. The function identifier may be any identification information that can represent each function, for example, a name of the function, a name of a data acquisition service corresponding to the function, an icon of the data acquisition service corresponding to the function, and the like, and may be specifically set by a user according to a requirement, which is not limited in the embodiment of the present invention.
In addition, in practical application, application scenes can be divided according to services, and data required by different services can be different, that is, data acquisition services required by different services can be different, and certainly, overlapping can also exist, and data acquisition requirements under the same service are generally similar. Therefore, in the embodiment of the present invention, data requirements for different services may also be preset, and the corresponding relationship between each service and the function is set respectively, so that a configuration file corresponding to each service may also be generated. Then, when the user triggers the data collection instruction, the configuration file under the corresponding service may be acquired as the configuration file of the corresponding data collection instruction according to the service to which the corresponding data collection instruction belongs.
Optionally, in an embodiment of the present invention, the step 140 may further include:
step 141, receiving a function selection instruction, and acquiring a service to which the function instruction belongs;
and 142, displaying a preset function publishing page, and displaying the function identifier of each function available under the service in the function publishing page.
As described above, in the embodiment of the present invention, the correspondence between different services and functions may be preset, so as to customize the function applicable to each service. However, at this time, the functions under the same service are set uniformly, and in the actual application process, different users under the same service or data requirements of the same user in different time periods may be different, and it is difficult to consider personalized requirements of different users in the form of uniformly configuring the functions under different services.
Therefore, in the embodiment of the present invention, in order to further improve the calling flexibility of each function and meet the personalized requirements of different users, the service to which the function instruction belongs may be acquired after the function selection instruction is received; furthermore, when a preset function publishing page is displayed, the function identifier of each function available under the service can be displayed only in the function publishing page, and at this time, the user of the user can select the currently required function identifier in the current function publishing page. The functions suitable for each service are limited, and meanwhile, the personalized requirements of users can be met.
Optionally, in the embodiment of the present invention, the method may further include:
step 160, receiving a function issuing request, wherein the function issuing request carries at least one function to be issued;
step 170, when the sender of the function publishing request has the function publishing authority, obtaining each function to be published, and recording at least one of address information and interface information of each function to be published.
In practical application, as the number of users increases, the data requirements of each user become more and more diversified, but for a management party, it is difficult to know newly generated data requirements of different users in time and to package a function matched with the corresponding data requirements in time. For the user, if each currently available function cannot meet the current data requirement of the user, the data requirement of the user can be reported to the management party to remind the management party to update the function, and the data requirement of the user cannot be met in time.
Therefore, in the embodiment of the present invention, in order to reduce the communication cost and the time cost, the function issuing authority may be issued to the user who passes the verification in advance, so as to facilitate the user to issue a new function according to the own requirement.
Specifically, a function publishing request may be received, where the function publishing request carries at least one function to be published; if the sender of the corresponding function issuing request has the function issuing authority, each function to be issued can be acquired, and at least one of address information and interface information of each function to be issued is recorded. Correspondingly, the function carried by the function issuing request is also obtained by performing functional encapsulation on any data acquisition service, and the data acquisition service corresponding to the issued function can be different from any existing data acquisition service corresponding to the function.
Of course, in the embodiment of the present invention, the user may also directly issue a new data acquisition service, and then after acquiring the data acquisition service issued by the user, the method may perform functional encapsulation on each data acquisition service to obtain an encapsulated function corresponding to each data acquisition service, so that the corresponding encapsulated function may be repeatedly called.
In addition, if the functions are displayed in the page form, after the newly released function is obtained, the function identifier of each currently available function displayed in the function release page may be updated accordingly, so as to increase the function identifier of the function which is successfully released last.
Secondly, in the embodiment of the present invention, each packaged function may be recorded through a preset storage area. The storage area may be set by user according to a requirement, and the embodiment of the present invention is not limited thereto. For example, the storage area may be set as a MySql database, and then the contents of each function, address information of the function, interface information, and the like, the correspondence between the function and the service, and the like may be recorded in the MySql database, so as to facilitate flexible multiplexing.
Optionally, in the embodiment of the present invention, the data acquisition service is functionally encapsulated in a manner that is not limited to at least one of a FaaS technology and a Groovy technology.
The FaaS is an abbreviation of Functions as a Service, and can be broadly understood as a functional Service or a functional Service. The FaaS is used only by paying attention to the service code logic and not to the server resources, so that the FaaS is closely related to a developer without paying attention to the server. FaaS can be said to provide a more subdivided and abstract servitization capability. FaaS gradually forms an event-driven architecture mainly based on code functions, and functions may be used as one online service or remote computing service when FaaS is used, and may be executed through an API (application programming interface), by mail, by Iot (Internet of things), by queue, and the like. Groovy is a dynamic language based on JVM (Java virtual Machine) and is also an object-oriented language, which is very similar to Java in syntax.
In the embodiment of the present invention, the functional encapsulation manner of different data acquisition services in different application scenarios may be different, and specifically, the functional encapsulation manner may be set by user according to requirements, which is not limited in the embodiment of the present invention. For example, in the case of recording parameters and/or finally acquiring the obtained data through the above hive data table, each data acquisition service may be packaged as a function through the FaaS technology; or, the data acquisition service may be packaged as a function by the FaaS technology when the parameter is offline data, and packaged as a function by the Groovy technology when the parameter is online data; and so on.
In addition, the FaaS technology also has call related information such as the number of calls, call time and the like for counting the functions after packaging, so that the FaaS technology can be adopted for packaging the functions under the condition of the requirements; moreover, the Faas technology can support common assembly languages such as Java, so that a user can issue a new function according to his own needs conveniently, and preferably, the user can issue a new function through the Faas technology, and the like.
Optionally, in an embodiment of the present invention, the step 120 further includes: responding to a plurality of functions corresponding to the configuration file, calling each function through multiple threads, and executing each function based on the parameters; each thread is used for calling and executing a function, and the functions called by the threads are different.
In practical applications, if a current data collection instruction needs to combine and call multiple functions, each function may be called and executed sequentially through one thread, or function calls may be made simultaneously through multiple threads. Furthermore, in order to improve the execution efficiency of the functions of the data acquisition process, each function may be called by a plurality of threads at the same time. Specifically, each of the functions may be called by multiple threads, and executed based on the parameters, and each of the threads is used to call and execute a function, and the functions called by the respective threads are different from each other. Then the total number of threads may be less than the total number of functions corresponding to the configuration file, but it is preferable that the number of threads is set to be the same as the total number of functions corresponding to the configuration file, so that the respective functions can be called and executed at the same time.
In addition, in the embodiment of the present invention, the function may be called in any available manner, and the embodiment of the present invention is not limited thereto. For example, if the Http address allocated to each function after encapsulation is completed is used as the address information of the corresponding function, and the address information of the function is included in the configuration file, the functions may be invoked in a multi-thread Http (hypertext transfer Protocol) manner, and so on.
In the embodiment of the invention, the relation of each data acquisition service is decoupled, so that each function can be independently or combinatively called, the flexibility can be improved, and the time cost and the communication cost can be reduced.
Moreover, the functions and the services are configured, so that the development cost can be reduced, a plurality of services can flexibly multiplex the functions, the data acquisition service is not required to be repeatedly published, only the functions need to be published independently, and the publishing cost is reduced. And the user can write and issue the function of the logic code required by the user, flexibly configure the relation between the service and the function and realize the multiplexing of the function.
Referring to fig. 3, a schematic structural diagram of a data acquisition apparatus in an embodiment of the present invention is shown.
The data acquisition device of the embodiment of the invention comprises: an instruction receiving module 210, a function calling module 220, and a data returning module 230.
The functions of the modules and the interaction relationship between the modules are described in detail below.
An instruction receiving module 210, configured to receive a data collection instruction, where the data collection instruction includes a parameter and a configuration file used for acquiring data, the configuration file includes at least one of address information and interface information of at least one function, the function is obtained after performing functional encapsulation on any data acquisition service, and data acquisition services corresponding to the functions are different from each other;
a function calling module 220, configured to call each function according to the configuration file, and execute each function based on the parameter;
and a data returning module 230, configured to obtain an execution result of each function and return the execution result.
Referring to fig. 4, in an embodiment of the present invention, the apparatus may further include:
a function display module 240, configured to receive a function selection instruction, and display a preset function publishing page, where the function publishing page includes a function identifier of each currently available function;
a configuration file generating module 250, configured to generate the configuration file according to the function identifier selected by the user through the function publishing page.
Optionally, in an embodiment of the present invention, the function display module 240 may further include:
the service determining submodule is used for receiving a function selection instruction and acquiring a service to which the function instruction belongs;
and the function display sub-module is used for displaying a preset function release page and displaying the function identification of each function available under the service in the function release page.
Optionally, in an embodiment of the present invention, the apparatus may further include:
the request receiving module is used for receiving a function publishing request, wherein the function publishing request carries at least one function to be published;
and the function issuing module is used for acquiring each function to be issued and recording at least one of address information and interface information of each function to be issued under the condition that a sender of the function issuing request has a function issuing authority.
Optionally, in the embodiment of the present invention, the data acquisition service is functionally encapsulated in a manner that at least one of a FaaS technology and a Groovy technology is used.
Optionally, in an embodiment of the present invention, the function call module is further configured to, in response to a plurality of functions corresponding to the configuration script, call each of the functions through multiple threads, and execute each of the functions based on the parameters;
each thread is used for calling and executing a function, and the functions called by the threads are different.
The data acquisition device provided by the embodiment of the present invention can implement each process implemented in the method embodiments of fig. 1 to fig. 2, and is not described herein again to avoid repetition.
Preferably, an embodiment of the present invention further provides an electronic device, including: the processor, the memory, and the computer program stored in the memory and capable of running on the processor, when executed by the processor, implement each process of the above-mentioned data acquisition method embodiment, and can achieve the same technical effect, and for avoiding repetition, details are not described here.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements each process of the data acquisition method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may be a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
Fig. 5 is a schematic diagram of a hardware structure of an electronic device implementing various embodiments of the present invention.
The electronic device 500 includes, but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and a power supply 511. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 5 does not constitute a limitation of the electronic device, and that the electronic device may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present invention, the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer, and the like.
It should be understood that, in the embodiment of the present invention, the radio frequency unit 501 may be used for receiving and sending signals during a message sending and receiving process or a call process, and specifically, receives downlink data from a base station and then processes the received downlink data to the processor 510; in addition, the uplink data is transmitted to the base station. In general, radio frequency unit 501 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 501 can also communicate with a network and other devices through a wireless communication system.
The electronic device provides wireless broadband internet access to the user via the network module 502, such as assisting the user in sending and receiving e-mails, browsing web pages, and accessing streaming media.
The audio output unit 503 may convert audio data received by the radio frequency unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output as sound. Also, the audio output unit 503 may also provide audio output related to a specific function performed by the electronic apparatus 500 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, a receiver, and the like.
The input unit 504 is used to receive an audio or video signal. The input Unit 504 may include a Graphics Processing Unit (GPU) 5041 and a microphone 5042, and the Graphics processor 5041 processes image data of a still picture or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 506. The image frames processed by the graphic processor 5041 may be stored in the memory 509 (or other storage medium) or transmitted via the radio frequency unit 501 or the network module 502. The microphone 5042 may receive sounds and may be capable of processing such sounds into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 501 in case of the phone call mode.
The electronic device 500 also includes at least one sensor 505, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor includes an ambient light sensor that can adjust the brightness of the display panel 5061 according to the brightness of ambient light, and a proximity sensor that can turn off the display panel 5061 and/or a backlight when the electronic device 500 is moved to the ear. As one type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of an electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), and vibration identification related functions (such as pedometer, tapping); the sensors 505 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.
The display unit 506 is used to display information input by the user or information provided to the user. The Display unit 506 may include a Display panel 5061, and the Display panel 5061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 507 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072. Touch panel 5071, also referred to as a touch screen, may collect touch operations by a user on or near it (e.g., operations by a user on or near touch panel 5071 using a finger, stylus, or any suitable object or attachment). The touch panel 5071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 510, and receives and executes commands sent by the processor 510. In addition, the touch panel 5071 may be implemented in various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 5071, the user input unit 507 may include other input devices 5072. In particular, other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein.
Further, the touch panel 5071 may be overlaid on the display panel 5061, and when the touch panel 5071 detects a touch operation thereon or nearby, the touch operation is transmitted to the processor 510 to determine the type of the touch event, and then the processor 510 provides a corresponding visual output on the display panel 5061 according to the type of the touch event. Although in fig. 5, the touch panel 5071 and the display panel 5061 are two independent components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 5071 and the display panel 5061 may be integrated to implement the input and output functions of the electronic device, and is not limited herein.
The interface unit 508 is an interface for connecting an external device to the electronic apparatus 500. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 508 may be used to receive input (e.g., data information, power, etc.) from external devices and transmit the received input to one or more elements within the electronic apparatus 500 or may be used to transmit data between the electronic apparatus 500 and external devices.
The memory 509 may be used to store software programs as well as various data. The memory 509 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 509 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
The processor 510 is a control center of the electronic device, connects various parts of the whole electronic device by using various interfaces and lines, performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 509 and calling data stored in the memory 509, thereby performing overall monitoring of the electronic device. Processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 510.
The electronic device 500 may further include a power supply 511 (e.g., a battery) for supplying power to various components, and preferably, the power supply 511 may be logically connected to the processor 510 via a power management system, so as to implement functions of managing charging, discharging, and power consumption via the power management system.
In addition, the electronic device 500 includes some functional modules that are not shown, and are not described in detail herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (14)
1. A method of data acquisition, comprising:
receiving a data collection instruction, wherein the data collection instruction comprises parameters and a configuration file for acquiring data, the configuration file comprises at least one of address information and interface information of at least one function, the function is obtained by performing functional encapsulation on any data acquisition service, and the data acquisition services corresponding to the functions are different from each other;
calling each function according to the configuration file, and executing each function based on the parameters;
and acquiring an execution result of each function and returning.
2. The method of claim 1, wherein prior to receiving the data collection instruction, further comprising:
receiving a function selection instruction, and displaying a preset function issuing page, wherein the function issuing page comprises a function identifier of each currently available function;
and generating the configuration file according to the function identifier selected by the user through the function publishing page.
3. The method according to claim 2, wherein the step of receiving a function selection command and displaying a preset function issue page comprises:
receiving a function selection instruction, and acquiring a service to which the function instruction belongs;
and displaying a preset function publishing page, and displaying the function identifier of each function available under the service in the function publishing page.
4. The method of claim 1, further comprising:
receiving a function issuing request, wherein the function issuing request carries at least one function to be issued;
and under the condition that a sender of the function issuing request has function issuing authority, acquiring each function to be issued, and recording at least one of address information and interface information of each function to be issued.
5. The method according to any one of claims 1-4, wherein the functional encapsulation of the data acquisition service comprises at least one of functional encapsulation using a FaaS technology and functional encapsulation using a Groovy technology.
6. The method according to any of claims 1-4, wherein said step of calling each of said functions according to said configuration file and executing each of said functions based on said parameters comprises:
responding to a plurality of functions corresponding to the configuration file, calling each function through multiple threads, and executing each function based on the parameters;
each thread is used for calling and executing a function, and the functions called by the threads are different.
7. A data acquisition apparatus, comprising:
the data acquisition system comprises an instruction receiving module, a data acquisition module and a data processing module, wherein the instruction receiving module is used for receiving a data acquisition instruction, the data acquisition instruction comprises parameters and a configuration file for acquiring data, the configuration file comprises at least one of address information and interface information of at least one function, the function is obtained by carrying out functional encapsulation on any data acquisition service, and the data acquisition services corresponding to the functions are different from each other;
the function calling module is used for calling each function according to the configuration file and executing each function based on the parameters;
and the data return module is used for acquiring the execution result of each function and returning the execution result.
8. The apparatus of claim 7, further comprising:
the function display module is used for receiving a function selection instruction and displaying a preset function release page, wherein the function release page comprises a function identifier of each currently available function;
and the configuration file generation module is used for generating the configuration file according to the function identifier selected by the user through the function publishing page.
9. The apparatus of claim 8, wherein the function exposure module comprises:
the service determining submodule is used for receiving a function selection instruction and acquiring a service to which the function instruction belongs;
and the function display sub-module is used for displaying a preset function release page and displaying the function identification of each function available under the service in the function release page.
10. The apparatus of claim 9, further comprising:
the request receiving module is used for receiving a function publishing request, wherein the function publishing request carries at least one function to be published;
and the function issuing module is used for acquiring each function to be issued and recording at least one of address information and interface information of each function to be issued under the condition that a sender of the function issuing request has a function issuing authority.
11. The apparatus according to any one of claims 7-10, wherein the data acquisition service is functionally encapsulated in at least one of a FaaS technology and a Groovy technology.
12. The apparatus according to any of claims 7-10, wherein the function call module is further configured to, in response to a plurality of functions corresponding to the configuration script, call each of the functions by multithreading and execute each of the functions based on the parameters;
each thread is used for calling and executing a function, and the functions called by the threads are different.
13. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the data acquisition method according to any one of claims 1 to 6.
14. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, carries out the steps of the data acquisition method according to one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010496532.4A CN111752688A (en) | 2020-06-03 | 2020-06-03 | Data acquisition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010496532.4A CN111752688A (en) | 2020-06-03 | 2020-06-03 | Data acquisition method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111752688A true CN111752688A (en) | 2020-10-09 |
Family
ID=72674364
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010496532.4A Pending CN111752688A (en) | 2020-06-03 | 2020-06-03 | Data acquisition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111752688A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113901315A (en) * | 2021-10-08 | 2022-01-07 | 北京字节跳动网络技术有限公司 | Data service method, device and related product |
| CN114663202A (en) * | 2022-05-19 | 2022-06-24 | 山东佳联电子商务有限公司 | Auction bidding system and method based on faas function service |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060282400A1 (en) * | 2005-06-10 | 2006-12-14 | Muralidhar Kalavacharla | Apparatus, system, and method for encapsulating logical units of work using business objects |
| US7788681B1 (en) * | 2003-09-16 | 2010-08-31 | Vignette Software, LLC | System and method for incorporating web services in a web site |
| US20180034924A1 (en) * | 2016-07-28 | 2018-02-01 | Polybit Inc. | System and method for a unified interface to networked webservices |
| CN109039751A (en) * | 2018-08-14 | 2018-12-18 | 深圳壹账通智能科技有限公司 | Configure method, apparatus, computer equipment and the storage medium of routing |
| CN110365751A (en) * | 2019-06-26 | 2019-10-22 | 口碑(上海)信息技术有限公司 | Method for processing business, device and the equipment of gateway system |
| US20190349447A1 (en) * | 2018-05-11 | 2019-11-14 | Jpmorgan Chase Bank, N.A. | Function as a service gateway |
| US20200057676A1 (en) * | 2018-08-17 | 2020-02-20 | Vmware, Inc. | Function as a service (faas) execution distributor |
-
2020
- 2020-06-03 CN CN202010496532.4A patent/CN111752688A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7788681B1 (en) * | 2003-09-16 | 2010-08-31 | Vignette Software, LLC | System and method for incorporating web services in a web site |
| US20060282400A1 (en) * | 2005-06-10 | 2006-12-14 | Muralidhar Kalavacharla | Apparatus, system, and method for encapsulating logical units of work using business objects |
| US20180034924A1 (en) * | 2016-07-28 | 2018-02-01 | Polybit Inc. | System and method for a unified interface to networked webservices |
| US20190349447A1 (en) * | 2018-05-11 | 2019-11-14 | Jpmorgan Chase Bank, N.A. | Function as a service gateway |
| CN109039751A (en) * | 2018-08-14 | 2018-12-18 | 深圳壹账通智能科技有限公司 | Configure method, apparatus, computer equipment and the storage medium of routing |
| US20200057676A1 (en) * | 2018-08-17 | 2020-02-20 | Vmware, Inc. | Function as a service (faas) execution distributor |
| CN110365751A (en) * | 2019-06-26 | 2019-10-22 | 口碑(上海)信息技术有限公司 | Method for processing business, device and the equipment of gateway system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113901315A (en) * | 2021-10-08 | 2022-01-07 | 北京字节跳动网络技术有限公司 | Data service method, device and related product |
| WO2023056841A1 (en) * | 2021-10-08 | 2023-04-13 | 北京字节跳动网络技术有限公司 | Data service method and apparatus, and related product |
| CN114663202A (en) * | 2022-05-19 | 2022-06-24 | 山东佳联电子商务有限公司 | Auction bidding system and method based on faas function service |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110752981B (en) | Information control method and electronic equipment | |
| CN110750189B (en) | Icon display method and device | |
| CN110703972B (en) | File control method and electronic equipment | |
| CN110442281B (en) | Application icon display method and terminal | |
| CN111049980A (en) | Application sharing method, electronic equipment and computer readable storage medium | |
| CN108984066B (en) | Application icon display method and mobile terminal | |
| CN110888705A (en) | Method for processing unread message corner mark and electronic equipment | |
| CN111163449B (en) | Application sharing method, first electronic device and computer-readable storage medium | |
| CN111752688A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN110995924B (en) | Account management method and electronic equipment | |
| CN113923515A (en) | Video production method and device, electronic equipment and storage medium | |
| CN111310165B (en) | Account switching or registering method and electronic equipment | |
| CN111163227B (en) | Sharing method and electronic equipment | |
| CN111200648B (en) | Service calling method, device, terminal equipment and storage medium | |
| CN112181442B (en) | Unloading page display method, device, terminal, server and storage medium | |
| CN111443968A (en) | Screenshot method and electronic equipment | |
| CN110620842A (en) | Application program control method and terminal | |
| CN110780795A (en) | Screen capturing method and electronic equipment | |
| CN111142998A (en) | Sharing method of background application and electronic equipment | |
| CN110825475A (en) | Input method and electronic equipment | |
| CN110851042A (en) | Interface display method and electronic equipment | |
| CN111045588B (en) | Information viewing method and electronic equipment | |
| CN111475141A (en) | List generation method and device and electronic equipment | |
| CN110855550A (en) | Communication method, electronic device, and storage medium | |
| CN113312249A (en) | Log data generation method, log data display method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |