CN110851280A - Automatic data acquisition method based on distributed intelligent edge computing technology - Google Patents
Automatic data acquisition method based on distributed intelligent edge computing technology Download PDFInfo
- Publication number
- CN110851280A CN110851280A CN201911104073.4A CN201911104073A CN110851280A CN 110851280 A CN110851280 A CN 110851280A CN 201911104073 A CN201911104073 A CN 201911104073A CN 110851280 A CN110851280 A CN 110851280A
- Authority
- CN
- China
- Prior art keywords
- data
- terminal
- component
- terminal equipment
- management platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000005516 engineering process Methods 0.000 title claims abstract description 32
- 230000005540 biological transmission Effects 0.000 claims abstract description 12
- 238000009434 installation Methods 0.000 claims abstract description 8
- 238000012795 verification Methods 0.000 claims abstract description 4
- 238000007726 management method Methods 0.000 claims description 68
- 238000012545 processing Methods 0.000 claims description 29
- 238000007781 pre-processing Methods 0.000 claims description 16
- 238000004458 analytical method Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 7
- 230000003993 interaction Effects 0.000 claims description 6
- 238000009877 rendering Methods 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 238000000586 desensitisation Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims 8
- 230000008901 benefit Effects 0.000 abstract description 6
- 238000003032 molecular docking Methods 0.000 abstract description 4
- 230000008859 change Effects 0.000 abstract description 3
- 230000009545 invasion Effects 0.000 abstract description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 8
- 230000006872 improvement Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 230000004927 fusion Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 235000008429 bread Nutrition 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005206 flow analysis Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 239000010931 gold Substances 0.000 description 1
- 229910052737 gold Inorganic materials 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Abstract
The invention provides an automatic data acquisition method based on a distributed intelligent edge computing technology, which comprises the following steps: s1, building a terminal management platform: a terminal management platform is set up on a physical server, and the terminal management platform mainly solves the problems of terminal data transmission and integrated control; s2, terminal equipment installation: installing a wide-angle camera outside a target service system, connecting terminal equipment, and interconnecting and intercommunicating the terminal equipment and a terminal management platform through a special network; s3, verification of acquisition results: and displaying the acquired data list and a log interacting with the data of the remote management platform on the terminal equipment, and displaying the terminal list, the returned data list and the related log on the terminal management platform. The invention has the beneficial effects that: the system docking and data acquisition method has the advantages of no invasion, no change of the existing system/program and no influence on the existing system/program.
Description
Technical Field
The invention relates to a data acquisition method, in particular to an automatic data acquisition method based on a distributed intelligent edge computing technology.
Background
At present, the data transmission problem among various systems is enough satisfied in the data exchange adopted modes (data transmission based on Socket and data transmission based on files), but in consideration of the investment of manpower and material resources and other objective factors, a novel data acquisition method which is integrally transplantable is scarce.
The prior art solves the problems by establishing a data platform, reconstructs a data interface of a service system and adopts a database interface mode to assemble a free system through the data platform.
The data platform is a data-oriented data mining fusion platform, a data interface of a service system is reconstructed by a software architecture reconstruction technology based on operation, mining system data is rapidly dug in real time, a multi-source data sharing pool is formed, and efficient platform support is provided for data opening, fusion and value-added innovation. And the platform generates an autonomous interface API through the source service system presentation layer and performs data interaction with the source system through the API. The bottom layer adopts a Socket data transmission mode, a virtual machine is deployed in the front of a source application system, the application system blockage is removed through a technical means, data interaction is formed between the virtual machine and a corresponding platform, and the functions of source data acquisition, data processing, warehousing and the like are completed.
For a certain application system, the application system is accessed according to the access mode of the application system, and the operation behavior of a user is captured according to memory mirror image analysis. Learning is understood as reproducible API model configuration by program understanding (including memory variable analysis, source code analysis, bytecode analysis, interface screenshot snapshot analysis and the like) and integrating webpage DOM tree analysis, behavior pattern learning, feature analysis extraction, corresponding data flow analysis, operation pointing node analysis and the like.
Second, a replayable API model configuration is run on the API running platform. Data of the application system is firstly cleaned, unnecessary data is filtered, and the required data is normalized. Constructing a calling and called logical relation between objects and reconstructing an application architecture Model (MVC) by a data access interface (API) reflection code generator, an adaptive model interpreter and the like based on an abstract syntax tree; and then according to the data requirement of the user on the application, under the condition of not changing the external behavior of the application system, adapting a data interface, and reflecting (reflecting) a software object for reading and writing data into a data access interface (API) conforming to the RESTful style.
The existing data acquisition has the following defects:
1. the technical style is unified: the problems exist in micro-service application and multi-user application, and the subsequent maintenance cost is high;
2. cracking friend provider software: the software of the friend has the problems of patents, confidential information, benefit division, high cost, complex technical system and product system and the like, and is not practical to crack one by one;
3. the implementation difficulty is high: under the actual condition, multiple systems exist in different networks, and the work of combing a transmission network, establishing a data channel and the like consumes quite long time;
4. the invasion is strong: the original system code needs to be modified, the use habit and the operation behavior of a user need to be captured, and a service code needs to be reconstructed;
5. poor portability: although the platform portability is strong, the service code reconstruction is designed by combining the implementation difficulty and the service requirements, and the portability is greatly reduced.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides an automatic data acquisition method based on a distributed intelligent edge computing technology.
The invention provides an automatic data acquisition method based on a distributed intelligent edge computing technology, which comprises the following steps:
s1, building a terminal management platform: a terminal management platform is set up on a physical server, and the terminal management platform mainly solves the problems of terminal data transmission and integrated control;
s2, terminal equipment installation: installing a wide-angle camera outside a target service system, connecting terminal equipment, and interconnecting and intercommunicating the terminal equipment and a terminal management platform through a special network;
s3, verification of acquisition results: displaying the acquired data list and a data interaction log with a remote management platform on the terminal equipment, and displaying the terminal list, a return data list and a related log on the terminal management platform;
s4, automatic data acquisition: the data of various systems are collected through terminal equipment, data preprocessing is completed inside the system, interconnection and intercommunication are performed between the system and a terminal management platform through a special network, and information is reported in real time.
As a further improvement of the invention, the terminal management platform has a device monitoring function, and the data acquired by the terminal management platform uses a TCP or UDP form storage guide database, or carries out message publishing and subscribing in a message queue form, or is directly connected with a service system for processing.
As a further improvement of the invention, the terminal management platform mainly comprises a disposal library, data torsion logic and gateway routing service, and waits for terminal equipment to access after the terminal equipment is started.
As a further improvement of the invention, a wide-angle camera with the resolution not less than 720p and the distance not more than 1.2m is adopted.
As a further improvement of the invention, after the terminal equipment is installed, the terminal equipment is started, the green light is turned on and has no abnormal sound to indicate that the terminal equipment is successfully electrified, the external display is connected, whether the wide-angle camera is normal or not is checked, and the data acquisition service is started.
As a further improvement of the present invention, the data preprocessing in step S4 includes encryption/decryption, desensitization, and deduplication data preprocessing.
As a further improvement of the present invention, in step S4, the acquiring, by the terminal device, data of various systems includes: the method comprises the steps of applying a chart identification component to content identification of the industrial control report, embedding a wide-angle camera into terminal equipment to read videos of the industrial control report, positioning collected videos, identifying the videos frame by using an Opencv technology, outputting the identified content, transmitting the content to short message dynamic data processing of the terminal equipment, and extracting key data and performing subsequent business operation by the content.
As a further improvement of the present invention, in step S4, the collecting of data of various systems by the terminal device includes the following sub-steps:
(1) video capture
The USB interface of the terminal equipment is butted, the wide-angle camera is connected through corresponding driving, corresponding video streams are obtained, and video acquisition work is finished;
(2) dynamic analysis
Comparing the content of the designated area in the video before and after, namely, the key feature area of the designated area of the previous frame and the next frame is changed, if the difference exists between the two areas, dynamically capturing the changed content through content identification and pushing the changed content;
(3) content identification
Identifying the acquired content by using an Opencv technology, including preprocessing, feature extraction, feature comparison and classification decision of each frame of image, specifically as follows:
image preprocessing: the method comprises the steps of carrying out denoising, smoothing, changing, gray level processing and binarization processing on an acquired image, wherein for the image, the binarization processing may have one or more optimal threshold values, the fixed value adopts a self-adaptive local binarization algorithm, namely comparison is carried out according to the size of an image identification area, the algorithm is used for carrying out dynamic increase and decrease on the gray level part, and finally the gray level part is converted into a binarization threshold value, and the threshold value is used for completing the comparison of the image;
characteristic extraction: the independent characters are divided after binarization processing to form connected domains and the connected domains are detected, namely the connected domain detection is to compare and detect the left two pixels on the pixel points of the two connected domains, and finally connected domain screening waves are used for screening the contained pixels and determining character areas;
and (3) feature comparison: comparing the extracted character of the feature with a feature library, and outputting an identification result;
(4) subscription push
And pushing the identified content through the chart identification component by calling and processing the integrally supported short message dynamic data.
As a further improvement of the invention, the automatic data acquisition method comprises the steps of acquiring data flow direction and controlling flow direction of a terminal, wherein terminal equipment is referred to as a terminal for short, and a terminal management platform is referred to as a management end for short;
the flow direction of the collected data is as follows: the flow direction represents the flow direction of data from the existing system to the use, the intelligent identification component deployed by the terminal identifies the software deployed by the PC client, the software deployed by the server and a series of industrial control icons, the identification result is transmitted to the basic component and the application component through the SDK, the data is uniformly received by short message dynamic data processing and distributed according to the configuration, part of the data is stored in the local file system of the terminal, and part of the data is transmitted to the management terminal through the information security component through the uniform gateway; after receiving the data, the management end processes the data by the information security component and then sends the data to a target database, a service system or a message queue by the short message dynamic data processing component of the management end to finish data acquisition in new and old system integration, or a unified management component of the management end carries out real-time rendering on the data acquisition condition by a D3 rendering component;
the terminal controls the flow direction: the flow direction represents a data flow direction of control operations such as component installation, policy setting and the like of an externally deployed terminal by a user; the unified gateway component of the management end authenticates the control command, and the unified management component of the management end is connected with the unified gateway component of each terminal after the operation is legal and distributes the unified gateway component to the corresponding unified management component; and after receiving the information, the unified management component of the terminal opens the comprehensive warehouse and performs component deployment or parameter and strategy setting. Since the terminal control flow is complicated, the upper diagram uses blue numbers to identify the control steps.
The invention has the beneficial effects that:
1. the technical styles are various, and the solutions are numerous: the invention adopts the micro-service architecture, uses the uniform deployment and management strategy, supports the participation of different technical manufacturers, has no limit on the disposal of complex problems, and can reduce the later maintenance cost;
2. no software cracking, no invasion to the system: the existing software is not cracked and invaded, and dispute problems of patents, confidential information, benefit division, high cost, complex technical system and product system and the like do not exist in the aspect of system integration;
3. the implementation difficulty is small, and the box can be opened for use: the terminal equipment is suitable for system butt joint of a plurality of systems existing among different networks, and can be used when one equipment is connected with the network without combing a transmission network, establishing a data channel and the like;
4. continuous upgrading, the extension degree is high: adopt the interface to communicate between the subassembly, can use the newest technique of a certain aspect to dock as required, terminal equipment has basic hardware such as USB, serial ports simultaneously, can expand and insert various intelligent products, can exert the advantage that has been unique in some specific areas.
Drawings
FIG. 1 is a flow chart of an automated data acquisition method based on distributed intelligent edge computing technology according to the present invention.
FIG. 2 is a schematic data acquisition diagram of an automated data acquisition method based on a distributed intelligent edge computing technology.
FIG. 3 is an overall network topology diagram of an automated data acquisition method based on distributed intelligent edge computing technology.
FIG. 4 is a functional diagram of a diagram recognition component of an automated data acquisition method based on distributed intelligent edge computing technology.
FIG. 5 is an overall data flow diagram of an automated data acquisition method based on distributed intelligent edge computing technology.
Detailed Description
The invention is further described with reference to the following description and embodiments in conjunction with the accompanying drawings.
As shown in fig. 1 and 2, an automated data acquisition method based on distributed intelligent edge computing technology adopts an integrated support platform/system, basic hardware, and an intelligent computing component (business logic) to perform the following steps:
a terminal management platform (management end for short) is set up: a terminal management platform is set up on a physical server, and the terminal management platform mainly solves the problems of terminal data transmission and integrated management and control. The method mainly comprises a disposal library, data torsion logic, gateway routing service and the like, and the method can be used after the terminal equipment is started and accessed.
Installation of terminal equipment (terminal for short): and installing a wide-angle camera (the resolution is not less than 720p and the distance is not more than 1.2 m) outside the target service system and connecting the terminal equipment. After the installation is finished, the terminal equipment is started, the green lamp is turned on and has no abnormal sound to indicate that the terminal equipment is successfully electrified, the external display is connected, whether the camera is normal or not is checked, and the data acquisition service is started.
And (3) verification of acquisition results: the collected data list and the log of data interaction with the remote management platform can be seen on the terminal equipment, and the terminal list, the returned data list and the related log can be seen by the control end. The data visualization component is used for rapidly knowing the conditions of data acquisition trend, terminal equipment state, data torsion and the like.
The automatic data acquisition method acquires data of various systems through a terminal, completes data preprocessing (encryption/decryption, desensitization, duplication removal and the like) inside, is interconnected and intercommunicated with a terminal management platform through a private network, and reports information in real time. And the terminal management platform has a device monitoring function. The data acquired by the terminal management platform can be stored in a database in a TCP or UDP form, can be published and subscribed in a message queue form, and can also be directly connected with a service system for processing.
The invention provides a small device for each system needing data exchange, and an integrated supporting platform is deployed for terminal device data acquisition and terminal control, and the system is combined through an intelligent computing component on the service logic of data interaction. Each small device is connected through an autonomous network and aggregates data back to the data resource pool. Form a distributed automatic data acquisition method of logic centralization and physical decentralization.
As shown in fig. 3, the terminals are connected by using an independent private network, and in order to ensure information security, security devices such as a gatekeeper, a firewall and the like are generally provided, so that the PC client and the industrial control diagram can still realize data acquisition without being in the same network.
The terminal device is a main carrier for realizing the method, and the terminal device is taken as hardware and forms a whole by matching with software. The terminal is similar to a 2Ux86 server, has functions of data storage, network and the like, but is not as large as the 2Ux86 server, is an ARM-based microcomputer, and can be provided with a corresponding operating system (such as Windows10 Iot, Centos, UbuntuMate, NOOBS, Raspbian and the like); supporting GPIO extension; the 4 USB interfaces can allow a second physical network card to be accessed and other intelligent hardware to be accessed; the method has the advantages that various sensors are well supported under the support of serial ports such as FT232 and CH 430; and simultaneously, other hardware such as a bread board, a DuPont wire, a diode, a resistor and the like are used for expansion according to requirements.
The terminal device can install corresponding development software such as Java, gold, Python and the like based on different operating systems, and has software development conditions, and the configuration number of the terminal is performed according to the number of systems to be accessed, which is generally 1: 1, a system needing to be accessed configures a terminal device, so a management end of the terminal device is needed to configure all terminals. In terms of architecture, the terminal device is a complete product and comprises two parts of hardware and software, and the terminal device management end is a pure software product and can be deployed at any place, such as a virtual machine, a cloud end, a physical machine and the like, and can also be deployed on the terminal device if site conditions do not support the product.
As shown in fig. 4, the graph recognition component is mainly applied to content recognition of an industrial control report, a wide-angle camera is embedded into the graph recognition component through terminal equipment, so that video reading of the industrial control report is achieved, and a collected video is positioned, namely a proper angle and a position of an icon in the video are selected for adjustment. And the Opencv technology is used for identifying the video frame by frame, outputting the identified content, transmitting the content to the short message dynamic data processing of the terminal equipment, and extracting key data and performing subsequent business operation by the short message dynamic data processing, wherein the functions of the system are as follows:
(1) video capture
And the USB interface of the terminal equipment is butted, the wide-angle camera is connected through corresponding driving, corresponding video streams are obtained, and the video acquisition work is completed.
(2) Dynamic analysis
The content of the designated area in the video is compared before and after, namely the pixels (key feature areas) of the designated area of the previous frame and the next frame are changed, if the difference exists between the two, the changed content can be dynamically captured through content identification and pushed.
(3) Content identification
And identifying the acquired content by using an Opencv technology, including preprocessing each frame of image, feature extraction, feature comparison and classification decision. The specific description is as follows:
image preprocessing: the method comprises the steps of carrying out denoising, smoothing, changing, gray level processing and binarization processing on an acquired image, wherein for the image, the binarization processing may have one or more optimal threshold values, carrying out comparison according to the size of an image identification area (for example, the part with the gray level higher than 40 is a foreground) by using a self-adaptive local binarization algorithm for a fixed value, carrying out dynamic increase and decrease on the gray level part by using the algorithm, finally converting into a binarization threshold value, and using the threshold value to complete the comparison of the image.
Characteristic extraction: and (4) separating the independent characters after binarization processing to form a connected domain and detecting the connected domain. The connected domain detection is to compare and detect two connected domains by using the left two pixels on the pixel point, and finally, a connected domain screening wave is used for screening the contained pixels and determining a character region.
And (3) feature comparison: and comparing the extracted characters of the features with the feature library, and outputting an identification result.
(4) Subscription push
And the identified content is pushed through the component by calling and processing the integrally supported short message dynamic data.
The overall dataflow graph is shown in fig. 5:
the data flow is divided into two flows, namely, the data flow is collected and the terminal control flow is controlled.
Collecting data flow direction: the flow direction represents the flow direction of data from the existing system to the use, the intelligent identification component deployed by the terminal identifies the software deployed by the PC client, the software deployed by the server and a series of industrial control icons, and the identification result is transmitted to the basic component and the application component through the SDK. The data is uniformly received by the dynamic data processing of the short message and is distributed according to the configuration (part of the data is stored in the local file system of the terminal, and part of the data is sent to the management terminal through the uniform gateway by the information security component). After receiving the data, the management end processes the data by the information security component and then sends the data to a target database, a service system or a message queue by the short message dynamic data processing component of the management end to complete data acquisition in new and old system integration, and the data acquisition condition can be rendered in real time by a D3 rendering component through the unified management component of the management end.
The terminal controls the flow direction: the flow direction indicates a data flow direction in which a user performs control operations such as component installation and policy setting on an externally deployed terminal. And after the operation is legal, the unified management component of the management end is connected with the unified gateway component of each terminal and distributes the unified gateway component to the corresponding unified management component. And after receiving the information, the unified management component of the terminal opens the comprehensive warehouse and performs component deployment or parameter and strategy setting. Since the terminal control flow is complicated, the upper diagram uses blue numbers to identify the control steps.
The invention provides an automatic data acquisition method based on a distributed intelligent edge computing technology, and discloses a system docking, data acquisition and data preprocessing method which is non-intrusive, does not change and does not influence the existing system/program in any network system. In the method, in the system data acquisition process, the data docking of the PLC system and the acquisition of liquid level chart data are realized by using a wide-angle camera and terminal equipment through a screen recognition method of mechanical vision, and meanwhile, the wireless data transmission is completed through the terminal equipment. The existing data information acquisition scheme needs to be implemented by invading the existing system and changing an interface structure in the system, the invention is not included, the method invading the existing system is used for implementing data acquisition, the use of the existing system is influenced, disputes are easily caused, and the mode of using data copy is large in limitation.
The invention provides an automatic data acquisition method based on a distributed intelligent edge computing technology, which is characterized in that the method is based on the distributed intelligent edge computing technology, and non-invasive and automatic data acquisition of the existing system and the existing system is realized through a plurality of fully distributed data acquisition devices with unified control management, operation coordination and information preprocessing. The intelligent community intelligent management system is suitable for various application scenes such as intelligent cities and intelligent communities, is simple and easy to use, and is rich in imagination and creativity.
The invention provides an automatic data acquisition method based on a distributed intelligent edge computing technology, which has the following characteristics:
1) the method has the advantages that the system docking and data acquisition method is non-invasive, does not change the existing and does not influence the existing system/program;
2) the system is provided with a fully distributed data acquisition device with unified control management, operation coordination and information preprocessing;
3) the programmable expansion component comprises screen recognition and chart recognition based on machine vision;
4) end-to-end real-time wireless data transmission, control and visual rendering are allowed;
5) and the combination of software secondary development, hybrid cloud deployment and intelligent hardware is supported.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (9)
1. An automatic data acquisition method based on a distributed intelligent edge computing technology is characterized by comprising the following steps:
s1, building a terminal management platform: a terminal management platform is set up on a physical server, and the terminal management platform mainly solves the problems of terminal data transmission and integrated control;
s2, terminal equipment installation: installing a wide-angle camera outside a target service system, connecting terminal equipment, and interconnecting and intercommunicating the terminal equipment and a terminal management platform through a special network;
s3, verification of acquisition results: displaying the acquired data list and a data interaction log with a remote management platform on the terminal equipment, and displaying the terminal list, a return data list and a related log on the terminal management platform;
s4, automatic data acquisition: the data of various systems are collected through terminal equipment, data preprocessing is completed inside the system, interconnection and intercommunication are performed between the system and a terminal management platform through a special network, and information is reported in real time.
2. The automated data collection method based on distributed intelligent edge computing technology of claim 1, wherein: the terminal management platform has a device monitoring function, and stores and guides a database in a TCP or UDP form through data acquired by the terminal management platform, or issues and subscribes messages in a message queue form, or is directly connected with a service system for processing.
3. The automated data collection method based on distributed intelligent edge computing technology of claim 1, wherein: the terminal management platform mainly comprises a disposal library, data torsion logic and gateway routing service, and waits for terminal equipment to be accessed after the terminal equipment is started.
4. The automated data collection method based on distributed intelligent edge computing technology of claim 1, wherein: and a wide-angle camera with the resolution not less than 720p and the distance not more than 1.2m is adopted.
5. The automated data collection method based on distributed intelligent edge computing technology of claim 1, wherein: after the terminal equipment is installed, the terminal equipment is started, the green lamp is turned on and has no abnormal sound to indicate that the terminal equipment is successfully electrified, the external display is connected, whether the wide-angle camera is normal or not is checked, and data acquisition service is started.
6. The automated data collection method based on distributed intelligent edge computing technology of claim 1, wherein: the data preprocessing in step S4 includes encryption/decryption, desensitization, and deduplication data preprocessing.
7. The automated data collection method based on distributed intelligent edge computing technology of claim 1, wherein: in step S4, the acquiring, by the terminal device, data of various systems includes: the method comprises the steps of applying a chart identification component to content identification of the industrial control report, embedding a wide-angle camera into terminal equipment to read videos of the industrial control report, positioning collected videos, identifying the videos frame by using an Opencv technology, outputting the identified content, transmitting the content to short message dynamic data processing of the terminal equipment, and extracting key data and performing subsequent business operation by the content.
8. The automated data collection method based on distributed intelligent edge computing technology of claim 7, wherein: in step S4, the collecting of data of various systems by the terminal device includes the following sub-steps:
(1) video capture
The USB interface of the terminal equipment is butted, the wide-angle camera is connected through corresponding driving, corresponding video streams are obtained, and video acquisition work is finished;
(2) dynamic analysis
Comparing the content of the designated area in the video before and after, namely, the key feature area of the designated area of the previous frame and the next frame is changed, if the difference exists between the two areas, dynamically capturing the changed content through content identification and pushing the changed content;
(3) content identification
Identifying the acquired content by using an Opencv technology, including preprocessing, feature extraction, feature comparison and classification decision of each frame of image, specifically as follows:
image preprocessing: the method comprises the steps of carrying out denoising, smoothing, changing, gray level processing and binarization processing on an acquired image, wherein for the image, the binarization processing may have one or more optimal threshold values, the fixed value adopts a self-adaptive local binarization algorithm, namely comparison is carried out according to the size of an image identification area, the algorithm is used for carrying out dynamic increase and decrease on the gray level part, and finally the gray level part is converted into a binarization threshold value, and the threshold value is used for completing the comparison of the image;
characteristic extraction: the independent characters are divided after binarization processing to form connected domains and the connected domains are detected, namely the connected domain detection is to compare and detect the left two pixels on the pixel points of the two connected domains, and finally connected domain screening waves are used for screening the contained pixels and determining character areas;
and (3) feature comparison: comparing the extracted character of the feature with a feature library, and outputting an identification result;
(4) subscription push
And pushing the identified content through the chart identification component by calling and processing the integrally supported short message dynamic data.
9. The automated data collection method based on distributed intelligent edge computing technology of claim 1, wherein: the automatic data acquisition method comprises the steps of acquiring data flow direction and terminal control flow direction, wherein terminal equipment is called a terminal for short, and a terminal management platform is called a management end for short;
the flow direction of the collected data is as follows: the flow direction represents the flow direction of data from the existing system to the use, the intelligent identification component deployed by the terminal identifies the software deployed by the PC client, the software deployed by the server and a series of industrial control icons, the identification result is transmitted to the basic component and the application component through the SDK, the data is uniformly received by short message dynamic data processing and distributed according to the configuration, part of the data is stored in the local file system of the terminal, and part of the data is transmitted to the management terminal through the information security component through the uniform gateway; after receiving the data, the management end processes the data by the information security component and then sends the data to a target database, a service system or a message queue by the short message dynamic data processing component of the management end to finish data acquisition in new and old system integration, or a unified management component of the management end carries out real-time rendering on the data acquisition condition by a D3 rendering component;
the terminal controls the flow direction: the flow direction represents a data flow direction of control operations such as component installation, policy setting and the like of an externally deployed terminal by a user; the unified gateway component of the management end authenticates the control command, and the unified management component of the management end is connected with the unified gateway component of each terminal after the operation is legal and distributes the unified gateway component to the corresponding unified management component; and after receiving the information, the unified management component of the terminal opens the comprehensive warehouse and performs component deployment or parameter and strategy setting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911104073.4A CN110851280A (en) | 2019-11-13 | 2019-11-13 | Automatic data acquisition method based on distributed intelligent edge computing technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911104073.4A CN110851280A (en) | 2019-11-13 | 2019-11-13 | Automatic data acquisition method based on distributed intelligent edge computing technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110851280A true CN110851280A (en) | 2020-02-28 |
Family
ID=69600384
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911104073.4A Pending CN110851280A (en) | 2019-11-13 | 2019-11-13 | Automatic data acquisition method based on distributed intelligent edge computing technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110851280A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111597116A (en) * | 2020-05-20 | 2020-08-28 | 浪潮软件集团有限公司 | Mobile terminal UI automatic test method, system, storage medium and electronic equipment |
| CN113094174A (en) * | 2021-04-09 | 2021-07-09 | 贵州电网有限责任公司 | Method for coordination control of power distribution network terminal autonomous networking based on edge computing technology |
| CN113596081A (en) * | 2021-06-21 | 2021-11-02 | 工业云制造(四川)创新中心有限公司 | Intelligent manufacturing open platform based on edge calculation |
| CN113613287A (en) * | 2021-06-21 | 2021-11-05 | 工业云制造(四川)创新中心有限公司 | Automatic data acquisition system based on edge calculation |
| CN114143446A (en) * | 2021-10-20 | 2022-03-04 | 深圳航天智慧城市系统技术研究院有限公司 | Histogram identification method, system, storage medium and equipment based on edge calculation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107990877A (en) * | 2017-12-06 | 2018-05-04 | 华中师范大学 | A kind of unmanned aerial vehicle remote sensing interpretation field investigation system and method based on internet |
| CN109451040A (en) * | 2018-12-10 | 2019-03-08 | 王顺志 | Internet of things networking system and network-building method based on edge calculations |
| CN109862087A (en) * | 2019-01-23 | 2019-06-07 | 深圳市康拓普信息技术有限公司 | Industrial Internet of things system and its data processing method based on edge calculations |
| CN110347134A (en) * | 2019-07-29 | 2019-10-18 | 南京图玩智能科技有限公司 | A kind of AI intelligence aquaculture specimen discerning method and cultivating system |
| CN110401262A (en) * | 2019-06-17 | 2019-11-01 | 北京许继电气有限公司 | GIS device state intelligent monitoring system and method based on edge calculations technology |
-
2019
- 2019-11-13 CN CN201911104073.4A patent/CN110851280A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107990877A (en) * | 2017-12-06 | 2018-05-04 | 华中师范大学 | A kind of unmanned aerial vehicle remote sensing interpretation field investigation system and method based on internet |
| CN109451040A (en) * | 2018-12-10 | 2019-03-08 | 王顺志 | Internet of things networking system and network-building method based on edge calculations |
| CN109862087A (en) * | 2019-01-23 | 2019-06-07 | 深圳市康拓普信息技术有限公司 | Industrial Internet of things system and its data processing method based on edge calculations |
| CN110401262A (en) * | 2019-06-17 | 2019-11-01 | 北京许继电气有限公司 | GIS device state intelligent monitoring system and method based on edge calculations technology |
| CN110347134A (en) * | 2019-07-29 | 2019-10-18 | 南京图玩智能科技有限公司 | A kind of AI intelligence aquaculture specimen discerning method and cultivating system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111597116A (en) * | 2020-05-20 | 2020-08-28 | 浪潮软件集团有限公司 | Mobile terminal UI automatic test method, system, storage medium and electronic equipment |
| CN111597116B (en) * | 2020-05-20 | 2023-06-09 | 浪潮软件集团有限公司 | Mobile terminal UI automatic test method, system, storage medium and electronic equipment |
| CN113094174A (en) * | 2021-04-09 | 2021-07-09 | 贵州电网有限责任公司 | Method for coordination control of power distribution network terminal autonomous networking based on edge computing technology |
| CN113094174B (en) * | 2021-04-09 | 2023-11-17 | 贵州电网有限责任公司 | Autonomous networking coordination control method for power distribution network terminal based on edge computing technology |
| CN113596081A (en) * | 2021-06-21 | 2021-11-02 | 工业云制造(四川)创新中心有限公司 | Intelligent manufacturing open platform based on edge calculation |
| CN113613287A (en) * | 2021-06-21 | 2021-11-05 | 工业云制造(四川)创新中心有限公司 | Automatic data acquisition system based on edge calculation |
| CN113613287B (en) * | 2021-06-21 | 2023-04-28 | 工业云制造(四川)创新中心有限公司 | Automatic data acquisition system based on edge calculation |
| CN114143446A (en) * | 2021-10-20 | 2022-03-04 | 深圳航天智慧城市系统技术研究院有限公司 | Histogram identification method, system, storage medium and equipment based on edge calculation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110851280A (en) | Automatic data acquisition method based on distributed intelligent edge computing technology | |
| CN101872524B (en) | Video monitoring method, system and device based on virtual wall | |
| CN109495520B (en) | Integrated network attack evidence obtaining and tracing method, system, equipment and storage medium | |
| CN107230066A (en) | A kind of method of payment and device, computer installation, readable storage medium storing program for executing | |
| CN101778264A (en) | Video monitoring method and video monitoring system | |
| CN104252592A (en) | Method and device for identifying plug-in application program | |
| CN109560996B (en) | Automatic testing system and method for terminal of Internet of things | |
| CN105338391A (en) | Intelligent television control method and mobile terminal | |
| CN106407078A (en) | An information interaction-based client performance monitoring device and method | |
| CN113140056A (en) | Intelligent lock management system for wireless communication | |
| CN104219197A (en) | Video conversation method, video conversation terminal, and video conversation system | |
| CN109510721A (en) | A kind of network log management method and system based on Syslog | |
| CN102754488B (en) | The control method of user's access, Apparatus and system | |
| CN113269039A (en) | On-duty personnel behavior identification method and system | |
| CN111340016A (en) | Image exposure method and apparatus, storage medium, and electronic apparatus | |
| CN103324673A (en) | Method for acquiring internet user behavior data | |
| CN104268193A (en) | Advertisement webpage intercepting method and device | |
| CN114296716A (en) | Function configuration method and device for intelligent operation of engineering machinery and electronic equipment | |
| CN110490118A (en) | Image processing method and device | |
| CN110807184B (en) | Method for intelligently recording screen and acquiring behavior data of computer and mobile terminal user | |
| US20210117686A1 (en) | Methods and systems for customized image and video analysis | |
| CN106469269A (en) | A kind of method of Password Management, device and terminal | |
| CN108563473A (en) | scanning module configuration method and device | |
| CN105405220A (en) | Method and device for automated test | |
| KR102437560B1 (en) | System and method for design task of architectural decoration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200228 |