CN113221866B - Equipment data acquisition system and method based on image recognition - Google Patents
Equipment data acquisition system and method based on image recognition Download PDFInfo
- Publication number
- CN113221866B CN113221866B CN202110487550.0A CN202110487550A CN113221866B CN 113221866 B CN113221866 B CN 113221866B CN 202110487550 A CN202110487550 A CN 202110487550A CN 113221866 B CN113221866 B CN 113221866B
- Authority
- CN
- China
- Prior art keywords
- module
- task
- local
- screen
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims abstract description 8
- 230000009471 action Effects 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 24
- 238000005516 engineering process Methods 0.000 claims description 8
- 238000003491 array Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 3
- 238000013518 transcription Methods 0.000 abstract description 2
- 230000035897 transcription Effects 0.000 abstract description 2
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000003754 machining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/96—Management of image or video recognition tasks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Information Transfer Between Computers (AREA)
- User Interface Of Digital Computer (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses an equipment data acquisition system based on image recognition, which comprises an interface module, a task management module, an acquisition strategy execution module and a content recognition module. Also discloses an equipment data acquisition method based on image recognition, which comprises the following steps: the external application calls an interface module, and the task management module acquires task parameters according to the interface module and screens out a task list; the acquisition strategy execution module is used for intercepting a screen according to the task list, performing local segmentation and pretreatment to obtain a local image and transmitting the local image to the content identification module; the content identification module identifies and calculates result data and transmits the result data back to the acquisition strategy execution module; and the acquisition strategy execution module feeds back the returned result data to the interface module according to a preset format. The invention realizes the automatic acquisition of screen data and avoids the error of artificial transcription; the system is customized according to the requirements of customers, simplifies the programs of the clients and meets the requirements of various screens and application scenes.
Description
Technical Field
The invention relates to the technical field of data acquisition, in particular to an equipment data acquisition system and method based on image recognition.
Background
With continuous innovation of information technology, the equipment data acquisition technology has become a main way for numerous enterprises to realize production of internet of things, improve management effect and improve production efficiency. However, from the actual operation perspective of an enterprise, the replacement of some traditional production and detection equipment needs to take into account the input and output conditions, the contradiction between the technical improvement and the profit guarantee is prominent, and in the aspect of traditional equipment data acquisition and transformation, the following problems often need to be faced: due to the innovation of the control system technology, the original communication interface and communication protocol of the traditional equipment are too old, direct communication conversion cannot be carried out with the current mainstream industrial Ethernet interface, and the physical layer lacks a corresponding intermediate conversion component, so that the acquisition of equipment data is difficult to carry out or cannot be realized at all; traditional industrial control equipment such as a machining lathe and detection equipment can be matched with a unique operation monitoring interface and a display to display key data to be monitored in real time, but as development platforms and versions of background monitoring software are different, and part of brands are in business secret guarantee, an external open interface cannot be provided, data collection and integration cannot be completed from a software layer, and recording of the key data in the process of equipment production operation or detection and verification can only depend on manual copying.
The image universal character recognition technology is often used for recognizing character contents in pictures, is widely applied to the fields of post and telecommunications, logistics and business office, replaces the work of manual input and copying to a certain extent, and realizes the quick conversion from paper to data. The problem of data acquisition of old equipment can be solved by applying the image recognition technology to key data recognition in a monitoring interface of the equipment.
Disclosure of Invention
In view of the above problems and deficiencies of the prior art, the present invention provides a device data acquisition system based on image recognition.
The technical scheme for solving the technical problems is as follows:
the invention provides an equipment data acquisition system based on image recognition, which comprises: the system comprises an interface module, a task management module, an acquisition strategy execution module and a content identification module;
the interface module is used for providing standard service for external application, receiving task parameters issued by the external application and returning result data to the external application;
the task management module is used for providing a user with a configuration, debugging and test self-defined acquisition strategy; the task list is called by the interface module, and the task list required to be executed is screened out according to the task parameters acquired by the interface module;
the acquisition strategy execution module intercepts a screen according to the task list, stores the screen as a screen capture picture, performs local segmentation and pretreatment on the screen capture picture to obtain a local image, and transmits the local image to the content identification module; transmitting the result data returned by the content identification module to the interface module;
the content identification module is used for preprocessing the received local image, identifying information in the local image by using an image identification technology to obtain result data, and transmitting the result data back to the acquisition strategy execution module.
On the basis of the above, the invention can be improved as follows:
furthermore, the acquisition strategy execution module comprises a virtual action module, a screen capture module and an image processing module;
the virtual action module is called by the task management module and executes a corresponding virtual preposed action according to the collection strategy screened by the task management module;
the screen capture module is called by the task management module, captures images according to the collection strategy screened by the task management module to obtain screen capture images, and transmits local identification tasks and the screen capture images to the image processing module;
the picture processing module is used for locally dividing and preprocessing the screen capture picture according to the local identification task transmitted by the screen capture module to generate a plurality of local images, transmitting the local images and the corresponding sequence label marks to the content identification module, receiving result data transmitted back by the content identification module, forming corresponding arrays according to the sequence of the sequence label marks, and feeding back the corresponding arrays to the interface module.
Preferably, the screen capture module and the virtual action module are located at the client and exist in the form of a background service.
The invention also provides an equipment data acquisition method based on image recognition, which comprises the following steps:
s1, calling an interface service issued by an interface module by an external application;
s2, the task management module acquires task parameters and screens out an executed task list; judging whether a preposed action is needed, if yes, executing a step S3, and if no, executing a step S4;
s3, the virtual action module executes the virtual prepositive action and transmits a screen capturing task to the screen capturing module;
s4, the screen capture module captures the current screen and stores the current screen as a screen capture picture, and transmits the task and the screen capture picture to the picture processing module;
s5, the picture processing module performs local segmentation and preprocessing according to the screenshot file corresponding to the task transmitted by the screenshot module, stores the screenshot file as a local picture and generates a sequence tag, and transmits the sequence tag to the content identification module;
s6, the content identification module identifies and calculates the pictures processed and transmitted by the pictures to obtain result data, and returns the result data to the picture processing module;
and S7, the image processing module feeds back result data returned by the identification module to the interface module according to a preset format.
The positive progress and beneficial effects of the invention are as follows:
the invention rapidly solves the problem of data acquisition and integration of the traditional equipment, realizes the automatic acquisition of screen data and avoids manual transcription errors; the system is customized according to the requirements of customers, simplifies the programs of the clients and meets the requirements of various screens and application scenes.
Drawings
FIG. 1 is a block diagram of the data acquisition system of the image recognition-based device of the present invention;
FIG. 2 is a flow chart of the data acquisition method of the device based on image recognition according to the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth to illustrate, but are not to be construed to limit the scope of the invention.
The embodiment provides an equipment data acquisition system based on image recognition, which comprises a client and a server. The client is installed on a monitoring software computer of the equipment needing to be collected, the address of a server for collecting data needs to be appointed during installation, and the starting background automatically runs after the software is installed.
As shown in fig. 1, the device data acquisition system based on image recognition includes an interface module, a task management module, a collection policy execution module, and a content recognition module; the acquisition strategy execution module further comprises a virtual action module, a screen capture module and an image processing module.
The interface module is positioned at the server end and used for providing standard service for the external application, receiving the task parameters issued by the external application and returning the result data to the external application.
The task management module is positioned at the server end and used for providing a user with a configuration, debugging and test self-defined acquisition strategy; and the task list required to be executed is screened out according to the task parameters acquired by the interface module when the task list is called by the interface module.
The acquisition strategy execution module intercepts a screen according to the task list, stores the screen as a screen capture picture, performs local segmentation and pretreatment on the screen capture picture to obtain a local image, and transmits the local image to the content identification module; transmitting the result data returned by the content identification module to the interface module;
the acquisition strategy execution module also comprises a virtual action module, a screen capture module and an image processing module;
the virtual action module is positioned at the client, exists in a background service form, is called by the task management module, executes a corresponding virtual prepositive action according to the collection strategy screened by the task management module, and transmits a screen capturing task to the screen capturing module;
the screen capture module is positioned at the client, exists in a background service form, is called by the task management module, captures a screen according to an acquisition strategy screened by the task management module to obtain a screen capture picture, and transmits a local identification task and the screen capture picture to the picture processing module;
the image processing module is positioned at the server side, and is used for locally dividing and preprocessing the screen capture image to generate a plurality of local images according to the local identification task transmitted by the screen capture module, transmitting the local images and the corresponding sequence label marks to the content identification module, receiving result data transmitted back by the content identification module, forming corresponding arrays according to the sequence of the sequence label marks, and feeding back the corresponding arrays to the interface module.
The content identification module is used for preprocessing the received local image, identifying information in the local image by using an image identification technology to obtain result data, and transmitting the result data back to the acquisition strategy execution module.
As shown in fig. 2, it is a flowchart of an apparatus data method based on image recognition according to the present invention, and the steps are as follows:
s1, calling an interface service issued by an interface module by an external application;
s2, the task management module obtains task parameters according to the interface module and screens out an executed task list; judging whether a preposed action is needed; specifically, the task management module provides configuration and debugging for a user, and provides task details of a reading process, such as reading area coordinate definition, simulation trigger action definition, execution sequence of each step of task and the like; on the other hand, the screen capture module is called by the interface module, screens all task lists executed at each time according to task parameters acquired by the interface module, then calls the virtual action module and the screen capture module to carry out prepositive action and screen capture operation, and after screen capture is acquired, the screen capture module can continuously transmit local identification tasks and screen capture pictures to the picture processing module;
s3, the virtual action module executes a virtual prepositive action and transmits a screen capturing task to the screen capturing module; specifically, when an application interface calls an interface, the interface is called by a task management module, corresponding virtual prepositive actions are executed according to a collection strategy screened by the task management module, and a screen capture task is transmitted to a screen capture module, wherein the prepositive actions are specified by a user through a keyboard key, a mouse click, a double click, a right key and a combination of the keyboard key, the mouse click, the double click and the right key;
s4, the screen capture module captures the current screen and stores the current screen as a screen capture picture, and transmits the task and the screen capture picture to the picture processing module; when the application interface calls the interface, the interface is called by the task management module, and screenshot is carried out according to the collection strategy screened by the task management module and the task and the picture file are transmitted back to the picture processing module;
s5, the picture processing module performs local segmentation and preprocessing according to the screenshot file corresponding to the task transmitted by the screenshot module, stores the screenshot file as a local picture and generates a sequence tag, and transmits the sequence tag to the content identification module;
s6, the content identification module identifies and calculates the pictures processed and transmitted by the pictures to obtain result data, and returns the result data to the picture processing module;
and S7, the image processing module feeds back result data returned by the identification module to the interface module according to a preset format.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (4)
1. An image recognition-based device data acquisition system, the system comprising: the system comprises an interface module, a task management module, an acquisition strategy execution module and a content identification module;
the interface module is used for providing standard service for external application, receiving task parameters issued by the external application and returning result data to the external application;
the task management module is used for providing a user with a configuration, debugging and test self-defined acquisition strategy; the task list is called by the interface module, and the task list required to be executed is screened out according to the task parameters acquired by the interface module;
the acquisition strategy execution module intercepts a screen according to the task list, stores the screen as a screen capture picture, performs local segmentation and pretreatment on the screen capture picture to obtain a local image, and transmits the local image and a sequence label mark corresponding to the local image to the content identification module; transmitting the result data returned by the content identification module to the interface module;
the content identification module is used for preprocessing the received local image, identifying information in the local image by using an image identification technology to obtain result data, forming a corresponding array according to the sequence of the sequence label marks, and transmitting the corresponding array back to the acquisition strategy execution module.
2. The image recognition-based device data acquisition system according to claim 1, wherein the acquisition strategy execution module comprises a virtual action module, a screen capture module and a picture processing module;
the virtual action module is called by the task management module, executes a corresponding virtual preposed action according to the collection strategy screened by the task management module, and transmits a screen capturing task to the screen capturing module;
the screen capture module is called by the task management module, captures images according to the collection strategy screened by the task management module to obtain screen capture images, and transmits local identification tasks and the screen capture images to the image processing module;
the picture processing module is used for carrying out local segmentation and pretreatment on the screen capturing picture according to the local identification task transmitted by the screen capturing module to generate a plurality of local images, transmitting the local images and the corresponding sequence label marks to the content identification module, receiving result data transmitted back by the content identification module, forming corresponding arrays according to the sequence of the sequence label marks, and feeding back the corresponding arrays to the interface module.
3. The image recognition-based device data acquisition system of claim 2, wherein the screen capture module and the virtual action module are located at a client in the form of a background service.
4. An equipment data acquisition method based on image recognition is characterized by comprising the following steps:
s1, calling an interface service issued by an interface module by an external application;
s2, the task management module acquires task parameters and screens out an executed task list; judging whether a preposed action is needed, if yes, executing a step S3, and if not, executing a step S4;
s3, the virtual action module executes a virtual prepositive action and transmits a screen capturing task to the screen capturing module;
s4, the screen capture module captures the current screen and stores the current screen as a screen capture picture, and transmits the task and the screen capture picture to the picture processing module;
s5, the picture processing module performs local segmentation and pretreatment according to the screenshot file corresponding to the task transmitted by the screenshot module, stores the local picture and the generated sequence label mark, and transmits the local picture and the generated sequence label mark to the content identification module;
s6, the content identification module carries out identification calculation on the pictures processed and transmitted by the pictures to obtain result data, corresponding arrays are formed according to the sequence of the sequence labels, and the result data are returned to the picture processing module;
and S7, the image processing module feeds back result data returned by the identification module to the interface module according to a preset format.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110487550.0A CN113221866B (en) | 2021-04-30 | 2021-04-30 | Equipment data acquisition system and method based on image recognition |
| EP21190977.5A EP4083935A1 (en) | 2021-04-30 | 2021-08-12 | Device data collection system and method based on image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110487550.0A CN113221866B (en) | 2021-04-30 | 2021-04-30 | Equipment data acquisition system and method based on image recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113221866A CN113221866A (en) | 2021-08-06 |
| CN113221866B true CN113221866B (en) | 2022-11-15 |
Family
ID=77090798
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110487550.0A Active CN113221866B (en) | 2021-04-30 | 2021-04-30 | Equipment data acquisition system and method based on image recognition |
Country Status (2)
| Country | Link |
|---|---|
| EP (1) | EP4083935A1 (en) |
| CN (1) | CN113221866B (en) |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102571828B (en) * | 2010-12-07 | 2014-09-10 | 新奥特（北京）视频技术有限公司 | Webpage-based remote multimedia monitoring method and system |
| US20160249106A1 (en) * | 2012-09-14 | 2016-08-25 | Appurify, Inc. | Remote Control of a Mobile Device |
| EP3112965A1 (en) * | 2015-07-02 | 2017-01-04 | Accenture Global Services Limited | Robotic process automation |
| CN105988688A (en) * | 2016-01-22 | 2016-10-05 | 腾讯科技（深圳）有限公司 | Screen capture method and apparatus |
| CN107977652B (en) * | 2017-12-21 | 2020-08-28 | 维沃移动通信有限公司 | Method for extracting screen display content and mobile terminal |
| CN108124487B (en) * | 2017-12-22 | 2023-04-04 | 达闼机器人股份有限公司 | Cloud meter reading method and device |
| CN110557417B (en) * | 2018-05-31 | 2022-04-19 | 西安中兴新软件有限责任公司 | Image processing method and device and storage medium |
| CN108898813A (en) * | 2018-06-13 | 2018-11-27 | 深圳市云识科技有限公司 | A kind of cloud identification kilowatt meter reading-out system based on Internet of Things framework |
| US11507772B2 (en) * | 2019-10-02 | 2022-11-22 | UiPath, Inc. | Sequence extraction using screenshot images |
| CN112256636A (en) * | 2020-11-10 | 2021-01-22 | 国网湖南省电力有限公司 | Data acquisition system for mobile application APP |
-
2021
- 2021-04-30 CN CN202110487550.0A patent/CN113221866B/en active Active
- 2021-08-12 EP EP21190977.5A patent/EP4083935A1/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| CN113221866A (en) | 2021-08-06 |
| EP4083935A1 (en) | 2022-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109995601B (en) | Network traffic identification method and device | |
| CN112966772A (en) | Multi-person online image semi-automatic labeling method and system | |
| CN107092913A (en) | A kind of spray printing character ONLINE RECOGNITION and interactive after-treatment system and method | |
| CN107733967A (en) | Processing method, device, computer equipment and the storage medium of pushed information | |
| CN103313289A (en) | Wap system automatic test system and method | |
| CN112685109B (en) | Method and system for dynamically identifying and recognizing remote application window | |
| CN112669110A (en) | Server label management method and related equipment | |
| US8914798B2 (en) | Production control for service level agreements | |
| US20160314543A1 (en) | System and method for improved inspection techniques | |
| CN115686280A (en) | Deep learning model management system, method, computer device and storage medium | |
| CN105446909A (en) | Caching management method and system | |
| CN113034421A (en) | Image detection method, device and storage medium | |
| CN110310028B (en) | Method and apparatus for crowdsourcing | |
| CN116010066A (en) | RPA robot and implementation method | |
| US20190384691A1 (en) | Methods for providing an enterprise synthetic monitoring framework | |
| CN113221866B (en) | Equipment data acquisition system and method based on image recognition | |
| CN207099116U (en) | Data acquisition and remote control | |
| CN114691782B (en) | Database table increment synchronization method, device and storage medium | |
| CN110032503A (en) | Data processing system, method, equipment and device based on UI automation and OCR | |
| CN115758005A (en) | Large-screen image processing method, device and medium | |
| TWI806004B (en) | AI process management system and method for automated visual inspection | |
| CN113641569A (en) | Robot process automation method | |
| CN112182413A (en) | Intelligent recommendation method and server based on big teaching data | |
| CN113434404B (en) | Automatic service verification method and device for verifying reliability of disaster recovery system | |
| CN112968941B (en) | Data acquisition and man-machine collaborative annotation method based on edge calculation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |