CN116935320A - Image modeling and recognition method based on image template - Google Patents
Image modeling and recognition method based on image template Download PDFInfo
- Publication number
- CN116935320A CN116935320A CN202310937654.6A CN202310937654A CN116935320A CN 116935320 A CN116935320 A CN 116935320A CN 202310937654 A CN202310937654 A CN 202310937654A CN 116935320 A CN116935320 A CN 116935320A
- Authority
- CN
- China
- Prior art keywords
- image
- recognition
- algorithm
- equipment
- modeling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000012545 processing Methods 0.000 claims abstract description 27
- 238000013135 deep learning Methods 0.000 claims abstract description 15
- 238000002372 labelling Methods 0.000 claims description 34
- 230000006870 function Effects 0.000 claims description 28
- 238000013507 mapping Methods 0.000 claims description 9
- 238000013461 design Methods 0.000 claims description 6
- 238000009434 installation Methods 0.000 claims description 5
- 238000013475 authorization Methods 0.000 claims description 2
- 230000002159 abnormal effect Effects 0.000 abstract description 5
- 238000012423 maintenance Methods 0.000 abstract description 5
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 230000000694 effects Effects 0.000 abstract description 3
- 230000003044 adaptive effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 7
- 238000007689 inspection Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 5
- 238000003672 processing method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000003825 pressing Methods 0.000 description 3
- 238000004146 energy storage Methods 0.000 description 2
- 239000007788 liquid Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/12—Geometric CAD characterised by design entry means specially adapted for CAD, e.g. graphical user interfaces [GUI] specially adapted for CAD
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
- G06F9/4482—Procedural
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
- G06F9/4488—Object-oriented
- G06F9/4492—Inheritance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Abstract
According to the image modeling and identification method based on the image template, a flexible and easily-extensible intelligent image identification algorithm management and cooperation framework is constructed; completing unified management of an artificial intelligence algorithm based on image processing and deep learning through image template modeling; the image recognition service performs unified scheduling and image recognition of a recognition algorithm according to the modeling information, and realizes concurrent recognition of states and readings of different types of equipment in the recognition object and recognition of abnormal scenes. The method can effectively solve the problem of the number of the data samples of different types of equipment of each transformer substation in different implementation stages of actual engineering, provides an adaptive recognition algorithm strategy, can simply, conveniently and quickly complete iterative maintenance of the recognition algorithm according to the gradually enriched data samples, and has the advantages of scientific and reasonable method, strong applicability, simplicity in operation, high recognition accuracy, good implementation effect and the like.
Description
Technical Field
The application relates to the field of intelligent image identification inspection of a track power supply substation, intelligent image identification inspection of a digital power plant and a power grid substation.
Background
At present, although a comprehensive automatic system is basically arranged in a railway traction substation, a digital transformer substation, a power plant, a rail transit vehicle control room and the like, the daily work demands of a dispatcher are met to a certain extent. However, the automation level of the operation and maintenance personnel work cannot be improved, and the daily inspection work such as whether the equipment room is leaked, whether the appearance of the equipment is good, whether the operating environment has foreign matters, the abnormal alarm of the equipment body, and the checking of the readings or the states of various types of equipment are still not improved. Even, a potential safety hazard problem may be brought. In addition, for operation and maintenance personnel, the working efficiency is low, and the risks such as artificial uncertainty and the like are also caused. Meanwhile, outdoor inspection work like a transformer substation can be influenced by weather and the like. Therefore, an automatic and intelligent inspection operation and maintenance means is needed to effectively improve the operation and maintenance level of the transformer substation, reduce security risks caused by leakage of equipment bodies and equipment rooms, and further promote unmanned on duty of the transformer substation.
With the continuous improvement of the performance of computer hardware equipment and the continuous development of artificial intelligence, image information processing and other technologies, the technologies can be innovated and applied in various traditional or emerging industrial fields. By carrying out visual image detection and identification on various types of equipment in a railway power supply transformer substation and a power grid transformer substation, the operations of manually opening an exhibition list, checking equipment conditions, detecting personnel invasion and the like in a high-voltage area in the past can be replaced, the manual introduction of data errors is avoided, the labor burden is lightened, the life safety of workers and the stable operation of a power system are ensured, the purposes of low cost, convenience and full-element unmanned detection are realized, the intelligent upgrading of the transformer substation is further promoted, and the related work of the transformer substation is developed to unmanned, reconfigurable, information digital, functional integrated, compact in structure and visual in state.
However, from practical engineering implementation experience, the types of power substations in various industrial fields are numerous, for example, in an urban rail transit power supply system, there are generally provided several types of power substations such as a main power substation, a traction step-down hybrid power substation, a step-down power substation, a following step-down power substation and the like; railway power supply, urban power grids also contain different types of substations. The transformer station comprises various kinds of equipment with different types.
From the technical point of view of image recognition, the technical means that can be used mainly include two aspects: image processing-based methods and deep learning-based methods. Based on the traditional image recognition algorithm, the method has lower requirements on the picture data, the processing result is mechanized, and the method has better effect on the object with simple background and small complexity of recognition targets. Because the method can only extract the shallow features such as the shape, the direction, the color and the like of the image, the identification generalization capability is weak, and the rich connotation information of the image is difficult to mine. Therefore, the object recognition effect for a complex scene is poor. The deep learning method based on data driving can effectively overcome subjectivity, ambiguity and uncertainty of artificial design features in the image processing method, and can fully mine deep semantic features of image data, and the application scene is wider than that of the image processing method, but the deep learning method is seriously dependent on data self factors such as size and diversity of data quantity, balance of sample quantity and the like.
At present, an image recognition for a track power supply transformer substation or a digital power plant transformer substation has the common problem that equipment with multiple equipment types on one image cannot be recognized accurately and conveniently at the same time. For example, an equipment cabinet image generally includes various types of equipment such as a switch-on/off indicator lamp, a certain number of open/close indicator lamps, a voltage meter, an ammeter, a certain number of pressing plates or connecting pieces (sometimes divided into pressing plates or connecting pieces which are put into use and standby), a gear knob, an LED digital display and the like. To cope with this problem, an engineering site basically sets a plurality of preset positions for one equipment cabinet, each preset position photographs a certain type of equipment on the equipment cabinet in a deviation angle manner, establishes a corresponding relation between the preset position and a corresponding equipment type, and then sequentially performs image recognition on the plurality of preset positions. The processing mode can multiply increase the preset number of cameras and the configuration workload of engineering personnel, so that the implementation efficiency of engineering projects is low. Even some factories cannot recognize the types of equipment in batches, and like the recognition software, the method is suitable for recognizing the states or analog quantities of fixed type equipment in equipment rooms of the substation in some subway stations.
In addition, the neural network deep learning technology is adopted completely, although the technology such as the algorithm of the comparison front edge can be adopted. However, in the early stage of project implementation, training data sets are lacking, and various devices are added, so that in particular, in actual situations, mutual exclusion state data or negative sample data of a specific device cannot be acquired at all. Thus, in this manner, particularly in the early stages of project implementation, the data set is essentially continuously constrained, and the goal of versatility is not achieved. Of course, the technical mode also has a suitable scene, namely the method can be used as an upgrading optimization means of image recognition when the later data set is sufficient. For example, if a person in a designated area wears a construction helmet (whether a worker wears a safety helmet or not is judged), a large number of test data sets exist, so that the technical mode is more suitable for being adopted.
In summary, in order to better adapt to the actual engineering situation, the method based on image processing and the method based on deep learning can be combined for application. However, how to uniformly manage and schedule advanced intelligent recognition algorithms or algorithms suitable for recognition of a specific device and allow human participation in adjustment, and apply the advanced intelligent recognition algorithms to a proper recognition scene is a main problem considered to be solved by the method.
Disclosure of Invention
In view of the above problems, the present application has an object of: by means of the image modeling and recognition method based on the image template, a general intelligent image recognition algorithm management and cooperation framework easy to expand is built, and unified management and scheduling of the artificial intelligent algorithm based on image processing and deep learning are completed.
In order to achieve the above purpose, the technical scheme adopted by the application is as follows: an image modeling and recognition method based on an image template comprises the following steps:
step one, image data of various devices of a power supply substation are collected through a data collection function.
The data acquisition function is completed by the cooperation of the camera holder management system and the image recognition request client. The camera cloud deck management system is used for setting a preset position of the ball machine to be inspected, and the image recognition request client is responsible for calling a camera cloud deck management system interface to inspect and shoot equipment images.
And step two, completing image recognition template modeling of different types of equipment of the transformer substation through an equipment image recognition template modeling tool.
The equipment image recognition template modeling tool firstly carries out image recognition auxiliary information labeling on the preset bitmap image acquired by the data acquisition function, then binds different image recognition algorithms to different recognized equipment through the binding image recognition algorithm function, and finally derives a substation equipment object image recognition template modeling labeling information file and a corresponding template image thereof.
The equipment image recognition template modeling tool has the functions of identifying auxiliary information labeling of a recognized image, binding an image recognition algorithm, pre-recognition of the image, storing labeling information and template images, and importing the image recognition template modeling labeling information and corresponding template images.
The equipment image recognition template modeling tool can respectively carry out information labeling, recognition algorithm binding and parameter setting on a plurality of equipment (such as an energy storage state table, a disconnecting link, a circuit breaker, an idle switch, a connecting piece, a temperature patch, a knob, a pressing plate, a liquid level meter, an indicator lamp, a single pointer meter, a double pointer meter and a character type indicator meter) of the same or different types on a recognized image.
The identification auxiliary annotation information comprises public annotation information and private annotation information.
The public annotation information comprises annotation coordinates, preset bit numbers, data point numbers agreed with an external system and data type information.
The private labeling information is designed according to the self characteristics of different types of equipment, such as pointer table type data, and can comprise: the initial range coordinate, the center coordinate, the initial range and whether mirror image exists; the liquid level apparatus includes: maximum scale value and minimum scale value.
The binding image recognition algorithm function comprises a core tuple, denoted < deviceclass, algorithm, ways, map >. The meaning is as follows: the equipment category, the algorithm number (the algorithm number corresponds to the equipment sub-category one by one), the identification method contained in the same algorithm number and the options with different values of the equipment state.
The value of deviceclass is used as a substation equipment major class, is named according to the definition of the actual class, and is used as a key word of the configuration item of the whole equipment class. For example, the indicator light type may be named as "light", the idle-on type may be named as "konskai", the pointer instrument type may be named as "meter", the switch knob type may be named as "xuanniu", and the transformer night may be named as: "yewei", energy storage device class named "chuneng", knife switch device class named: the breaker equipment category is named "duanluqi".
algorithm is used as a sub-category of a large category of equipment, and each sub-category corresponds to a specific algorithm number. The configuration sub-item is a configuration object, and the configuration sub-item takes the value of: "Key" and algorithm number (algorithm-number). Wherein, the key is named according to the definition of the actual category, and the algorithm-number is correspondingly assigned with a non-repeated number. Assuming that the knife switch device class contains two subcategories, an algorithm object corresponding to the knife switch device class may be defined in the form of { "algorithm": { "daozha":2, "gekai":45 }.
The different recognition methods contained in the same algorithm number are Ways, the method is a tuple structure, each element of the tuple is an object, and the value of the object is as follows: "{ identification method (key): identification algorithm number (identification-way-number), name: identification method display name }". The identification-way-number corresponds to a number that is not repeated. Assume that the indicator lamp type device has 4 different identification modes, so that 4 groups of values are taken in total, and the form is as follows:
[ { "light_dl":0, "name": "indicator light (deep learning)" }, { "light_lite 1":1, "name": "small light one" }, { "light_lite 2":2, "name": small light two "}, {" light_color ":3," name ": color judgment" }, { "light":4, "name": light "}. When the first group is selected in the modeling process, the recognition of the on-off state of the indicator light is performed by adopting a deep learning algorithm. Other modes adopt an image processing method for identification. The second last group is selected in the modeling process, and the identification of the color of the indicator lamp is performed by using an image processing method.
The algorithm and the wax are mainly used for controlling granularity and flexibility of a binding recognition algorithm, generally different equipment sub-categories correspond to one recognition algorithm, and when the recognition requirement is met, the wax can be not further specified. When the same equipment subclass can adopt a plurality of recognition methods of deep learning or image processing, the optional image recognition algorithm set can be further expanded by configuring the way so as to meet the requirements of equipment image recognition at different implementation stages of engineering and facilitate later algorithm updating and replacement.
The map mainly converts the identification result into understandable content and marks the content on the identified picture. The system is of an array structure, each element of the array is an object, and the value of the object is as follows: "recognition result in digital form (key): recognition result map name (map-name)", wherein key is the result of image recognition expressed in digital form, map-name corresponds to map names of different meanings. And the modeling is configured into a plurality of groups according to actual demands, and one group is selected according to the demands during modeling. Taking knob class as an example, 9 groups of values are taken in total, and the form is as follows:
[ { "0": "input", "1": "exit" }, { "0": "distant", "1": local ", { 0": local ","1": distant", { 0":" brake off ","1":" brake off "}, { 0": manual ","1": local", { 0": remote", { 0": 1": inhibit ","2": local", { 0": brake off (green red light)", "-1": abnormal ", { 1": automatic ","0": 0", "1": 1"," 2": 2", "" 3"," "4": 4"," 5": 5", "6": 6 "}, { 0" "brake off (green red light)", "1": brake off (green red light) "," abnormal "" "" "be" "be".
When the last group is selected in the modeling process, the knob at the corresponding position on the identified image is selected by a frame when the image identification result is 0, and a switch-on (green, red, bright) is displayed.
When the binding image recognition algorithm function carries out recognition algorithm binding, firstly, setting a device major class, then setting an algorithm class (device subclass), then, if the device to be bound with the recognition algorithm is configured with different recognition methods, setting one of the recognition methods according to configuration selectable items, and finally, setting the mapping information of the recognition result, namely finishing the mapping setting between the recognition result of the digital type and the understandable display content.
The image recognition algorithm binding function designs an image recognition algorithm interface specification by an object-oriented design method, and builds an extended algorithm management module, firstly, an algorithm processing abstract interface is defined, and then, the image algorithm interface is realized for each image recognition algorithm (algorithm subclass) of the newly added equipment type, so that the corresponding image recognition algorithm is completed.
The image recognition algorithm interface specification provides a method function for a specific recognition algorithm, and comprises affine transformation matrix solving, drawing results, JSON format data extracting, image capturing, image matching, image cutting, coordinate selection and image recognition execution.
Different algorithm subclasses inherit from the algorithm interface specifications, and different methods of the same algorithm class can be processed by adopting different branch functions according to actual design, and different processing classes can be defined for processing. Thereby meeting the open-close principle and facilitating the updating and replacement of the later algorithm.
After information labeling, recognition algorithm binding and parameter setting are completed for a plurality of devices to be recognized on the recognized image. Through the image pre-recognition function, the image recognition result is previewed, and after confirming, the information of the current annotation can be saved to the annotation result file, and meanwhile, the image is saved as an image recognition template.
And thirdly, completing image recognition and result feedback of different types of substation equipment through the image recognition service.
The image recognition service may multiplex the image recognition algorithm interface specification and its different algorithm subclasses.
The image recognition service needs an authorized license to be started normally, and then enters a loop to wait for an image recognition request of an external client. After that, if an image recognition request is received, the following processing flow is performed.
Firstly, analyzing the request data, and if the request data accords with the agreed parameter request format, loading the identified picture.
Secondly, loading an image modeling template, and if template information exists, performing feature matching on the identified picture and the image modeling template.
And then, according to the information marked by the image modeling, determining the identification algorithm and parameters set by different types of equipment in the image, and acquiring corresponding identification algorithm processing types according to the algorithm numbers to identify the equipment state or equipment range and reading.
And finally, returning the identification result to the client.
The recognition result comprises two forms, wherein one form is a recognition result in a picture form, and the other form is a recognition result in a text format. And selecting different types of equipment frames on the identified picture by the picture form result, and respectively marking the identification results such as the state, the measuring range, the reading and the like of the equipment above the position of the equipment. The main key items of the text format recognition result comprise:
channel id: the channel number of the camera, representing a uniquely encoded one of the camera devices, is typically provided by the video manufacturer.
channel name: the picture name, also commonly set to a unique identifier, is the rule "installation site_installation location_camera type and preset bit number", such as "control room_row of wall mounted 4# ball machines 46".
result: is an array of data objects, wherein each element of the array is identification result data of all different types of devices on a predetermined bit. The key items mainly comprise:
PresetNum: the number of the preset bit;
presetName: the name of the preset bit;
the key item of the identification result is used for representing different types of equipment, and the rule is as follows: "device class" (key) and "result-value" are recognized.
The identification category (key) comprises a value set of deviceclass in the binding image identification algorithm function, and the specific value of the identification category (key) depends on the equipment category in the loaded image modeling annotation information when a client request is processed.
The "result-value" is a tuple structure. Each element of the array is an identification result object of the same type of equipment, and the main values comprise:
objectId: a number representing an external system data point object;
attrId: the method is mainly used for the attribute number of the data point object of the third party system, and if the external system is not needed, the item is used as a reserved field;
raw: representing the original value of the identification result;
value: and representing the display value of the identification result, namely, the mapping value corresponding to the original value of the identification result.
dataType: the type of the identified result value is represented, and the preferable values include: bool, int, float, long, string for client to decode data.
The values of objectId, attrId, dataType are all from common annotation information in image modeling.
Compared with the prior art, the technical scheme of the application has the following advantages:
1. according to the method, intelligent image recognition algorithms aiming at different equipment image recognition scenes can be customized according to the data sample conditions of different implementation stages of engineering, and the recognition algorithm strategies can be flexibly adjusted according to the gradual perfection of the data samples. Facing many different types of equipment in rail power substations.
2. According to the collaborative framework, modeling and feature labeling of the image template are adopted, concurrent identification and anomaly detection of different types of equipment contained in the identification object can be flexibly achieved, engineering implementation is facilitated, and compared with a mode that equipment outline areas are determined from a plurality of different types of images by means of completely relying on gray level processing, edge detection, image information extraction and the like, and then matching identification is performed, the method has higher identification efficiency and accuracy.
3. The method does not need to care whether a specific recognition algorithm adopts an image processing technology or a deep learning technology, but mainly focuses on how to uniformly manage the specific recognition algorithm through agreement of an algorithm interface standard and modeling of an image template, and uses the specific recognition algorithm in a recognition scene suitable for application of the specific recognition algorithm to realize simultaneous recognition and anomaly detection of various equipment images on an engineering site in batches.
Drawings
FIG. 1 is an overall collaborative framework for image modeling and recognition in accordance with an embodiment of the present application.
Fig. 2 is a flowchart of an image recognition process of a different type of device according to an embodiment of the present application.
Fig. 3 is a schematic diagram of global configuration of different device types according to an embodiment of the present application.
Fig. 4 is a flow chart of accident picture processing according to an embodiment of the present application.
Detailed Description
So that those skilled in the art can further understand the features of the present application and the technical content thereof, refer to the following detailed description of the application and the accompanying drawings, which are provided for reference and illustration and not for limitation.
In this embodiment, the image recognition service program operation device information acquisition program is written as "machineInfo", the image recognition service program authorization program is written as "genLicense", the image recognition modeling tool, and the image recognition service program. The image recognition modeling tool is realized by using Python and PyQt, and the image recognition service program is realized by using Python, so that the patrol instruction request processing, the image recognition and the result feedback are completed.
In this embodiment, the image template modeling and recognition overall collaboration framework is shown in fig. 1. In order to more briefly describe the implementation process, in this embodiment, the devices in the screen cabinet in the substation equipment room are selected, and the simultaneous identification of the indicator light, the idle-open device, the single pointer table and the knob device is taken as an example, and the characteristics and the technical embodiments of the present application are described with reference to the accompanying drawings.
Step one, authorizing an image identification service program.
Firstly, running 'machineInfo' on a machine needing to run an image recognition service program, completing information acquisition of the running machine, and generating an encrypted 'computer_info.dat' file.
The "computer_info. Dat" file is then used as input to the "genelicense" program, and the "genelicense" program is run on a machine dedicated to authorization, generating an encrypted "license. Dat" file.
Finally, the image recognition service program needs to be started normally based on "license.
And step two, acquiring image data of various devices of the power supply substation through a data acquisition function.
The data acquisition function is completed by the cooperation of the camera holder management system and the image recognition request client. The camera holder management system is used for setting preset sites of the ball inspection machine to be inspected, and the image recognition request client is responsible for calling a camera holder management system interface to inspect and shoot image data of equipment.
Step1 selects two cameras, and sets a preset bit for each camera, and the preset bit naming rule is "camera installation area_according to the position_camera name+preset bit number". For convenience of the subsequent description of this embodiment, the preset positions of the two ball machines are respectively denoted as "preset position one" and "preset position two", and correspond to the indicator light and the idle device in the "screen cabinet one" and the knob and the voltage and current meter device in the "screen cabinet two" respectively.
Step2, setting the preset bit of the first dome camera to be 'preset bit one' through the control of a holder, performing image snapshot operation, and storing the image as an image, wherein the naming rule is as follows: and adding shooting time and channel number information on the basis of a naming rule of a preset bit. For example, the actual designation of a snap shot image is: "control Chamber wall mounted 3 ball machine 02_20221222131946_003.Jpg". For convenience of the following description of the present embodiment, the Image captured by "preset bit one" in this step is denoted as "presetcone_image". Then the second ball machine is controlled to set its preset bit to "preset bit two" and take a snapshot of the Image and record the snapshot of the Image as "presettwo_image".
And thirdly, completing image recognition template modeling of different types of equipment of the transformer substation through an equipment image recognition template modeling tool.
Step1 is aimed at setting modeling tool global configuration information of different types of equipment, and mainly comprises equipment category, algorithm number, identification method contained in the same algorithm number, setting of key contents such as different value options of equipment state, and the global configuration schematic diagram of different identified equipment types in the embodiment is shown in fig. 3. The configuration information is only required to be set once, and can be flexibly expanded or updated according to the newly added equipment type and algorithm.
Step2 loads "presetcone_image" and "presetwo_image" into the Image recognition modeling tool. A schematic diagram of the image template modeling tool in this embodiment is shown in fig. 4.
Step3, performing feature labeling and recognition algorithm binding operation on the recognized equipment in the image.
Step3_1 first performs characteristic information labeling and recognition algorithm binding on the indicator light type device in the Image of 'presetone_image'. And then, marking the characteristic information of the idle type equipment and binding the characteristic information with an identification algorithm.
The Step3_1_1 indicator lamp type device performs the following characteristic information labeling and recognition algorithm binding process.
1. The "major class" (i.e. device class) selection box in the modeling tool right side selects "indicator light", and selects the corresponding minor class (i.e. device subclass, if the corresponding device class has no multiple recognition methods, the corresponding recognition algorithm number of the device subclass) according to the indicator light type in the actual image.
2. Selecting an 'indicator light-DL' option in a 'method' (namely identification algorithm number) selection frame, wherein the selection represents that the on-off state of indicator light equipment in an image is identified based on a deep learning intelligent identification algorithm in the embodiment; meanwhile, in the 'mapping' selection frame, a recognition result with display significance is set.
3. Clicking the labeling operation, performing clicking, positioning and framing the periphery of the indicator lamp by using a mouse in the left preset position image column, confirming labeling information, and displaying the labeled information in a labeling result display table.
After the labeling operation, auxiliary parameter information is set in the popped information frame, and only the device ID is needed to be filled into the indicator lamp device.
The characteristic information labeling and recognition algorithm binding process of the Step3_1_2 open type device is similar, and the specific steps are as follows.
1. In the modeling tool right side selection operation field, a "major class" (i.e., device class) selection frame selects "open switch", and a minor class selection frame selects "open switch".
2. Selecting a null-on-DL option in a method selection frame, which means that in the embodiment, the on-off state of the null-on equipment in the image is identified based on a deep learning intelligent identification algorithm; meanwhile, in the 'mapping' selection frame, a recognition result with display significance is set.
3. Clicking the mark, performing point positioning frame selection on the periphery of a blank switch by using a mouse in a left preset position image column, then confirming mark information and displaying the marked information in a mark result display table.
After the "labeling" operation, the device ID is filled in the pop-up information box, and "OK" is clicked.
Step3_2 first performs characteristic information labeling and recognition algorithm binding on the voltage and current meter type equipment in the 'presettwo_image' Image. And then, carrying out characteristic information labeling and recognition algorithm binding on the knob type equipment.
The characteristic information labeling and recognition algorithm binding process of the Step3_2_1 voltage and current meter type device is as follows.
1. In this embodiment, the pointer table is identified by adopting an image processing-based manner, and the single pointer table and the double pointer table of different types correspond to different identification algorithms. And selecting a 'pointer table' by a major category selection box in a right side selection operation field of the modeling tool, and selecting a corresponding minor category according to the type of the single pointer table in the image.
2. Selecting a corresponding mapping relation according to the identification position in the actual picture;
3. clicking the mark, namely clicking and positioning the center of the single pointer table and the starting range point by using a mouse in the left preset position image column;
4. and filling the device ID and the start-stop range value into the popped information frame.
The characteristic information labeling and recognition algorithm binding process of the Step3_2_2 knob type device is similar, and the specific steps are as follows.
1. In this embodiment, the knob is identified by adopting an image processing-based manner, and different gears and different shapes of the knob correspond to different identification algorithms. And selecting a 'knob' from the major category in the right selection operation column, and selecting a corresponding minor category according to the knob type in the actual image.
2. Selecting a corresponding mapping relation according to the identification position in the actual picture;
3. clicking the mark, and using a mouse to position around the frame selection knob in the left preset position image column and confirm.
Step4, pre-identifying the image, clicking an identification button, identifying all marked devices on the image, checking whether modeling identification is accurate or not through the operation, and readjusting marking information if individual identification is not ideal.
Step5, clicking the right side of the interface to store the template image and store the labeling information after the accurate verification of the image pre-recognition.
Step6, the image template and the feature labeling information file are distributed to the 'template img' and 'configs' of the image recognition service program.
And step four, completing image recognition and result feedback of different types of substation equipment through the image recognition service.
The overall processing flow of the image recognition service program in this embodiment is shown in fig. 2.
Step1, simulating a timing inspection function by an image recognition simulation request program. Firstly, invoking the data acquisition function to sequentially take pictures of preset positions set by the control room wall-mounted 3# ball machine and the control room wall-mounted 3# ball machine, and storing the pictures to a specified path for loading by the image recognition service program. Then, an image recognition request is initiated to the image recognition service program. The request parameters include picture names (including camera and preset bit coding information), inquiry time periods and identified image storage paths.
And Step2, the image recognition service program starts an image recognition working thread for each request to perform image recognition and feeds back a processing result.
Step2_1 loads all preset bitmap pieces to be identified in the requested time period under the corresponding path according to the request parameters.
And analyzing the Step2_2 through the picture name information, and acquiring corresponding preset bit image template images and labeling information established by an image modeling tool from the catalogues of 'template img' and 'configs'.
And Step2_3, acquiring an object to be identified according to the parsed labeling information, and identifying the equipment state or the indication according to the algorithm code call corresponding to the binding algorithm.
The processing procedure of the identification algorithm adopted in the present embodiment mainly includes two main procedures. First, feature matching is performed on the identified image and the template image. Then, the subsequent specific recognition processing is performed again for the recognized device.
And directly labeling and storing the identification result in the image by the Step2_4, forming contracted JSON format text data at the same time, and then returning the result to the client.
Aiming at the problems of shooting angles of various different types of equipment and various cameras of a track power supply substation and environmental interference factors, and considering the problems of the number of data samples of various substation equipment and the like in the engineering implementation stage, in order to further ensure that detection results can still be obtained accurately in various non-ideal scenes, the embodiment constructs a flexible and easily-expanded intelligent image recognition algorithm management and cooperation framework; completing unified management of an artificial intelligence algorithm based on image processing and deep learning through image template modeling; the image recognition service performs unified scheduling and image recognition of recognition algorithms according to modeling information, so that an adaptive recognition algorithm strategy can be provided for the problem of the number of data samples of different types of equipment of each transformer substation in different implementation stages of actual engineering, and the iterative optimized recognition algorithm can be maintained simply, conveniently and rapidly according to the gradually enriched data samples. Meanwhile, the concurrent recognition of the states and readings of different types of equipment in the recognition object and the recognition of abnormal scenes can be more effectively realized.
The above embodiments are only for illustrating the technical idea of the present application, and the protection scope of the present application is not limited by the above embodiments, and any modification made on the basis of the technical scheme according to the technical idea of the present application falls within the protection scope of the claims of the present application. The technology not related to the application can be realized by the prior art.
Claims (10)
1. An image modeling and recognition method based on an image template is characterized by comprising the following steps:
step one, acquiring image data of different types of equipment of a power supply substation through a data acquisition function;
step two, completing image recognition template modeling of different types of equipment of the transformer substation through an equipment image recognition template modeling tool;
and thirdly, completing image recognition and result feedback of different types of substation equipment through the image recognition service.
2. The image modeling and recognition method based on the image template according to claim 1, wherein the data acquisition function in the first step is completed by the camera pan-tilt management system and the image recognition request client in a matched manner; the camera cloud deck management system is used for setting a preset position of the ball machine to be inspected, and the image identification request client is responsible for calling a camera cloud deck management system interface to inspect and shoot equipment images.
3. The image modeling and recognition method based on the image template according to claim 1, wherein the device image recognition template modeling tool in the second step has a function of labeling recognition auxiliary information of the recognized image, a function of binding an image recognition algorithm, a function of pre-recognition of the image, a function of storing labeling information and template images, and a function of importing the labeling information of the image recognition template modeling and the corresponding template images, and can respectively label information, bind recognition algorithm and set parameters for a plurality of devices of the same or different types on the recognized image.
4. The image modeling and recognition method based on the image template according to claim 3, wherein the device image recognition template modeling tool in the second step firstly performs image recognition auxiliary information labeling on the preset bitmap image acquired by the data acquisition function, then binds different image recognition algorithms to different recognized devices through the binding image recognition algorithm function, and finally derives a substation device object image recognition template modeling labeling information file and a corresponding template image thereof.
5. A method of image modeling and recognition based on image templates as defined in claim 3, wherein: the identification auxiliary annotation information comprises public annotation information and private annotation information; the public labeling information comprises labeling coordinates, preset bit numbers, data point numbers agreed by an external system and data type information; the private annotation information is designed according to the characteristics of different types of equipment;
the binding image recognition algorithm function comprises a core tuple which is recorded as < deviceclass, algorithm, ways, map >; the meaning is as follows: the equipment category, the algorithm number, the identification method contained in the same algorithm number and the equipment state different value options.
6. The image modeling and recognition method based on the image template according to claim 4, wherein: when the binding image recognition algorithm function carries out recognition algorithm binding, firstly, setting equipment major categories, then setting algorithm categories, then, if equipment to be bound with the recognition algorithm is configured with different recognition methods, setting one of the recognition methods according to configuration selectable items, and finally, setting the mapping information of the recognition result, namely finishing the mapping setting between the recognition result of the digital type and the understandable display content.
7. The image modeling and recognition method based on the image template according to claim 6, wherein: the image recognition algorithm binding function designs an image recognition algorithm interface specification by an object-oriented design method, and builds an extended algorithm management module, firstly, an algorithm processing abstract interface is defined, then, the image recognition algorithm interface is realized for each newly added equipment type image recognition algorithm, and the corresponding image recognition algorithm is completed.
8. A method of image modeling and recognition based on image templates as defined in claim 3, wherein:
the value of deviceclass is used as a substation equipment major class, named according to the definition of the actual class and used as a key word of the configuration item of the whole equipment class;
algorithm and ways are used for controlling granularity and flexibility of binding identification algorithms, generally different device subcategories correspond to one identification algorithm, and when the identification requirement is met, ways can not be further specified; when the same equipment subclass can adopt a plurality of recognition methods of deep learning or image processing, the optional image recognition algorithm set can be further expanded by configuring the wax;
algorithm is used as a sub-category of a large category of equipment, and each sub-category corresponds to a specific algorithm number; the configuration sub-item is a configuration object, and the configuration sub-item takes the value of: "Key" in the form of an algorithm number (algorithm-number); wherein, the key is named according to the definition of the actual category, and the algorithm-number is correspondingly assigned with a non-repeated number;
the different recognition methods contained in the same algorithm number are Ways, the method is a tuple structure, each element of the tuple is an object, and the value of the object is as follows: "{ identification method (key): identification algorithm number (identification-way-number), name: identification method display name }"; the identification-way-number corresponds to a non-repeated number;
map converts the identification result into understandable content and marks the content on the identified picture; the system is of an array structure, each element of the array is an object, and the value of the object is as follows: a "recognition result (key) in digital form" form of a recognition result map name (map-name), wherein the key is a result of image recognition expressed in digital form, and the map-name corresponds to map names of different meanings; and the modeling is configured into a plurality of groups according to actual demands, and one group is selected according to the demands during modeling.
9. The image modeling and recognition method based on the image template according to claim 1, wherein: the image recognition service in the step three can multiplex the image recognition algorithm interface specification and different algorithm subclasses; the image recognition service can be started normally only by authorization permission, and then enters a loop to wait for an image recognition request of an external client; after that, if an image recognition request is received, the following processing flow is performed:
firstly, analyzing request data, and loading an identified picture if the request data accords with a contract parameter request format;
secondly, loading an image modeling template, and if template information exists, performing feature matching on the identified picture and the image modeling template;
then, according to the information marked by the image modeling, determining the identification algorithm and parameters set by different types of equipment in the image, and according to the algorithm number, acquiring the corresponding identification algorithm processing class to identify the equipment state or equipment range and reading;
and finally, returning the identification result to the client.
10. The image modeling and recognition method based on the image template according to claim 9, wherein: the identification result comprises two forms, wherein one form is an identification result in a picture form, and the other form is an identification result in a text format; the result of the picture form is to select different types of equipment frames on the identified picture, and the state, the measuring range and the reading of the equipment are respectively marked above the position of the equipment; the text format recognition result comprises:
channel id: the channel number of the video camera represents a uniquely coded image capturing device, which is generally provided by a video manufacturer;
channel name: the picture name is also generally set as a unique identifier, and the rule is 'installation place_installation position_camera type and preset bit number';
result: is an array of data objects, wherein each element of the array is identification result data of all different types of devices on a predetermined bit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310937654.6A CN116935320A (en) | 2023-07-28 | 2023-07-28 | Image modeling and recognition method based on image template |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310937654.6A CN116935320A (en) | 2023-07-28 | 2023-07-28 | Image modeling and recognition method based on image template |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116935320A true CN116935320A (en) | 2023-10-24 |
Family
ID=88389500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310937654.6A Pending CN116935320A (en) | 2023-07-28 | 2023-07-28 | Image modeling and recognition method based on image template |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116935320A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117610105A (en) * | 2023-12-07 | 2024-02-27 | 上海烜翊科技有限公司 | Model view structure design method for automatically generating system design result |
-
2023
- 2023-07-28 CN CN202310937654.6A patent/CN116935320A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117610105A (en) * | 2023-12-07 | 2024-02-27 | 上海烜翊科技有限公司 | Model view structure design method for automatically generating system design result |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110084526B (en) | BIM + WBS-based visual project management monitoring system | |
| CN109446664B (en) | On-site progress management display equipment system | |
| CN109242439A (en) | Feature extraction recognition methods based on substation equipment associated data | |
| CN112633535A (en) | Photovoltaic power station intelligent inspection method and system based on unmanned aerial vehicle image | |
| CN109543992A (en) | Intelligent polling method, device, intelligent terminal and server | |
| CN103578154A (en) | PDA-based realization method for intelligent patrol of electric equipment | |
| CN110648420A (en) | Intelligent inspection system for scheduling master station system equipment | |
| CN116935320A (en) | Image modeling and recognition method based on image template | |
| CN104567985A (en) | Bridge information processing method and device | |
| CN111369220A (en) | Teacher's cloud platform is built to wisdom | |
| CN114118831A (en) | Power transmission line management system based on digital twins | |
| CN105389381A (en) | Electronic map sign drawing system and method for city street lamps | |
| CN114862821A (en) | Automatic monitoring method, system and equipment for relay protection hard pressing plate | |
| CN114580957A (en) | Power grid infrastructure process control method, system and medium based on digital twin technology | |
| CN115858714B (en) | Unmanned aerial vehicle collected GIS data automatic modeling management system and method | |
| CN112418231A (en) | System algorithm for collecting operation and maintenance and AI model | |
| CN117172749A (en) | Rail transit inspection maintenance worker management method, device, equipment and storage medium | |
| CN117094660A (en) | Construction monitoring method and system based on digital twin technology | |
| CN116307595A (en) | Sanitation operation automatic planning method and system using unmanned plane and road monitoring | |
| CN207571783U (en) | A kind of LNG filling machines fault diagnosis removal system | |
| CN115689206A (en) | Intelligent monitoring method for transformer substation infrastructure progress based on deep learning | |
| CN113742441A (en) | Rapid updating method and system for electronic house map | |
| CN116363397A (en) | Equipment fault checking method, device and inspection system | |
| CN114897501A (en) | Intelligent rural industry digital treatment service platform | |
| CN107480797A (en) | A kind of LNG filling machines fault diagnosis removal system and its operating method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |