WO2023082061A1

WO2023082061A1 - Smart agent visual dispatching method based on augmented reality image processing

Info

Publication number: WO2023082061A1
Application number: PCT/CN2021/129629
Authority: WO
Inventors: 陈奂; 虢韬; 周海; 吕政�; 方曦; 肖林; 黄玉辉; 林先堪; 龙燕; 黄磊; 汪适; 喻群; 张宇红; 张亚维; 吴寿长; 禹天润; 邹胤; 杨秋; 艾丹; 田川
Original assignee: 贵州电网有限责任公司
Priority date: 2021-11-09
Filing date: 2021-11-09
Publication date: 2023-05-19

Abstract

Disclosed in the present invention is a smart agent visual dispatching method based on augmented reality (AR) image processing. The method comprises: collecting a dispatching instruction of a dispatcher and an operation instruction of an operation ticket, textualizing the dispatching instruction by means of image recognition technology, and performing standardized decomposition on a text instruction; a field operator finding a corresponding operation device according to the decomposed dispatching instruction and scanning a two-dimensional code on the operation device by means of an AR device, then, reading information of the operation device, and thereafter, automatically performing information comparison; if the comparison indicates that the information is correct, the field operator taking safety protection measures on the operation device before an operation, taking pictures of the safety protection measures and uploading same, and automatically determining, by means of artificial intelligence (AI) image recognition, whether the field safety protection measures are standard; if the field safety protection measures are standard, illustrating the operation instruction by means of the AR device, and an operator operating the operation device according to the illustrated operation instruction, taking pictures of the operation device and uploading same after the operation is completed; otherwise, reconfiguring the safety protection measures and checking same; taking pictures of the operation device and uploading same, screening out key data of an image collected by the AR device, and performing discrete cosine transform (DCT) compressed transmission; performing decompression in a terminal server; processing a compressed video image by means of improved Laplacian image sharpening technology, and classifying and learning the compressed video image and storing same; using the processed image for an application of a dispatching system, and proposing a regulation and control operation assistant decision-making suggestion by means of an AI algorithm for reference and selection by the dispatcher; then, automatically determining, by means of AI image recognition, whether the state of the operation device satisfies the operation instruction; if so, determining whether the operation instruction is fully completed; otherwise, re-operating the operating device; and if the operation instruction is fully completed, the system sending a completion instruction to the AR device, so as to inform the field operator that the device operation of the dispatching instruction has been completed. By means of the present invention, the mistransmissibility of information is relatively low, the problem of complex and tedious operation in existing electric power dispatching systems is effectively solved, and the training of new employees can be promoted by means of a visual operation, thereby improving the operation flexibility and safety of an electric power dispatching system.

Description

A Visual Scheduling Method for Smart Agents Based on Augmented Reality Image Processing

technical field

The invention relates to the technical field of electric power dispatching, in particular to a visual dispatching method for intelligent agents based on augmented reality image processing.

Background technique

Power scheduling is a very important link in the power system. The normal operation of the power system is inseparable from the real-time scheduling between the power supply company and the load. In addition, the maintenance and repair of power equipment also requires power scheduling. The most dangerous part of power dispatching is the on-site operation of power equipment. Once misoperation occurs, especially in high-voltage systems, very serious personal injury accidents will occur.

Augmented reality (Augmented reality, referred to as AR), also known as augmented reality. Augmented reality technology is a new technology that "seamlessly" integrates real world information and virtual world information. Taste, touch, etc.), through computer and other science and technology, simulate and then superimpose, apply virtual information to the real world, and be perceived by human senses, so as to achieve a sensory experience beyond reality. The real environment and virtual objects are superimposed on the same screen or space in real time. Augmented reality technology not only shows real world information, but also displays virtual information at the same time, and the two kinds of information complement and superimpose each other. In visual augmented reality, users use AR equipment to multiplex the real world and computer graphics, and then they can see the real world surrounding it. In recent years, smart wearable devices based on AR technology have been applied in the power industry, and innovative solutions have been proposed to solve traditional problems in the power system.

In 2020, the State Grid Corporation proposed the strategic goal of building an "internationally leading energy Internet enterprise with Chinese characteristics", with the grid as the center and the strong and smart grid as the basic platform, integrating advanced information and communication technology, control technology and advanced energy technology Integrate applications to build a smart energy system featuring clean and low-carbon, safe and reliable, ubiquitous interconnection, efficient interaction, and intelligent openness. On this basis, with the rapid development of 5G, AR and other technologies, the trend of integrating new technologies into power systems will become more and more obvious.

However, in the existing power system dispatching operation, it mainly relies on manual verbal repetition of various instructions, and each step is repeated at least twice, which is cumbersome and inefficient, and requires personnel who are extremely familiar with the site and equipment to operate. Training new employees requires a lot of manpower and material resources and there are uncertain risk factors. Using the visual scheduling method of smart seats based on augmented reality image processing has effectively changed the complex and cumbersome operation of the existing power dispatching system. Visualized operations can promote the training of new employees and improve the flexibility and safety.

Contents of the invention

The purpose of this section is to outline some aspects of embodiments of the invention and briefly describe some preferred embodiments. Some simplifications or omissions may be made in this section, as well as in the abstract and titles of this application, to avoid obscuring the purpose of this section, abstract and titles, and such simplifications or omissions should not be used to limit the scope of the invention.

In view of the above existing problems, the present invention is proposed.

Therefore, the present invention provides a visual scheduling method for intelligent agents based on augmented reality image processing, which can solve the problems of cumbersome operation steps for equipment operators and insufficient accuracy and safety of scheduling instructions in the power scheduling process in the prior art.

In order to solve the above-mentioned technical problems, the present invention provides the following technical solutions: including collecting dispatcher's dispatching instructions and operating instructions of operation tickets, textualizing dispatching instructions through image recognition technology, and standardizing and decomposing textual instructions; on-site operators according to The decomposed scheduling instruction finds the corresponding operating device, and scans the QR code on the operating device through the AR device to read the information of the operating device, and then automatically compares the information; if the information is compared correctly, the on-site operation The personnel will take the security measures before the operation of the operation equipment, take photos of the security measures and upload them, and automatically judge whether the on-site security measures are standardized through AI image recognition; if the on-site security measures are standardized, then pass the The AR device visualizes the operation instructions, and the operator operates the operation equipment according to the illustrated operation instructions. After the operation is completed, the operation equipment is photographed and uploaded; The security measures are described and checked; the operating device is photographed and uploaded, and the key image of the image collected by the AR device is screened out and compressed and transmitted by DCT (discrete cosine transform technology); decompressed in the terminal server; The Plasian operator image sharpening technology processes the compressed video images, classifies them, learns them, and stores them; the processed images are used in the application of the dispatching system, and artificial intelligence algorithms are used to propose control and operation assistance. Decision-making suggestions for the dispatcher to refer to and choose; then automatically judge through the AI image recognition whether the state of the operating equipment meets the operating instructions; if so, judge whether the operating instructions are all completed; otherwise, re-operate the operating equipment ; If all the operation instructions are completed, the system sends a completion instruction to the AR device, informing the on-site operator that the device operation of the dispatch instruction has been completed.

As a preferred solution of the intelligent agent visual scheduling method based on augmented reality image processing according to the present invention, wherein: the standardized decomposition of the text instructions includes,

Decomposition result = action + voltage level + device name.

As a preferred solution of the smart agent visualization scheduling method based on augmented reality image processing in the present invention, the AR device includes AR glasses, an intelligent image acquisition device, a 5G mobile phone, and the like.

As a preferred solution of the smart agent visualization dispatching method based on augmented reality image processing in the present invention, wherein: the information comparison also includes checking whether the operation device and the The corresponding situation of the scheduling instruction.

As a preferred solution of the smart agent visualization scheduling method based on augmented reality image processing in the present invention, the security measures include setting isolation fences and hanging warning signs.

As a preferred solution of the intelligent agent visualization scheduling method based on augmented reality image processing in the present invention, wherein: the AI image recognition includes preprocessing images, installing Tensorflow and Pillow libraries; defining convolutional network models, loss functions and An optimizer; execute image recognition training to obtain convolutional network model parameters; use the convolutional network model to perform the image recognition.

As a preferred solution of the smart agent visualization scheduling method based on augmented reality image processing in the present invention, wherein: taking pictures and uploading the security measures includes taking pictures of the security measures through an AR device and uploading them.

As a preferred solution of the smart agent visualization scheduling method based on augmented reality image processing in the present invention, wherein: the images required for the collection of power scheduling include all the images collected by the AR device and are directly stored in the local server, and automatically overwrites every fifteen days.

As a preferred solution of the smart agent visualization scheduling method based on augmented reality image processing in the present invention, wherein: the screening out of the key images collected by the AR device and compressing includes the selection of the key images collected by the AR device is The images in the real-time collection of the on-site AR equipment are processed by similarity filtering, and the duplicate images are filtered out, and then the DCT image compression processing is performed and transmitted.

As a preferred scheme of the smart agent visual scheduling method based on augmented reality image processing in the present invention, wherein: the transmission includes that the transmission method is to use 4G, 5G, SMS, etc. for transmission, which is suitable for short-distance , Medium-distance and long-distance transmission.

As a preferred solution of the smart agent visual scheduling method based on augmented reality image processing in the present invention, wherein: the improved Laplacian image sharpening technology includes the compressed key image passing through the scheduling terminal Perform decompression processing, use threshold denoising technology to perform noise reduction processing on the key image, and then perform Laplacian image enhancement. The selection of the threshold value varies with different application scenarios and different brightness.

As a preferred solution of the smart agent visual scheduling method based on augmented reality image processing in the present invention, wherein: the filtered images are classified and stored, including compressed images collected and transmitted by the AR device, Perform decompression processing, use the improved Laplacian operator image sharpening technology for image processing and analysis, and classify and store the processed pictures in the intelligent agent terminal server.

As a preferred scheme of the intelligent agent visual dispatching method based on augmented reality image processing in the present invention, wherein: the artificial intelligence algorithm is used to provide an auxiliary decision-making suggestion for regulation and operation, including the artificial intelligence algorithm, and extracting the compressed image , and can further maintain system voltage stability through equipment identification, text recognition, and fault diagnosis for power flow calculation and regional load compensation, and complete the deep reinforcement learning work of the power dispatching system, and propose auxiliary decision-making suggestions for regulation and operation through artificial intelligence algorithms for dispatching Staff reference selection.

As a preferred solution of the smart agent visual scheduling method based on augmented reality image processing in the present invention, wherein: the transmission of the operation completion instruction includes that when the operation instruction is sent to the AR device, voice broadcast operation content is required And use the AR device to mark the part of the device that needs to be operated; after the operation is completed, the AR device needs to be used to collect the password of the on-site operator to complete the operation, and give feedback information at the same time.

As a preferred scheme of the intelligent agent visual scheduling method based on augmented reality image processing in the present invention, wherein: the system includes that the system is a system in which text instructions and pictures correspond to each other; the steps of constructing the system are as follows: : First, take pictures of electrical equipment of different models, categories, and functions in sequence according to different switching states; then match the pictures of the equipment with the text one by one.

Beneficial effects of the present invention: the present invention combines the instruction content of the operation ticket with the operation device itself for visual processing, and the dispatcher and the operator establish contact through the AR device. Even the operator who is not familiar with the state of the device can scan the QR code, AI image recognition technology and image processing technology can also quickly locate the equipment to be operated and issue instructions for correct operation; the possibility of mistransmission of information is less than the existing traditional mode of manual broadcast, and the operation speed has also been greatly improved; At the same time, the present invention uses the remote expert cooperation function of the AR device to transmit the real-time on-site picture, and through the cooperation of multiple experts, the actual problems on the site can be solved online, and the AR device can be used to inform the operator in time of the part of the equipment that needs to be operated, so as to realize remote scheduling deal with.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without any creative effort. in:

Fig. 1 is a schematic flow chart of the image processing part described in the first embodiment of the present invention;

FIG. 2 is a schematic flow diagram of the method for visually dispatching intelligent agents based on augmented reality image processing described in the second embodiment of the present invention;

Fig. 3 is a schematic diagram of the operation instruction visualization process of the augmented reality image processing-based intelligent agent visualization scheduling method described in the second embodiment of the present invention;

Fig. 4 is the comparison of the distribution network image of the third embodiment of the present invention after three processing methods, wherein (a) is the compressed color image, (b) is the unimproved Laplacian image sharpness (c) is the effect diagram of the improved Laplacian image sharpening technology image processing.

Detailed ways

In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Example. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, a lot of specific details are set forth in order to fully understand the present invention, but the present invention can also be implemented in other ways different from those described here, and those skilled in the art can do it without departing from the meaning of the present invention. By analogy, the present invention is therefore not limited to the specific examples disclosed below.

Second, "one embodiment" or "an embodiment" referred to herein refers to a specific feature, structure or characteristic that may be included in at least one implementation of the present invention. "In one embodiment" appearing in different places in this specification does not all refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

The present invention is described in detail in conjunction with schematic diagrams. When describing the embodiments of the present invention in detail, for the convenience of explanation, the cross-sectional view showing the device structure will not be partially enlarged according to the general scale, and the schematic diagram is only an example, which should not limit the present invention. scope of protection. In addition, the three-dimensional dimensions of length, width and depth should be included in actual production.

At the same time, in the description of the present invention, it should be noted that the orientation or positional relationship indicated by "upper, lower, inner and outer" in the terms is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention. The invention and the simplified description do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and thus should not be construed as limiting the present invention. In addition, the terms "first, second or third" are used for descriptive purposes only, and should not be construed as indicating or implying relative importance.

Unless otherwise specified and limited in the present invention, the term "installation, connection, connection" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integrated connection; it can also be a mechanical connection, an electrical connection or a direct connection. A connection can also be an indirect connection through an intermediary, or it can be an internal communication between two elements. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention in specific situations.

Example 1

Referring to FIG. 1 , it is the first embodiment of the present invention, which provides a method for processing images collected by an AR device. Including: screening out the key images of the images collected by the AR device and compressing and transmitting them; processing the compressed video images through the improved Laplacian image sharpening technology, and classifying, learning and storing them; The processed pictures are used in the application of the dispatching system, and the artificial intelligence algorithm is used to propose auxiliary decision-making suggestions for control and operation for the dispatcher to refer to and choose.

Example 2

With reference to Fig. 2～Fig. 3, it is the second embodiment of the present invention, this embodiment provides a kind of intelligent agent visual scheduling method based on augmented reality image processing, including:

S1: Collect the scheduling instructions of the dispatcher and the operation instructions of the operation ticket, textualize the dispatching instructions through speech recognition technology, and standardize and decompose the text instructions.

The basic principles of speech recognition technology are as follows:

(1) Training (Training): Analyze the speech feature parameters in advance, make a speech template, and store it in the speech parameter library.

(2) Recognition: The speech to be recognized undergoes the same analysis as in training to obtain the speech parameters; compare it with the reference templates in the library one by one, and use the judgment method to find the template closest to the speech features, and obtain out the recognition result.

(3) Distortion Measures: There must be a standard for comparison, which is the "distortion measure" between the speech feature parameter vectors.

(4) Main recognition framework: Dynamic Time Warping (DTW) based on pattern matching and Hidden Markov Model (HMM) based on statistical model.

Further, the text instruction is standardized and decomposed by the following formula:

Decomposition result = action + voltage level + device name.

S2: On-site operators find the corresponding operating equipment according to the decomposed scheduling instructions, and scan the QR code on the operating equipment through the AR device, and then read the information of the operating equipment, and then automatically compare the information.

According to the name of the equipment to be operated on the operation ticket, the switch name is automatically compared with the text on the equipment information. If the content is the same, the automatic comparison is correct, otherwise the automatic comparison fails.

If the information is automatically compared correctly, the on-site operators will take security measures before the operation of the operating equipment, and use the AR device to take pictures of the security measures and upload them, and automatically judge whether the on-site security measures are standardized through AI (Artificial Intelligence) image recognition. Preferably, if the on-site environment is complex, a positioning label can be pasted on the device in the early stage to ensure the speed and accuracy of AI image recognition; if the information comparison is incorrect, the AR device will correspond to the display device incorrectly. The operator should re-check whether the information on the operation ticket is consistent with the actual information on site, and then scan the QR code again until the information corresponds to each other before the system proceeds.

Specifically, the steps of AI image recognition are as follows:

(1) Preprocess images, install Tensorflow and pillow libraries;

Mark the image, normalize it, and then divide it into a training set and a test set. The ratio of the training set to the test set is 3:1; it should be noted that TensorFlow is a data flow graph (data flow graphs) , an open source software library for numerical calculations; the Pillow library is a Python image library that supports a large number of image formats and is the best choice for image processing and batch processing. It can be used to create thumbnails, convert between file formats, and print images , size conversion, color conversion and other operations.

The program code for installing Tensorflow and pillow libraries is as follows:

import os#Image reading library

from PIL import Image#Matrix operation library

import numpy as np

import tensorflow as tf

(2) Define the convolutional network model, loss function and optimizer;

#Define the convolution layer, 20 convolution kernels, the convolution kernel size is 5, and activate with Relu

conv0 = tf.layers.conv2d(datas_placeholder, 20, 5, activation = tf.nn.relu)

#Define the max-pooling layer, the pooling window is 2x2, and the step size is 2x2

pool0=tf.layers.max_pooling2d(conv0,[2,2],[2,2])

#Define the convolution layer, 40 convolution kernels, the convolution kernel size is 4, and activate with Relu

conv1=tf.layers.conv2d(pool0,40,4,activation=tf.nn.relu)

pool1=tf.layers.max_pooling2d(conv1,[2,2],[2,2])

#Use cross entropy to define loss

losses = tf.nn.softmax_cross_entropy_with_logits(

labels=tf.one_hot(labels_placeholder,num_classes),

logits=logits

)

# average loss

mean_loss = tf.reduce_mean(losses)

#Define the optimizer, specify the loss function to be optimized

optimizer = tf.train.AdamOptimizer(learning_rate=1e-2).minimize(losses)

(3) Perform image recognition training to obtain convolutional network model parameters;

Training needs to use sess.run(tf.global_variables_initializer()) to initialize parameters. After training, saver.save(sess,model_path) needs to be used to save model parameters.

The test needs to use saver.restore(sess, model_path) to read the parameters.

(4) Use the convolutional network model for image recognition.

If the on-site security measures are standardized, the operation instructions will be graphically displayed through the AR device, and the operation equipment will be operated according to the illustrated operation instructions. After the operation is completed, the operation equipment will be photographed and uploaded; otherwise, the security measures will be rearranged and examine.

It should be noted that security measures include setting up isolation fences and hanging warning signs.

S3: Use the AR device to take photos of the operating equipment and upload them, and then automatically judge whether the operating equipment status meets the operating instructions through AI image recognition.

On the premise that the system determines that the security measures are in place, the system will sequentially send the illustrated operation instructions to the AR device according to the operation ticket instructions. Use the AR device to take photos and upload them again, and then use the AI image recognition function to automatically judge whether the operation device status meets the operation instructions. Only when the operation is consistent with the standard operation picture, the system will issue the next operation ticket instruction to the AR device. When emergency operations are required due to unexpected circumstances at the operation site, the AR device can use the high-speed and low-latency characteristics of the 5G network to transmit the on-site images to the dispatcher in time, and the temporary operation plan can be communicated immediately after the professional judges to the operator.

It should be noted that the system is a system in which text instructions and pictures correspond to each other; the steps to build the system are: ① take pictures of electrical equipment of different models, categories, and functions in sequence according to different switch states; ② combine the pictures of the equipment with the text One to one correspondence.

When the system issues operation instructions, it needs to broadcast the operation content by voice and mark the part of the equipment that needs to be operated through the AR device; after the operation is completed, it needs to use the AR device to collect the password of the on-site operator to complete the operation, and give feedback information at the same time.

Specifically, if the state of the operating device satisfies the operating instruction, it is judged whether all the operating instructions are completed; otherwise, the operating device is re-operated.

If all the operation instructions are completed, the artificial intelligence dispatching system will send the completion instruction to the AR device, and inform the on-site operator that the operation of the dispatch instruction equipment has been completed.

Preferably, the AR device has the capability of a remote expert collaboration system, and can conduct multi-person calls when the network communication is good. Operators can use AR equipment to timely put forward collaboration requirements to remote experts, combined with 5G technology, to achieve real-time technical support, reduce downtime losses, and reduce travel expenses.

Example 3

Referring to Fig. 4, it is the third embodiment of the present invention. In order to verify and illustrate the technical effect of the present invention, the present embodiment adopts the traditional technical scheme and the method of the present invention to carry out comparative tests, and compares the test results by means of scientific demonstration to verify the method the real effect it has.

In order to ensure that the experiment can be implemented, it is necessary to build a test platform for experimental comparison, wherein the test environment is to select C++ engine and java database for testing, and randomly select 100 pictures in the distribution network image to carry out image processing test, in order to verify the benefits of the present invention Effect, image processing of distribution network image compression, image processing of traditional Laplacian image sharpening technology and image processing of Laplacian image sharpening technology improved by this method, three methods of processing After comparing the images, refer to Figure 4 for the results.

Figure 4 is the comparison of distribution network images after three processing methods, where (a) is the compressed color image, and (b) is the image processing effect of the unimproved Laplacian operator image sharpening technology , (c) is the effect diagram of the improved Laplacian image sharpening technology image processing, it can be clearly seen that the image processing using the method of the present invention has a higher image definition, and only the compressed image is clear The degree is the lowest, and this method has a better effect of image processing due to the elimination of noise interference.

Example 4

In order to verify and explain the technical effect adopted in this method, this embodiment chooses the traditional technical scheme and adopts this method to conduct a comparative test, and compares the test results by means of scientific demonstration to verify the real effect of this method.

The traditional technical solution uses the traditional purely manual operation, that is, the dispatcher communicates the instructions one by one to the on-site personnel through the phone, and the operator needs to repeat the instruction; after the operation is completed, the operator and the dispatcher also need to complete the instruction. Repeat it twice again. This technology is time-consuming and labor-intensive. It not only requires dispatchers and operators to be extremely familiar with the site and equipment, but also is affected by external factors such as environmental noise and communication interference.

In order to verify that this method has faster operation speed, stronger anti-interference and lower information mistransmission than the traditional technical solution, in this embodiment, the traditional technical solution and this method will be used to execute the operation ticket respectively. The situation is compared in real time.

Among the 2138 face checks performed by a company in the year, 58 irregular tickets were found, accounting for 2.713%; 38 irregular operations were found during the follow-up inspection of the operation process using traditional technical solutions, a total of 17 times, accounting for 2.713%. 0.795%; among the 58 irregular votes, there were 10 inaccurate operational tasks, accounting for 17.241%, irregular application of operational terms, 28 missing inspection items, accounting for 48.276% of the irregular votes, and missing signatures , Missing "√" in total 6 pieces, accounting for 10.345%, and 4 pieces with inaccurate time registration, accounting for 6.897%.

In the non-scheduled follow-up spot checks on the behavior of operators during the execution of operation tickets using traditional technical solutions, 17 of them found that the operation behavior of the switching operation process was not standardized, a total of 38 person-times, accounting for 0.795% of the total number of votes in the year; However, the number of people who use this method to execute the operation ticket and find that the operation behavior is irregular is 0, and the specific data is shown in the following table.

Table 1: A comparison table of the results of executing operation tickets using two different methods.

It can be seen from the above table that this method can completely avoid the problems of non-standard operation tickets such as non-standard operation terms, lack of signature and inaccurate time registration.

It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention without limitation, although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be carried out Modifications or equivalent replacements without departing from the spirit and scope of the technical solution of the present invention shall be covered by the claims of the present invention.

Claims

A method for visual scheduling of intelligent agents based on augmented reality image processing, characterized in that: comprising,

Collect the scheduling instructions of the dispatcher and the operation instructions of the operation ticket, textualize the dispatching instructions through image recognition technology, and standardize and decompose the text instructions;

On-site operators find the corresponding operating equipment according to the decomposed scheduling instructions, and scan the QR code on the operating equipment through the AR device, and then read the information of the operating equipment, and then automatically compare the information;

If the comparison of the information is correct, the on-site operator will take security measures before the operation of the operation equipment, take photos of the security measures and upload them, and automatically judge whether the on-site security measures are standardized through AI image recognition;

If the on-site security measures are standardized, the AR device will graphically display the operating instructions, and the operator will operate the operating equipment according to the graphically displayed operating instructions. After the operation is completed, the operator will The device takes pictures and uploads them; otherwise, rearranges the security measures and checks;

The operating device is photographed and uploaded, the key image of the image collected by the AR device is screened out and compressed and transmitted by DCT (discrete cosine transform technology); decompressed in the terminal server; The sharpening technology processes the compressed video images, classifies them, learns them, and stores them; uses the processed images for the application of the scheduling system, and proposes auxiliary decision-making suggestions for control and operation through artificial intelligence algorithms for the dispatcher Refer to the selection; then automatically judge whether the state of the operating device meets the operating instruction through the AI image recognition; if so, judge whether all the operating instructions are completed; otherwise, re-operate the operating device;

If all the operation instructions are completed, the system sends a completion instruction to the AR device to inform the on-site operator that the device operation of the dispatch instruction has been completed.
The method for visual scheduling of smart agents based on augmented reality image processing according to claim 1, wherein: the standardized decomposition of the text instructions includes,

Decomposition result = action + voltage level + device name.
The method for visual scheduling of intelligent agents based on augmented reality image processing according to claim 1, wherein the AR device includes,

AR glasses, smart image acquisition devices, 5G mobile phones, etc.
The method for visual scheduling of smart agents based on augmented reality image processing according to claim 1 or 2, wherein the information comparison further includes,

If the comparison of the information is incorrect, check the correspondence between the operating device and the scheduling instruction.
The method for visual dispatching of intelligent agents based on augmented reality image processing according to claim 4, wherein the security measures include setting isolation fences and hanging warning signs.
The method for visual scheduling of smart agents based on augmented reality image processing according to claim 1 or 5, wherein the AI image recognition includes,

Preprocess images, install Tensorflow and Pillow libraries;

Define the convolutional network model, loss function and optimizer;

Perform image recognition training to obtain convolutional network model parameters;

The image recognition is performed using the convolutional network model.
The method for visual scheduling of intelligent agents based on augmented reality image processing according to claim 1, wherein: taking pictures and uploading said security measures includes,

Take pictures of the security measures through the AR device and upload them.
The method for visual scheduling of smart agents based on augmented reality image processing according to claim 1, wherein the images required for collecting power scheduling include:

All images collected by the AR device are directly stored in the local server without processing, and are automatically overwritten every fifteen days.
The method for visual scheduling of smart agents based on augmented reality image processing according to claim 1 or 8, wherein the filtering out and compressing the key images collected by the AR device includes,

The screening out of the key images collected by the AR device is to perform similarity filtering processing on the images being collected by the on-site AR device in real time, and perform DCT image compression processing and transmission after filtering out duplicate images.
The method for visual scheduling of intelligent agents based on augmented reality image processing according to claim 1, characterized in that: said transmission includes,

The transmission method is 4G, 5G, SMS, etc., which is suitable for short-distance, medium-distance and long-distance transmission.
The method for visual scheduling of smart agents based on augmented reality image processing according to claim 1, wherein the improved Laplacian image sharpening technology includes,

The compressed key image is decompressed by the scheduling terminal, and the key image is denoised by using the threshold denoising technology, and then the Laplacian image is enhanced. The selection of the threshold varies with different application scenarios and different Bright and different.
The method for visual dispatching of smart agents based on augmented reality image processing according to claim 1, wherein: classifying and storing the screened images includes,

Decompress the compressed images collected and transmitted by the AR device, use the improved Laplacian image sharpening technology to perform image processing and analysis, and classify and store the processed images in the intelligent agent terminal server middle.
The method for visual scheduling of intelligent agents based on augmented reality image processing according to claim 1, characterized in that: the proposed artificial intelligence algorithm to provide auxiliary decision-making suggestions for regulating operation includes,

The artificial intelligence algorithm extracts the compressed and processed pictures, and can further perform power flow calculation and regional load compensation through equipment identification, character recognition, and fault diagnosis to maintain system voltage stability and complete the work of deep reinforcement learning for power dispatching systems. The artificial intelligence algorithm proposes auxiliary decision-making suggestions for regulation and operation for the dispatcher to refer to and choose.
The method for visual scheduling of intelligent agents based on augmented reality image processing according to claim 1, wherein: said operation completion command transmission includes,

When sending the operation instruction to the AR device, it is necessary to broadcast the operation content by voice and mark the part of the device that needs to be operated through the AR device; after the operation is completed, it is necessary to use the AR device to collect the completion of the on-site operator. Operation password, and give feedback information at the same time.
The method for visual scheduling of smart agents based on augmented reality image processing according to claim 10 or 11, characterized in that: the system includes,

The system is a system in which text instructions and pictures correspond to each other;

The steps for constructing the system are as follows: firstly, take pictures of electrical equipment of different models, categories, and functions in sequence according to different switch states;