CN111949270A - Process robot's operating environment change perception method and device - Google Patents
Process robot's operating environment change perception method and device Download PDFInfo
- Publication number
- CN111949270A CN111949270A CN202010729743.8A CN202010729743A CN111949270A CN 111949270 A CN111949270 A CN 111949270A CN 202010729743 A CN202010729743 A CN 202010729743A CN 111949270 A CN111949270 A CN 111949270A
- Authority
- CN
- China
- Prior art keywords
- screenshot
- corner
- corner point
- process robot
- operating environment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 157
- 230000008569 process Effects 0.000 title claims abstract description 123
- 230000008859 change Effects 0.000 title claims abstract description 34
- 230000008447 perception Effects 0.000 title claims description 9
- 238000004590 computer program Methods 0.000 claims description 17
- 238000005316 response function Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 2
- 230000003252 repetitive effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
- G06F8/43—Checking; Contextual analysis
- G06F8/433—Dependency analysis; Data or control flow analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/008—Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
- G06N5/025—Extracting rules from data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Robotics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Numerical Control (AREA)
- Manipulator (AREA)
Abstract
本发明提供了一种流程机器人的运行环境变化感知方法及装置，该方法包括：获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；分别提取第一截图的多个角点和第二截图的多个角点；根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，每个角点对包括第一截图的一个角点和第二截图的一个角点；根据多组角点对，判断流程机器人的运行环境变化。本发明可以随时获得流程机器人的运行环境变化，提高流程机器人的运行效率。
The present invention provides a method and device for sensing changes in the operating environment of a process robot. The method includes: acquiring a first screenshot of the operating environment when the process robot was started last time, and a second screenshot of the operating environment when the process robot was started this time; respectively; Extract multiple corner points of the first screenshot and multiple corner points of the second screenshot; generate multiple sets of corner point pairs according to the multiple corner points of the first screenshot and the multiple corner points of the second screenshot, and each corner point pair It includes a corner point of the first screenshot and a corner point of the second screenshot; according to multiple sets of corner point pairs, it is judged that the operating environment of the process robot changes. The invention can obtain the operation environment change of the process robot at any time, and improve the operation efficiency of the process robot.
Description
技术领域technical field
本发明涉及计算机技术领域，尤其涉及一种流程机器人的运行环境变化感知方法及装置。The invention relates to the field of computer technology, and in particular, to a method and device for sensing changes in the operating environment of a process robot.
背景技术Background technique
流程机器人为软件机器人，即根据某些事先设定好的规则从而可以自动化执行的软件工具。Process robots are software robots, that is, software tools that can be executed automatically according to certain pre-set rules.
在现代企业中，出于运营管理及安全考虑等需求，存在如报销人员信息处理，报表生成等操作。从企业的角度出发，这些工作存在较高的重复性，使用流程机器人来代替人工操作将减少人工操作，解放人力，缩短成本。In modern enterprises, there are operations such as information processing of reimbursement personnel and report generation due to operational management and security considerations. From the perspective of enterprises, these tasks are highly repetitive, and the use of process robots to replace manual operations will reduce manual operations, liberate manpower, and reduce costs.
流程机器人代替人工的重复性工作虽然会大大的提升工作效率，但是流程机器人是基于规则生成的代码，并不会像人一样自动识别界面是否发生变化，只是一味的按照既定的规则或流程去相应的界面获取元素，进行类似于登录，点击，下载，查询或者填写的一系列操作。完成这些操作的必要条件之一就是流程机器人需要在整个界面中根据这些元素的属性或者坐标位置准确快速的找到这些元素。Although process robots replace manual repetitive work, it will greatly improve work efficiency, but process robots are code generated based on rules, and will not automatically recognize whether the interface has changed like a human, but just blindly follow the established rules or processes to respond. The interface obtains elements and performs a series of operations similar to login, click, download, query or fill-in. One of the necessary conditions for completing these operations is that the process robot needs to find these elements accurately and quickly according to the attributes or coordinate positions of these elements in the entire interface.
随着电子化程度的逐步提高，软件或者网站等更新的速度也逐渐加快，然而，为了美工，软件或者网站更新之后界面往往会发生较大的变化，某些关键性的元素的属性也可能发生变化，这就可能导致流程中某些元素获取不到，从而流程失败，这种失败并不是技术问题或者业务问题，属于环境问题，但无论如何，一旦流程失败就会大大的影响该流程的成功率，且技术人员在进行错误分析时，并不会第一时间想到是元素找不到，需要进行一步一步的排查才会判断是环境发生了变化导致流程失败。With the gradual improvement of the degree of electronization, the update speed of software or website is also gradually accelerated. However, for the sake of art, the interface often changes greatly after the software or website is updated, and the attributes of some key elements may also change. Changes, which may cause some elements in the process to be unavailable, and thus the process fails. This failure is not a technical problem or a business problem, but an environmental problem, but in any case, once the process fails, it will greatly affect the success of the process. In addition, when performing error analysis, technicians will not immediately think that the element cannot be found. It needs to be checked step by step to determine that the process has failed due to changes in the environment.
为了解决上述问题，目前都是业务人员定期会登录软件或者网站看其是否有更新，如果有更新的话就去报告给技术人员，由技术人员判断元素是否发生变化，如果元素发生变化，则修改代码，反之则正常运行流程。但这种方法不仅会大大的降低流程机器人的运行效率，且会极大的影响业务人员的使用体验。In order to solve the above problems, at present, business personnel regularly log in to the software or website to see if there is an update. If there is an update, report it to the technician, who will judge whether the element has changed. If the element has changed, modify the code. , otherwise the process runs normally. However, this method will not only greatly reduce the operation efficiency of process robots, but also greatly affect the experience of business personnel.
发明内容SUMMARY OF THE INVENTION
本发明实施例提出一种流程机器人的运行环境变化感知方法，用以随时获得流程机器人的运行环境变化，提高流程机器人的运行效率，该方法包括：An embodiment of the present invention proposes a method for sensing changes in the operating environment of a process robot, which is used to obtain changes in the operating environment of the process robot at any time and improve the operation efficiency of the process robot. The method includes:
获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；Obtain the first screenshot of the running environment when the process robot was started last time, and the second screenshot of the running environment when the process robot was started this time;
分别提取第一截图的多个角点和第二截图的多个角点；Extracting multiple corners of the first screenshot and multiple corners of the second screenshot respectively;
根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，每个角点对包括第一截图的一个角点和第二截图的一个角点；According to the multiple corner points of the first screenshot and the multiple corner points of the second screenshot, multiple sets of corner point pairs are generated, and each corner point pair includes a corner point of the first screenshot and a corner point of the second screenshot;
根据多组角点对，判断流程机器人的运行环境变化。According to multiple sets of corner point pairs, it is judged that the operating environment of the process robot changes.
本发明实施例提出一种流程机器人的运行环境变化感知装置，用以随时获得流程机器人的运行环境变化，提高流程机器人的运行效率，该装置包括：The embodiment of the present invention proposes an operating environment change sensing device of a process robot, which is used to obtain the operating environment change of the process robot at any time and improve the operation efficiency of the process robot. The device includes:
截图获取模块，用于获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；The screenshot obtaining module is used to obtain the first screenshot of the running environment when the process robot was started last time, and the second screenshot of the running environment when the process robot was started this time;
角点提取模块，用于分别提取第一截图的多个角点和第二截图的多个角点；a corner extraction module for extracting a plurality of corners of the first screenshot and a plurality of corners of the second screenshot respectively;
角点对生成模块，用于根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，每个角点对包括第一截图的一个角点和第二截图的一个角点；The corner point pair generation module is used to generate multiple sets of corner point pairs according to the plurality of corner points of the first screenshot and the plurality of corner points of the second screenshot, and each corner point pair includes a corner point of the first screenshot and a second corner point of the second screenshot. a corner of the screenshot;
判断模块，用于根据多组角点对，判断流程机器人的运行环境变化。The judgment module is used for judging changes in the operating environment of the process robot according to multiple sets of corner point pairs.
本发明实施例还提出了一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述流程机器人的运行环境变化感知方法。An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor implements the operating environment of the process robot when the processor executes the computer program Change perception method.
本发明实施例还提出了一种计算机可读存储介质，所述计算机可读存储介质存储有执行上述流程机器人的运行环境变化感知方法的计算机程序。An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for executing the above-mentioned method for sensing a change in an operating environment of a process robot.
在本发明实施例中，获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；分别提取第一截图的多个角点和第二截图的多个角点；根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，每个角点对包括第一截图的一个角点和第二截图的一个角点；根据多组角点对，判断流程机器人的运行环境变化。在上述过程中，根据上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图，自动判断流程机器人的运行环境变化，而无需技术人员自己判断，从而随时获得流程机器人的运行环境变化，大大提高了流程机器人的运行效率。In the embodiment of the present invention, a first screenshot of the running environment when the process robot was started last time, and a second screenshot of the running environment when the process robot was started this time are obtained; multiple corners of the first screenshot and the second screenshot are extracted respectively. Multiple corner points; according to multiple corner points of the first screenshot and multiple corner points of the second screenshot, multiple sets of corner point pairs are generated, and each corner point pair includes one corner point of the first screenshot and one corner point of the second screenshot Corner point; according to multiple sets of corner point pairs, determine the change of the operating environment of the process robot. In the above process, according to the first screenshot of the running environment when the process robot was started last time, and the second screenshot of the running environment when the process robot was started this time, the change of the running environment of the process robot is automatically judged without the need for technicians to judge by themselves, thereby The change of the operating environment of the process robot can be obtained at any time, which greatly improves the operation efficiency of the process robot.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts. In the attached image:
图1为本发明实施例中电子雷管的位数据读取方法的流程图；Fig. 1 is the flow chart of the bit data reading method of electronic detonator in the embodiment of the present invention;
图2为本发明实施例流程机器人的运行环境变化感知方法的详细流程图；FIG. 2 is a detailed flowchart of a method for sensing changes in an operating environment of a process robot according to an embodiment of the present invention;
图3为本发明实施例中流程机器人的运行环境变化感知装置的示意图；3 is a schematic diagram of an operating environment change sensing device of a process robot in an embodiment of the present invention;
图4为本发明实施例中计算机设备的示意图。FIG. 4 is a schematic diagram of a computer device in an embodiment of the present invention.
具体实施方式Detailed ways
为使本发明实施例的目的、技术方案和优点更加清楚明白，下面结合附图对本发明实施例做进一步详细说明。在此，本发明的示意性实施例及其说明用于解释本发明，但并不作为对本发明的限定。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention more clearly understood, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but not to limit the present invention.
在本说明书的描述中，所使用的“包含”、“包括”、“具有”、“含有”等，均为开放性的用语，即意指包含但不限于。参考术语“一个实施例”、“一个具体实施例”、“一些实施例”、“例如”等的描述意指结合该实施例或示例描述的具体特征、结构或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。各实施例中涉及的步骤顺序用于示意性说明本申请的实施，其中的步骤顺序不作限定，可根据需要作适当调整。In the description of this specification, the use of "comprising", "including", "having", "containing" and the like are all open-ended terms, that is, meaning including but not limited to. Description with reference to the terms "one embodiment", "one particular embodiment", "some embodiments", "for example" etc. means that a particular feature, structure or characteristic described in connection with the embodiment or example is included in at least one of the present application examples or examples. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The sequence of steps involved in each embodiment is used to schematically illustrate the implementation of the present application, and the sequence of steps is not limited and can be appropriately adjusted as required.
图1为本发明实施例中电子雷管的位数据读取方法的流程图，如图1所示，该方法包括：1 is a flowchart of a method for reading bit data of an electronic detonator in an embodiment of the present invention. As shown in FIG. 1 , the method includes:
步骤101，获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；
步骤102，分别提取第一截图的多个角点和第二截图的多个角点；
步骤103，根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，每个角点对包括第一截图的一个角点和第二截图的一个角点；
步骤104，根据多组角点对，判断流程机器人的运行环境变化。
在本发明实施例中，根据上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图，自动判断流程机器人的运行环境变化，而无需技术人员自己判断，从而随时获得流程机器人的运行环境变化，大大提高了流程机器人的运行效率。In the embodiment of the present invention, according to the first screenshot of the operating environment when the process robot was started last time, and the second screenshot of the operating environment when the process robot was started this time, the change of the operating environment of the process robot is automatically determined without the need for technicians to judge by themselves , so as to obtain the changes of the operation environment of the process robot at any time, which greatly improves the operation efficiency of the process robot.
具体实施时，在流程机器人每次启动流程时，都会获取本次启动流程机器人时运行环境的的截图，即第二截图，而上次启动流程机器人时运行环境的截图为第一截图。During specific implementation, each time the process robot starts a process, a screenshot of the running environment when the process robot is started this time, that is, the second screenshot, is obtained, and the screenshot of the running environment when the process robot is started last time is the first screenshot.
之后，进入步骤102，分别提取第一截图的多个角点和第二截图的多个角点，在一实施例中，分别提取第一截图的多个角点和第二截图的多个角点，包括：After that, go to step 102, extract multiple corners of the first screenshot and multiple corners of the second screenshot, respectively, in one embodiment, extract multiple corners of the first screenshot and multiple corners of the second screenshot respectively points, including:
分别计算第一截图和第二截图中每个像素点的Harris角点响应函数值；Calculate the Harris corner response function value of each pixel in the first screenshot and the second screenshot respectively;
对于每个像素点，若该像素点的Harris角点响应函数值大于第一阈值，确定该像素点为角点。For each pixel, if the Harris corner response function value of the pixel is greater than the first threshold, the pixel is determined to be a corner.
其中，Harris角点响应函数是1988年Harris提出的，如果考虑窗口内不同点贡献权重的差异，角点响应函数可以写为：Among them, the Harris corner response function was proposed by Harris in 1988. If the difference in the contribution weights of different points in the window is considered, the corner response function can be written as:
其中，
CRF(u,v)＝(A·B-C2)-k(A+B)2 CRF(u,v)=(A·BC 2 )-k(A+B) 2
其中，CRF(u,v)为像素点(u,v)的Harris角点响应函数值；Among them, CRF(u, v) is the Harris corner response function value of the pixel point (u, v);
Ix和Iy分别为像素点(u,v)在水平和垂直方向的一阶导数，
k为常数。k is a constant.
在上述实施例中，k一般取0.04～0.06。In the above embodiment, k generally takes 0.04 to 0.06.
需要注意的是，第一截图获得的角点数与第二截图获得的角点数可能不同。It should be noted that the number of corner points obtained in the first screenshot may be different from the number of corner points obtained in the second screenshot.
在一实施例中，根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，包括：In one embodiment, multiple sets of corner point pairs are generated according to multiple corner points of the first screenshot and multiple corner points of the second screenshot, including:
对于第一截图的每个角点，分别计算第一截图的该角点与第二截图的每个角点的归一化相关系数，从多个归一化相关系数中确定最大归一化相关系数对应的第二截图的角点与第一截图的该角点组成角点对。For each corner of the first screenshot, calculate the normalized correlation coefficient between the corner of the first screenshot and each corner of the second screenshot, and determine the maximum normalized correlation from the multiple normalized correlation coefficients The corner point of the second screenshot corresponding to the coefficient forms a corner point pair with the corner point of the first screenshot.
在上述实施例中，通过计算第一截图的该角点与第二截图的每个角点的归一化相关系数来组成角点对，来确定角点匹配的程序的方法称为归一化互相关(NormalizedCross Correlation method,NCC)匹配算法，在一实施例例中，采用如下公式，对于第一截图的每个角点，分别计算第一截图的该角点与第二截图的每个角点的归一化相关系数：In the above-mentioned embodiment, the method of determining the program of the corner point matching is called normalization by calculating the normalized correlation coefficient between the corner point of the first screenshot and each corner point of the second screenshot to form a corner point pair. The cross-correlation (Normalized Cross Correlation method, NCC) matching algorithm, in one embodiment, adopts the following formula, for each corner point of the first screenshot, respectively calculates the corner point of the first screenshot and each corner of the second screenshot Normalized correlation coefficient for points:
其中，(a,b)分别为水平方向和垂直方向的偏移量；Among them, (a, b) are the offsets in the horizontal and vertical directions, respectively;
R(a,b)为归一化相关系数；R(a,b) is the normalized correlation coefficient;
xi+a,j+b为第一截图的(i+a,j+b)处的角点的像素值，yij为第二截图的(i,j)处的角点的像素值；x i+a, j+b are the pixel values of the corners at (i+a, j+b) of the first screenshot, and y ij are the pixel values of the corners at (i, j) of the second screenshot;
N1和N2为常量。 N1 and N2 are constants.
在上述实施例中，R(a,b)值越大，证明两个角点的相似性越高，因此，在第一截图的某个角点计算的多个归一化相关系数中，最大归一化相关系数对应的第二截图的角点与第一截图的该角点组成角点对。In the above embodiment, the larger the value of R(a,b), the higher the similarity between the two corner points is proved. Therefore, among the multiple normalized correlation coefficients calculated at a certain corner point of the first screenshot, the largest The corner point of the second screenshot corresponding to the normalized correlation coefficient and the corner point of the first screenshot form a corner point pair.
另外，为了提高本发明感知运行环境变化的准确性，需要删除误匹配的角点对，在一实施例中，在生成多组角点对之后，还包括：In addition, in order to improve the accuracy of perceiving changes in the operating environment of the present invention, it is necessary to delete incorrectly matched corner point pairs. In one embodiment, after generating multiple sets of corner point pairs, the method further includes:
对每组角点对，若该组角点对的两个角点的归一化相关系数小于第二阈值，剔除该组角点对。For each group of corner point pairs, if the normalized correlation coefficient of the two corner points of the group of corner point pairs is less than the second threshold, the group of corner point pairs is eliminated.
在上述实施例中，确定了归一化相关系数小于第二阈值的角点对为误匹配的角点对。In the above-mentioned embodiment, it is determined that the corner point pair whose normalized correlation coefficient is smaller than the second threshold is a wrongly matched corner point pair.
在一实施例中，根据多组角点对，判断流程机器人的运行环境变化，包括：In one embodiment, judging changes in the operating environment of the process robot according to multiple sets of corner point pairs, including:
计算每组角点对的像素差值；Calculate the pixel difference of each set of corner pairs;
若任意一组角点对的像素差值小于第三阈值，确定流程机器人的运行环境发生变化。If the pixel difference of any set of corner point pairs is less than the third threshold, it is determined that the operating environment of the process robot has changed.
在上述实施例中，可采用如下公式计算每组角点对的像素差值：In the above embodiment, the following formula can be used to calculate the pixel difference value of each group of corner point pairs:
change＝abs(imt1-imt2)change=abs(imt1-imt2)
其中，change为每组角点对的像素差值，imt1为每组角点对中第一截图中的角点的像素值，imt2为每组角点对中第二截图中的角点的像素值。Among them, change is the pixel difference value of each group of corner point pairs, imt1 is the pixel value of the corner point in the first screenshot of each group of corner point pairs, and imt2 is the pixel value of the corner point in the second screenshot of each group of corner point pairs value.
若存在5个角点对，则可得到五个像素差值，若这其中任意一组角点对的像素差值小于第三阈值，确定流程机器人的运行环境发生变化。之后，可将小于第三阈值的角点对形成的差异图像发送至技术人员，用于指导技术人员修改流程机器人的代码。若流程机器人的运行环境没有发生变化，则流程机器人继续执行流程，如果流程报错，则错误并不属于环境错误。这既减少了由环境变化而给业务人员或技术人员带来的不必要的工作量，也缩小了错误排查的范围，缓解了业务人员需要定期查看软件或者网站是否有更新的压力，也能够大大降低由环境问题导致的流程失败的次数，更加能够缩减技术人员对错误流程的排查范围，释放了人力。If there are 5 corner point pairs, five pixel difference values can be obtained. If the pixel difference value of any one group of corner point pairs is less than the third threshold, it is determined that the operating environment of the process robot has changed. Afterwards, the difference image formed by the pair of corner points smaller than the third threshold can be sent to the technician, so as to guide the technician to modify the code of the process robot. If the operating environment of the process robot does not change, the process robot continues to execute the process. If the process reports an error, the error is not an environmental error. This not only reduces the unnecessary workload for business personnel or technical personnel caused by changes in the environment, but also reduces the scope of error investigation, relieves the pressure of business personnel to regularly check whether the software or website is updated, and can greatly Reducing the number of process failures caused by environmental problems can further reduce the scope of troubleshooting for faulty processes by technicians, freeing up manpower.
基于上述实施例，本发明提出如下一个实施例来说明流程机器人的运行环境变化感知方法的详细流程，图2为本发明实施例流程机器人的运行环境变化感知方法的详细流程图，如图2所示，包括：Based on the above embodiment, the present invention proposes the following embodiment to illustrate the detailed process of the method for sensing changes in the operating environment of a process robot. FIG. 2 is a detailed flowchart of the method for sensing changes in the operating environment of a process robot according to an embodiment of the present invention. display, including:
步骤201，获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；
步骤202，分别计算第一截图和第二截图中每个像素点的Harris角点响应函数值；
步骤203，对于每个像素点，若该像素点的Harris角点响应函数值大于第一阈值，确定该像素点为角点；
步骤204，对于第一截图的每个角点，分别计算第一截图的该角点与第二截图的每个角点的归一化相关系数，从多个归一化相关系数中确定最大归一化相关系数对应的第二截图的角点与第一截图的该角点组成角点对；
步骤205，对每组角点对，若该组角点对的两个角点的归一化相关系数小于第二阈值，剔除该组角点对；
步骤206，计算每组角点对的像素差值；
步骤207，若任意一组角点对的像素差值小于第三阈值，确定流程机器人的运行环境发生变化，通知技术人员；否则，继续执行流程。
当然，可以理解的是，上述详细流程还可以有其他变化例，相关变化例均应落入本发明的保护范围。Of course, it can be understood that the above detailed process may also have other variations, and relevant variations should all fall within the protection scope of the present invention.
综上所述，在本发明实施例提出的方法中，获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；分别提取第一截图的多个角点和第二截图的多个角点；根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，每个角点对包括第一截图的一个角点和第二截图的一个角点；根据多组角点对，判断流程机器人的运行环境变化。在上述过程中，根据上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图，自动判断流程机器人的运行环境变化，而无需技术人员自己判断，从而随时获得流程机器人的运行环境变化，大大提高了流程机器人的运行效率。To sum up, in the method proposed in the embodiment of the present invention, a first screenshot of the operating environment when the process robot was started last time, and a second screenshot of the operating environment when the process robot was started this time is obtained; a plurality of corner points and a plurality of corner points of the second screenshot; according to the plurality of corner points of the first screenshot and the plurality of corner points of the second screenshot, multiple sets of corner point pairs are generated, and each corner point pair includes one corner point of the first screenshot A corner point and a corner point of the second screenshot; according to multiple sets of corner point pairs, determine the change of the operating environment of the process robot. In the above process, according to the first screenshot of the running environment when the process robot was started last time, and the second screenshot of the running environment when the process robot was started this time, the change of the running environment of the process robot is automatically judged without the need for technicians to judge by themselves, thereby The change of the operating environment of the process robot can be obtained at any time, which greatly improves the operation efficiency of the process robot.
本发明实施例还提出一种流程机器人的运行环境变化感知装置，其原理与流程机器人的运行环境变化感知方法类似，这里不再赘述。The embodiment of the present invention also provides an operating environment change sensing device for a process robot, the principle of which is similar to the operating environment change sensing method for a process robot, and will not be repeated here.
图3为本发明实施例中流程机器人的运行环境变化感知装置的示意图，包括：3 is a schematic diagram of an operating environment change sensing device of a process robot in an embodiment of the present invention, including:
截图获取模块301，用于获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；The
角点提取模块302，用于分别提取第一截图的多个角点和第二截图的多个角点；
角点对生成模块303，用于根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，每个角点对包括第一截图的一个角点和第二截图的一个角点；The corner point
判断模块304，用于根据多组角点对，判断流程机器人的运行环境变化。The judging
在一实施例中，角点提取模块302具体用于：In one embodiment, the
分别计算第一截图和第二截图中每个像素点的Harris角点响应函数值；Calculate the Harris corner response function value of each pixel in the first screenshot and the second screenshot respectively;
对于每个像素点，若该像素点的Harris角点响应函数值大于第一阈值，确定该像素点为角点。For each pixel, if the Harris corner response function value of the pixel is greater than the first threshold, the pixel is determined to be a corner.
在一实施例中，角点提取模块302具体用于：In one embodiment, the
采用如下公式，分别计算第一截图和第二截图中每个像素点的Harris角点响应函数值：Use the following formula to calculate the Harris corner response function value of each pixel in the first screenshot and the second screenshot respectively:
CRF(u,v)＝(A·B-C2)-k(A+B)2 CRF(u,v)=(A·BC 2 )-k(A+B) 2
其中，CRF(u,v)为像素点(u,v)的Harris角点响应函数值；Among them, CRF(u, v) is the Harris corner response function value of the pixel point (u, v);
Ix和Iy分别为像素点(u,v)在水平和垂直方向的一阶导数，
k为常数。k is a constant.
在一实施例中，角点对生成模块303具体用于：In one embodiment, the corner
对于第一截图的每个角点，分别计算第一截图的该角点与第二截图的每个角点的归一化相关系数，从多个归一化相关系数中确定最大归一化相关系数对应的第二截图的角点与第一截图的该角点组成角点对。For each corner of the first screenshot, calculate the normalized correlation coefficient between the corner of the first screenshot and each corner of the second screenshot, and determine the maximum normalized correlation from the multiple normalized correlation coefficients The corner point of the second screenshot corresponding to the coefficient forms a corner point pair with the corner point of the first screenshot.
在一实施例中，角点对生成模块303具体用于：In one embodiment, the corner
采用如下公式，对于第一截图的每个角点，分别计算第一截图的该角点与第二截图的每个角点的归一化相关系数：Using the following formula, for each corner of the first screenshot, calculate the normalized correlation coefficient between the corner of the first screenshot and each corner of the second screenshot:
其中，(a,b)分别为水平方向和垂直方向的偏移量；Among them, (a, b) are the offsets in the horizontal and vertical directions, respectively;
R(a,b)为归一化相关系数；R(a,b) is the normalized correlation coefficient;
xi+a,j+b为第一截图的(i+a,j+b)处的角点的像素值，yij为第二截图的(i,j)处的角点的像素值；x i+a, j+b are the pixel values of the corners at (i+a, j+b) of the first screenshot, and y ij are the pixel values of the corners at (i, j) of the second screenshot;
N1和N2为常量。 N1 and N2 are constants.
在一实施例中，角点对生成模块303还用于：In one embodiment, the corner
对每组角点对，若该组角点对的两个角点的归一化相关系数小于第二阈值，剔除该组角点对。For each group of corner point pairs, if the normalized correlation coefficient of the two corner points of the group of corner point pairs is less than the second threshold, the group of corner point pairs is eliminated.
在一实施例中，判断模块304具体用于：In one embodiment, the judging
计算每组角点对的像素差值；Calculate the pixel difference of each set of corner pairs;
若任意一组角点对的像素差值小于第三阈值，确定流程机器人的运行环境发生变化。If the pixel difference of any set of corner point pairs is less than the third threshold, it is determined that the operating environment of the process robot has changed.
综上所述，在本发明实施例提出的装置中，获取上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图；分别提取第一截图的多个角点和第二截图的多个角点；根据第一截图的多个角点和第二截图的多个角点，生成多组角点对，每个角点对包括第一截图的一个角点和第二截图的一个角点；根据多组角点对，判断流程机器人的运行环境变化。在上述过程中，根据上次启动流程机器人时运行环境的第一截图，以及本次启动流程机器人时运行环境的第二截图，自动判断流程机器人的运行环境变化，而无需技术人员自己判断，从而随时获得流程机器人的运行环境变化，大大提高了流程机器人的运行效率。To sum up, in the device proposed in the embodiment of the present invention, a first screenshot of the operating environment when the process robot was started last time, and a second screenshot of the operating environment when the process robot was started this time is obtained; a plurality of corner points and a plurality of corner points of the second screenshot; according to the plurality of corner points of the first screenshot and the plurality of corner points of the second screenshot, multiple sets of corner point pairs are generated, and each corner point pair includes one corner point of the first screenshot A corner point and a corner point of the second screenshot; according to multiple sets of corner point pairs, determine the change of the operating environment of the process robot. In the above process, according to the first screenshot of the running environment when the process robot was started last time, and the second screenshot of the running environment when the process robot was started this time, the change of the running environment of the process robot is automatically judged without the need for technicians to judge by themselves, thereby The change of the operating environment of the process robot can be obtained at any time, which greatly improves the operation efficiency of the process robot.
本申请的实施例还提供一种计算机设备，图4为本发明实施例中计算机设备的示意图，该计算机设备能够实现上述实施例中的流程机器人的运行环境变化感知方法中全部步骤，所述电子设备具体包括如下内容：The embodiment of the present application also provides a computer device. FIG. 4 is a schematic diagram of the computer device in the embodiment of the present invention. The equipment specifically includes the following:
处理器(processor)401、存储器(memory)402、通信接口(CommunicationsInterface)403和总线404；a processor (processor) 401, a memory (memory) 402, a communication interface (CommunicationsInterface) 403 and a
其中，所述处理器401、存储器402、通信接口403通过所述总线404完成相互间的通信；所述通信接口403用于实现服务器端设备、检测设备以及用户端设备等相关设备之间的信息传输；Wherein, the
所述处理器401用于调用所述存储器402中的计算机程序，所述处理器执行所述计算机程序时实现上述实施例中的流程机器人的运行环境变化感知方法中的全部步骤。The
本申请的实施例还提供一种计算机可读存储介质，能够实现上述实施例中的流程机器人的运行环境变化感知方法中全部步骤，所述计算机可读存储介质上存储有计算机程序，该计算机程序被处理器执行时实现上述实施例中的流程机器人的运行环境变化感知方法的全部步骤。Embodiments of the present application further provide a computer-readable storage medium capable of implementing all the steps in the method for sensing changes in the operating environment of a process robot in the above-mentioned embodiments, where a computer program is stored on the computer-readable storage medium, and the computer program When executed by the processor, all steps of the method for sensing changes in the operating environment of the process robot in the above embodiments are implemented.
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.
以上所述的具体实施例，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本发明的具体实施例而已，并不用于限定本发明的保护范围，凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above-mentioned specific embodiments are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010729743.8A CN111949270B (en) | 2020-07-27 | 2020-07-27 | Method and device for sensing changes in operating environment of process robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010729743.8A CN111949270B (en) | 2020-07-27 | 2020-07-27 | Method and device for sensing changes in operating environment of process robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111949270A true CN111949270A (en) | 2020-11-17 |
| CN111949270B CN111949270B (en) | 2025-01-14 |
Family
ID=73338151
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010729743.8A Active CN111949270B (en) | 2020-07-27 | 2020-07-27 | Method and device for sensing changes in operating environment of process robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111949270B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117437187A (en) * | 2023-10-13 | 2024-01-23 | 深圳市锐巽自动化设备有限公司 | Detonator detection method, detonator detection system and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017049994A1 (en) * | 2015-09-25 | 2017-03-30 | 深圳大学 | Hyperspectral image corner detection method and system |
| CN107341803A (en) * | 2017-07-25 | 2017-11-10 | 电子科技大学 | A kind of digital picture object minute movement monitoring algorithm based on Corner Detection |
| CN107590816A (en) * | 2017-09-08 | 2018-01-16 | 哈尔滨工业大学 | A kind of Water-Body Information approximating method based on remote sensing images |
| CN109919095A (en) * | 2019-03-07 | 2019-06-21 | 北京石油化工学院 | Monitoring method and electronic equipment for palletizing in hazardous chemicals warehouse |
-
2020
- 2020-07-27 CN CN202010729743.8A patent/CN111949270B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017049994A1 (en) * | 2015-09-25 | 2017-03-30 | 深圳大学 | Hyperspectral image corner detection method and system |
| CN107341803A (en) * | 2017-07-25 | 2017-11-10 | 电子科技大学 | A kind of digital picture object minute movement monitoring algorithm based on Corner Detection |
| CN107590816A (en) * | 2017-09-08 | 2018-01-16 | 哈尔滨工业大学 | A kind of Water-Body Information approximating method based on remote sensing images |
| CN109919095A (en) * | 2019-03-07 | 2019-06-21 | 北京石油化工学院 | Monitoring method and electronic equipment for palletizing in hazardous chemicals warehouse |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117437187A (en) * | 2023-10-13 | 2024-01-23 | 深圳市锐巽自动化设备有限公司 | Detonator detection method, detonator detection system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111949270B (en) | 2025-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6099793B2 (en) | Method and system for automatic selection of one or more image processing algorithms | |
| CN108229313B (en) | Face recognition method and apparatus, electronic device, computer program, and storage medium | |
| CN110458184B (en) | Optical character recognition assistance method, device, computer equipment and storage medium | |
| US20230196825A1 (en) | Face key point detection method and apparatus, and electronic device | |
| US9971954B2 (en) | Apparatus and method for producing image processing filter | |
| CN109993222B (en) | Data anomaly detection system and method | |
| CN114553591B (en) | Training method of random forest model, abnormal flow detection method and device | |
| CN113902956B (en) | Training method of fusion model, image fusion method, device, equipment and medium | |
| CN111949270A (en) | Process robot's operating environment change perception method and device | |
| CN112765606A (en) | Malicious code homology analysis method, device and equipment | |
| CN116188917A (en) | Defect data generation model training method, defect data generation method and device | |
| CN113205090B (en) | Picture correction method, device, electronic equipment and computer readable storage medium | |
| US10320636B2 (en) | State information completion using context graphs | |
| CN114511696A (en) | Control positioning method and device, electronic equipment and readable storage medium | |
| CN114119990A (en) | Method, apparatus and computer program product for image feature point matching | |
| CN112597662A (en) | Method and system for checking correctness and mistakes of building model | |
| CN116071625B (en) | Training method of deep learning model, target detection method and device | |
| CN111737067A (en) | A kind of hard disk failure prediction model interpretation method and device | |
| US20220335708A1 (en) | Data cleaning device, data cleaning method and face verification method | |
| CN116824707A (en) | Signature verification method, device, equipment, storage medium and product | |
| CN114090119B (en) | Control flow verification method, device, equipment and storage medium | |
| CN110716817B (en) | System operation fault processing method and device, storage medium and electronic equipment | |
| CN115344495A (en) | Data analysis method, device, computer equipment and medium for batch task testing | |
| US12040948B2 (en) | Method and electronic device for grouping objects in data management system | |
| JPWO2021038840A5 (en) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220921 Address after: 25 Financial Street, Xicheng District, Beijing 100033 Applicant after: CHINA CONSTRUCTION BANK Corp. Address before: 25 Financial Street, Xicheng District, Beijing 100033 Applicant before: CHINA CONSTRUCTION BANK Corp. Applicant before: Jianxin Financial Science and Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |