CN111598913B

CN111598913B - Image segmentation method and system based on robot vision

Info

Publication number: CN111598913B
Application number: CN202010352152.3A
Authority: CN
Inventors: 陈佳期; 陈旭; 李密; 颜茂春; 陈嘉华; 罗伟华
Original assignee: Fujian Strait Zhihui Technology Co ltd
Current assignee: Fujian Strait Zhihui Technology Co ltd
Priority date: 2020-04-28
Filing date: 2020-04-28
Publication date: 2023-03-17
Anticipated expiration: 2040-04-28
Also published as: CN111598913A

Abstract

The invention provides an image segmentation method and system based on robot vision, which comprises the steps of obtaining a target image of the robot vision and obtaining the definition of the target image by using opencv; responding to the fact that the definition of the target image is larger than a first threshold value, and judging whether the target image is shot in a specified area or not by using a feature matching algorithm; performing similarity matching on the part contained in the target image and the corresponding part in the template in response to the target image being shot to the designated area and the matching probability of the target image and the template being greater than a second threshold value; in response to the result of the similarity matching being a complete match or the type and/or number of the components contained in the target image being greater than the type and/or number of the corresponding components in the template, performing lateral arrangement segmentation on the coordinates of the target image based on the ordering of the corresponding components in the template; a component included in the target image is assigned a notation identification. The method and the system can accurately segment the target image so as to facilitate subsequent identification work.

Description

Image segmentation method and system based on robot vision

Technical Field

The invention relates to the technical field of image processing of robot vision, in particular to an image segmentation method and system based on robot vision.

Background

The electric power instrument is used as a terminal unit for power grid construction and is widely applied to transformer substation places. Because the instruments in the transformer substation are various in types, the working efficiency of artificial verification is extremely low, the method is difficult to adapt to the condition that a large number of instruments need to be verified at present, and the problem of subjective observation errors possibly exists. The pointer instrument has the advantages of visual reading, simple structure, high precision, low manufacturing cost, strong anti-electromagnetic interference capability, convenient maintenance and the like, and is widely applied in production practice. Usually, the pointer instrument has no digital communication interface, can not convert the measuring signal into a digital signal, and needs to rely on a manual mode to identify the instrument number. Pointer instrument reading identification is a tedious, boring and high-repeatability work. In some occasions needing a large amount of identification of the readings of the pointer instrument, such as electric power systems, pointer instrument detection and the like, the accuracy of the acquired readings of the instrument depends on the responsibility and the visual fatigue degree of an operator to a great extent, errors and reading errors are easy to occur in the identification process, if the errors are found in time, the workload needs to be increased, otherwise serious consequences can be caused. The traditional manual identification pointer instrument reading mode not only causes the waste of human resources, but also cannot achieve the ideal identification effect.

With the development of scientific technology, more and more fields begin to use intelligent systems. The meter recognition technology is widely used in the industrial field as an intelligent processing technology, and is receiving more and more attention. At present, most of instruments are fixed, fixed image acquisition equipment is used for shooting, although the identification effect is good, more image acquisition equipment is needed in the arrangement of a plurality of groups of instruments, and the use cost is greatly increased. The movable robot is adopted to carry the image acquisition equipment to acquire the instrument, so that the input cost of the equipment can be effectively reduced, but the image needs to be processed and segmented to facilitate the identification of the instrument, the current segmentation algorithm is complex, the identification rate is low, and the phenomena of omission, error and the like are easy to occur.

Disclosure of Invention

The invention provides an image segmentation method and system based on robot vision, which are used for solving the technical problems that in the prior art, the efficiency of manually calibrating an instrument is low, errors or errors are easy to occur, and an algorithm in an intelligent instrument identification technology is complex, the identification rate is low, the cost is high, and the like.

In one aspect, the present invention provides an image segmentation method based on robot vision, including the following steps:

s1: acquiring a target image of robot vision, and acquiring the definition of the target image by using opencv;

s2: responding to the fact that the definition of the target image is larger than a first threshold value, and judging whether the target image is shot in a designated area or not by using a feature matching algorithm based on a template in the gallery;

s3: performing similarity matching on the components contained in the target image and the corresponding components in the template in response to the target image being shot to the designated area and the matching probability of the target image and the template obtained by using the target detection algorithm being greater than a second threshold value;

s4: in response to the result of the similarity matching being a complete match or the type and/or number of the components contained in the target image being greater than the type and/or number of the corresponding components in the template, performing lateral arrangement segmentation on the coordinates of the target image based on the ordering of the corresponding components in the template; and

s5: and assigning a representation identification to the component contained in the target image, wherein the representation identification is characterized by the representation identification of the component corresponding to the target image in the template.

Preferably, the target detection algorithm in step S1 is a tensflow image recognition algorithm.

Preferably, the method for determining the sharpness in step S2 is an algorithm for detecting picture blurriness by opencv, where the first threshold is set to 100. By means of the ambiguity algorithm and the setting of the first threshold value, target images which are not clear can be filtered, the operation pressure of a system is reduced, and the image segmentation efficiency is improved.

Further preferably, the feature matching algorithm includes a SIFT scale invariant feature transform algorithm, a SURF accelerated robust feature algorithm, or a FAST feature detection algorithm. The feature points of the target image can be acquired more accurately by virtue of the selectivity of various feature matching algorithms.

Further preferably, the second threshold value is 0.7. The target image which is the same as or similar to the template can be conveniently screened out through the setting of the second threshold value, and the segmentation operation is conveniently carried out on the target image.

Preferably, step S3 further comprises: and in response to the type and/or number of the components contained in the target image being smaller than the type and/or number of the corresponding components in the template, adjusting the second threshold value to a third threshold value, wherein the third threshold value is set to be 0.5, and performing step S2 again. And the condition of missing detection is avoided by adjusting the second threshold value, so that the validity of the data is ensured.

Further preferably, step S4 is preceded by: the excess features in the target image compared to the template are removed, and features that match prior features are retained in response to the features included in the target image overlapping features in the template. By means of the setting, the situations of repeated identification and the like possibly existing in the algorithm process can be eliminated, and data collision is avoided.

Preferably, the components include a dashboard, indicator lights and switches. The electric power meter element segmentation method can segment various types of electric power meter elements, and facilitates reading and identification of subsequent meter data.

According to a second aspect of the present invention, a computer-readable storage medium is presented, having stored thereon one or more computer programs which, when executed by a computer processor, implement the above-described method.

According to a third aspect of the present invention, there is provided a robot vision-based image segmentation system, comprising:

a target image acquisition unit: configuring a target image for acquiring robot vision, and acquiring the definition of the target image by using a target opencv;

a target image screening unit: the image processing method comprises the steps that the image processing method is configured to respond to the fact that the definition of a target image is larger than a first threshold value, and whether the target image is shot in a specified area or not is judged through a feature matching algorithm based on a template in a gallery;

a similarity matching unit: the method comprises the steps that the similarity matching between a component contained in a target image and a corresponding component in a template is carried out in response to the fact that the target image is shot to a specified area and the matching probability of the target image and the template is larger than a second threshold value obtained by using a target detection algorithm;

an image segmentation unit: the method comprises the steps that in response to the result of similarity matching being complete matching or the type and/or number of components contained in a target image being larger than the type and/or number of corresponding components in a template, the coordinates of the target image are transversely arranged and segmented based on the ordering of the corresponding components in the template; and;

a table identification giving unit: the method includes assigning a token identification to a component included in the target image, where the token identification is characterized as a token identification of a component in the template that corresponds to the target image.

The invention provides an image segmentation method and system based on robot vision, which utilize opencv and tensoflow target detection algorithms to realize the definition screening of robot vision images and the operation of feature matching probability, further correspond the robot vision target images to a template in a gallery, segment components such as corresponding instruments in the target images, and endow the corresponding instrument components in the template with the representation identification, thereby facilitating the subsequent acquisition of data information of the instrument components through the corresponding representation identification algorithms. The method and the system are used for carrying out image segmentation of robot vision, so that the accuracy and the speed of instrument image identification are greatly improved, instrument images are obtained by matching with image acquisition equipment carried by movable robots such as a hanging rail or a wheel, the image segmentation of various instruments is realized, and the data of different instruments can be conveniently read by adopting an instrument reading identification algorithm in a subsequent pertinence manner.

Drawings

The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and together with the description serve to explain the principles of the invention. Other embodiments and many of the intended advantages of embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow chart of a method of image segmentation based on robot vision according to an embodiment of the present application;

FIG. 3 is a flowchart of a method for image segmentation based on robot vision in accordance with a specific embodiment of the present application;

FIG. 4 is a block diagram of an image segmentation system based on robot vision in accordance with an embodiment of the present application;

FIG. 5 is a schematic block diagram of a computer system suitable for use in implementing an electronic device according to embodiments of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.

It should be noted that, in the present application, the embodiments and features of the embodiments may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Fig. 1 illustrates an exemplary system architecture 100 to which the image segmentation method based on robot vision of the present application may be applied.

As shown in FIG. 1, the system architecture 100 may include a data server 101, a network 102, and a main server 103. Network 102 serves as a medium for providing a communication link between data server 101 and host server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The main server 103 may be a server that provides various services, such as a data processing server that processes information uploaded by the data server 101. The data processing server may perform image segmentation based on robot vision.

It should be noted that the image segmentation method based on robot vision provided in the embodiment of the present application is generally executed by the host server 103, and accordingly, the apparatus of the image segmentation method based on robot vision is generally disposed in the host server 103.

The data server and the main server may be hardware or software. When the hardware is used, the hardware can be implemented as a distributed server cluster consisting of a plurality of servers, or can be implemented as a single server. When software, it may be implemented as a plurality of software or software modules (e.g., software or software modules for providing distributed services) or as a single software or software module.

It should be understood that the number of data servers, networks, and host servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Fig. 2 shows a flowchart of an image segmentation method based on robot vision according to an embodiment of the present application. As shown in fig. 2, the method comprises the steps of:

s201: and acquiring a target image of the robot vision, and acquiring the definition of the target image by using opencv. The primary screening of the target image can be conveniently carried out by calculating and acquiring the definition, and the calculation amount is reduced.

In specific embodiment, the target image of the robot vision can be acquired by adopting image acquisition equipment integrated on a hanging rail or a wheel type inspection robot, time and labor consumption and error are consumed and caused easily compared with manual verification, more image acquisition equipment needs to be arranged on a plurality of groups of instruments for fixed instrument identification, the use cost is greatly increased, and the movable robot is adopted to carry the image acquisition equipment to acquire the instruments, so that the investment cost of the equipment can be effectively reduced.

In a specific embodiment, the obtaining of the sharpness by using opencv specifically includes: performing laplacian mask convolution operation on a certain channel in the target image, preferably a gray value channel, then calculating a variance (namely the square of a standard deviation), and judging whether the target image is fuzzy or not according to the variance. This method works because the laplacian definition itself. It is used to measure the second derivative of the picture, highlighting areas of rapid intensity change in the picture, much like Sobel and Scharr operators. Also, as with the above operators, the laplacian operator is also often used for edge detection. Furthermore, this algorithm is based on the following assumptions: if the picture has a high variance, it has a wide frequency response range, which represents a normal, accurately focused picture. But if a picture has a small variance, it has a narrow frequency response range, meaning that the number of edges in the picture is small. The more blurred the picture, the fewer its edges. The algorithm provides us with a floating point number to represent the "blur" of a particular image. The algorithm is fast, simple and easy to use-it is only necessary to convolve the input image with the laplacian operator and then calculate the variance.

Alternatively, in addition to using the above-mentioned sharpness algorithm, other sharpness algorithms, such as Tenengrad gradient function, laplacian gradient function, or a combination thereof, may be selected, and the technical effects of the present invention may also be achieved. The Tenengrad gradient function adopts a Sobel operator to respectively extract gradient values in the horizontal direction and the vertical direction, and the definition of the image of the basis and the Tenengrad gradient function is defined as follows: d (f) = ∑ E _y ∑ _x G (x, y) | (G (x, y) > T), the form of G (x, y) is as follows:

where T is a given edge detection threshold, G _x And G _y Respectively, the convolution of Sobel operator in horizontal and vertical directions at pixel points (x, y) detects edges by using the following Sobel operator templates:

the Laplacian gradient function is basically consistent with the Tenengrad gradient function, and a Laplacian operator is used for replacing a Sobel operator, and the operator is defined as follows:

the definition of image star sharpness based on the Laplacian gradient function is as follows: d (f) = ∑ E _y ∑ _x G (x, y) | (G (x, y) > T), where G (x, y) is the convolution of the Laplacian operator at pixel point (x, y).

S202: and in response to the definition of the target image being larger than a first threshold value, judging whether the target image is shot in a specified area or not by using a feature matching algorithm based on the template in the image library. And screening and filtering unclear target images by using the first threshold value, reducing the operation amount, judging whether the target images reach the appointed shooting area by using a feature matching algorithm, and further filtering invalid target images.

In a particular embodiment, a picture may be considered blurred if the picture variance is below a predefined first threshold. Above the first threshold, it is not ambiguous. Preferably, the first threshold may be set to 100. It should be appreciated that the sharpness algorithm is tricky in setting a suitable first threshold, which is very dependent on the applied domain, too low of which may cause the normal picture to be misinterpreted as a blurred picture, and too high of which may cause the blurred picture to be misinterpreted as a normal picture, so the first threshold should be set to a suitable value according to the actual applied domain scene. The method tends to play a role in the range environment of calculating the acceptable definition evaluation value, and can detect abnormal photos.

In a specific embodiment, the feature matching algorithm may adopt one or a combination of a SIFT scale invariant feature transform algorithm, a SURF accelerated robust feature algorithm, or a FAST feature detection algorithm. The SIFT is fully called as follows: the scale-invariant feature transform is an algorithm widely used to extract and represent features in images. The SIFT algorithm has the advantages of remarkable extracted features, strong matching capability, strong stability in the face of other interference factors such as scale, noise, rotation and illumination and the like, and wide application due to strong robustness. The SURF algorithm, i.e., the accelerated robust feature algorithm, is a feature algorithm improved on the SIFT basis, which accelerates the computation by using an integral image and a box filter. The FAST algorithm, like its name, is a feature detection algorithm that is extremely FAST in computation. In general, most of the feature points of the FAST algorithm appear at the corner positions, so the FAST feature detection algorithm is also called as FAST corner detection algorithm. The main calculation idea of the FAST corner detection algorithm is as follows: selecting a pixel point on the image, taking a circular neighborhood around the pixel point, respectively solving gray value differences of all the pixel points on the circular neighborhood and a central pixel, and if the absolute value of the gray value difference between the central pixel point and more than n pixel points on the circular neighborhood is greater than a threshold value t, determining that the pixel point is an angular point. Colloquially, the gray value of the central pixel is brighter or darker than the gray value of most surrounding pixels by a threshold.

In a specific embodiment, a feature matching algorithm is used for identifying feature points in the target image and comparing the feature points with the template to judge whether the shooting position of the target image reaches the specified position, and if the target image is not shot at the specified position, the target image is filtered.

S203: and performing similarity matching on the part contained in the target image and the corresponding part in the template in response to the target image being shot to the designated area and the matching probability of the target image and the template obtained by using the target detection algorithm being greater than a second threshold value.

In a specific embodiment, in response to the type and/or number of the components included in the target image being smaller than the type and/or number of the corresponding components in the template, the step S202 is repeated by adjusting the second threshold to be a third threshold, where the third threshold is set to 0.5. If the number of types of components included in the target image is greater than or equal to the number of types of corresponding components in the template, the target image can be divided. Preferably, the second threshold is set to 0.7 and the third threshold is set to 0.5. Alternatively, the second threshold and the third threshold may be set to other values according to practical applications, for example, the second threshold is set to 0.8, the third threshold is set to 0.6, and the like, and the technical effects of the present invention can also be achieved.

In a specific embodiment, the target detection algorithm specifically adopts Tensorflow to construct target detection, and a general flow for implementing the machine learning algorithm by using Tensorflow is as follows: loading a data set; defining an algorithm formula, namely a calculation graph of forward calculation; defining a loss function (loss function), selecting an optimizer, and specifying the optimizer to optimize the loss function; performing iterative training on the data; accuracy assessments are made on either the test set or the cross-validation data set. And training and optimizing the model by using the own data on the basis of the pre-training model.

S204: and in response to the result of the similarity matching being complete matching or the type and/or number of the components contained in the target image being greater than the type and/or number of the corresponding components in the template, performing horizontal arrangement segmentation on the coordinates of the target image based on the ordering of the corresponding components in the template.

In a specific embodiment, the method further comprises the following steps: the excess features in the target image compared to the template are removed, and features that match prior features are retained in response to the features included in the target image overlapping features in the template. In the process of identifying the segmentation, overlapped parts possibly caused by circulation need to be removed, and only the matched prior parts are reserved, so that the influence of data collision on the segmentation result is avoided.

S205: and assigning a representation identification to the component contained in the target image, wherein the representation identification is characterized by the representation identification of the component corresponding to the target image in the template. The identification of the representation of the component correspondingly contained in the target image, for example, the characterization of the component as a switch, a voltmeter, an indicator light, etc., is assigned based on the representation in the template. Therefore, the segmentation of the target image is completed, the segmented image can be subjected to state or reading by using the recognition algorithm of the corresponding component, and the recognition result is more accurate and quicker.

Fig. 3 illustrates an image segmentation method based on robot vision according to a specific embodiment of the present invention, as shown in fig. 3, the method includes the following steps:

301: and starting.

302: the tensoflow image recognition algorithm detects all images. All target images are detected by using a tensoflow image recognition algorithm.

303: and judging whether the characteristic points are compared between the pictures in the picture library and the shot pictures, whether the pictures are shot to the specified positions or not and whether the picture quality reaches the identification standard or not. If so, entering a step 305, otherwise, entering a step 304, wherein the picture quality is characterized by the definition of the picture, the definition is calculated by using opencv, and the feature points are obtained by using a feature matching algorithm.

304: at which position the shot is likely to be. Acquiring the position that is currently possible may facilitate selecting the appropriate image in a forward or backward recursion. And the process continues to return to step 302.

305: obtaining a segmented image by obtaining a segmentation target probability, wherein the probability: pre. Probability: pre > 0.7.

306: and carrying out similarity matching on the type and the number of the image segmentation and the corresponding template, and calculating the score. And judging the similarity matching result, if the similarity matching result is completely matched, entering 307, and if the similarity matching result is not completely matched, entering 308.

307: and acquiring the segmented target probability to obtain the segmented image.

308: and judging whether the types and the number are larger than the set number of the templates. If so, step 309 is entered, otherwise step 310 is entered.

309: and transversely arranging and dividing the image coordinates.

310: and judging whether the probability pre is greater than 0.5. If not, the process goes to 314 and ends, if yes, the process returns to 305 to judge the segmentation again.

311: and acquiring the types and the number of the instrument panels of the corresponding templates, and arranging the belonged rows and the sequence.

312: and comparing the row sequence of the predicted values with the types in the template in sequence, removing redundant predicted values, and removing the overlapped predicted values by only using the predicted values which are matched firstly if the overlapped predicted values appear. If not, then the tag identification is directly assigned without removal and ends at 314.

313: and distributing to the corresponding token identification. And correspondingly distributing the template to the corresponding table identification according to the template, so that the subsequent data reading and other operations aiming at the table identification are facilitated.

314: and (6) ending.

According to the method, the target images can be identified and segmented quickly, according to the multiple tests of the method, the average time for identifying and segmenting each of 200 target images is only 0.05-0.5 second according to the image types with different complexity, and the efficiency of image identification and segmentation is greatly improved.

With continued reference to FIG. 4, FIG. 4 illustrates a robot vision based image segmentation system according to an embodiment of the present invention. The system specifically includes a target image acquisition unit 401, a target image screening unit 402, a similarity matching unit 403, an image segmentation unit 404, and a table identification assignment unit 405.

In a specific embodiment, the target image acquisition unit 401: configuring a target image for acquiring robot vision, and acquiring the definition of the target image by using a target opencv; target image screening unit 402: the image processing method comprises the steps that the image processing method is configured to respond to the fact that the definition of a target image is larger than a first threshold value, and whether the target image is shot in a specified area or not is judged through a feature matching algorithm based on a template in a gallery; similarity matching unit 403: the method comprises the steps that the similarity matching between a component contained in a target image and a corresponding component in a template is carried out in response to the fact that the target image is shot to a specified area and the matching probability of the target image and the template is larger than a second threshold value obtained by using a target detection algorithm; image segmentation unit 404: the method comprises the steps that in response to the result of similarity matching being complete matching or the type and/or number of components contained in a target image being larger than the type and/or number of corresponding components in a template, the coordinates of the target image are transversely arranged and segmented based on the ordering of the corresponding components in the template; the entry identifier assigning unit 405: the method includes assigning a token identification to a component included in the target image, where the token identification is characterized as a token identification of a component in the template that corresponds to the target image.

Referring now to FIG. 5, shown is a block diagram of a computer system 500 suitable for use in implementing the electronic device of an embodiment of the present application. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the use range of the embodiments of the present application.

As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU) 501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the system 500 are also stored. The CPU501, ROM502, and RAM503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Liquid Crystal Display (LCD) and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.

In particular, the processes described above with reference to the flow diagrams may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer-readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the method of the present application when executed by the Central Processing Unit (CPU) 501. It should be noted that the computer readable storage medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present application may be implemented by software or hardware.

As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may be separate and not incorporated into the electronic device. The computer readable storage medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a target image of the robot vision, and acquiring the definition of the target image by using opencv; responding to the fact that the definition of the target image is larger than a first threshold value, and judging whether the target image is shot in a designated area or not by using a feature matching algorithm based on a template in the gallery; performing similarity matching on the components contained in the target image and the corresponding components in the template in response to the target image being shot to the designated area and the matching probability of the target image and the template obtained by using the target detection algorithm being greater than a second threshold value; in response to the result of the similarity matching being a complete match or the type and/or number of the components contained in the target image being greater than the type and/or number of the corresponding components in the template, performing lateral arrangement segmentation on the coordinates of the target image based on the ordering of the corresponding components in the template; and assigning a representation identification to the component contained in the target image, wherein the representation identification is characterized by the representation identification of the component corresponding to the target image in the template.

The foregoing description is only exemplary of the preferred embodiments of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims

1. An image segmentation method based on robot vision is characterized by comprising the following steps:

s2: responding to the fact that the definition of the target image is larger than a first threshold value, and judging whether the target image is shot in a specified area or not by using a feature matching algorithm based on a template in a gallery;

s3: performing similarity matching on components contained in the target image and corresponding components in the template in response to the target image being shot to a specified area and the matching probability of the target image and the template obtained by the target detection algorithm being greater than a second threshold;

s5: assigning a token identification to a component included in the target image, wherein the token identification is characterized as a token identification of a component in the template that corresponds to the target image.

2. The robot vision-based image segmentation method according to claim 1, wherein the target detection algorithm in step S1 is a tensoflow image recognition algorithm.

3. The image segmentation method based on robot vision according to claim 1, wherein the sharpness determination method in step S2 is an opencv detection picture ambiguity algorithm, and the first threshold is set to 100.

4. The robot vision-based image segmentation method according to claim 1 or 3, wherein the feature matching algorithm comprises a SIFT scale invariant feature transform algorithm, a SURF speeded up robust feature algorithm, or a FAST feature detection algorithm.

5. The robot vision-based image segmentation method according to claim 4, wherein the second threshold is 0.7.

6. The image segmentation method based on robot vision according to claim 1, wherein the step S3 further comprises: and in response to the type and/or number of the components contained in the target image being smaller than the type and/or number of the corresponding components in the template, adjusting the second threshold value to a third threshold value, wherein the third threshold value is set to 0.5, and performing the step S2 again.

7. The image segmentation method based on robot vision according to claim 6, wherein the step S4 is preceded by the steps of: removing excess features in the target image compared to the template, and in response to a feature included in the target image overlapping a feature in the template, retaining a matching prior feature.

8. The robot vision-based image segmentation method according to claim 1, wherein the components include a dashboard, an indicator light, and a switch.

9. A computer-readable storage medium having one or more computer programs stored thereon, which when executed by a computer processor perform the method of any one of claims 1 to 8.

10. An image segmentation system based on robot vision, characterized in that the system comprises:

a target image acquisition unit: the method comprises the steps of configuring a target image for obtaining robot vision, and obtaining definition of the target image by using a target opencv;

a target image screening unit: the image processing method comprises the steps that the image processing method is configured to respond to the fact that the definition of the target image is larger than a first threshold value, and whether the target image is shot in a specified area or not is judged through a feature matching algorithm based on a template in a gallery;

a similarity matching unit: the method comprises the steps of configuring and responding to the condition that a target image is shot to a specified area and the matching probability of the target image and the template is obtained by utilizing the target detection algorithm and is larger than a second threshold value, and carrying out similarity matching on components contained in the target image and corresponding components in the template;

an image segmentation unit: the method comprises the steps that the coordinates of the target image are transversely arranged and segmented based on the sorting of corresponding parts in the template in response to the similarity matching result being complete matching or the type and/or number of the parts contained in the target image being larger than the type and/or number of the corresponding parts in the template;

a table identification giving unit: configured to assign a token identification to a component included in the target image, wherein the token identification is characterized as a token identification of a component in the template corresponding to the target image.