CN113255455A - Monocular camera object identification and positioning method based on vector illumination influence removing algorithm - Google Patents
Monocular camera object identification and positioning method based on vector illumination influence removing algorithm Download PDFInfo
- Publication number
- CN113255455A CN113255455A CN202110463617.7A CN202110463617A CN113255455A CN 113255455 A CN113255455 A CN 113255455A CN 202110463617 A CN202110463617 A CN 202110463617A CN 113255455 A CN113255455 A CN 113255455A
- Authority
- CN
- China
- Prior art keywords
- image
- target object
- illumination
- robot
- brightness
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000005286 illumination Methods 0.000 title claims abstract description 129
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000004364 calculation method Methods 0.000 claims abstract description 30
- 238000001914 filtration Methods 0.000 claims abstract description 9
- 238000006243 chemical reaction Methods 0.000 claims abstract description 7
- 230000008569 process Effects 0.000 claims description 15
- 230000003287 optical effect Effects 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 5
- 238000012935 Averaging Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 4
- 238000005260 corrosion Methods 0.000 claims description 4
- 230000007797 corrosion Effects 0.000 claims description 4
- 230000000877 morphologic effect Effects 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000002834 transmittance Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a monocular camera object identification and positioning method based on a vector illumination influence removing algorithm, which comprises the following steps: s1, acquiring an image by a robot, and performing light removal treatment on the image; s2, filtering the image after the light removal treatment to obtain a filtered image; s3, performing color space conversion on the image, and then obtaining a binary image according to the color threshold of the target object; s4, eliminating the interferent outside the target area, determining the target object, and obtaining the image coordinate of the target object; s5, judging whether the target object exists in the target area or not according to the image coordinates of the target object, if not, returning to the null state and finishing the identification and positioning; if the target object exists, returning the center coordinate of the target object, adjusting the head angle and the body pose of the robot to enable the center coordinate of the target object to be overlapped with the center position of the image, and then calculating the distance from the target object to the robot according to the calculation model to obtain the recognition result of the target object.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a monocular camera object identification and positioning method based on a vector illumination influence removing algorithm.
Background
The positioning and recognition research based on computer vision is a leading-edge scientific and technological subject which develops rapidly in recent years, and has various applications and very wide development prospects in various fields. However, the illumination has a great influence on the recognition of the object, and the recognition success rate of the same object is far from the success rate of the same object at different illumination intensities and different illumination angles. In the aspect of positioning and ranging of objects, equipment such as a binocular camera, an RGBD depth camera or a laser radar is generally required, and the equipment is expensive and is easily interfered by external factors.
Therefore, it is necessary to develop a monocular camera object recognition and positioning method based on a vector operation and illumination elimination algorithm, the illumination influence of an image is eliminated through the vector algorithm, and then the interference of a target area is eliminated, so that the target object in the target area is accurately recognized.
Disclosure of Invention
The invention aims to solve the technical problem of providing a monocular camera object identification and positioning method based on a vector illumination influence removing algorithm, which removes the illumination influence of an image through the vector algorithm, eliminates the interference of a target area, improves the identification efficiency of an object, and accurately identifies the target object in the target area.
In order to solve the technical problems, the invention adopts the technical scheme that: the monocular camera object identification and positioning method based on the vector illumination influence removing algorithm specifically comprises the following steps:
the method specifically comprises the following steps:
s1: acquiring an image map by using a robot, and performing light removal treatment on the image map;
s2: filtering the image map subjected to the light removal processing in the step S1 to obtain a filtered image;
s3: performing color space conversion on the image obtained in the step S2, and obtaining a binary image according to a color threshold of a target object;
s4: eliminating interference objects outside the target area, determining a target object, and obtaining the image coordinates of the target object;
s5: judging whether a target object exists in the target area or not according to the image coordinates of the target object, if not, returning to the empty state, and finishing the identification and positioning; if the target object exists, returning the center coordinate of the target object, adjusting the head angle and the body pose of the robot to enable the center coordinate of the target object to be overlapped with the center position of the image, and then calculating the distance from the target object to the robot according to a calculation model to obtain the identification result of the target object.
By adopting the technical scheme, effective image data are obtained by means of the acquisition device of the robot; then, the automatic analysis, identification and positioning technology of the image data is carried out; identifying, analyzing and positioning an image acquired by the robot, classifying the image by using the image acquired by the robot as a data source by using an automatic image identification technology, and finally identifying and judging a target object for the classified image; the monocular camera object identification and positioning method based on the vector de-illumination influence algorithm can effectively reduce illumination influence and improve the identification efficiency of objects, can be effectively applied to life scenes such as table tennis ball picking and the like of a table tennis court, and has the advantages that the application effect shows that the method is accurate and reliable, the identification rate is high, the identification speed is high, the positioning error is within a range of 3 cm, and convenience is brought to life.
The invention is further improved in that the method for recognizing and locating a monocular camera object based on the vector de-illumination influencing algorithm further comprises a step S6 of correcting the recognition result of the target object obtained in the step S5 and outputting the corrected recognition result.
As a preferred technical solution of the present invention, in step S1, the robot acquires an image through the acquisition device, and performs a de-illumination process on the acquired image through an algorithm of vector de-illumination influence, which includes the specific steps of:
s11: firstly, assuming that illumination in the shadow image consists of balanced illumination and unbalanced illumination, the shadow image is converted into a vector I according to coordinates, namely the shadow image is represented by the following formula:
wherein the content of the first and second substances,
representing a balanced illumination of the light,
representing uneven illumination;
representing the reflection power of the object, P represents the coefficient,
vector coordinates of the image map; further converting the formula (1) into the following formula (2):
setting:
；
；
solving to obtain the formula (3):
（3）；
wherein
Is the image of the object under the balanced illumination;
the image of an object under unbalanced illumination belongs to an interference part and shows unbalanced brightness;
calculating the brightness distribution of the image to obtain the unbalanced illumination component, and defining the brightness of the image
Is the brightness V in the image HSV color space, i.e.:
(ii) a And then calculating the average brightness:
wherein N and M are averaging range constants, and N and M are equal or different and are related to the resolution of the image;
then, defining darkness D, namely D = D (x, y), and calculating the formula as follows:
the unbalanced illumination component is obtained by weighting the Gaussian kernel function of the brightness V, the average brightness Hv and the darkness D, and the calculation formula is as follows:
wherein, σ is a constant attenuation speed, α is a compensation parameter, the value of α determines the illumination balance condition, i.e. it represents insufficient or over-bright illumination, and its value is determined by the average brightness Hv and the brightness V; | D-V | represents the absolute value of the unbalanced illumination component, and when the difference between the brightness V and the darkness D is large, the larger the value is, the more unbalanced the illumination is; exp { - | V-Hv | | ^2/(2 ^ σ ^2) } is a Gaussian kernel function, which represents the compensation speed; when the illumination is unbalanced, namely the difference between the maximum value of the brightness V and the average brightness Hv is larger, the iteration can reach convergence faster; when the difference between the maximum value of the brightness V and the average brightness Hv is large, the illumination is considered to be unbalanced, and compensation is needed; in contrast, when the maximum value of the luminance V is the same as the average luminance Hv, no compensation is required;
when compensation is needed, when the maximum value of the brightness V is lower than 50% of the average brightness Hv, the illumination is considered to be insufficient, and a positive 0.5 gain is obtained; when the maximum value of the luminance V exceeds 50% of the average luminance Hv, it is considered that the light is too bright, and it should be suppressed, and a gain of minus 0.5 should be obtained; otherwise, the linear change is carried out at minus 0.5 to 0.5 according to the brightness V and the average brightness Hv; therefore, the calculation formula of the compensation parameter alpha is shown in formula (7),
α={0.5, V≤0.5ΗvΗv-VΗv, 0.5Ηv<V<1.5Ηv-0.5, V≥1.5Ηv} （7）；
s12: calculating I2(x, y) according to equation (6);
s13: calculating I1 (x, y) from I2(x, y); i.e., I1 (x, y) = I (x, y) -I2 (x, y);
s14: setting I (x, y) = I1 (x, y), returning to the step S12 for iteration until the iteration condition is not met, and ending the iteration;
s15: and restoring the vector I into an image according to the coordinates, namely obtaining the image with the illumination removed.
The obtained ground image after the illumination influence is removed is still an RGB image, when a vector illumination influence removing calculation method is used, the interference part is removed to obtain an image under balance, but the unbalanced illumination component cannot be completely removed through one-time estimation, and multiple iterations are required through setting a threshold value.
As a preferred technical solution of the present invention, the specific process of performing the filtering process on the image map after the light removal process in step S2 is as follows: firstly, performing Gaussian blur processing to smooth an image; and performing morphological operation, and adopting corrosion expansion to eliminate small noise points to obtain a filtered image.
As a preferred technical solution of the present invention, the step S4 includes the following steps:
s41: boundary lines are used for dividing the inside and the outside of the target area, and the boundary is extracted by utilizing Hough line transformation;
s42: in the image coordinate system, objects on the different side of the boundary line with the robot are excluded by utilizing the dotted-line relation, and objects on the same side are confirmed as target objects;
s43: and then, judging according to the upper and lower relations between the central coordinate point of the target object and the boundary line in the image coordinate system, wherein the expression of a straight line is L: ax + By + C =0, the central coordinate point of the target object is P (m, n), since the image coordinate system is opposite to the y-axis of the conventional coordinate axis; therefore, when the center coordinates P (m, n) of the target object are substituted into the boundary line, if Am + Bn + C >0 is satisfied, the center coordinates of the target object that satisfy the requirement are obtained, and the center coordinates of the target object in the image coordinate system are obtained.
As a preferred technical solution of the present invention, in step S5, when a target object exists in the target area and the head angle and the body pose of the robot are adjusted, a calculation formula of a vertical included angle between the center of the target object and the acquisition device of the robot is:
g1 is a vertical included angle between the center of the target object and the optical axis of the acquisition device of the robot; v is the ordinate of the center of the target object in the image coordinate system, and p1 is the image vertical pixel size; a1 is the vertical visual angle of the collecting device;
the calculation formula of the horizontal included angle between the center of the target object and the optical axis of the acquisition device of the robot is as follows:
g2 is a horizontal included angle between the center of the target object and the optical axis of the acquisition device of the robot; u is the abscissa of the center of the target object in the image coordinate system; p2 is image horizontal pixel size; a2 is the horizontal visual angle of the acquisition device; adjusting the angle of the robot head by g1 and g2 may center the target object in the center of the robot's field of view.
As a preferred embodiment of the present invention, the formula of the calculation model in step S5 is:
wherein alpha represents the included angle between the acquisition device and the horizontal plane, gamma represents half of the vertical view angle of the acquisition device, H is the height of the acquisition device from the ground, H is the radius of the ball, and d is the distance from the robot to the center of the target object.
As a preferred technical solution of the present invention, the converting of the image color space in step S3 is to convert the image from RGB color space to HSV color space, and then obtain a binary image according to the color threshold of the target object.
As a preferred embodiment of the present invention, the capturing device in step S1 is a camera, and is disposed on the head of the robot.
As the preferred technical scheme of the invention, the algorithm for converting the RGB color space into the HSV color space comprises the following steps:
wherein R is red in the RGB color space, G is green in the RGB color space, and B is blue in the RGB color space; h is Hue (Hue) of the HSV color space, S is Saturation (Saturation) of the HSV color space, and V is brightness (Value) of the HSV color space; and then obtaining a binary image according to the color threshold value of the target object.
Compared with the prior art, the invention has the beneficial effects that: in computer vision, illumination always has great influence on the identification of objects, and the success rate of identification of the same object is far from each other at different illumination intensities and different illumination angles; by utilizing a vector-based illumination removing algorithm, illumination influence can be effectively reduced, the detection accuracy is improved, and a specific calculation model is applied, so that an interference object is eliminated, and the detection efficiency is improved; the monocular camera object identification and positioning method based on the vector illumination removing influence algorithm can be effectively applied to life scenes such as table tennis ball picking and the like of a table tennis field, and the application effect shows that the method is accurate and reliable, high in identification rate, high in identification speed, and convenient for life, and the positioning error is within a range of 3 cm.
Drawings
FIG. 1 is a flowchart of the operation of the object recognition and positioning method of the monocular camera based on the vector illumination influence removing algorithm according to the present invention;
FIG. 2 is a schematic diagram of an image and illumination vector algorithm of the monocular camera object recognition and positioning method based on the vector illumination influence removing algorithm of the present invention;
FIG. 3 is a vertical view diagram of an NAO camera of the monocular camera object recognition and positioning method based on the vector de-illumination impact algorithm of the present invention;
FIG. 4 is a horizontal view diagram of an NAO camera of the monocular camera object recognition and positioning method based on the vector de-illumination impact algorithm of the present invention;
FIG. 5 is an object distance solution model diagram of the monocular camera object recognition and positioning method based on the vector illumination influence removal algorithm of the present invention;
FIG. 6 is a partial image generated during the image processing process of the monocular camera object recognition and positioning method based on the vector illumination influence removing algorithm of the present invention, wherein: (a) taking the picture as an original picture; (b) a white portion representing the recognized red color is a binary image generated based on the recognition of the red region of the object; (c) extracting boundary lines and marking the boundary lines as black lines; (d) for the recognition result graph, the finally recognized sphere is marked with a black circle to form a contour, and the position of the sphere center is marked in black;
FIG. 7 is a program operation result diagram of the monocular camera object recognition and positioning method based on the vector de-illumination influencing algorithm of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the drawings of the embodiments of the present invention.
Example (b): as shown in fig. 1, the method for recognizing and positioning an object of a monocular camera based on a vector illumination influence removing algorithm specifically includes the following steps:
s1: acquiring an image map by using a robot, and performing light removal treatment on the image map;
as shown in fig. 2, in step S1, the robot acquires an image through an acquisition device, that is, a camera disposed at the head of the robot, and performs a de-illumination process on the acquired image through an algorithm of vector de-illumination influence, which includes the specific steps of:
s11: firstly, assuming that illumination in the shadow image consists of balanced illumination and unbalanced illumination, the shadow image is converted into a vector I according to coordinates, namely the shadow image is represented by the following formula:
wherein the content of the first and second substances,
representing a balanced illumination of the light,
representing uneven illumination;
representing the reflection power of the object, P represents the coefficient,
vector coordinates of the image map; further converting the formula (1) into the following formula (2):
setting:
；
；
solving to obtain the formula (3):
（3）；
wherein
Is the image of the object under the balanced illumination;
the image of an object under unbalanced illumination belongs to an interference part and shows unbalanced brightness;
calculating the brightness distribution of the image to obtain the unbalanced illumination component, and defining the brightness of the image
Is the brightness V in the image HSV color space, i.e.:
(ii) a And then calculating the average brightness:
wherein N and M are averaging range constants, and N and M are equal or different and are related to the resolution of the image;
then, defining darkness D, namely D = D (x, y), and calculating the formula as follows:
the unbalanced illumination component is obtained by weighting the Gaussian kernel function of the brightness V, the average brightness Hv and the darkness D, and the calculation formula is as follows:
wherein, σ is a constant attenuation speed, α is a compensation parameter, the value of α determines the illumination balance condition, i.e. it represents insufficient or over-bright illumination, and its value is determined by the average brightness Hv and the brightness V; | D-V | represents the absolute value of the unbalanced illumination component, and when the difference between the brightness V and the darkness D is large, the larger the value is, the more unbalanced the illumination is; exp { - | V-Hv | | ^2/(2 ^ σ ^2) } is a Gaussian kernel function, which represents the compensation speed; when the illumination is unbalanced, namely the difference between the maximum value of the brightness V and the average brightness Hv is larger, the iteration can reach convergence faster; when the difference between the maximum value of the brightness V and the average brightness Hv is large, the illumination is considered to be unbalanced, and compensation is needed; in contrast, when the maximum value of the luminance V is the same as the average luminance Hv, no compensation is required;
when compensation is needed, when the maximum value of the brightness V is lower than 50% of the average brightness Hv, the illumination is considered to be insufficient, and a positive 0.5 gain is obtained; when the maximum value of the luminance V exceeds 50% of the average luminance Hv, it is considered that the light is too bright, and it should be suppressed, and a gain of minus 0.5 should be obtained; otherwise, the linear change is carried out at minus 0.5 to 0.5 according to the brightness V and the average brightness Hv; therefore, the calculation formula of the compensation parameter alpha is shown in formula (7),
α={0.5, V≤0.5ΗvΗv-VΗv, 0.5Ηv<V<1.5Ηv-0.5, V≥1.5Ηv} （7）；
s12: calculating I2(x, y) according to equation (6);
s13: calculating I1 (x, y) from I2(x, y); i.e., I1 (x, y) = I (x, y) -I2 (x, y);
s14: setting I (x, y) = I1 (x, y), returning to the step S12 for iteration until the iteration condition is not met, and ending the iteration;
s15: restoring the vector I into an image according to the coordinates to obtain a deluminated image map;
removing the influence of illumination to obtain a terrestrial image which is still an RGB image;
s2: filtering the image map subjected to the light removal processing in the step S1 to obtain a filtered image;
the specific process of performing the filtering process on the image map after the light removal process in the step S2 is as follows: firstly, performing Gaussian blur processing to smooth an image; performing morphological operation, adopting corrosion expansion to eliminate small noise points and obtaining a filtered image;
s3: performing color space conversion on the image obtained in the step S2, and obtaining a binary image according to a color threshold of a target object;
the step S3, converting the image color space from RGB to HSV, and obtaining a binary image according to the color threshold of the target object;
the algorithm for converting from the RGB color space to the HSV color space is:
wherein R is red in the RGB color space, G is green in the RGB color space, and B is blue in the RGB color space; h is Hue (Hue) of the HSV color space, S is Saturation (Saturation) of the HSV color space, and V is brightness (Value) of the HSV color space; then obtaining a binary image according to the color threshold value of the target object;
s4: eliminating interference objects outside the target area, determining a target object, and obtaining the image coordinates of the target object;
the specific steps of step S4 are:
s41: boundary lines are used for dividing the inside and the outside of the target area, and the boundary is extracted by utilizing Hough line transformation;
s42: in the image coordinate system, objects on the different side of the boundary line with the robot are excluded by utilizing the dotted-line relation, and objects on the same side are confirmed as target objects;
s43: and then, judging according to the upper and lower relations between the central coordinate point of the target object and the boundary line in the image coordinate system, wherein the expression of a straight line is L: ax + By + C =0, the central coordinate point of the target object is P (m, n), since the image coordinate system is opposite to the y-axis of the conventional coordinate axis; therefore, when the central coordinate P (m, n) of the target object is substituted into the boundary line, if Am + Bn + C >0 is satisfied, the central coordinate of the target object is satisfied, and the central coordinate of the target object in the image coordinate system is obtained;
s5: judging whether a target object exists in the target area or not according to the image coordinates of the target object, if not, returning to the empty state, and finishing the identification and positioning; if the target object exists, returning the center coordinate of the target object, adjusting the head angle and the body pose of the robot to enable the center coordinate of the target object to be overlapped with the center position of the image, and then calculating the distance from the target object to the robot according to a calculation model to obtain the identification result of the target object;
as shown in fig. 3, when a target object exists in the target area and the head angle and the body pose of the robot are adjusted in step S5, a calculation formula of a vertical angle between the center of the target object and the acquisition device of the robot is:
g1 is a vertical included angle between the center of the target object and the optical axis of the acquisition device of the robot; v is the ordinate of the center of the target object in the image coordinate system, and p1 is the image vertical pixel size; a1 is the vertical visual angle of the collecting device;
the calculation formula of the horizontal included angle between the center of the target object and the optical axis of the acquisition device of the robot is as follows:
g2 is a horizontal included angle between the center of the target object and the optical axis of the acquisition device of the robot; u is the abscissa of the center of the target object in the image coordinate system; p2 is image horizontal pixel size; a2 is the horizontal visual angle of the acquisition device; the center of the target object can be placed in the center of the vision of the robot by adjusting the angle of the robot head through g1 and g 2;
as shown in fig. 5, the formula of the calculation model in step S5 is:
wherein alpha represents an included angle between the acquisition device and a horizontal plane, gamma represents a half of a vertical view angle of the acquisition device, H is the height of the acquisition device from the ground, H is the radius of the ball, and d is the distance from the robot to the center of the target object; the calculation results are shown in fig. 7;
s6: the recognition result of the target object obtained in step S5 is corrected, and the corrected recognition result is output.
The application example is as follows: taking an NAO robot golf game as an example, the monocular camera object identification and positioning method based on the vector de-illumination influence algorithm specifically comprises the following steps:
s1: acquiring an image map (as shown in (a) in fig. 6) by using a robot, and performing a delumination treatment on the image map; in the step S1, the robot acquires an image through an acquisition device, namely, a camera arranged on the head of the robot;
because the reflection capability of the object is related to illumination, under the same illumination condition, the larger the reflection capability of the object is, the higher the brightness of the image is; with the same reflective power, the stronger the illumination, the higher the brightness of the image. Namely:
；
wherein the reflection power
The lighting device is determined by factors such as material, shape and posture of an object and is irrelevant to lighting; while
Representing illumination, P representing a coefficient;
the original image can be seen from the calculation formula
And reflection power
And illumination of light
Solving a problem of an indefinite equation set; the general algorithm is solved by adding a constraint condition, so that the solutions obtained by the difference of the constraint condition are different; it is difficult to completely extract the reflectance of an object from the image by separating it from the illumination; if the illumination is completely removed, the object is placed in a dark room, and a completely dark image is obtained; therefore, the reflection coefficients of the reaction objects need to be illuminated and need to be uniformThe balance illumination is used for balancing illumination, so that the influence of unbalanced illumination on the image is reduced, and the essence of processing the illumination problem is realized;
therefore, in the vector illumination influence removing algorithm, it is firstly assumed that the illumination in the shadow image consists of balanced illumination and unbalanced illumination, and the vector illumination influence removing algorithm specifically comprises the following steps:
s11: the image map is converted into a vector I in terms of coordinates, i.e., the image is represented by the following formula (1):
wherein the content of the first and second substances,
representing a balanced illumination of the light,
representing uneven illumination;
representing the reflection power of the object, P represents the coefficient,
vector coordinates of the image map; further converting the formula (1) into the following formula (2):
setting:
；
；
solving to obtain the formula (3):
（3）；
wherein
Is the image of the object under the balanced illumination;
the image of an object under unbalanced illumination is represented as unbalanced brightness and belongs to an interference part; the relationship in the high-dimensional space can be illustrated by a two-bit vector diagram, as shown in fig. 2 (the arc in the diagram represents the iterative process);
in the sample image, the uneven illumination appears as uneven brightness. Calculating the brightness distribution of the image to obtain the unbalanced illumination component, and defining the brightness of the image
Is the brightness V in the image HSV color space, i.e.:
(ii) a And then calculating the average brightness:
wherein N and M are averaging range constants, and N and M are equal or different and are related to the resolution of the image; 1/32, which is generally the size of an image, is suitable;
then, defining darkness D, namely D = D (x, y), and calculating the formula as follows:
the unbalanced illumination component is obtained by weighting the Gaussian kernel function of the brightness V, the average brightness Hv and the darkness D, and the calculation formula is as follows:
wherein, σ is an attenuation speed constant, and the value of σ is determined according to practical experiments and is generally 4-8; alpha is a compensation parameter, the value of alpha is determined by illumination balance condition, namely, the illumination is insufficient or over-bright, and the value of alpha is determined by average brightness Hv and brightness V; | D-V | represents the absolute value of the unbalanced illumination component, and when the difference between the brightness V and the darkness D is large, the larger the value is, the more unbalanced the illumination is; exp { - | V-Hv | | ^2/(2 ^ σ ^2) } is a Gaussian kernel function, which represents the compensation speed; when the illumination is unbalanced, namely the difference between the maximum value of the brightness V and the average brightness Hv is larger, the iteration can reach convergence faster; when the difference between the maximum value of the brightness V and the average brightness Hv is large, the illumination is considered to be unbalanced, and compensation is needed; in contrast, when the maximum value of the luminance V is the same as the average luminance Hv, no compensation is required;
when compensation is needed, when the maximum value of the brightness V is lower than 50% of the average brightness Hv, the illumination is considered to be insufficient, and a positive 0.5 gain is obtained; when the maximum value of the luminance V exceeds 50% of the average luminance Hv, it is considered that the light is too bright, and it should be suppressed, and a gain of minus 0.5 should be obtained; otherwise, the linear change is carried out at minus 0.5 to 0.5 according to the brightness V and the average brightness Hv; therefore, the calculation formula of the compensation parameter alpha is shown in formula (7),
α={0.5, V≤0.5ΗvΗv-VΗv, 0.5Ηv<V<1.5Ηv-0.5, V≥1.5Ηv} （7）；
s12: calculating I2(x, y) according to equation (6);
s13: calculating I1 (x, y) from I2(x, y); i.e., I1 (x, y) = I (x, y) -I2 (x, y);
s14: setting I (x, y) = I1 (x, y), returning to the step S12 for iteration until the iteration condition is not met, and ending the iteration;
s15: restoring the vector I into an image according to the coordinates to obtain a deluminated image map;
the iteration end condition can be set as the modulus or norm of I2(x, y) (i.e. max { I2(x, y) } < = threshold; when the modulus of I2(x, y) is small, I (x, y) and I1 (x, y) are very close to each other and represent that the illumination of the image is uniform; the norm condition of the other I2(x, y) is to control the image noise, and the iteration should be stopped when the noise is generated in the iteration process; the obtained ground shadow image is still an RGB image after the illumination influence is removed;
s2: filtering the image map subjected to the light removal processing in the step S1 to obtain a filtered image; the specific process of performing the filtering process on the image map after the light removal process in the step S2 is as follows: firstly, performing Gaussian blur processing to smooth an image; performing morphological operation, adopting corrosion expansion to eliminate small noise points and obtaining a filtered image;
s3: performing color space conversion on the image obtained in the step S2, and obtaining a binary image according to a color threshold of a target object;
the conversion of the image color space in step S3 is to convert the image from RGB color space to HSV color space, HSV being Hue (Hue), Saturation (Saturation), and Value (Value), where Value V represents the brightness of the color, and for the source color, the brightness Value is related to the brightness of the illuminant; for object colors, this value is related to the transmittance or reflectance of the object; then obtaining a binary image according to the color threshold value of the target object;
the algorithm for converting from the RGB color space to the HSV color space is:
wherein R is red in the RGB color space, G is green in the RGB color space, and B is blue in the RGB color space; h is Hue (Hue) of the HSV color space, S is Saturation (Saturation) of the HSV color space, and V is brightness (Value) of the HSV color space; then obtaining a binary image according to the color threshold value of the target object; we can get a binary image as shown in (b) of fig. 6, where the white part in the binary image corresponds to the candidate ball (which may be the ball we are looking for);
s4: eliminating interference objects outside the target area, determining a target object, and obtaining the image coordinates of the target object;
the specific steps of step S4 are:
s41: boundary lines are used for dividing the inside and the outside of the target area, and the boundary is extracted by using Hough line transformation, as shown in (c) of FIG. 6, a black line is an extracted white boundary line;
s42: in the image coordinate system, objects on the different side of the boundary line with the robot are excluded by utilizing the dotted-line relation, and objects on the same side are confirmed as target objects;
s43: and then, judging according to the upper and lower relations between the central coordinate point of the target object and the boundary line in the image coordinate system, wherein the expression of a straight line is L: ax + By + C =0, the central coordinate point of the target object is P (m, n), since the image coordinate system is opposite to the y-axis of the conventional coordinate axis; therefore, when the central coordinate P (m, n) of the target object is substituted into the boundary line, if Am + Bn + C >0 is satisfied, the central coordinate of the target object is satisfied, and the central coordinate of the target object in the image coordinate system is obtained;
in a general xy rectangular coordinate system, the equation of any straight line can be represented by Ay + Bx + C =0, the coordinate of a point M is (a, b), the point M is substituted into the equation, if Ab + Ba + C >0, the point M is above the straight line, and if Ab + Ba + C <0, the point M is below the straight line; when a white sideline exists in the image, the robot is below the white sideline, and the robot is always in the field, so that the center of the ball can be shown to be in the court as long as the coordinates of the center of the ball are also below the white sideline; in the coordinates of the image, the origin is the upper left corner of the image, the right of the horizontal axis is the increasing direction of the y axis, and the lower of the vertical axis is the increasing direction of the x circumference; this is in contrast to the general coordinate axis; therefore, when the coordinates (m, n) of the sphere center are substituted into the white edge line, the coordinates of the sphere center which meet the requirement when An + Bm + C >0 are met; as shown in (d) in fig. 6, the target sphere detection result is marked with a black circle;
s5 locating the target object: judging whether a target object exists in the target area or not according to the image coordinates of the target object, if not, returning to the empty state, and finishing the identification and positioning; if the target object exists, returning the center coordinate of the target object, adjusting the head angle and the body pose of the robot to enable the center coordinate of the target object to be overlapped with the center position of the image, and then calculating the distance from the target object to the robot according to a calculation model to obtain the identification result of the target object;
as shown in fig. 3, the vertical angle of view range of the NAO camera is 34.80 degrees, as shown in fig. 4, the horizontal angle of view range of the NAO camera is 60.97 degrees, and since the intersection point of the optical axis of the NAO camera and the image plane is the central point of the image plane, the vertical and horizontal included angles between the target and the optical axis of the camera can be derived from the position of the center of the target in the image coordinate system according to the angle of view range imaged by the NAO camera and the size of the image; therefore, in step S5, when the target object exists in the target area and the head angle and the body pose of the robot are adjusted, the calculation formula of the vertical angle between the center of the target object and the acquisition device of the robot is:
g1 is a vertical included angle between the center of the target object and the optical axis of the acquisition device of the robot; v is the ordinate of the center of the target object in the image coordinate system, and p1 is the image vertical pixel size; since the experiment used a size of 640 x 480 separation rate when capturing images, the value of p1 was 480; a1, the vertical visual angle of the NAO camera is 47.64 degrees;
the calculation formula of the horizontal included angle between the center of the target object and the optical axis of the acquisition device of the robot is as follows:
g2 is a horizontal included angle between the center of the target object and the optical axis of the acquisition device of the robot; u is the abscissa of the center of the target object in the image coordinate system; p2 is the image horizontal pixel size, value 640; a2, the horizontal visual angle of the NAO camera is 60.97 degrees; the center of the red ball can be placed in the center of the robot vision by adjusting the angle of the robot head through g1 and g 2;
as shown in fig. 5, the formula of the calculation model in step S5 is:
wherein alpha represents an included angle between the acquisition device and a horizontal plane, gamma represents a half of a vertical view angle of the acquisition device, H is the height of the acquisition device from the ground, H is the radius of the ball, and d is the distance from the robot to the center of the target object; the calculation results are shown in fig. 7;
s6: the recognition result of the target object obtained in step S5 is corrected, and the corrected recognition result is output.
The application effect shows that the method is accurate and reliable, the recognition rate is high, the recognition speed is high, and the positioning error is within the range of 3 cm.
The above description is only exemplary of the present invention and should not be taken as limiting the invention, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A monocular camera object identification and positioning method based on a vector de-illumination influence algorithm is characterized by specifically comprising the following steps:
s1: acquiring an image map by using a robot, and performing light removal treatment on the image map;
s2: filtering the image map subjected to the light removal processing in the step S1 to obtain a filtered image;
s3: performing color space conversion on the image obtained in the step S2, and obtaining a binary image according to a color threshold of a target object;
s4: eliminating interference objects outside the target area, determining a target object, and obtaining the image coordinates of the target object;
s5: judging whether a target object exists in the target area or not according to the image coordinates of the target object, if not, returning to the empty state, and finishing the identification and positioning; if the target object exists, returning the center coordinate of the target object, adjusting the head angle and the body pose of the robot to enable the center coordinate of the target object to be overlapped with the center position of the image, and then calculating the distance from the target object to the robot according to a calculation model to obtain the identification result of the target object;
in the step S1, the robot acquires an image through the acquisition device, and performs a de-illumination process on the acquired image through an algorithm for vector de-illumination influence, which includes the following specific steps:
s11: firstly, assuming that illumination in the shadow image consists of balanced illumination and unbalanced illumination, the shadow image is converted into a vector I according to coordinates, namely the shadow image is represented by the following formula:
wherein the content of the first and second substances,
representing a balanced illumination of the light,
representing uneven illumination;
representing the reflection power of the object, P represents the coefficient,
vector coordinates of the image map; further converting the formula (1) into the following formula (2):
setting:
；
；
solving to obtain the formula (3):
（3）；
wherein
Is the image of the object under the balanced illumination;
the image of an object under unbalanced illumination belongs to an interference part and shows unbalanced brightness;
calculating the brightness distribution of the image to obtain the unbalanced illumination component, and defining the brightness of the image
Is the brightness V in the image HSV color space, i.e.:
(ii) a And then calculating the average brightness:
wherein N and M are averaging range constants, and N and M are equal or different and are related to the resolution of the image;
then, defining darkness D, namely D = D (x, y), and calculating the formula as follows:
the unbalanced illumination component is obtained by weighting the Gaussian kernel function of the brightness V, the average brightness Hv and the darkness D, and the calculation formula is as follows:
wherein, σ is a constant attenuation speed, α is a compensation parameter, the value of α determines the illumination balance condition, i.e. it represents insufficient or over-bright illumination, and its value is determined by the average brightness Hv and the brightness V; | D-V | represents the absolute value of the unbalanced illumination component, and when the difference between the brightness V and the darkness D is large, the larger the value is, the more unbalanced the illumination is; exp { - | V-Hv | | ^2/(2 ^ σ ^2) } is a Gaussian kernel function, which represents the compensation speed; when the difference between the maximum value of the brightness V and the average brightness Hv is large, the illumination is considered to be unbalanced, and compensation is needed; in contrast, when the maximum value of the luminance V is the same as the average luminance Hv, no compensation is required;
when compensation is needed, when the maximum value of the brightness V is lower than 50% of the average brightness Hv, the illumination is considered to be insufficient, and a positive 0.5 gain is obtained; when the maximum value of the luminance V exceeds 50% of the average luminance Hv, it is considered that the light is too bright, and it should be suppressed, and a gain of minus 0.5 should be obtained; otherwise, the linear change is carried out at minus 0.5 to 0.5 according to the brightness V and the average brightness Hv; therefore, the calculation formula of the compensation parameter alpha is shown in formula (7),
α={0.5, V≤0.5ΗvΗv-VΗv, 0.5Ηv<V<1.5Ηv-0.5, V≥1.5Ηv} （7）；
s12: calculating I2(x, y) according to equation (6);
s13: calculating I1 (x, y) from I2(x, y); i.e., I1 (x, y) = I (x, y) -I2 (x, y);
s14: setting I (x, y) = I1 (x, y), returning to the step S12 for iteration until the iteration condition is not met, and ending the iteration;
s15: and restoring the vector I into an image according to the coordinates, namely obtaining the image with the illumination removed.
2. The method of claim 1, further comprising a step S6 of performing a site verification on the recognition result of the target object obtained in the step S5, modifying the recognition result according to the site verification, and outputting the modified recognition result.
3. The method for recognizing and locating a monocular camera object based on a vector de-illumination influence algorithm as claimed in claim 2, wherein the specific process of filtering the image map after the de-illumination processing in step S2 is as follows: firstly, performing Gaussian blur processing to smooth an image; and performing morphological operation, and adopting corrosion expansion to eliminate small noise points to obtain a filtered image.
4. The method for recognizing and locating an object of a monocular camera based on a vector de-illumination influence algorithm as claimed in claim 3, wherein the specific step of the step S4 is:
s41: boundary lines are used for dividing the inside and the outside of the target area, and the boundary is extracted by utilizing Hough line transformation;
s42: in the image coordinate system, objects on the different side of the boundary line with the robot are excluded by utilizing the dotted-line relation, and objects on the same side are confirmed as target objects;
s43: and then, judging according to the upper and lower relations between the central coordinate point of the target object and the boundary line in the image coordinate system, wherein the expression of a straight line is L: ax + By + C =0, the central coordinate point of the target object is P (m, n), since the image coordinate system is opposite to the y-axis of the conventional coordinate axis; therefore, when the center coordinates P (m, n) of the target object are substituted into the boundary line, if Am + Bn + C >0 is satisfied, the center coordinates of the target object that satisfy the requirement are obtained, and the center coordinates of the target object in the image coordinate system are obtained.
5. The method for identifying and positioning an object of a monocular camera according to claim 4, wherein when the target object exists in the target area and the head angle and the body pose of the robot are adjusted in step S5, the calculation formula of the vertical angle between the center of the target object and the acquisition device of the robot is:
g1 is a vertical included angle between the center of the target object and the optical axis of the acquisition device of the robot; v is the ordinate of the center of the target object in the image coordinate system, and p1 is the image vertical pixel size; a1 is the vertical visual angle of the collecting device;
the calculation formula of the horizontal included angle between the center of the target object and the optical axis of the acquisition device of the robot is as follows:
g2 is a horizontal included angle between the center of the target object and the optical axis of the acquisition device of the robot; u is the abscissa of the center of the target object in the image coordinate system; p2 is image horizontal pixel size; a2 is the horizontal visual angle of the acquisition device; adjusting the angle of the robot head by g1 and g2 may center the target object in the center of the robot's field of view.
6. The method for recognizing and locating a monocular camera object based on a vector de-illumination influencing algorithm according to claim 5, wherein the formula of the calculation model in step S5 is:
wherein alpha represents the included angle between the acquisition device and the horizontal plane, gamma represents half of the vertical view angle of the acquisition device, H is the height of the acquisition device from the ground, H is the radius of the ball, and d is the distance from the robot to the center of the target object.
7. The method for object recognition and location based on vector de-illumination influencing algorithm of claim 6, wherein the image color space transformation in step S3 is to transform the image from RGB color space to HSV color space, and then obtain the binary image according to the color threshold of the target object.
8. The method for recognizing and locating a monocular camera object based on a vector de-illumination influence algorithm according to claim 6, wherein the capturing device in step S1 is a camera, and is disposed at the head of the robot.
9. The method for recognizing and locating a monocular camera object according to claim 7, wherein the algorithm for converting from RGB color space to HSV color space is:
wherein R is red in the RGB color space, G is green in the RGB color space, and B is blue in the RGB color space; h is Hue (Hue) of the HSV color space, S is Saturation (Saturation) of the HSV color space, and V is brightness (Value) of the HSV color space; and then obtaining a binary image according to the color threshold value of the target object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110463617.7A CN113255455B (en) | 2021-04-28 | 2021-04-28 | Monocular camera object identification and positioning method based on vector illumination influence removing algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110463617.7A CN113255455B (en) | 2021-04-28 | 2021-04-28 | Monocular camera object identification and positioning method based on vector illumination influence removing algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113255455A true CN113255455A (en) | 2021-08-13 |
| CN113255455B CN113255455B (en) | 2021-10-22 |
Family
ID=77222325
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110463617.7A Active CN113255455B (en) | 2021-04-28 | 2021-04-28 | Monocular camera object identification and positioning method based on vector illumination influence removing algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113255455B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114558308A (en) * | 2022-03-04 | 2022-05-31 | 东莞理工学院 | Control method and system of ball door aiming device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107009357A (en) * | 2017-04-05 | 2017-08-04 | 武汉科技大学 | A kind of method that object is captured based on NAO robots |
| CN110648367A (en) * | 2019-08-15 | 2020-01-03 | 大连理工江苏研究院有限公司 | Geometric object positioning method based on multilayer depth and color visual information |
-
2021
- 2021-04-28 CN CN202110463617.7A patent/CN113255455B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107009357A (en) * | 2017-04-05 | 2017-08-04 | 武汉科技大学 | A kind of method that object is captured based on NAO robots |
| CN110648367A (en) * | 2019-08-15 | 2020-01-03 | 大连理工江苏研究院有限公司 | Geometric object positioning method based on multilayer depth and color visual information |
Non-Patent Citations (2)
| Title |
|---|
| 张墨逸等: "移动单目相机下基于单应的动态手势轨迹识别", 《华中科技大学学报》 * |
| 朱君波等: "基于矢量运算的图像光照处理算法", 《计算机应用》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114558308A (en) * | 2022-03-04 | 2022-05-31 | 东莞理工学院 | Control method and system of ball door aiming device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113255455B (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10607106B2 (en) | Object symmetry axis detection method based on RGB-D camera | |
| WO2021138995A1 (en) | Fully automatic detection method for checkerboard corners | |
| CN108319973B (en) | Detection method for citrus fruits on tree | |
| CN111415363B (en) | Image edge identification method | |
| WO2022205525A1 (en) | Binocular vision-based autonomous underwater vehicle recycling guidance false light source removal method | |
| CN110866903B (en) | Ping-pong ball identification method based on Hough circle transformation technology | |
| CN110232389B (en) | Stereoscopic vision navigation method based on invariance of green crop feature extraction | |
| CN111915704A (en) | Apple hierarchical identification method based on deep learning | |
| CN112819094A (en) | Target detection and identification method based on structural similarity measurement | |
| CN112991302A (en) | Flexible IC substrate color-changing defect detection method and device based on super-pixels | |
| CN111144207A (en) | Human body detection and tracking method based on multi-mode information perception | |
| CN112861654A (en) | Famous tea picking point position information acquisition method based on machine vision | |
| CN114241438B (en) | Traffic signal lamp rapid and accurate identification method based on priori information | |
| CN113255455B (en) | Monocular camera object identification and positioning method based on vector illumination influence removing algorithm | |
| CN115661110A (en) | Method for identifying and positioning transparent workpiece | |
| Han et al. | Target positioning method in binocular vision manipulator control based on improved canny operator | |
| CN102024264B (en) | Dimensional-histogram-statistic-based touch pen color recognition method | |
| CN111161219B (en) | Robust monocular vision SLAM method suitable for shadow environment | |
| CN111290582A (en) | Projection interaction area positioning method based on improved linear detection | |
| KR100703528B1 (en) | Apparatus and method for recognizing an image | |
| Xiang et al. | Research on target detection method based on HSV fusion Gaussian mixture model | |
| CN112541471B (en) | Multi-feature fusion-based shielding target identification method | |
| CN114066993A (en) | Power distribution cabinet control panel segmentation method based on machine vision | |
| Chen | A PCB Image Self-adaption Threshold Segmentation Method Fusing Color Information and OTSU Theory | |
| CN113096188B (en) | Visual odometer pose optimization method based on highlight pixel detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 210042 8 Blocks 699-22 Xuanwu Avenue, Xuanwu District, Nanjing City, Jiangsu Province Patentee after: Speed Technology Co.,Ltd. Address before: 210042 8 Blocks 699-22 Xuanwu Avenue, Xuanwu District, Nanjing City, Jiangsu Province Patentee before: SPEED TIME AND SPACE INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder |