CN115810117A - Method and device for detecting brick laying condition of robot, electronic equipment and storage medium - Google Patents
Method and device for detecting brick laying condition of robot, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115810117A CN115810117A CN202211635525.3A CN202211635525A CN115810117A CN 115810117 A CN115810117 A CN 115810117A CN 202211635525 A CN202211635525 A CN 202211635525A CN 115810117 A CN115810117 A CN 115810117A
- Authority
- CN
- China
- Prior art keywords
- brick
- machine vision
- vision image
- quadrant
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of industrial robots, in particular to a method and a device for detecting brick laying conditions of a robot, electronic equipment and a storage medium, wherein the method comprises the following steps: determining a reference brick and a brick to be paved; acquiring an initial machine vision image; configuring detection parameters; in the working process of the robot, acquiring a current machine vision image according to the configured detection parameters; and performing visual line detection based on the current machine visual image, if the first quadrant brick corner does not exist, judging that brick pressing or over-view occurs, and reporting a judgment result. The method can judge the conditions of brick pressing and over-vision based on the visual line, and has higher accuracy.
Description
Technical Field
The embodiment of the invention relates to the technical field of industrial robots, in particular to a method and a device for detecting brick laying conditions of a robot, electronic equipment, a storage medium and a brick laying method of the robot.
Background
The industrial robot replaces manpower to carry out brick paving and pasting, so that the working efficiency can be improved, and the precision can be effectively controlled. At present, the mainstream scheme of brick paving of a robot is to use machine vision to combine with a mechanical arm to pave bricks, but because errors exist when a machine navigates to a specified station, brick angle information may not be seen in the visual field of the machine vision, and difficulty is caused to visual positioning. This requires timely detection of the robot tile condition for appropriate handling. In the prior art, the brick paving condition is generally judged based on the change of the gray value, and the accuracy is low.
Disclosure of Invention
Based on the problem that the detection accuracy of the brick paving condition of the robot in the prior art is low, the embodiment of the invention provides a method and a device for detecting the brick paving condition of the robot, electronic equipment, a storage medium and a brick paving method of the robot, and the conditions of brick pressing and over-vision can be judged based on visual lines.
In a first aspect, an embodiment of the present invention provides a method for detecting a brick laying condition by a robot, including:
determining a reference brick and a brick to be paved;
acquiring an initial machine vision image; a reference brick corner and a brick corner to be tiled are shot in the initial machine vision image, the state of the reference brick corner in the initial machine vision image is a first-quadrant brick corner, and the state of the brick corner to be tiled in the initial machine vision image is a fourth-quadrant brick corner;
configuring detection parameters;
in the working process of the robot, acquiring a current machine vision image according to the configured detection parameters;
performing visual line detection based on a current machine visual image, if the brick corner of the first quadrant does not exist, judging that brick pressing or over-view occurs, and reporting a judgment result; the first quadrant brick corner exists under the condition that the horizontal direction of the current machine vision image changes from left to right, the vertical direction changes from upper to lower, and the horizontal and vertical direction boundaries intersect.
Optionally, the determining that brick pressing or super view occurs includes:
detecting whether a fourth quadrant brick angle does not exist in the current machine vision image, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the fourth quadrant brick corner has the condition that a boundary line changing from light to dark is arranged from the left side to the right side in the transverse direction of the current machine vision image, and a boundary line changing from dark to light is arranged from the upper side to the lower side in the longitudinal direction;
detecting whether a first quadrant vertical line exists, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the first quadrant vertical line exists on the condition that a boundary line changing from light to dark is arranged from the left side to the right side of the horizontal direction of the current machine vision image;
detecting whether a third quadrant vertical line exists, if so, judging that the field of view is exceeded upwards and finishing the judgment, otherwise, continuing to execute the operation; the third quadrant vertical line exists under the condition that a dark-to-light boundary line exists from the left side to the right side in the transverse direction of the current machine vision image;
detecting whether a first quadrant transverse line exists, if so, judging to exceed the visual field leftwards, and ending the judgment, otherwise, continuing to execute; the condition that the first quadrant horizontal line exists is that a boundary line which changes from light to dark is formed from the upper side to the lower side of the longitudinal direction of the current machine vision image;
and judging that the brick pressing or the super view of the specific situation cannot be determined.
Optionally, the acquiring an initial machine vision image comprises:
determining the brick paving direction of the robot;
if the brick paving direction of the robot is longitudinal downward brick paving, directly shooting an initial machine vision image;
if the brick paving direction of the robot is not vertical downward, performing direction conversion after shooting to obtain an initial machine vision image;
when the direction is changed, after the brick pressing or super view is judged to appear, and before the judgment result is reported, the method further comprises the following steps:
and performing inverse transformation on the direction in the judgment result to obtain a judgment result suitable for the brick laying direction of the robot.
In a second aspect, an embodiment of the present invention further provides a device for detecting a brick laying condition of a robot, including:
the determining module is used for determining a reference brick and a brick to be paved;
the system comprises an initial module, a processing module and a display module, wherein the initial module is used for acquiring an initial machine vision image; a reference brick corner and a brick corner to be tiled are shot in the initial machine vision image, the state of the reference brick corner in the initial machine vision image is a first-quadrant brick corner, and the state of the brick corner to be tiled in the initial machine vision image is a fourth-quadrant brick corner;
the configuration module is used for configuring detection parameters;
the acquisition module is used for acquiring a current machine vision image according to the configured detection parameters in the working process of the robot;
the judging module is used for carrying out visual line detection based on the current machine visual image, judging whether brick pressing or over-view occurs if the brick corner in the first quadrant does not exist, and reporting a judging result; the first quadrant brick corner exists under the condition that the horizontal direction of the current machine vision image changes from left to right, the vertical direction changes from upper to lower, and the horizontal and vertical direction boundaries intersect.
Optionally, the determining module determines that brick pressing or over-view occurs, including performing the following operations:
detecting whether a fourth quadrant brick angle does not exist in the current machine vision image, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the fourth quadrant brick corner has the condition that a boundary line changing from light to dark is arranged from the left side to the right side in the transverse direction of the current machine vision image, and a boundary line changing from dark to light is arranged from the upper side to the lower side in the longitudinal direction;
detecting whether a first quadrant vertical line exists, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the first quadrant vertical line exists under the condition that a boundary line changing from light to dark is arranged from the left side to the right side of the transverse direction of the current machine vision image;
detecting whether a third quadrant vertical line exists, if so, judging that the field of vision is exceeded upwards and finishing the judgment, otherwise, continuing to execute the operation; the third quadrant vertical line exists on the condition that a dark-to-light boundary line exists from the left side to the right side of the transverse direction of the current machine vision image;
detecting whether a first quadrant transverse line exists, if so, judging to exceed the visual field leftwards, and ending the judgment, otherwise, continuing to execute; the condition that the first quadrant horizontal line exists is that a boundary line which changes from light to dark is formed from the upper side to the lower side of the longitudinal direction of the current machine vision image;
and judging that the brick pressing or the super-vision of the specific situation cannot be determined.
Optionally, the initial module acquires an initial machine vision image, including performing the following operations:
determining the brick paving direction of the robot;
if the brick paving direction of the robot is longitudinal downward brick paving, directly shooting an initial machine vision image;
if the brick paving direction of the robot is not vertical downward, performing direction conversion after shooting to obtain an initial machine vision image;
when the direction is changed, the judging module further executes the following operations after the brick pressing or super view is judged to appear and before the judgment result is reported:
and performing inverse transformation on the direction in the judgment result to obtain a judgment result suitable for the brick laying direction of the robot.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the method for detecting a brick paving status of a robot according to any embodiment of this specification is implemented.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed in a computer, the computer program causes the computer to execute the method for detecting the brick paving status of the robot according to any embodiment of the present specification.
In a fifth aspect, an embodiment of the present invention further provides a robot tile laying method, where tile laying is performed based on guidance of machine vision, and detection is performed in a tile laying process by using the robot tile laying condition detection method described in any embodiment.
The embodiment of the invention provides a method and a device for detecting the brick laying condition of a robot, electronic equipment, a storage medium and a brick laying method of the robot.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for detecting a brick laying condition of a robot according to an embodiment of the present invention;
FIG. 2 is a schematic view of a machine vision image taken with reference and to-be-tiled corners;
FIG. 3 is a flow chart of another method for detecting a brick laying status of a robot according to an embodiment of the present invention;
FIG. 4 (a) shows a situation where the fourth quadrant tile corner is not detected;
FIG. 4 (b) shows a situation where a third quadrant vertical line can be detected;
FIG. 4 (c) shows a situation where the first quadrant horizontal line can be detected;
FIG. 5 is a diagram of a hardware architecture of an electronic device according to an embodiment of the present invention;
fig. 6 is a structural diagram of a brick laying status detection device of a robot according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, it is obvious that the described embodiments are some, but not all embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
As mentioned above, the mainstream scheme of robot tile paving is to use machine vision in combination with a mechanical arm to tile, but because there is an error when a machine navigates to a specified station, tile corner information may not be seen in the visual field of the machine vision, which makes visual positioning difficult. This requires timely detection of the robot tile condition for appropriate handling. In the prior art, whether bricks are pressed or over-view is generally judged based on the change of gray values, and judgment is carried out based on the change of gray values, so that the brick pressing or over-view is easily influenced by factors such as light, ground gray information and the like, error judgment is caused, and the accuracy is low. In view of the above, the present invention provides a method and an apparatus for detecting a brick laying status of a robot, an electronic device, and a storage medium, wherein the method and the apparatus identify the remaining brick corners and brick edge information in an image by using machine vision, and determine the conditions of brick pressing and over-view according to the brick corners and the brick edge information.
Specific implementations of the above concepts are described below.
Referring to fig. 1, an embodiment of the present invention provides a method for detecting a brick laying condition of a robot, including:
102, acquiring an initial machine vision image; a reference brick corner and a brick corner to be tiled are shot in the initial machine vision image, the state of the reference brick corner in the initial machine vision image is a first-quadrant brick corner, and the state of the brick corner to be tiled in the initial machine vision image is a fourth-quadrant brick corner;
this step 102 may generate instructions to move the robot (or arm) to capture an initial machine vision image that meets the requirements; if the initial machine vision image does not meet the requirements, the judgment result of the subsequent judgment of brick pressing or over-vision possibly deviates from the actual result;
FIG. 2 is a schematic diagram showing a machine vision image with reference tile corners and tile corners to be tiled captured, where a first quadrant tile corner, i.e., one corner of a tile in the image, is the lower left corner of the tile, and the tile is "inserted" into the image in a manner of being inserted from the first quadrant orientation, and similarly, a second quadrant tile corner, i.e., one corner of a tile in the image, is the lower right corner of the tile, a third quadrant tile corner, i.e., one corner of a tile in the image, is the upper right corner of the tile, a fourth quadrant tile corner, i.e., one corner of a tile in the image, is the upper left corner of the tile, and the tile is "inserted" into the image in a manner of being inserted from the fourth quadrant orientation; it should be noted that the gray scale shown in fig. 2 is convenient for display, in the actual machine vision image, the area with the brick (reference brick or brick to be laid) is a bright color area, the ground part without the brick is a dark color area, and the reference brick and the brick to be laid are not distinguished by obvious depth;
106, acquiring a current machine vision image according to configured detection parameters in the working process of the robot;
at least shooting a brick corner to be tiled by using a current machine vision image;
the first quadrant brick angle exists on the condition that the horizontal direction of the current machine vision image has a bright-dark boundary from the left side to the right side, the vertical direction of the current machine vision image has a bright-dark boundary from the upper side to the lower side, and the horizontal and vertical boundaries intersect at an angle of about 90 ° (e.g., 90 ° ± 10 °), i.e., the complete brick angle is formed. If the first quadrant brick angle can be detected, the first quadrant brick angle is judged to be in a normal state, brick paving work and detection can be continuously executed, and the step 106 is returned to continuously acquire the current machine vision image.
As shown in fig. 2, in a normal working state, a reference brick corner and a brick corner to be tiled should be provided in a machine vision image, a state of the reference brick corner in the machine vision image is a first quadrant brick corner, a state of the brick corner to be tiled in the machine vision image is a fourth quadrant brick corner, the brick to be tiled is clamped by a mechanical arm and can always exist in a visual field of the machine vision, but after an error occurs in machine navigation, the reference brick may be blocked (i.e., pressed) or deviated from the visual field range (i.e., beyond the visual field), and needs to be reported in time to remind a user that the tile tiling state is abnormal.
In the embodiment of the invention, the visual line detection is carried out based on the machine vision image, the brick corner information in the current machine vision image is determined through the boundary from the left side to the right side in the image transversely and the boundary from the upper side to the lower side in the image longitudinally so as to judge whether brick pressing or over-view appears in time and report the judgment result, thereby quickly finding the brick paving condition of the robot. In addition, the detection method provided by the invention does not need to clearly and specifically identify the characteristic points of the reference brick and the brick to be paved and acquire the moving parameters of the robot, and can acquire the brick corner and brick side information through the boundary line of the brightness change in the machine vision, thereby realizing the judgment of the brick paving condition and having higher calculation speed.
Optionally, the determining in step 108 that brick pressing or over-view occurs further includes:
step 108-1, detecting whether a fourth quadrant brick angle does not exist in the current machine vision image, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute step 108-3; the fourth quadrant brick corner has the condition that a boundary line changing from light to dark is arranged from the left side to the right side in the transverse direction of the current machine vision image, and a boundary line changing from dark to light is arranged from the upper side to the lower side in the longitudinal direction;
step 108-3, detecting whether a first quadrant vertical line exists in the current machine vision image, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the first quadrant vertical line exists on the condition that a boundary line changing from light to dark is arranged from the left side to the right side of the horizontal direction of the current machine vision image;
step 108-5, detecting whether a third quadrant vertical line exists in the current machine vision image, if so, judging that the field of vision is exceeded upwards and finishing the judgment, otherwise, continuing to execute the operation; the third quadrant vertical line exists on the condition that a dark-to-light boundary line exists from the left side to the right side of the transverse direction of the current machine vision image;
step 108-7, detecting whether a first quadrant transverse line exists in the current machine vision image, if so, judging to exceed the visual field leftwards and finishing the judgment, otherwise, continuing to execute the operation; the condition that the first quadrant horizontal line exists is that a boundary line which changes from light to dark is formed from the upper side to the lower side of the longitudinal direction of the current machine vision image;
and 108-9, judging that the brick pressing or the super view field of the specific situation cannot be determined.
By adopting the embodiment, the judgment can be further carried out according to the brick side information in the machine vision image so as to determine the concrete condition of brick pressing or super vision, which is beneficial for a user to obtain a more accurate detection result so as to process the brick paving condition. It should be noted that the aforementioned orientation words "left", "right", "upper" and "lower" are all relative to the orientation of the machine vision image. Alternatively, in order to improve the processing efficiency, if the step 108 performs the visual line detection, and determines which boundaries and bright-dark relationships on two sides of the boundaries exist from the left side to the right side in the horizontal direction of the current machine vision image, and which boundaries and bright-dark relationships on two sides of the boundaries exist from the upper side to the lower side in the vertical direction, the subsequent step needs to perform the repeated detection on the current machine vision image, screen which boundaries and bright-dark relationships on two sides of the boundaries exist from the left side to the right side in the horizontal direction of the current machine vision image, and which boundaries and bright-dark relationships on two sides of the boundaries exist from the upper side to the lower side in the vertical direction of the current machine vision image, and determine whether the corresponding boundaries exist.
Optionally, for step 102, "said acquiring an initial machine-vision image" further comprises:
determining the brick paving direction of the robot;
if the brick paving direction of the robot is longitudinal downward brick paving, directly shooting an initial machine vision image;
if the brick paving direction of the robot is not in a longitudinal downward brick paving direction, performing direction transformation after shooting to obtain an initial machine vision image, and enabling the state of a reference brick corner in the initial machine vision image to be a first quadrant brick corner and the state of a brick corner to be paved in the initial machine vision image to be a fourth quadrant brick corner;
correspondingly, when the brick paving direction of the robot is not the longitudinal downward brick paving direction in step 102, and the azimuth change is performed after the shooting, after the brick pressing or super-field is determined to occur in step 108, before the reporting of the determination result, the method further includes: and performing inverse transformation on the direction in the judgment result to obtain a judgment result suitable for the brick laying direction of the current robot.
By adopting the embodiment, for different brick paving directions of the robot, the embodiment can be adopted for detection to determine the concrete conditions of brick paving, and corresponding conversion is carried out after determination, so that the judgment result aiming at the brick paving direction of the current robot can be obtained, and the detection flow does not need to be adjusted repeatedly. The transformation of the orientation of the image in step 102 corresponds to the inverse transformation of the orientation of the result in step 108, i.e. if the right side of the image is transformed to the left side in step 102, the left-hand hyper-view in step 108 is the right-hand hyper-view in real space. For a specific transformation, reference may be made to the prior art, and further description is omitted here.
As shown in fig. 3, the invention also provides a method for detecting the brick laying condition of the robot, which comprises the following steps:
308, performing visual line detection based on the current machine visual image, if the first quadrant brick angle does not exist, continuing to execute the step 310, otherwise, returning to the step 306; the first quadrant brick corner exists under the condition that the horizontal direction of the current machine vision image changes from left to right, the vertical direction of the current machine vision image changes from upper to lower, and the horizontal direction and the vertical direction intersect;
and step 320, reporting the judgment result.
Referring to fig. 4 (a) to 4 (c), fig. 4 (a) shows a case where the fourth quadrant brick corner cannot be detected, and the corresponding determination result is brick pressing, fig. 4 (b) shows a case where the third quadrant vertical line can be detected, and the corresponding determination result is upward-beyond-field, fig. 4 (c) shows a case where the first quadrant horizontal line can be detected, and the corresponding determination result is leftward-beyond-field, in fig. 4 (a) to 4 (c), a square with a lighter gray scale represents a tiled brick and can be used as a reference brick, a square with a darker gray scale represents a to-be-tiled brick, and a black square represents a range of a machine vision image.
The embodiment provided by the invention can utilize the brick corner and brick edge information in the machine vision image to quickly judge the common brick pressing or over view, and in order to improve the processing efficiency, if the visual line detection is carried out in the step 308, the bright-dark relation of which boundary lines and the bright-dark relation of two sides of the boundary lines are determined from the left side to the right side in the transverse direction of the current machine vision image, and the bright-dark relation of which boundary lines and the bright-dark relation of two sides of the boundary lines are determined from the upper side to the lower side in the longitudinal direction of the current machine vision image, the subsequent steps 310 to 316 need to repeatedly detect the current machine vision image, screen which boundary lines and the bright-dark relation of two sides of the boundary lines are determined from the left side to the right side in the transverse direction of the current machine vision image, and which boundary lines and the bright-dark relation of two sides of the boundary lines are determined from the upper side to the lower side in the longitudinal direction of the current machine vision image, and determine whether the corresponding boundary lines exist.
As shown in fig. 5 and 6, an embodiment of the present invention provides a robot tile laying condition detection apparatus. The apparatus embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. In terms of hardware, as shown in fig. 5, a hardware architecture diagram of an electronic device where a robot tile laying status detection apparatus according to an embodiment of the present invention is located is shown, and besides the processor, the memory, the network interface, and the nonvolatile memory shown in fig. 5, the electronic device where the apparatus is located may also include other hardware, such as a forwarding chip responsible for processing a message. As shown in fig. 6, a logical device is formed by reading a corresponding computer program in a non-volatile memory into a memory by a CPU of an electronic device in which the device is located and running the computer program. The brick paving condition detection device of the robot provided by the embodiment comprises:
a determining module 601, configured to determine a reference brick and a brick to be tiled;
an initial module 602 for acquiring an initial machine vision image; a reference brick corner and a brick corner to be tiled are shot in the initial machine vision image, the state of the reference brick corner in the initial machine vision image is a first-quadrant brick corner, and the state of the brick corner to be tiled in the initial machine vision image is a fourth-quadrant brick corner;
a configuration module 603, configured to configure detection parameters;
an obtaining module 604, configured to obtain a current machine vision image according to configured detection parameters during a robot working process;
a judging module 605, configured to perform visual line detection based on the current machine vision image, if a brick corner in the first quadrant does not exist, judge that brick pressing or over view occurs, and report a judgment result; the first quadrant brick corner exists under the condition that the horizontal direction of the current machine vision image changes from left to right, the vertical direction changes from upper to lower, and the horizontal and vertical boundary lines intersect.
In an embodiment of the present invention, the determining module 601 may be configured to perform step 100 in the foregoing method embodiment, the initiating module 602 may be configured to perform step 102 in the foregoing method embodiment, the configuring module 603 may be configured to perform step 104 in the foregoing method embodiment, the obtaining module 604 may be configured to perform step 106 in the foregoing method embodiment, and the determining module 605 may be configured to perform step 108 in the foregoing method embodiment.
Optionally, the determining module 605 determines that brick pressing or super view occurs, including performing the following operations:
detecting whether a fourth quadrant brick angle does not exist in the current machine vision image, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the fourth quadrant brick corner has the condition that a boundary line changing from light to dark is arranged from the left side to the right side in the transverse direction of the current machine vision image, and a boundary line changing from dark to light is arranged from the upper side to the lower side in the longitudinal direction;
detecting whether a first quadrant vertical line exists, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the first quadrant vertical line exists on the condition that a boundary line changing from light to dark is arranged from the left side to the right side of the horizontal direction of the current machine vision image;
detecting whether a third quadrant vertical line exists, if so, judging that the field of view is exceeded upwards and finishing the judgment, otherwise, continuing to execute the operation; the third quadrant vertical line exists on the condition that a dark-to-light boundary line exists from the left side to the right side of the transverse direction of the current machine vision image;
detecting whether a first quadrant transverse line exists, if so, judging to exceed the visual field leftwards, and ending the judgment, otherwise, continuing to execute; the condition that the first quadrant horizontal line exists is that a boundary line which changes from light to dark is formed from the upper side to the lower side of the longitudinal direction of the current machine vision image;
and judging that the brick pressing or the super-vision of the specific situation cannot be determined.
The initialization module 602 acquires an initial machine vision image, including performing the following operations:
determining the brick paving direction of the robot;
if the brick paving direction of the robot is longitudinal downward brick paving, directly shooting an initial machine vision image;
if the brick paving direction of the robot is not vertical downward, performing direction conversion after shooting to obtain an initial machine vision image;
correspondingly, when the direction is changed, the determining module 605 further performs the following operations after the brick pressing or super view is determined to occur and before the determination result is reported:
and performing inverse transformation on the direction in the judgment result to obtain a judgment result suitable for the brick paving direction of the robot.
It is to be understood that the illustrated structure of the embodiments of the present invention does not constitute a specific limitation to a robotic tile condition detecting apparatus. In other embodiments of the invention, a robotic tile condition detection device may include more or fewer components than shown, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
For the information interaction, execution process and other contents between the modules in the above-mentioned apparatus, because the same concept is based on as the method embodiment of the present invention, specific contents can refer to the description in the method embodiment of the present invention, and are not described herein again.
The embodiment of the invention also provides a robot brick laying method, which comprises the following steps:
and paving bricks based on the guidance of machine vision, and detecting by adopting the robot brick paving condition detection method in any embodiment of the invention in the brick paving process.
The embodiment of the invention also provides electronic equipment which comprises a memory and a processor, wherein the memory is stored with a computer program, and when the processor executes the computer program, the method for detecting the brick laying condition of the robot in any embodiment of the invention is realized.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program causes the processor to execute a method for detecting a tile paving status of a robot in any embodiment of the present invention.
Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the embodiments described above are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer by a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then a CPU or the like mounted on the expansion board or the expansion module is caused to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the embodiments described above.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (9)
1. A method for detecting brick paving conditions by a robot is characterized by comprising the following steps:
determining a reference brick and a brick to be paved;
acquiring an initial machine vision image; a reference brick corner and a brick corner to be tiled are shot in the initial machine vision image, the state of the reference brick corner in the initial machine vision image is a first-quadrant brick corner, and the state of the brick corner to be tiled in the initial machine vision image is a fourth-quadrant brick corner;
configuring detection parameters;
in the working process of the robot, acquiring a current machine vision image according to configured detection parameters;
performing visual line detection based on the current machine visual image, if the brick corner of the first quadrant does not exist, judging whether brick pressing or over-view occurs, and reporting a judgment result; the first quadrant brick corner exists under the condition that the horizontal direction of the current machine vision image changes from left to right, the vertical direction changes from upper to lower, and the horizontal and vertical direction boundaries intersect.
2. The method of claim 1,
the judging of the occurrence of brick pressing or super-visual field comprises the following steps:
detecting whether a fourth quadrant brick angle does not exist in the current machine vision image, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the fourth quadrant brick corner has the condition that a boundary line changing from light to dark is arranged from the left side to the right side in the transverse direction of the current machine vision image, and a boundary line changing from dark to light is arranged from the upper side to the lower side in the longitudinal direction;
detecting whether a first quadrant vertical line exists, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the first quadrant vertical line exists under the condition that a boundary line changing from light to dark is arranged from the left side to the right side of the transverse direction of the current machine vision image;
detecting whether a third quadrant vertical line exists, if so, judging that the field of vision is exceeded upwards and finishing the judgment, otherwise, continuing to execute the operation; the third quadrant vertical line exists on the condition that a dark-to-light boundary line exists from the left side to the right side of the transverse direction of the current machine vision image;
detecting whether a first quadrant transverse line exists, if so, judging to exceed the visual field leftwards, and ending the judgment, otherwise, continuing to execute; the condition that the first quadrant horizontal line exists is that a boundary line which changes from light to dark is formed from the upper side to the lower side of the longitudinal direction of the current machine vision image;
and judging that the brick pressing or the super-vision of the specific situation cannot be determined.
3. The method of claim 2,
the acquiring an initial machine vision image, comprising:
determining the brick paving direction of the robot;
if the brick paving direction of the robot is longitudinal downward brick paving, directly shooting an initial machine vision image;
if the brick paving direction of the robot is not vertical downward, performing direction conversion after shooting to obtain an initial machine vision image;
when the direction is changed, after the brick pressing or super view is judged to appear, and before the judgment result is reported, the method further comprises the following steps:
and performing inverse transformation on the direction in the judgment result to obtain a judgment result suitable for the brick paving direction of the robot.
4. A robot tile laying condition detection device, comprising:
the determining module is used for determining a reference brick and a brick to be paved;
the system comprises an initial module, a processing module and a display module, wherein the initial module is used for acquiring an initial machine vision image; a reference brick corner and a brick corner to be tiled are shot in the initial machine vision image, the state of the reference brick corner in the initial machine vision image is a first-quadrant brick corner, and the state of the brick corner to be tiled in the initial machine vision image is a fourth-quadrant brick corner;
the configuration module is used for configuring detection parameters;
the acquisition module is used for acquiring a current machine vision image according to the configured detection parameters in the working process of the robot;
the judging module is used for carrying out visual line detection based on the current machine visual image, judging whether brick pressing or over-view occurs if the brick corner in the first quadrant does not exist, and reporting a judging result; the first quadrant brick corner exists under the condition that the horizontal direction of the current machine vision image changes from left to right, the vertical direction changes from upper to lower, and the horizontal and vertical direction boundaries intersect.
5. The apparatus of claim 4,
the judging module judges whether brick pressing or over-view occurs, and comprises the following operations:
detecting whether a fourth quadrant brick angle does not exist in the current machine vision image, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the fourth quadrant brick corner has the condition that the horizontal direction of the current machine vision image changes from bright to dark from the left side to the right side, and the vertical direction of the current machine vision image changes from dark to bright from the upper side to the lower side;
detecting whether a first quadrant vertical line exists, if so, judging that the brick is pressed and finishing the judgment, otherwise, continuing to execute the operation; the first quadrant vertical line exists on the condition that a boundary line changing from light to dark is arranged from the left side to the right side of the horizontal direction of the current machine vision image;
detecting whether a third quadrant vertical line exists, if so, judging that the field of vision is exceeded upwards and finishing the judgment, otherwise, continuing to execute the operation; the third quadrant vertical line exists on the condition that a dark-to-light boundary line exists from the left side to the right side of the transverse direction of the current machine vision image;
detecting whether a first quadrant transverse line exists, if so, judging to exceed the visual field leftwards, and ending the judgment, otherwise, continuing to execute; the condition that the first quadrant horizontal line exists is that a boundary line which changes from light to dark is formed from the upper side to the lower side of the longitudinal direction of the current machine vision image;
and judging that the brick pressing or the super view of the specific situation cannot be determined.
6. The apparatus of claim 5,
the initialization module acquires an initial machine vision image, including performing the following operations:
determining the brick paving direction of the robot;
if the brick paving direction of the robot is longitudinal downward brick paving, directly shooting an initial machine vision image;
if the brick paving direction of the robot is not longitudinal downward brick paving, performing orientation transformation after shooting to obtain an initial machine vision image;
when the direction is changed, the judging module further executes the following operations after the brick pressing or super view is judged to appear and before the judgment result is reported:
and performing inverse transformation on the direction in the judgment result to obtain a judgment result suitable for the brick laying direction of the robot.
7. An electronic device comprising a memory and a processor, the memory having stored therein a computer program, characterized in that the processor, when executing the computer program, implements the method according to any of claims 1-3.
8. A storage medium having stored thereon a computer program, characterized in that the computer program, when executed in a computer, causes the computer to execute the method of any of claims 1-3.
9. A robotic tile laying method, comprising:
tiling is performed based on machine vision based guidance and detected during tiling using the robotic tile condition detection method of any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211635525.3A CN115810117B (en) | 2022-12-19 | 2022-12-19 | Robot brick laying condition detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211635525.3A CN115810117B (en) | 2022-12-19 | 2022-12-19 | Robot brick laying condition detection method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115810117A true CN115810117A (en) | 2023-03-17 |
| CN115810117B CN115810117B (en) | 2023-06-23 |
Family
ID=85486169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211635525.3A Active CN115810117B (en) | 2022-12-19 | 2022-12-19 | Robot brick laying condition detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115810117B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110280496A1 (en) * | 2010-05-11 | 2011-11-17 | Mitutoyo Corporation | Image measuring apparatus, program, and teaching method of image measuring apparatus |
| CN108195381A (en) * | 2017-12-26 | 2018-06-22 | 中国科学院自动化研究所 | Indoor robot vision alignment system |
-
2022
- 2022-12-19 CN CN202211635525.3A patent/CN115810117B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110280496A1 (en) * | 2010-05-11 | 2011-11-17 | Mitutoyo Corporation | Image measuring apparatus, program, and teaching method of image measuring apparatus |
| CN108195381A (en) * | 2017-12-26 | 2018-06-22 | 中国科学院自动化研究所 | Indoor robot vision alignment system |
Non-Patent Citations (1)
| Title |
|---|
| 张艳丽，等: "基于图像处理的自动铺砖机设计", 西安文理学院学报( 自然科学版) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115810117B (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11105624B2 (en) | Methods and apparatus to track a blade | |
| CN101663561B (en) | Multiple-point measuring method and survey instrument | |
| US20050195384A1 (en) | Surveying method and surveying instrument | |
| CN107390679B (en) | Storage device and laser navigation forklift | |
| TWI413937B (en) | Image recognition methods and devices | |
| CN103299177A (en) | Solder-attachment inspection method, PCB-inspection system, and solder-attachment inspection device | |
| JP2012230041A (en) | Position detection method, position detector and robot system | |
| CN105354816A (en) | Electronic element positioning method and apparatus | |
| CN109591051B (en) | Method and system for compensating operation precision of composite robot and storage medium | |
| CN115810117A (en) | Method and device for detecting brick laying condition of robot, electronic equipment and storage medium | |
| CN112956961A (en) | Sweeping robot, repositioning method and repositioning device thereof, and storage medium | |
| US8396681B2 (en) | Prediction algorithm for scanning an object | |
| CN109540022B (en) | Method for planning and deciding path of flat-cabin robot based on TOF depth camera | |
| CN115014205B (en) | Visual detection method and detection system for tower tray and automatic welding guiding system thereof | |
| KR102135567B1 (en) | Apparatus, system, and method for positioning compensation using block shape information on welding process | |
| CN107567275A (en) | Component is positioned over the method, apparatus and component patch/inserter of substrate | |
| CN117446520B (en) | Container stacking method, device, electronic equipment and storage medium | |
| JP2013010160A (en) | Robot control system, robot system, and marker processing method | |
| CN108262583A (en) | The type judgement of weld seam and localization method and system | |
| US20230314133A1 (en) | Laser scanning method, laser scanning system, and laser scanning program | |
| JP2562047B2 (en) | Position and orientation recognition method of target object | |
| CN114043531B (en) | Table tilt angle determination, use method, apparatus, robot, and storage medium | |
| CN110421292A (en) | A kind of method and apparatus to objects' contour positioning of welding robot | |
| CN103218822B (en) | Based on the image characteristic point automatic testing method of disappearance importance | |
| JP7210171B2 (en) | IMAGE PROCESSING DEVICE AND CONTROL METHOD THEREOF, IMAGING DEVICE, AND PROGRAM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |