CN109230120A - It a kind of mechanical palms control method based on image recognition and takes book and puts book apparatus - Google Patents

It a kind of mechanical palms control method based on image recognition and takes book and puts book apparatus Download PDF

Info

Publication number
CN109230120A
CN109230120A CN201810864955.XA CN201810864955A CN109230120A CN 109230120 A CN109230120 A CN 109230120A CN 201810864955 A CN201810864955 A CN 201810864955A CN 109230120 A CN109230120 A CN 109230120A
Authority
CN
China
Prior art keywords
angle
bookend
books
straight line
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810864955.XA
Other languages
Chinese (zh)
Other versions
CN109230120B (en
Inventor
邓俊坚
余弘正
孙明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhuhai Jizhu Small And Medium Enterprise Advanced Technology Research Institute Co Ltd
Zhuhai College of Jilin University
Original Assignee
Zhuhai Jizhu Small And Medium Enterprise Advanced Technology Research Institute Co Ltd
Zhuhai College of Jilin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhuhai Jizhu Small And Medium Enterprise Advanced Technology Research Institute Co Ltd, Zhuhai College of Jilin University filed Critical Zhuhai Jizhu Small And Medium Enterprise Advanced Technology Research Institute Co Ltd
Priority to CN201810864955.XA priority Critical patent/CN109230120B/en
Publication of CN109230120A publication Critical patent/CN109230120A/en
Application granted granted Critical
Publication of CN109230120B publication Critical patent/CN109230120B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B65CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65GTRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G1/00Storing articles, individually or in orderly arrangement, in warehouses or magazines
    • B65G1/02Storage devices
    • B65G1/04Storage devices mechanical
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1679Programme controls characterised by the tasks executed
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/04Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/181Segmentation; Edge detection involving edge growing; involving edge linking

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Robotics (AREA)
  • Automation & Control Theory (AREA)
  • Image Analysis (AREA)

Abstract

It the mechanical palms control method based on image recognition that the invention discloses a kind of and takes and book and puts book apparatus, the method includes obtaining picture, identification straight line, the inclined degree for calculating straight line and corresponding angle, adjust the angle of bookend and pick and place books using bookend;Described device includes controller, camera system, mechanical palms and steering engine, and the controller is connect with camera system, mechanical palms and steering engine respectively.The mechanical structure that control mechanical palms or bookend etc. are used to pick and place book accurately rotates by a certain angle, allow mechanical palms or bookend parallel with the books or position to be picked and placed, bookend is accurately inserted into the gap between books, completes to pick and place book operation.In short, through the invention, the angle for the mechanical structure that bookend etc. is used to pick and place book can match with putting for books, damage of the mechanical structure to books is avoided on the basis of accurately can pick and place book.The present invention is widely used in technical field of automation equipment.

Description

It a kind of mechanical palms control method based on image recognition and takes book and puts book apparatus
Technical field
The present invention relates to automation equipment field, especially a kind of mechanical palms control method based on image recognition and take Book and put book apparatus.
Background technique
Currently, it is fast-developing with the large-scale library in society in the school, there is the lending of up to ten thousand books daily and gives back, This is undoubtedly a huge burden, not only heavy workload, but also low efficiency for librarian.Manpower management is to books Arrangement have hysteresis quality, greatly reduce library operation efficiency.
Intelligent robot of library can substitute manpower and be managed to books.Taking book on the shelf and putting book is books The important link of management is that the physics of books management executes step.Existing robot pick and place books technology be using mechanical arm, Mechanical palms or the pick-and-place book mechanical structure of other simulation manpowers need to develop control for books are picked and placed come what is realized The control algolithm of these mechanical structures.
In existing technology, by setting coordinate, color tracking and to functions such as books bar code scans, machine can control People reaches the bookshelf position that target books are placed/will be placed, and also can control mechanical palms and is raised or lowered to phase The height answered, but the above process is the general positioning for completing mechanical palms.During taking book, the step of most critical It is to be crawled out target books from book heap with mechanical palms, during putting book, the step of most critical is to use manipulator Target books are put into book heap by the palm.And in actual application environment, the mostly all not all book of book heap and horizontal plane " proper alignment " in vertical namely intuitive meaning, a part of usually book heap belongs to " proper alignment ", and part books Be with " inclination " in the angled namely intuitive meaning of horizontal plane, or even having part books is to be parallel to the horizontal plane also just It is " falling down " in intuitive meaning, while every book thickness in a book heap may be different, it is universal in these reality There are the problem of be all that mechanical palms take book and put the realization of book and bring difficulty.Book is taken at the two and puts the critical process of book In, mechanical palms must be physically contacted with books, but in the prior art the posture of mechanical palms can not with it is " whole in book heap The books matching of neat arrangement ", " inclination " and " falling down ", causes mechanical palms to operate by force and books is caused to damage.
Applicant has applied for that inventing a key name is " a kind of integrated mechanical palms of pick-and-place books " on August 1st, 2018 Patent of invention comprising a bookend being made of fixed baffle, flapper and butterfly hinge, the opening degree of bookend are Adjustable, therefore can clamp the books of different-thickness to realize the function of taking book and put book, bookend and mechanical palms are whole Translation and rotation can also be controlled by the steering engine of steering engine or outside inside mechanical palms.This mechanical palms have General positioning function in the prior art, that is, the bookshelf institute that target books are placed/will be placed can be accurately arrived at In position, it is raised or lowered to certain altitude then to reach corresponding bookshelf layer, then moves to target books in this layer of book The position placed/will placed on frame.But under the control of existing control algolithm, there are still cannot be accurate for this mechanical palms Adapt to book heap arranging situation and the problem of be easy to cause books to damage, need to be accurately controlled algorithm come to this mechanical palms into Row control.
Summary of the invention
In order to solve the above-mentioned technical problem, the object of the invention is that providing a kind of mechanical palms control based on image recognition It method and takes and book and puts book apparatus.
First technical solution adopted by the present invention is:
A kind of mechanical palms control method based on image recognition takes book, the mechanical palms for controlling mechanical palms Including the bookend being made of first baffle and second baffle, for the bookend for clamping books, the control method includes following Step:
It obtains and carries out horizontal shooting the first picture obtained for target book heap;The target book heap includes being taken out Target books;
The boundary of target books is identified from the first picture and then first straight line is gone out by the edge fitting of target books;
First angle is calculated according to inclined degree of the first straight line in the first picture;
The tilt angle of bookend is adjusted to angle corresponding with first angle;
Target books are taken out from target book heap using through over-angle bookend adjusted.
Further, the boundary that target books are identified from the first picture and then the edge fitting by target books It is to be carried out by Hough transformation the step for first straight line out.
Further, described that this step of first angle is calculated according to inclined degree of the first straight line in the first picture Suddenly, it specifically includes:
Obtain coordinate of any two o'clock in first straight line in the first picture;
According to the coordinate of described two points, the slope of first straight line is calculated;
According to the slope of first straight line, first angle is calculated.
Further, the slope according to first straight line, the step for calculating first angle, formula used are as follows:
θ in formula1For first angle, (x1,x2) and (y1,y2) it is respectively any two o'clock in first straight line in the first picture In coordinate,For the slope of first straight line.
Second technical solution adopted by the present invention is:
A kind of mechanical palms control method based on image recognition puts book, the mechanical palms for controlling mechanical palms Including the bookend being made of first baffle and second baffle, for the bookend for clamping books, the control method includes following Step:
Obtain the second picture for the shooting of target book heap level;The target book heap includes the books target to be put into Position, the target position are the gap in the edge or target book heap of target book heap between two books;
Target position is identified from second picture and then second straight line is fitted by target position;
Second angle is calculated according to inclined degree of the second straight line in second picture;
The tilt angle of bookend is adjusted to angle corresponding with second angle;
Utilize target position books being put into through over-angle bookend adjusted in target book heap.
Further, it is described identified from second picture target position so that by target position fit second straight line this One step is carried out by Hough transformation.
Further, the inclined degree according to second straight line in second picture calculates this step of second angle Suddenly, it specifically includes:
Obtain coordinate of any two point in second straight line in second picture;
According to the coordinate of described two points, the slope of second straight line is calculated;
According to the slope of second straight line, second angle is calculated.
Further, the slope according to second straight line, the step for calculating second angle, formula used are as follows:
θ in formula2For second angle, (x3,x4) and (y3,y4) it is respectively any two point in second straight line in second picture In coordinate,For the slope of second straight line.
Third technical solution adopted by the present invention is:
It is a kind of that book apparatus, including controller, camera system, mechanical palms and steering engine, the control are taken based on image recognition Device processed is connect with camera system, mechanical palms and steering engine respectively;
The controller is for running program to execute control method described in the first technical solution;
The camera system is used to shoot the first picture for target book heap for controller processing;
The mechanical palms include the bookend being made of first baffle and second baffle, and the bookend is used to clamp books, The bookend from target book heap for taking out target books under the control of the controller;
The steering engine is used under the control of the controller by the tilt angle of bookend or the tilt angle of mechanical palms entirety It is adjusted to angle corresponding with first angle.
4th technical solution adopted by the present invention is:
It is a kind of that book apparatus, including controller, camera system, mechanical palms and steering engine, the control are put based on image recognition Device processed is connect with camera system, mechanical palms and steering engine respectively;
The controller is for running program to execute control method described in the second technical solution;
The camera system is used for for target book heap shooting second picture for controller processing;
The mechanical palms include the bookend being made of first baffle and second baffle, and the bookend is used to clamp books, The bookend is used for the target position being under the control of the controller put into books in target book heap;
The steering engine is used under the control of the controller by the tilt angle of bookend or the tilt angle of mechanical palms entirety It is adjusted to angle corresponding with second angle.
The beneficial effects of the present invention are: control mechanical palms or bookend etc. are for picking and placing book by efficient control method Mechanical structure accurately rotate by a certain angle, allow mechanical palms or bookend and the books or position to be picked and placed flat Row, is inserted into bookend accurately in the gap between books, completes to pick and place book operation.In short, through the invention, book The angle of the mechanical structure for picking and placing book such as folder can match with putting for books, on the basis of accurately can pick and place book Avoid damage of the mechanical structure to books.
Detailed description of the invention
Fig. 1 is a kind of structure chart for picking and placing the integrated mechanical palms of books used in the method for the present invention;
Fig. 2 is a kind of schematic diagram that can apply book heap of the invention;
Fig. 3 is the flow chart of 1 method of embodiment;
Fig. 4 is the flow chart of 2 method of embodiment.
Specific embodiment
In addition patent of invention " a kind of pick-and-place books integration that applicant applies can be used in all embodiments of the invention Mechanical palms " realize, but the present invention is not limited to use mechanical palms disclosed in above-mentioned patent document to realize.
A kind of structure of the integrated mechanical palms of pick-and-place books disclosed in above-mentioned patent document referring to Fig.1, be into Row picks and places the book heap of book operation referring to Fig. 2.In practical application, more books stack book heap be all spine outwardly, so just In pick-and-place, various embodiments of the present invention are illustrated so that robot and camera system are towards spine as an example, other stack mode Book heap is also referred to be applicable in.It is contemplated that the present invention may be use with the situation that " the book heap " that only constitutes to a book is picked and placed.
In Fig. 1, the bookend clamping books that mechanical palms are made up of first baffle A and second baffle B, and first baffle A It is tabular with second baffle B, therefore first baffle A and second baffle B can be inserted in the gap in book heap between each book, To carry out taking book or put the operation of book.The method illustrated by following each embodiments, adjustable first baffle A and second The tilt angle of baffle B, so that between first baffle A and second baffle B and the edge or books of the books to be picked and placed Gap is parallel, so that first baffle A and second baffle B can smoothly clamp books and complete to pick and place the movement of books, keeps away Exempt to cause books to damage because bookend tilt angle is inconsistent with books.
Embodiment 1
The present embodiment control method takes book for controlling mechanical palms, and referring to Fig. 3, the control method includes following step It is rapid:
It obtains and carries out horizontal shooting the first picture obtained for target book heap;The target book heap includes being taken out Target books;
The boundary of target books is identified from the first picture and then first straight line is gone out by the edge fitting of target books;
First angle is calculated according to inclined degree of the first straight line in the first picture;
The tilt angle of bookend is adjusted to angle corresponding with first angle;
Target books are taken out from target book heap using through over-angle bookend adjusted.
Preferably, spine one side and camera lens picture and ground level of the camera towards book heap, institute when shooting the first picture It is as shown in Figure 2 to shoot the first picture come.
In target book heap shown in Fig. 2, the mechanical palms target books to be taken out are contained.Preferably, using suddenly Husband's transformation handles the first picture, so as to which the Boundary Extraction of target books in the first picture is come in, further intends First straight line is synthesized, the inclined degree of first straight line is just equal to the inclined degree of target books.Hough transformation is that one kind can be examined The algorithm of discontinuous point boundary shape is surveyed, it is by transforming to parameter space for image coordinate, to realize the fitting of straight line and curve.
The inclined degree of first straight line is converted into being described with angle, to obtain first angle.According to first angle The tilt angle of the bookend on mechanical arm is adjusted, it can so that bookend be parallel with the target books to be taken out, in turn So that bookend can be inserted in the gap between target books and its side books, to smoothly take out books.
It is further used as preferred embodiment, it is described to be calculated according to inclined degree of the first straight line in the first picture The step for first angle, specifically includes:
Obtain coordinate of any two o'clock in first straight line in the first picture;
According to the coordinate of described two points, the slope of first straight line is calculated;
According to the slope of first straight line, first angle is calculated.
It is further used as preferred embodiment, the slope according to first straight line calculates this step of first angle Suddenly, formula used are as follows:
θ in formula1For first angle, (x1,x2) and (y1,y2) it is respectively any two o'clock in first straight line in the first picture In coordinate,For the slope of first straight line.
In the present embodiment, can from selecting any two pixel in the first straight line in the first picture, with this two The coordinate of a pixel calculates the corresponding first angle of first straight line.If the coordinate of two pixels extracted is respectively (x1,x2) and (y1,y2), then first angle is just
In the case where the first picture is horizontal shooting, the first picture referential is earth reference system in fact, therefore the One angle is the angular dimension that bookend should be formed with ground.The mechanical palms rotation where bookend or bookend is driven, is made Bookend and ground angulation are equal to first angle, can be so that bookend be smoothly inserted into the gaps of target books Yu side books In, and then complete that book is taken to operate.
Embodiment 2
The present embodiment control method puts book for controlling mechanical palms, referring to Fig. 4, comprising the following steps:
Obtain the second picture for the shooting of target book heap level;The target book heap includes the books target to be put into Position, the target position are the gap in the edge or target book heap of target book heap between two books;
Target position is identified from second picture and then second straight line is fitted by target position;
Second angle is calculated according to inclined degree of the second straight line in second picture;
The tilt angle of bookend is adjusted to angle corresponding with second angle;
Utilize target position books being put into through over-angle bookend adjusted in target book heap.
What mechanical palms to be completed puts book operation, actually puts books into the gap between two books, or Person is placed on the edge of book heap, therefore the edge in the gap or book heap between two books is the target position to be put into;And machine Tool palm itself there are the books by target position both sides to push open to form the function of putting book space, it is therefore desirable to which what is done is adjustment The angle of mechanical palms matches the direction of bookend with book heap edge or gap.
By Hough transformation, it can identify target position and convert thereof into the identical second straight line of inclined degree.Ginseng According to 1 the method for embodiment, inclined degree of the second straight line in second picture can be calculated, so that second angle is obtained, And then bookend or mechanical palms are integrally adjusted according to second angle.
Embodiment 3
The present embodiment is a kind of to take book apparatus, including controller, camera system, mechanical palms and rudder based on image recognition Machine, the controller are connect with camera system, mechanical palms and steering engine respectively;
The controller is for running program to execute control method described in embodiment 1;
The camera system is used to shoot the first picture for target book heap for controller processing;
The mechanical palms include the bookend being made of first baffle and second baffle, and the bookend is used to clamp books, The bookend from target book heap for taking out target books under the control of the controller;
The steering engine is used under the control of the controller by the tilt angle of bookend or the tilt angle of mechanical palms entirety It is adjusted to angle corresponding with first angle.
The picture that controller is taken by obtaining camera system executes related program code to execute described in embodiment 1 Control method handled, and control signal is issued to mechanical palms and steering engine, to realize automatic book taking.
Wherein, steering engine may be mounted at the corresponding position in mechanical palms shown in Fig. 1, and steering engine is allowed individually to control The rotation of one baffle A and second baffle B, to control the tilt angle of the bookend of first baffle A and second baffle B composition.? Can by mechanical palms integral installation shown in Fig. 1 on steering engine, so that steering engine can control the rotation of mechanical palms entirety, thus Control the tilt angle of bookend.
Camera system used in the present embodiment can be OpenMV.
Embodiment 4
The present embodiment is a kind of to put book apparatus, including controller, camera system, mechanical palms and rudder based on image recognition Machine, the controller are connect with camera system, mechanical palms and steering engine respectively;
The controller is for running program to execute control method described in embodiment 2;
The camera system is used for for target book heap shooting second picture for controller processing;
The mechanical palms include the bookend being made of first baffle and second baffle, and the bookend is used to clamp books, The bookend is used for the target position being under the control of the controller put into books in target book heap;
The steering engine is used under the control of the controller by the tilt angle of bookend or the tilt angle of mechanical palms entirety It is adjusted to angle corresponding with second angle.
The realization of the present embodiment is referred to the explanation of embodiment 3 to carry out.
Embodiment 5
The present embodiment is illustrated controller used in embodiment 3 and embodiment 4 and camera system.In the present embodiment, Use Arduino master control borad as controller, uses OpenMV camera as camera system.
The control method of Arduino master control borad: initially setting up structural body, the structural body include floating type array data and Integer variable len is storage to data and the number for judging data respectively, initializes to steering engine, need to call # This header file library include<Servo.h>, establishes SERVO CONTROL object servo_8.Call the attach interface letter in the library Number, the function are used to set the interface of steering engine, such as No. 8 pins.Write () function in the library is called, the function is for setting The rotation angle of steering engine, the angle that can be set are 0-180 degree.Steering engine is initialized, so that steering engine moves to initial angle.Then Serial ports is configured, #include<SoftwareSerial.h>header file library is called, calls the begin () function in the library, Realize that, in order to communicate with OpenMV, suction parameter is set as 115200 to the opening of serial ports.And call the library Whether available () function, detection serial ports receive data, and when receiving data from OpenMV, available () value is big In 0, judged with if sentence, when if () is true, execution getList (), which is used to collect data, stores data into number In group, data mode that Arduino master control borad receives is { [x1, y1, x2, y3] }, call #include < In read () function and #include<Arduino.h>header file library in SoftwareSerial.h>header file library While () function reads a character receiving of serial ports, when receiving ' { ' character, while () function be it is false, jump out While () circulation, otherwise waits always, calls in #include<SoftwareSerial.h>header file library ReadStringUntil () function, the function are used to read the data of serial ports, until receiving ' } ', define a word at this time S is accorded with, for storing the string data in { }, the variable str an of character types is re-defined, is recycled with for () to character string Each character in s is handled, with length () function measurement character in #include<Arduino.h>header file library The character number gone here and there in s is arranged an integer variable i, is set as 0, when i is less than the character number in character string s, execution for () Sentence in circulation, when receiving ' [' character, str is null character, and when the character received is not ' [", "] ', character is deposited Storage defines the integer variable that one is 0, when receiving first ', ', by the toInt () letter of the character in Str in str Number is converted into integer and stores into array, and it is null character that k, which becomes 1, str, similarly, when encountering second with third ', ', point Cun Chu not be into array, at this point, the data stored in array are respectively x1, y1, x2, y2. is obtained by (y2-y1)=k (x2-x1) Slope k obtains an angle by atan () function, and because being once equal to 57.29, is mostly multiplied with the angle and obtained in 57.29 The angle of the mechanical arm rotation defines a variable _ _ var__35282_24230 for storing the angle and uses write at this time () function, respectively to 4 pin assignment, No. 8 pins are _ _ var__35282_24230, and No. 9 pins are that 90, No. 10 pins are 90, No. 11 pins are 90.Finally, memset () function in #include<string.h>header file library is called to empty array Data in data, and the end () function in #include<SoftwareSerial.h>header file library is called to close serial ports.
The control method of OpenMV camera: this algorithm with python language realize, first have to OpenMV camera into Row configuration, imports sensor module, is called to reset () function of sensor module, realizes at the beginning of OpenMV camera Then beginningization configures the pixel format of OpenMV camera, call set_pixformat () function of sensor module, pixel There are two types of formats, and one kind is RGB565 chromaticity diagram (every pixel 16), another is CRAYSCALE grayscale image (every pixel 8 Position).The invention is RGB565, so set_pixformat () function entrance parameter is sensor.RGB565, is then matched The pixel size of OpenMV is set, set_framesize () function of sensor module is called, suction parameter has respectively Sensor.QQVGA:160x120, sensor.QQVGA2:128x160, sensor.QVGA:320x240, sensor.QQCIF: 88x72, sensor.QCIF:176x144, sensor.CIF:352x288, the invention be resolution ratio is 320x240, i.e., QVGA, in order to allow arrangement above to play a role, calls the skip_ of sensor module so suction parameter is sensor.QVGA Frames () function, suction parameter 10 skip 10 frames.In order to realize contacting for OpenMV camera and master control borad, import UART (uart_idx, baud_rate) function of pyb module, suction parameter uart_idx is serial ports ID, because OpenMV is imaged Head only has serial ports 3, so suction parameter uart_idx is 3, suction parameter baud_rate is baud rate, and the invention uses The transmission rate of 115200baud/s.Then it is acquisition data, initializes an img object, for saving image, in order to Obtain image, call sensor module snapshot () function, OpenMV acquire image be placed in img, in order to from Image scanning goes out gap, calls find_lines () function of image module, suction parameter threshold, theta_ Margin, rho_margin, roi.`threshold` controls the straight line monitored from Hough transformation.It only returns and is greater than or waits In the straight line of threshold value.The threshold value right value of application program depends on image.That is the pixel size of straight line is more than or equal to 1500.theta_margin is straight line angle, and rho_margin is linear radial, if scanning the straight line angle come and radius Difference is less than marginal value, then merges two straight lines, is set as 25 pixels herein, and roi is the field of vision that color is arranged and identifies Domain, roi are a tuples, and roi=(x, y, w, h) represents the width since left upper apex (x, y) as the rectangle region of a height of h of w Domain, roi are defaulted as the whole image visual field if being not provided with, be set as (150,20,20,200), that is, central area herein, A blobs object is initialized, for storing the information of straight line.It is recycled with a for, a sheet by a sheet image is handled, when When the angle offset angle of straight line is less than 20 degree, the data of straight line are sent to master control borad, have used theta () function herein, are used In measurement straight line angle, write () function, for writing data to serial ports.However, the data type of blobs storage is dict, So dumps () function of json module is imported, for the data of dict type to be changed into str, because if directly will It can report an error in the data write-in json file of dict type, therefore need to use the function when writing data into.
The present embodiment can be used as the programmed method in embodiment 1-4 implementation process to particular hardware.
The present invention can achieve following technical effect: by efficient control method, controlling mechanical palms or bookend etc. and uses Accurately rotate by a certain angle in the mechanical structure for picking and placing book, allow mechanical palms or bookend and the books to be picked and placed or Position is parallel, and bookend is accurately inserted into the gap between books, completes to pick and place book operation.In short, passing through this The angle of invention, the mechanical structure that bookend etc. is used to pick and place book can match with putting for books, can accurately pick and place book On the basis of avoid damage of the mechanical structure to books.
It is to be illustrated to preferable implementation of the invention, but the implementation is not limited to the invention above Example, those skilled in the art can also make various equivalent variations on the premise of without prejudice to spirit of the invention or replace It changes, these equivalent deformations or replacement are all included in the scope defined by the claims of the present application.

Claims (10)

1. a kind of mechanical palms control method based on image recognition takes book, the mechanical palms packet for controlling mechanical palms The bookend being made of first baffle and second baffle is included, the bookend is for clamping books, which is characterized in that the control method The following steps are included:
It obtains and carries out horizontal shooting the first picture obtained for target book heap;The target book heap includes the mesh to be taken out Bidding documents sheet;
The boundary of target books is identified from the first picture, and then first straight line is gone out by the edge fitting of target books;
First angle is calculated according to inclined degree of the first straight line in the first picture;
The tilt angle of bookend is adjusted to angle corresponding with first angle;
Target books are taken out from target book heap using through over-angle bookend adjusted.
2. a kind of mechanical palms control method based on image recognition according to claim 1, which is characterized in that it is described from The step for identifying the boundary of target books in first picture, and then going out first straight line by the edge fitting of target books is logical Cross Hough transformation progress.
3. a kind of mechanical palms control method based on image recognition according to claim 1, which is characterized in that described The step for calculating first angle according to inclined degree of the first straight line in the first picture, specifically includes:
Obtain coordinate of any two o'clock in first straight line in the first picture;
According to the coordinate of described two points, the slope of first straight line is calculated;
According to the slope of first straight line, first angle is calculated.
4. a kind of mechanical palms control method based on image recognition according to claim 3, which is characterized in that described According to the slope of first straight line, the step for calculating first angle, formula used are as follows:
θ in formula1For first angle, (x1,x2) and (y1,y2) it is respectively any two o'clock in first straight line in the first picture Coordinate,For the slope of first straight line.
5. a kind of mechanical palms control method based on image recognition puts book, the mechanical palms packet for controlling mechanical palms The bookend being made of first baffle and second baffle is included, the bookend is for clamping books, which is characterized in that the control method The following steps are included:
Obtain the second picture for the shooting of target book heap level;The target book heap includes the books target position to be put into It sets, the target position is the gap in the edge or target book heap of target book heap between two books;
Target position is identified from second picture and then second straight line is fitted by target position;
Second angle is calculated according to inclined degree of the second straight line in second picture;
The tilt angle of bookend is adjusted to angle corresponding with second angle;
Utilize target position books being put into through over-angle bookend adjusted in target book heap.
6. a kind of mechanical palms control method based on image recognition according to claim 5, which is characterized in that it is described from Target position is identified in second picture so the step for second straight line is fitted by target position be by Hough transformation into Capable.
7. a kind of mechanical palms control method based on image recognition according to claim 5, which is characterized in that described The step for calculating second angle according to inclined degree of the second straight line in second picture, specifically includes:
Obtain coordinate of any two point in second straight line in second picture;
According to the coordinate of described two points, the slope of second straight line is calculated;
According to the slope of second straight line, second angle is calculated.
8. a kind of mechanical palms control method based on image recognition according to claim 7, which is characterized in that described According to the slope of second straight line, the step for calculating second angle, formula used are as follows:
θ in formula2For second angle, (x3,x4) and (y3,y4) it is respectively any two point in second straight line in second picture Coordinate,For the slope of second straight line.
9. a kind of take book apparatus based on image recognition, which is characterized in that including controller, camera system, mechanical palms and rudder Machine, the controller are connect with camera system, mechanical palms and steering engine respectively;
The controller requires any one of the 1-4 control method for running program with perform claim;
The camera system is used to shoot the first picture for target book heap for controller processing;
The mechanical palms include the bookend being made of first baffle and second baffle, and the bookend is described for clamping books Bookend from target book heap for taking out target books under the control of the controller;
The steering engine is under the control of the controller adjusting the tilt angle of the tilt angle of bookend or mechanical palms entirety For angle corresponding with first angle.
10. a kind of put book apparatus based on image recognition, which is characterized in that including controller, camera system, mechanical palms and rudder Machine, the controller are connect with camera system, mechanical palms and steering engine respectively;
The controller requires any one of the 5-8 control method for running program with perform claim;
The camera system is used for for target book heap shooting second picture for controller processing;
The mechanical palms include the bookend being made of first baffle and second baffle, and the bookend is described for clamping books Bookend is used for the target position being under the control of the controller put into books in target book heap;
The steering engine is under the control of the controller adjusting the tilt angle of the tilt angle of bookend or mechanical palms entirety For angle corresponding with second angle.
CN201810864955.XA 2018-08-01 2018-08-01 Mechanical palm control method based on image recognition and book taking and placing device Active CN109230120B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810864955.XA CN109230120B (en) 2018-08-01 2018-08-01 Mechanical palm control method based on image recognition and book taking and placing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810864955.XA CN109230120B (en) 2018-08-01 2018-08-01 Mechanical palm control method based on image recognition and book taking and placing device

Publications (2)

Publication Number Publication Date
CN109230120A true CN109230120A (en) 2019-01-18
CN109230120B CN109230120B (en) 2021-01-15

Family

ID=65073420

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810864955.XA Active CN109230120B (en) 2018-08-01 2018-08-01 Mechanical palm control method based on image recognition and book taking and placing device

Country Status (1)

Country Link
CN (1) CN109230120B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109956251A (en) * 2019-04-19 2019-07-02 深圳蓝胖子机器人有限公司 A kind of cargo identification grasping means, equipment and storage medium
CN113830481A (en) * 2021-10-26 2021-12-24 成都航空职业技术学院 Automatic book taking and placing device and method for library based on image scanning

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1589483B1 (en) * 2004-04-23 2009-12-09 Fanuc Ltd Object picking system
CN202400531U (en) * 2011-07-22 2012-08-29 方建军 An automatic fetch and return device for books in libraries
CN105069786B (en) * 2015-07-31 2018-05-08 小米科技有限责任公司 Line detection method and device
US9996805B1 (en) * 2016-09-30 2018-06-12 Amazon Technologies, Inc. Systems and methods for automated shipping optimization
CN108202328A (en) * 2017-12-29 2018-06-26 深圳市越疆科技有限公司 Robot adaptively picks and places method, apparatus, mechanical arm and the storage medium of object
CN108326850A (en) * 2018-01-10 2018-07-27 温州大学 A kind of accurate mobile mechanical arm of robot reaches the method and system of designated position
JP2019520586A (en) * 2016-07-14 2019-07-18 シーメンス・ヘルスケア・ダイアグノスティックス・インコーポレーテッドSiemens Healthcare Diagnostics Inc. Method and apparatus for dynamic position adjustment of robot gripper based on sample rack imaging data

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1589483B1 (en) * 2004-04-23 2009-12-09 Fanuc Ltd Object picking system
CN202400531U (en) * 2011-07-22 2012-08-29 方建军 An automatic fetch and return device for books in libraries
CN105069786B (en) * 2015-07-31 2018-05-08 小米科技有限责任公司 Line detection method and device
JP2019520586A (en) * 2016-07-14 2019-07-18 シーメンス・ヘルスケア・ダイアグノスティックス・インコーポレーテッドSiemens Healthcare Diagnostics Inc. Method and apparatus for dynamic position adjustment of robot gripper based on sample rack imaging data
US9996805B1 (en) * 2016-09-30 2018-06-12 Amazon Technologies, Inc. Systems and methods for automated shipping optimization
CN108202328A (en) * 2017-12-29 2018-06-26 深圳市越疆科技有限公司 Robot adaptively picks and places method, apparatus, mechanical arm and the storage medium of object
CN108326850A (en) * 2018-01-10 2018-07-27 温州大学 A kind of accurate mobile mechanical arm of robot reaches the method and system of designated position

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109956251A (en) * 2019-04-19 2019-07-02 深圳蓝胖子机器人有限公司 A kind of cargo identification grasping means, equipment and storage medium
CN113830481A (en) * 2021-10-26 2021-12-24 成都航空职业技术学院 Automatic book taking and placing device and method for library based on image scanning

Also Published As

Publication number Publication date
CN109230120B (en) 2021-01-15

Similar Documents

Publication Publication Date Title
CN109409366B (en) Distorted image correction method and device based on angular point detection
CN107953329B (en) Object recognition and attitude estimation method and device and mechanical arm grabbing system
JP6378701B2 (en) Hall slide image registration and inter-image annotation device, system, and method
CN109079788B (en) Chess playing method based on humanoid robot and humanoid robot
CN109325444B (en) Monocular texture-free three-dimensional object posture tracking method based on three-dimensional geometric model
CN110163211B (en) Image recognition method, device and storage medium
CN108875730B (en) Deep learning sample collection method, device, equipment and storage medium
CN109389555B (en) Panoramic image splicing method and device
CN1550297A (en) Image processing apparatus
CN105809711B (en) A kind of pig movement big data extracting method and its system based on video frequency tracking
CN110992422B (en) Medicine box posture estimation method based on 3D vision
CN109230120A (en) It a kind of mechanical palms control method based on image recognition and takes book and puts book apparatus
CN108229587A (en) A kind of autonomous scan method of transmission tower based on aircraft floating state
CN110400278A (en) A kind of full-automatic bearing calibration, device and the equipment of color of image and geometric distortion
CN111325051A (en) Face recognition method and device based on face image ROI selection
CN111860527A (en) Image correction method, image correction device, computer device, and storage medium
CN112040198A (en) Intelligent water meter reading identification system and method based on image processing
CN110838086A (en) Outdoor image splicing method based on correlation template matching
CN112581540A (en) Camera calibration method based on human body posture estimation in large scene
CN112605993A (en) Automatic file grabbing robot control system and method based on binocular vision guidance
CN112364805A (en) Rotary palm image detection method
CN110069655B (en) Face searching method for private photo album
WO2022152050A1 (en) Object detection method and apparatus, computer device and storage medium
CN114511592A (en) Personnel trajectory tracking method and system based on RGBD camera and BIM system
CN112488110A (en) Method and system for accurately capturing local information in picture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant