TW444499B - Binocular mechanical head monitoring system - Google Patents

Abstract

The present invention relates to a binocular mechanical head monitoring system consisting of a binocular mechanical head, a visual tracking controller, and a video processing system, The binocular mechanical head has a base. The first transmission axle on the top of the base drives the rotation stand above it. The second transmission axle is pivotally connected the top of the rotation stand and the head. By cooperating with the visual tracking controller and the video process system, two CCD cameras driven by the third transmission axle and the fourth transmission axle on two sides of the head drive can be used to monitor a moving object. Thus, a system with 3D visual monitoring effect is provided.

Description

Printed by the Ministry of Economic Affairs and the Central Government Bureau of Real Industrial Consumer Cooperatives ^ 444499 V. Description of the Invention (/) The present invention relates to a binocular manipulator monitoring system, especially a binocular manipulator with four auto & degrees. The CCD camera can obtain excellent stereo vision to form an excellent monitoring system. Although the technology of automation has been very advanced, most mechanical transmission mechanisms can only perform fixed tasks in a fixed working environment. In order to make the mechanical transmission mechanism work more flexible and intelligent, the most direct way is It imitates human's ability to recognize and track the target with the eyes' and then cooperates with the mechanical transmission mechanism to complete the task; only for a three-dimensional target, a single camera can only obtain the two-dimensional image information of the target projected on the image plane In order to obtain the spatial information of the target object, it must be calculated through the two image information of stereo vision, just as human beings have two clear eyes and can sense the distance of the object. Therefore, the inventor thought that if a The system of monitoring the effect of the erection of ships should improve the efficiency of monitoring. Therefore, in order to provide a better monitoring system, the inventor has designed the "binocular manipulator monitoring system" of the present invention, and hopes to obtain excellent monitoring by monitoring the binocular manipulator like human visual behavior. The main purpose of the design of the present invention is to provide a "binocular manipulator monitoring system", which is composed of a binocular manipulator, a vision tracking controller, and an image processing system. The binocular manipulator has a base, and the top of the base is provided with a first A transmission shaft drives the upper rotation seat, the top of the rotation seat is pivoted by the second transmission shaft, and two CCD cameras are driven by the third transmission shaft and the fourth transmission shaft on both sides of the head. And image processing system, which can monitor the moving target and provide a system with stereoscopic monitoring effect. (Please read the note on the back ^ before filling in this page j

Af B7 V. Explanation of the Invention (above) In order for your review committee to better understand the structure and application mode of the monitoring system of the present invention, it is explained in conjunction with the drawings: (1) Schematic part: The first diagram is a pair of the invention Schematic diagram of the appearance of the eye robot head. The first circle is a schematic front view of the binocular manipulator of the present invention, and the second circle is a schematic side view of the binocular manipulator of the present invention. The fourth figure is a schematic top view of the binocular mechanical head of the present invention. The fifth circle is the speed reduction ratio of the worm and worm of each axis of the double-headed mechanical head of the present invention. The sixth circle is the circle indicated by the pulse command control block of the present invention. The seventh circle is a schematic diagram of the pulse wave output of the present invention. The eighth-eighth is a schematic diagram of the network structure of the image processing unit of the present invention. The ninth aspect is an image processing flowchart of the present invention. The tenth circle is a schematic diagram of the system control architecture of the present invention. The eleventh figure is a definition diagram of the joint angle of the binocular mechanical head of the present invention. The twelfth series is a schematic diagram of the SO IM network architecture applied by the present invention. The thirteenth figure is a schematic diagram of the S 0 I M network learning architecture of the present invention. The fourteenth figure is a system control frame $ # intention of the present invention integrated with a robot arm. The fifteenth figure is a table showing the relationship between the maximum speed of each axis of the binocular mechanical head of the present invention and the relationship between the & @ out angle and the pulse wave input. The sixteenth circle is a schematic diagram of the first axis of the present invention. The seventeenth figure is a schematic diagram of the second shaft of the present invention. t (CNS) ('210 X 297 ^ *' (Jing Xian first read the notes on the back and then fill out this page) > »1T:-A44499 Central Standards Bureau of the Ministry of Economics β; Industrial and consumer cooperatives printed A? ·, B7 5 Explanation of the invention (j) Figure 18 is a schematic diagram of the response of the third axis of the present invention. Figure 19 is a schematic diagram of the response of the fourth axis of the present invention. Figure 20 is a binocular mechanical head of the present invention learning on the SO I M network Table of the range of rotation of each axis in the process. Figure 21 is a schematic diagram of the error I 1 of the present invention. Figure 22 is a schematic diagram of the error I 2 of the present invention. Figure 23 is a schematic diagram of the I 3 error of the present invention. Twenty The fourth figure is a schematic diagram of the hardware architecture of the system of the present invention. The twenty-fifth (the present invention is a change in the position of the moving target in the left-eye image plane of the present invention). The twenty-sixth circle is the present invention for the moving target in the right-eye image plane. The position change diagram of the 27th circle is the position change diagram of the moving object in the u-direction of the left-eye image plane of the present invention. The 28th circle is the position of the moving target object in the v-direction of the left-eye image plane of the present invention. Change diagram "The twenty-ninth diagram is for the invention The position change diagram of the moving target in the u direction of the right eye image plane. The thirtieth circle is the position change of the moving target object in the ν direction of the right eye image plane. The thirty-first diagram is the integration of the invention with a robotic arm. The hardware structure of the system is unintentional. The thirty-second circumference is the change in the left eye image coordinate of the moving target and the robot arm terminator. / {Please read the precautions on the back before filling in this. Ding.- I— I— 5 Seal A7 B7, Consumer Work Cooperative of the Central Government Bureau of the Ministry of Economic Affairs, V. 5. Description of the Invention (p) Figure 23 shows the changes in the coordinates of the right eye image of the moving target and the robot arm terminator. (1) Part No. * (10) Base (11) First drive shaft (12) Rotary seat (13) Second drive shaft C14) Head (15) Third drive shaft (1 6) Fourth drive Axis (17) camera (18) camera The binocular manipulator monitoring system of the present invention is composed of a binocular manipulator, a vision tracking controller, and an image processing system. The structure of the binocular manipulator is as shown in the first figure. As shown, it has a base (10), the top of the base (1 0) starts with the first The transmission shaft (1 1) drives the upper rotation seat (1 2), and the top of the rotation seat (1 2) is pivoted by the second transmission shaft (1 3 ·) to the joint portion (14), and each side of the head (14) Drive the two CCD cameras (χ 7) (18) with the third drive shaft (15) and the fourth drive shaft (16). Please refer to the second, third, and fourth pictures. The base ( 1 0) The first transmission shaft (1 1) provided on it can be rotated in a wide range, 'the rotation range is about 180 degrees', as a human neck can be rotated horizontally, and the second transmission cymbal (1 3) Partial rotation parallel to the horizontal plane can be done 'the range of rotation can be about one Baidu' as the head of a person can do pitching movement 'and the third transmission shaft (1 5) and the fourth transmission shaft (1 can be independently rotated about one hundred Eighty degrees, so that the camera can move left and right as if imitating the eyeball. The first transmission shaft (1 1), the second transmission shaft (1 3), and the third 6 paper method are selected from Chinese national standards. (CNS > A4 wash grid (210X297) * > (Please read the back (Please note this item and fill in this page) f. Central sample rate of the Ministry of Economic Affairs, India Industrial Cooperatives Cooperative Seal A7 B7 V. Description of the invention (f) — ^ The drive shaft (1 5) and the fourth drive shaft (16) are all motors Driven by fans and actuators, it mainly provides the entire mechanism for servo control. The motor and reducer are installed on the body of the mechanism, and the driving benefits of each motor are placed in the control box to drive the binocular mechanical head. The reducers of the four motor output shafts are all design devices of wok gear worms (wom gear b〇x). The main advantages are as follows: 1. The worm gear is a type of reduction mechanism, which can provide the mechanism output in position. The higher resolution is expected to achieve more accurate servo control. 2 · The most common problem encountered by general deceleration mechanisms is the backlash phenomenon, which often causes control problems, and the backlash phenomenon of the worm gear group is far away. Less than ordinary reduction mechanism β 3, MA It can increase the torque output after installing the worm gear group. 4. Because the worm gear group has the feature of non-reversible driving, it can increase the rigidity of the system. For each axis, the worm gear group is equipped with a reduction ratio (gear rati〇 The choice of ') depends on different considerations, as shown in the fifth figure, where the first drive shaft and the second drive shaft are considered for low speed and high torque output, because the entire mechanism requires the maximum output of the first and second drive shafts. In particular, the second transmission shaft must also consider the gravity factor, so the reduction ratio is the largest. Although the third and fourth transmission shafts only need to physically move the two CCD cameras, if the worm gear unit is added, because Can increase the torque of the output shaft, so you can use a smaller moving motor 'in addition to increase the resolution of the CCD for vergence. _ _7 This paper is applicable from the national Mingjia Kuangquan (CNS) 八 E # · (ⅩΙχ × 297mm) ~ ------!. _ ¥ i A ------ J ·. (Please read the notes on the back before filling in this page) ^: Λ 4 9 9 Α7 --- Β7 5. Description of the invention (What) In the control interface, binocular machinery The control interface used by the head mainly includes digital input / output cards, interrupt cards and decoding cards, as follows: 1. Digital input / output cards, digital input / output cards used by binocular mechanical heads can do up to 16 sets of digital input And 16 sets of digital output. The digital output part is used to do the pulse wave generated by the pulse wave generator 'will be sent to the motor driver, as the command input of the joints of the binocular mechanical head. The digital input function is used as light The detection of the interruption of the interrupter is based on β 2. The interrupt card is mainly using its intei 8254 chip for interrupt control of the control program. The interrupt frequency of this card can reach 10MHz at the fastest. 3. Decoding card ’is used to read the encoder information as the angle feedback of the joints of the binocular mechanical head, which can provide a 24-bit resolution count of each joint. Consumption cooperation between employees of the Central Standards Bureau of the Ministry of Economic Affairs (please read the precautions on the back before filling out this page) For the input command signal of the motor of each axis of the binocular mechanical head, in conjunction with the selection of the position control mode of the driver, pulse input The command is carried out. The main consideration is that when the binocular mechanical head performs active visual tracking, it hopes to have the characteristics of rapid positioning of stepping horses. In conjunction with the position control mode provided by the driver, it can not only meet the requirements of rapid positioning. It also has a resolution that ordinary stepping horses can't match. The control principle is shown in the sixth circle. The controller generates forward (CW) and reverse (CCW) pulse wave instructions to the driver. The deviation register inside the driver will then communicate with the motor encoder based on this pulse wave number. The number of feedback pulses is subtracted. This differential output is converted to an analog signal by D / A as an actuation command of the motor. This can prevent out-of-step phenomenon.

S The scale of this paper is applicable to the standard of 81 units in China (CNS) A4 specifications (210 X 297 males — Λ44499 Α7 Β7) Five Central Government Standards Bureau of the Ministry of Economy Λ Industrial Consumption Cooperation Du Yincai, Invention Description) (Please read the back Please fill in this page again for the matters needing attention) As for the pulse wave output part, you can use C language to write a program, cooperate with the chip of the interrupt card to do t-break, and use the digital output function to achieve the pulse wave command output. As shown in the seventh figure, the width of the pulse wave will affect the speed of the motor. The narrower the width of the pulse wave, the more frequently the frequency of the pulse wave is not generated, and the faster the motor speed. If the width of the pulse wave is fixed, the The speed will be fixed. The way to modulate the pulse width is to first give a fixed time Γ every time the motor reaches the expected positioning. If the number of pulse waves required by the motor is obtained by the control law, the pulse width is Δί = i .

The 2N binocular manipulator is a four-degree-of-freedom servo mechanism. It is used in conjunction with actuators and servo controllers to achieve positioning control effects. It can be used by the camera to perform pan, tilt, and vergence motions. In image processing, the main purpose of the visual system is to obtain the image information of the target object. The image information is obtained just like humans, and the environment must be sensed through both eyes, and then the stimulation of the visual nerves is transmitted to the brain, and then Obtain the required image information according to the rule of thumb to take corresponding measures, and the CCD camera and image processing unit are like the parts of human eyes and brain responsible for processing visual perception. The image processing system of the binocular manipulator is composed of two CCDs and an image processing unit. The image processing unit can be two or more, which can be planned as required. After the target is captured by the two CCDs, The NTSC video signal is output and then sent to the image processing unit through the signal line, converted into a digital full-color signal and stored in VRAM 'for the program to do digital image processing. If two image processing units are used in the image processing system, the HB furniture ratio <) Α4 size (2 丨 0 × 297 mm) applicable to 9 paper sizes is printed by the Central Laboratories Bureau of the Ministry of Economic Affairs. A7 ------B7 V. Description of the invention (^) ~ ~~~~ ~-The PC connected is called the Master image processing unit, and the other is called the Slave image processing unit. As shown in the eighth figure of the processing unit network 2, the main image processing unit and the slave image processing unit are respectively connected to the cameras of the left and right eyes of the double-headed mechanical head. As for the synchronization of the two image processing units, before processing the image information, The signal is sent from the main image processing unit to the slave image processing unit and then starts the operation, which can achieve the purpose of capturing the target image at the same time. As for the image processing steps, from the capture of the image of the spatial target by the CCD camera to obtaining the coordinates on the required image plane, many image processing procedures must be passed. The parallel C programming language can be used to make the image processing steps in the image. In the processing unit, in order to obtain the image information of the space target, the image processing steps can be roughly divided into four parts: video conversion, target identification, noise removal and finding feature points: 1. Video conversion: After the target is shot by the camera, Image processing unit The image processing unit converts the image analog signals it sends into digital signals of the three primary colors of RGB, which is the so-called true color mode. »Each pixel is stored in three bytes. One pixel value 'and each primary color occupies 8 bits.

2. Target identification: In the method of identifying multiple targets, the traditional method is to use geometric figures through neural-like identification or cooperate with other identification rules. However, it often takes a large amount of computing time and is not suitable for active visual tracking. You can try to distinguish by the color of the target. For red targets, the R 10 paper size applies to the Chinese national standard (CNS > A4 size (210X297 mm)) (Please read the note on the back first) (Please fill in this page again)

Printed by the Central Ministry of Economics and Consumers ’Cooperatives« L 444 ^ 99 A7 __B7 V. Description of the invention (^) The value will be higher than both the G value and the B value, so when scanning the image, for each pixel, first compare The size of the R, G, and B values is taken as the largest one. If it is the background, the pixel value is set to zero. Because the R, G, and B values each occupy 8 bits, this way, not only can it be quickly identified. Targets of different colors, and each pixel value is only 8-bit size "3 · Remove noise (noise): For any visual system, noise interference of image information is inevitable, and image information Accompanying noise often severely affects the performance of the system. "Because noise is scattered in the image as white spots, according to this feature, the entire image can be scanned in a small window. When the pixel value of a point is less than a threshold, the point is regarded as the background and the pixel value is made zero; otherwise, this point may be the target or noise. The method of judgment is to this The neighborhood of the point for inspection, if there is no other super in the neighborhood Tan door point value, or has but few, i.e. noise it is determined at this point, and allowed the pixel value of zero 'thus can effectively eliminate the problem of image noise. 4 · Finding feature points: After identifying the target and removing the noise, in order to obtain the position of the target on the image plane, the centroid of the target can be used as the feature point. The method is to add the image coordinate values of each pixel of the target, and then divide by the total number of pixels of the target ^ LT. For the image information captured by the left and right eyes, the two images are processed in the two papers respectively. «National Standards (CNS) A4 is now available (210x297 mm). (Please read the precautions on the back before filling this page. )

V Order in the Ministry of Economic Affairs and Economics_Printed by the Consumers 'Cooperatives of the Lost Prospective Bureau 4 444 9 9 Α7 Β7 V. Description of the Invention (/ 〇) ~~ Perform the above steps in the unit' Each image processing unit image processing process As shown in the ninth figure, 'the centroid coordinates of the target in the image plane obtained by the two image processing units are finally transmitted to the PC. In order for the binocular mechanical head to achieve the purpose of monitoring, the present invention proposes a control architecture 'as shown in the tenth figure, the entire & control architecture includes the 螓 eye_ &% subsystem, image processing subsystem, visual tracking control And predictive write. Among them, the visual tracking controller part adopts a SOIM (SeH-〇rganizing Invertible Map) neural network architecture. The advantage is that learning through neural-like can avoid tedious correction work, and the purpose is to learn a spatial expression / (spatial representation), this expression is a three-dimensional vector, which can uniquely represent the position of the target in the working space, and will not change due to the change of the camera attitude. The SO IM network uses the image coordinates of the target and The joint angle of the servo mechanism is input and output. In addition, based on the consideration of real-time control, the predictor uses the gray prediction method to predict the trajectory of the target. In the visual tracking controller part, the binocular mechanical head The definition of the joint angle is shown in the eleventh circle. '丨, & respectively represent the angular displacement of the left and right joints' is called the vergence angle; ^ is the angular displacement of the first axis joint, which is called pan angle' this joint Can be fixed; & represents the angular displacement of the second axis joint is called the tilt angle. The three joint angles of name, & and & can be independently controlled by the computer to track the target in space. When the intersection of the optical axes of the two cameras falls on the surface of the target, the target is considered to be locked (Fixated) 'Its image falls on the center of two image planes at the same time. 〇12 This paper size applies the Chinese national standard (> A4 specifications (2 | 297X297)) II: W 1 order ^ ----- -(Please read the “Notes on the back page before filling out this page”) WC Industry Consumer Cooperation Du Printed by the Central Bureau of Standards, Ministry of Economic Affairs 444499 A7 _____B7-~ ——------ V. Description of Invention (^ /)- When performing active visual tracking, in order to lock the target, the camera changes the state of its mechanism as the target moves, and the position of the target in space can be derived from the spatial geometric relationship according to the joint angle of the camera, but this requires Accurate calibration of the binocular manipulator system, but neither external mechanism parameters nor internal image parameters can be easily calibrated accurately. The following is a description of the soim neural network. And explain its reversibility: ( ―) The main spirit of the S0IM network architecture is to learn a mapping relationship. The input of this mapping is the image coordinates of the target in the left and right eyes and the joint angle of each axis of the binocular manipulator. The output is a spatial expression. This spatial expression The formula can uniquely describe the position of the object in three-dimensional space. Soim is a three-layer network architecture, as shown in the twelfth circle. The first layer is the input layer (input layer), which contains two different forms of input : (1) Image coordinates input: f represents the coordinates of the spatial target projected on the left and right camera image planes respectively, and W ,, V ,, & have been normalized to 0 to 1 (2) Angle vector input: 殳 一 ～ 心 'corresponding to the instantaneous angle of each sleeve of the binocular mechanical head' and normalized between 0 and 1, where min, & are the three- and four-axis joint angles,% Is the joint angle of the second axis. The second layer: The image coordinate input part first passes the fuzzy adaptive resonance of the second layer. 13 This paper size is based on the Chinese national standard (CNS) A4 (210X297 mm) 1 ^; V 1 J ^ 1 Please read the back Note: Please fill in this education.) 444499 Central Bureau of Economic Affairs, Ministry of Economic Affairs, W: Industrial and consumer cooperation, Du Yinzhong A7, B7 V. Description of invention (/ >) fUZZy adaptive resonance theory network for classification processing The adaptive resonance theory network belongs to unsupervised learning. The biggest difference from the traditional ART1 network is that ART1 can only accept binary input, while fuzzy art can accept binary and continuous input. The input of analog, the input type of the present invention is any value between 0 and 1. The fuzzy adaptive resonance theory network has the following two biggest characteristics: (1) stability: when new things are input, old things can be properly retained "(2) plasticity: when new things are input, they should be quickly To learn. However, there is some conflict between stability and plasticity. The adaptive resonance theory network uses a vigilance test to solve this contradiction. The basic principle is: (1) If the characteristics of a new thing and an old thing Have similar characteristics (that is, pass the guard value test), then only modify a part of the memory of the old thing in the system, so that it can meet the characteristics of the old and new things at the same time 'make the old things can be properly retained, so that the stability of the Claim. (2) If the characteristics of the new thing are not sufficiently similar to the characteristics of all the old things (that is, the guard value test cannot be passed), the system establishes a brand-new memory for this new thing to quickly learn the new thing. Meet the requirements of plasticity. From the twelfth figure, the fuzzy adaptive resonance theory network can be divided into six and six parts and connected by weights. With regard to its learning rules, the left eye can be used for 14 papers using the Chinese national standard (CNS > (210X297 public envy) — --- — (Please read the notes on the back before filling in this page)

-1T t-i. 444 ^ 99 A7 B7 V. Explanation of the invention (/ multiple) Examples (the right eye and the left eye are the same), the following are listed 12 4 The initial value of the weight (W) is set to L 〇. The input vector (Α, ν,) is increased to a four-dimensional vector, and the purpose is to prevent the classification expansion problem (category proliferation problem). According to the four-dimensional input vector of the K layer, each node (node) of the full layer is found to have its four matching values f. \ ^ AWf \ '" «ψτΤ where α is a constant approaching zero β Λ represents fuzzy AND' (xA > 〇, .Smin (u). 2M 1 j ·! Is defined as | x | = 'M is X J > * 1 Select the node with the largest matching value. If there is a node with the same matching value, then select the small node.// Check the knowing point // whether the vigilance criterion is met as follows: (Formula A .) Seal of the Central Government Bureau of the Ministry of Economy

PLj X1 ΆίΓ; XL ^ PL (Form B) where pt is a vigilance parameter β 〆 represents the similarity between the input and the clever layer's • / th classification node. 6 · If the / node passes the alert value test, then modify the weight value of 15 paper waves scale using the China II family standard rate (CNS) Α4 specifications (210 X 297 mm) (Please read the note on the back first (Fill in this page)

444499 Λ7 V. Description of the invention The learning rule of C ′ is W = | xW | (to) 7. If the node / fails the guard value test, the node j is dropped and the process returns to step 3. If there are no nodes in layer 6 that can pass the guard value test, a new node is generated, and the process returns to step 5 to set the initial values of the weights of the new node and the f-layer connection to 1.0. "Therefore each set of input vectors P and y corresponds to the classification generated by the fuzzy adaptive resonance theory network. The third layer: for the output layer, a spatial representation is generated / = [/ p / 2, / 3] / = 1,2,3 of which (D type) (Please read the precautions on the back before filling out this page), 11 The ruler of the Central Government Bureau of the Ministry of Economic Affairs prints the ruler, / for both left and right After the eye image input passes the fuzzy adaptive resonance theory network, the corresponding classification nodes β <, 4 are the connection weights of the clever classification layer and the output layer, and the initial value is set to a random value between 0 and 1. In each learning cycle, each set of input vectors can correspond to a set of outputs / such as (D type), and the most important work of the third layer is the learning space expression J, which is characterized by the process of learning is self-organized ( seH_ organized) method. For a fixed target in space, '/ t and eight' are two consecutive space expressions obtained by (D formula), and / * and // calculated for the same position in space should be the same , But because the initial value of < 4 is 0 Lemon rate (CNS) Α4 now grid (2 Shu 0) < 297 A7 B7 five well toss

Du Yin A, Consumer Cooperation of the Central Bureau of the Ministry of Economic Affairs, Consumer Instruction (Random value between / 51 and 1, so 4 and // will not be the same, and 4 and β must be modified. Its learning rule is: (Please first 闳Read the notes on the back and fill out this page) ^ = r (i, -ii) pj (E formula) where the parameter γ represents the learning rate. The reason why this learning is self-organizing is because (/ — < ') Is self-generated (sel f-generated) by the network in the process of learning 1. The learning architecture of the SOIM network application in the binocular mechanical head is shown in Figure 13' in offline learning (〇f f-1 ine learning In the process, aiming at a fixed target in the working space, the axes of the binocular manipulator are randomly rotated to a position, and then the image of this target is taken by the CCD, and the joint angles and the images of the binocular manipulator are stored. After processing the image coordinates, iteratively repeats the above actions to the desired learning times, and then inputs the stored data to the S0IM network. Each input can correspond to a set of rotations ^, according to the (E formula) learning rule ( learning rule) to adjust the weight values 4 and 4, this is fixed The spatial expression (/) of the target object will eventually converge to a small interval, and the weight and /) after learning will be recorded to complete the learning of S0IM. (1) The invertible property of the S0IM network can be represented by (F-type). For any point in space, within the range that can be captured by two cameras, a set of spatial representations can be obtained through the S0IM network. formula/. The reversibility of the S0IM network means that the output angle of the image server can be calculated so that the spatial target is imaged at any specified position in the image plane. Therefore, if we determine the total image coordinates of the target, then the image The corresponding angle output of the servo mechanism can be obtained by (F-type) 17 papers ΛΛϋ 财 __Household material (CNS) from the threat (21 () > < 297 公 # > Central Bureau of Standards, Ministry of Economic Affairs β: Industrial Consumption Printed by the cooperative Λ7 B7______ 5. Obtained from the & Description of the invention. ((M) where / 'and f represent the output of the corresponding image coordinates A in the fuzzy adaptive resonance theory network. The purpose of the predictor is to predict the trajectory of the moving object to compensate for the time lag in the operation of the controller and the movement of the mechanism. Because the trajectory of the target is irregular, it is difficult to establish its motion model, so gray can be used. Theory to predict the trajectory of the target. For an engineering system, observe its behavior and find that the internal and external environment of the system have fairly clear input and output relations. When the system is connected, the so-called mapping function can be used to represent this relationship. 'The internal and external relationship information of this type of system is completely clear.' It is called a white system. The opposite is called a black system if it is not clear at all. The relationship information is partly clear and partly ambiguous, so it is called a gray system. Generally, in the process of engineering problems, the system is modeled first, and then the behavior of the entire system is studied based on the dynamic characteristics of the system. Promote the performance of the system with appropriate control theory. This process of establishing a mathematical model based on the relationship between various state variables in the system, and then starting to analyze and control the system is called "the forward process"; otherwise, If you first know the system's dynamic characteristics data and data, and then build its system model, this process is called the "reverse process". For the process of model establishment, the input and output data acquisition includes 18 books-paper "LAA 遑" Using China • National Rubbing Standard (CNS) from the state (ϋ297 mm --------- — ^ -------- ^ ------ ^ ir-T ----- Λ'-Γ (Please read the note on the back first Please fill in this page again for details) 44449 ^ Printed by the Ministry of Economic Affairs of the Central Bureau of Associate Bureau, Consumer Cooperatives, A7 B7 V. Description of Invention (/ 7) When there are certain uncertain factors, the original system is modeled as gray "Inverse process" and the model created by this inverse process is called the Grey Model (referred to as GM. The gray model expresses the continuous behavior inside the system.) So the gray system theory establishes a differential equation model, It is not a difference equation model established by general system identification. In a broad sense, for the "order; gray model of the z-variable (Gray Model), it is said that different" have different meanings from Λ's GM model Common uses of gray models are: 1 · GM («, 1) model (prediction model)«-generally less than 3 ', that is, the gray model of the system is below the third order, «The larger the greater the meaning of the system, but because The huge amount of calculations causes a loss in calculation time, but it is often not suitable for dynamic real-time prediction. The GM (1,1) model with "= 1 is simple to calculate, but it cannot reflect the process of disturbance ', so it also has the effect of crossing waves. 2 · C? M (1, A) model (state model) GM (l, / i) model can reflect the influence of other (A-ι) variables on the first derivative of a variable, but this requires & amp Time series, and must do as much analysis as possible in advance to determine which factors the time series should be included in this variable. 3 <? M (0, Α) model (static model) «= 〇 'means that the time factor is not considered, so it is a static model 〇 The present invention uses the GM (1,1) model in the experimental stage, The (^^, 丨) template paper wave scale is applicable to China's a family rate (CNS) A4 specification (210X297 mm) (Please read the precautions on the back before filling this page)

V order-444 4 9 9 Α7 Β7 (/ f: V. In the invention description type ("k 1) 'GM (U) model has the advantage of the shortest operation time and meets the requirements of real-time control, because it is a first-order prediction, so If it does not reflect high-frequency interference (please read the precautions on the back before filling this page) — there is a disturbance at an obvious location or track, and you do n’t need to make a detailed forecast, just the obvious location (Nominal location) prediction. Before performing gray modeling, you must first explain some basic definitions in gray theory. 'For a set of original sequences such as (G formula), net, =:;: 定义 First define (G The operation of an accumulation sequence of (formula) is like (η formula): ^ = (Η) For the sake of convenience, let's write the one accumulation sequence of (G) as (I) The price (I) Then define the meaning The sub-decrement sequence of ^ is as follows: (j-form): ί = 1,2,.,., ΛΓ {au \ Xil \ i)} · = (J-form) «/ = 1, 2". · ，， "Among them, printed by the Central Government Bureau of the Ministry of Economic Affairs, Λ Industrial Consumer Cooperative, α⑴« ,,) = α (0) (〇-α (0 > (Especial 1) " · -1), W, 0 = αϋ · Κ, /)-, ι · -1) Basically, the general form of (? Λ / (1,1) is a first-order single-variable differential equation. I can be shown as follows:楳 Ratio (CNS) 8 4 specifications (210X297 mm >) V. Description of the invention (⑴ {f dt axt0 = uu, A7 B7 ux = (〇 = 〇⑴ (0)-: Ka, +-aa (K formula ) Α) In order to realize gray modeling with a computer, considering the convenience of numerical calculation, the first-order differential equation representing GM (1,1) was changed to an incremental type. Available: Δχ (1) (ί) + αχ (1) (ί) = u (M formula) cr0) (jc0 > (/), / t) + three (χ (1) (/ 5γ) + λγ⑴ (丨 _1)) = «(Ν formula) 2) (female) + αζ () (moxibustion) = w (◦ formula) (Please read the precautions on the back before filling this page) > where z (!) ( ^) = ^ (x {,) (A :) + x (,) (/ tl)) So we can get discrete 飨 should look like this: χ (ι) (众 +1) = 〇 (0 > (1 ) — +-Aa —— Order the printing and production of Papier consumer cooperation of the Central Bureau of Standards of the Ministry of Economic Affairs (P-type) (Q-type) For the parameters α and w in GA / (1,1), the identification method is generally Least square (Least square method), the matrix formula is as follows (R formula): ά = (ΒτΒ) ~ 'ΒτΥκ (r formula) Among which 21 paper sizes are applicable to China National Kneading Rate (CNS) Α4 specification (2l0x297 mm). «£ The Central Ministry of Economic Affairs ΛΛ 工 消 t Cooperative Seal *-A7 _____B7 V. Invention Description θ) aά-， u「 -ζπ) (2) Γ R-^ υ (3) 1 iS = > -zw ( n) \ Γχ (0) (2) Ί r ^ < 〇 > (3) YN = For the above derivation 'For example, the following shows how to build a model based on a given set of data: No. ⑺ 1 2 3 4 xco > (0 9 0 8 4 7 1 6 0 According to (R formula), after the gray modeling process, [a, μ] = [0.1678,106.1556] can be obtained by substituting the values of α and μ into (P formula) The discrete response of GA / (1,1) is as follows: X ⑴ (A: +1) = (90 — 032.6317) / stomach + 632.6317 The result is the sequence number ⑺ 2 3 4 original χί1) (〇174 245 305 Estimate χω (〇173.824 244. 699 304.626 From the comparison of the data χ (1) (〇 and ί (ι) (ί_), we can see that the accuracy is quite good. ’22 (Please read the notes on the back before filling in this tile)

V-Book Paper Family (CNS) (2 丨 〇) < 297 gong) 444499 A7 B7 5M Printed by the Ministry of Economic Affairs of the Central Government Bureau of Industrial and Consumer Cooperatives, Description of Invention (~ y) Please read the back of Jing Matter t. Write w This _ page. With the cooperation of binocular manipulator, image processing system and vision tracking controller, a four-degree-of-freedom binocular manipulator can be used to perform active visual tracking tasks. Vision feedback, and obtain excellent stereoscopic vision perception. Therefore, this binocular manipulator monitoring system can be applied to various mechanical transmission mechanisms. If the binocular manipulator monitoring system is coordinated with a robotic arm, that is, Form an active vision robot system. The active vision robot mainly uses a binocular manipulator monitoring system to track a moving target, and at the same time, feedbacks its image information to enable the robot arm to perform grasping tasks. Its control architecture As shown in Figure 14 ^ Order the system control architecture shown in Figure 14 can be divided into two parts, the upper part is the grab control of the robot arm, and the lower part is the binocular robot head monitor The tracking control β robotic arm of the control system performs grasping tasks through visual image feedback, while the binocular robotic head is dedicated to visual tracking of space targets. The entire control architecture includes a robot arm subsystem, a binocular robot head subsystem, an image processing subsystem, and a vision tracking controller and predictor. In the control part of the robotic arm, the image information of the robotic arm and the target is simultaneously captured during the visual feedback process to obtain an individual spatial expression, and the error (Δ7) is used as the feedback signal. The difference is the target There is a gray prediction but the robot arm does not, that is, the error (that is, the error between the current robot arm space expression (jr) and the target object space expression (/,) at the next moment. The input command for the robot arm control is- The expected error (A / d) is subtracted from the feedback signal (μ-μ), and then the robotic arm is driven by the robotic arm controller to capture the target β. To prove that the present invention and its application can be implemented, it is specifically targeted to both eyes Machine 23 CNS) aW (2.0X297 ^ *) Λ44499 Printed by Aberdeen Consumer Cooperative, A7 B7, Central Bureau of Economic Affairs, Ministry of Economic Affairs 5. Description of the invention (»Mechanical head monitoring system and active vision robot system for operation experiments, first with both eyes Manipulator positioning control experiment to test the performance of the binocular manipulator. 'Another SOIM network learning experiment, the binocular manipulator learns its spatial expression at a fixed point in space' After learning all the weight values in the network, then perform active visual tracking experiments, and finally integrate the binocular manipulator with the robotic arm to perform an active vision robot system grab experiment. 1 · Binocular manipulator positioning control performance test both eyes The control principle of the manipulator is to output the pulse wave from the PC to the drive of each axis to drive the motor to the position. 'This experiment will perform angle positioning control of each joint angle of the binocular manipulator' and observe its performance in time domain response. In the experiment, the PCL-818 card is used as an interrupt program to control the frequency of pulse wave output. The pulse wave output can reach 50 KHz at the fastest. The pulse wave output with the highest frequency is used as the input of the motor of each axis of the binocular mechanical head to perform positioning control of 20 · for each axis motor, and the sampling time (samp) _ing time is 5xl (T5 sec ' The time domain response is shown in Figure 16, Figure 17, Figure 18, and Figure 19. 2 · S0IM network learning experiment The method of S0IM network learning is to place a black background board in the double Eye machine In front of the robot head, a fixed white small block on the board is used as the target of the binocular robot head. The image of the target object captured by the camera can be obtained after image processing ^ its: the coordinates of the centroid on the image plane, the image plane The size of the set is 400x400 (^ _, ^) 2. The S0IM network training data is generated by making the two, three, and four axes of the robot head within a limited range of 24 CNS > (210X297 public漦 J ： {Please read the notes on the back before filling this page)

* 1T Printed by the Central Government of the Ministry of Economic Affairs and Consumer Cooperatives 4 TU 4 9 9 A7 ----——__ B7 V. Description of the invention (¾ ^) ~ --- 8000 random rotations ^ ^ The axis is solid.) The rotation range of each axis is shown in the twentieth chart, and the image coordinates of the target and the target angle of the binocular manipulator are recorded. Road learning training materials. As for the parameter setting of the network, the alert parameter (vigilance

Param6ter) ^ = ^ = 0.88, learning rate Uearrnng rate) Wide 0.39 ’parameters. During the learning process, the relationship between the change in the expression of the fixed object space expression and the number of times of learning is shown in Figure 21, Figure 22, and Figure 23, respectively. According to the results, all three components of the spatial expression 7 converge to about 0. 3. Active visual tracking experiment The active visual tracking experiment uses the SOIM network as a visual tracking controller and uses a binocular manipulator to track a moving target in the space without trajectory planning, as shown in the twenty-fourth circle. It is a schematic diagram of its hardware architecture. If the binocular robot head can successfully track this target, the target will be imaged in a fixed area of the left and right image planes, and the sampling time for image processing is 0.26 seconds . In the experiment, the desired coordinates of the target object on the left and right image planes are set to (200, 200), and the time required for the binocular manipulator to reach the positioning position is set to 0-5 seconds. The variable pulse wave output frequency is used as the control signal of the binocular manipulator. The target moves at a movement rate of about 5 cm per second within the range that the binocular manipulator can see. While the binocular manipulator performs active visual tracking, Recording the coordinates of the two left and right image planes. From the 25th circle and the 26th figure, it can be found that the binocular mechanical head can maintain the imaging position of the moving target in the middle of the image plane in the left and right eyes. , Figures 27 to 30 list the moving targets 25 (please read the notes on the back before filling this page) Order 'tBB material rate (CNS) for this paper AAii (210X297 mm) 83.5 5,000 0.88 444499 A7 B7 Economic Central Bureau X. Consumption Cooperative Association Printing%: V. The change of objects in the image plane W and V direction with time, it can be seen that the binocular mechanical head has a good time in active visual tracking. Domain response4. The binocular manipulator monitoring system guides the robotic arm to grasp the experiment. The hardware architecture diagram of the entire active vision robot system is shown in Figure 3-as shown in the picture. 'The binocular manipulator is responsible for obtaining image information and active vision tracking. For vertical tasks, the image processing part is completed by the image processing unit, and the robotic arm performs the grabbing action. The main task of the computer is to communicate the binocular robotic head subsystem with the robotic arm subsystem, including driving the pulses of the binocular robotic head. The generation of wave instructions, the realization of the SOIM network and the operation of gray prediction. The visual guidance robotic arm capture experiment uses the image information obtained by the binocular manipulator as a feedback to guide the manipulator to the target location. The robotic arm subsystem controller includes a traditional PID controller and a method of calculating torque combined with a neural network controller. The parallel processing is performed by a calculator network, and its sampling time is 3.0 ms. The image processing subsystem part is processed by the image. The unit network architecture is responsible for all processing and calculations on the image. The sampling time is 0.26 seconds. The target object moves at a rate of about 2 cm / sec, and the expected value of the space coordinate expression of the robot arm terminator and the target object is set. for. The binocular manipulator performs active visual tracking of moving targets, but the robot arm terminator must be kept in the visible range. In the experiment, the target and the robot arm terminator are on the left and right eyes of the binocular manipulator. The changes of the coordinates are shown in the position change diagrams of the thirty-second and thirty-third centuries, respectively. Therefore, from the above experiments, it can be known that ‘the binocular manipulator monitoring system of the present invention is applied to a mechanical transmission mechanism’ and can obtain excellent monitoring results, especially in the case of (please note the precautions on the back before completing this page)

V 26 A7 B7 V. Description of the invention A four-degree-of-freedom double-eye mechanical head, combined with two (: CD cameras, can obtain erection vision as two eyeballs', where the CCD can be set on the third and fourth axes of the mechanism The camera 'does a movement similar to that of an eyeball that can be turned to the left / right', while the first axis and the second axis provide neck-like functions, so with the image processing system, you can obtain excellent monitoring results. In summary, The specially designed binocular manipulator monitoring system of the present invention can provide a system with excellent monitoring effect, which can complement and multiply the automation system. Therefore, the design of the present invention is very industrially applicable and should meet the requirements of the invention patent. The application was submitted in accordance with the law. (Please read the precautions on the back before filling out this page} r Printed by the Central Laboratories of the Ministry of Carriage and Labor Cooperatives Du Yin «. This paper size is in accordance with China National Standards < CNS) A4 Specification ( 210X297 mm

Claims (1)

444499 A8 Ββ C8 D8 ---— 1 W 3 days to amend and supplement the patent application scope No. 8112 197r binocular manipulator monitoring system ”Application for patent scope amendment 1. A binocular manipulator monitoring The monitoring system is composed of a binocular manipulator's vision tracking controller and an image processing system. The binocular manipulator has a base, and the top of the base drives the upper rotation base with a first drive shaft. The top of the rotation base uses a second drive shaft. Pivot joint, and drive the two cameras with a second drive shaft and a fourth drive shaft on both sides of the head, cooperate with the image processing system to predict the motion trajectory of the target, and use a som neural network as the framework. The visual tracking controller monitors the moving target and provides a system with three-dimensional visual monitoring effect: The aforementioned SOIM is a three-layer network architecture, which is used to learn a mapping relationship. The input of this mapping is the target's Left and right. The image coordinates of the eye and the joint angle of each axis of the binocular manipulator are output as a spatial expression. This spatial expression can uniquely describe the target in the working space. In position; of the image processing system may be two CCD and an image processing unit configured; the motion trajectory of the object belongs to the gray differential equation model of a predictive model to predict the order. 2 ♦ The binocular mechanical head monitoring system described in item 1 of the scope of patent application, in which each drive shaft is driven by a motor, reducer and driver. 3. The binocular mechanical head monitoring system as described in item 2 of the patent application, where the reducer can be designed for worm and worm gear sets. 4 _As the binocular manipulator monitoring system described in item 1 of the scope of patent application, it enables the binocular manipulator to perform pulse wave input through the control interface (please read the precautions on the back before filling this page}-Order The Ministry of Commerce " 'Cai 4 ^ Employees Co-operative Cooperative Co., Ltd. Printed Good Paper Standards Applicable to Chinese National Standards (Oyang) 8 Specifications (2 丨 0 Fathers 297 Public 癀) Α8 Β8 C8 D8 Scope 5 · According to the binocular manipulator monitoring system described in item 4 of the scope of patent application, the control interface must be a digital input / output card, an interrupt card, and a decoding card (please read the precautions on the back before filling out this page). Printed on the paper by the Bureau ’s Consumer Cooperatives. Applicable to 11 countries in China (CNS) A4 # L grid (2I0X297 mm)