JP2011010109A - Image transmitting system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve such a problem that, for locating a plurality of cameras in a robot arm to reduce the number of image transmission lines more than the number of cameras, an image signal is required to be multiplexed in a single transmission line, and such multiplexing as in a conventional example causes to deteriorate the transfer efficiency of image data, and in addition, an image photographed by the plurality of cameras at the same time cannot be obtained, and furthermore, a circuit is complicated in an image data division system.SOLUTION: This system includes: an image transmission line selecting means and a sending means for selecting the image data of a plurality of image transmission lines input for each frame and sending the image data to the single transmission line for multiplexing; a write circuit and an image memory for selecting the image data of the plurality of image transmission lines and writing the image data in a memory for each frame; and a readout circuit for reading out the image data of a required portion. Therefore, the image transmission lines are reduced in number while transmission efficiency can be improved, and also the image data photographed at the same time can be used for image processing.

Description

  The present invention relates to improvement in the efficiency of image data transmission through a single transmission path when two cameras are mounted on one robot arm, and simplification of the circuit configuration.

  When you want to perform tasks that require precise movements, such as assembly of optical equipment, automatically and quickly with a robot arm that has fingers for work, the relative position of the workpiece with respect to the robot arm is used as three-dimensional information. It is necessary to grasp accurately. In order to obtain three-dimensional information, it is necessary to install a plurality of cameras on one robot arm.

  A signal related to the image obtained by the camera is transmitted to an image processing apparatus in a computer for controlling the robot arm through a transmission line.

  When a plurality of cameras are installed on one robot arm, the number of image transmission paths for transmitting signals related to the images to an image processing apparatus that performs image processing is required for the number of cameras.

  If the transmission line is passed through the robot arm, it is necessary to secure the transmission line inside the robot arm, which increases the size of the robot arm in the radial direction. If the transmission line is arranged so as to cover the exterior of the robot arm, it becomes a factor that hinders the movement of the robot.

  For this reason, conventionally, as disclosed in Japanese Patent Application Laid-Open No. 2003-274374 (Patent Document 1), a signal related to an image is multiplexed and transmitted using a multiplexer to reduce the image transmission path.

JP 2003-274374 A

  However, the technique disclosed in Patent Document 1 can reduce the image transmission path, but simply transmits image information alternately. Therefore, when transmitting a plurality of image data sequentially, there is a drawback that it takes a long transmission time. It was.

  Furthermore, in the technique disclosed in Patent Document 1, images taken by a plurality of cameras cannot be simultaneously sent to the image processing apparatus, and when performing image measurement by the stereo method, a time difference corresponding to the frame transmission time between the images of the plurality of cameras. Will occur. For this reason, there is a possibility that the stereo measurement accuracy may be lowered due to the photographing position being shifted due to the vibration of the apparatus.

  Therefore, the present invention provides a technique for transmitting image data having a large amount of information compared to the bandwidth of the transmission path without increasing the transmission path or using a complicated circuit.

To solve the above problem,
In an image transmission device that transmits image data acquired by two imaging devices through a single transmission path,
At least one selection circuit that selects one of the image data acquired by each of the two imaging devices;
A transmission circuit for transmitting image data to a single transmission line;
A memory for storing the other image data;
A readout circuit that reads out the other image data stored in the memory and transmits the image data to a transmission circuit,
All of the selected one image data is transmitted to a single transmission line, all the other image data is stored in a memory, and after the one image data has been transmitted, the other image data is transmitted. An image transmission device is provided.

  As a result, according to the image transmission apparatus configured as described above, a plurality of image data can be transmitted through a single transmission line without using a complicated circuit.

  Note that the image data can be transmitted at this time using a clock of a high-speed clock oscillator, and the transmission efficiency of multiplexed image data on a single transmission path can be further increased. Also, the image data selected and transmitted above and the image data transmitted after being stored in the memory are taken at the same time, and are less susceptible to errors caused by fine movement of the imaging position for each camera due to the vibration of the apparatus. Therefore, deterioration of stereo measurement accuracy can be prevented.

It is a block diagram which shows the structure of the robot by embodiment of this invention. It is an internal block diagram of the image processing apparatus which receives image data and performs image processing It is a figure explaining the internal structure of an image transmission apparatus. It is a flowchart figure explaining operation | movement of the multiplex transmission process of a several image. It is a timing chart figure explaining operation | movement of the multiplex transmission process of a several image.

  FIG. 1 shows a configuration example in which two cameras are mounted on a robot arm equipped with a transmission apparatus according to the present invention.

  In the robot arm 1, a single transmission path 12 is wired in the robot arm 1. The robot arm 1 is drawn from the tip of the robot arm 1 to the outside of the arm and connected to the image transmission apparatus 2. The single transmission path is taken out from the robot arm. It is connected to the image processing device 8.

  Both the image data transmission path 10 of the camera 1 and the image data transmission path 11 of the camera 2 are connected to the image transmission apparatus 2.

  The format of the image data signal transmitted to the image data transmission path 10 and the image data transmission path 11 may be a camera link standard signal, for example. The robot control device 9 controls the robot arm 1 to perform a predetermined operation. The robot control device 9 is connected to the image processing device 8 and transmits / receives data to / from the image processing device.

  The illumination device 7 is controlled by a signal from the image processing device 8 and illuminates the workpiece 6 when an image is captured by the camera 1 or the camera 2. The robot control device 9 controls the robot arm 1 according to a program that prescribes an operation created in advance, positions the workpiece at a position where the image of the workpiece can be acquired by the camera, and controls the image processing device 2 to perform image processing of the workpiece. Order through. When receiving the command, the image processing apparatus turns on the illumination and transmits the image transmission command to the image transmission apparatus through the control line 1.

  The control line 1 is, for example, CAN, RS232C, or camera link. When the image transmission apparatus receives the image transmission command, the image data acquired by the camera 1 and the image data acquired by the camera 2 are sequentially transmitted to the image processing apparatus via a single transmission path as described later.

  Upon receiving the image data, the image processing device 8 performs image processing based on the image data of the camera 1 and the image data of the camera 2 using a known stereo method, and calculates the three-dimensional position of the work 6. Information on the calculated three-dimensional position is transmitted to the robot controller. The robot control device 9 transmits and controls a signal for positioning the robot arm 1 or the finger 5 according to the received three-dimensional position information. Since the robot arm 1 or the finger 5 is controlled based on the calculated three-dimensional position information, the workpiece 6 can be accurately and accurately positioned.

  In FIG. 1, a state in which a single transmission path 12 for sending image data is arranged inside the robot arm 1 is schematically shown. However, in addition to this, an arm is driven inside the robot arm 1. Power signal line to actuator, feedback signal line for actuator position information from actuator to robot controller, sensor signal line from finger 5, air piping to arm tip, etc. It is very difficult to increase the number from the viewpoint of space and cost.

  FIG. 2 is a block diagram of the image processing apparatus 8 according to the embodiment. The image data receiving circuit 31 is connected to the single transmission line 12 and stores the image data transmitted from the image transmission apparatus 2 in the memory 32 and stores it. The communication circuit 35 transmits and receives commands to and from the robot control device 9 via the control line 2. The communication circuit 34 transmits and receives commands to and from the image transmission apparatus 2 via the control line 1. The CPU 33 is a central processing unit that controls the entire image processing apparatus and performs image processing.

  FIG. 3 is a block diagram of the image transmission apparatus 2 connected to the camera 1 and the camera 2 according to the embodiment.

  As for the control circuit 27 that controls the circuit in the image transmission apparatus and the camera, the control circuit 27 schematically illustrates only the connection wiring to the communication circuit 3 and the trigger signal to the camera. (Wiring to individual circuits in the image transmission apparatus is not shown).

  The selection circuit 1 (21a) has a plurality of image data transmission paths for receiving image data, and receives from one image transmission path of the image data transmission path 1 or the image data transmission path 2 input by an instruction from the control circuit 27. The selected image data is selected, and all the image data is sent to the transmission circuit 22. The transmission circuit 22 sends image data to the single transmission line 12 with a signal specification (for example, LVDS specification) suitable for the transmission line.

  The selection circuit 2 (21 b) transfers all the image data received from the image transmission path not selected by the selection circuit 1 (21 a) to the writing circuit 23. The writing circuit 23 writes the received image data into the memory 24.

  The readout circuit 25 reads out the image designated by the control circuit 27 from the image data written in the memory, reads out the image data transmission path 25a as the readout image data in the format of a predetermined image data signal, for example, as a camera link standard signal signal. To the selection circuit 1.

  When performing work with the robot arm 1, when there is one piece of image data captured at a certain timing, it is not always necessary to specify the three-dimensional position of the image information of the entire region. This is because when performing an assembling operation, if only the three-dimensional position of a specific region is known in detail, the other regions are often unrelated to work.

  For example, in the operation of inserting a shaft-shaped workpiece into the pore, the position of the pore and the position of the tip of the shaft need only be specified. Therefore, if the three-dimensional position of the pore and the shaft is specified, other members The three-dimensional position is almost unrelated to the work.

  In the case of work that requires only the three-dimensional position of a specific area, only the image data of the specific area of the image data written in the memory 24 is partially read out and transmitted to the selection circuit 1. Good. When the actual work is performed, the robot arm 1 repeats the operation of the same cycle. Therefore, in a specific work in one cycle, the position of the camera and the work placement position are often almost fixed. Therefore, the necessary image data area is determined in advance by determining the image data of the necessary area in the image data written in the memory 24 such as “only the left half” or “upper right 1/4” of the captured image data. It is easy to keep.

  The read image data transmission path 25a is an image data transmission path according to the format of the image data signal, for example, a camera link standard signal. The clock oscillation circuit 26 is an oscillation circuit that generates a clock faster than the transfer clock when the image data of the camera 1 and the camera 2 is transmitted from the camera, and is 60 MHz or higher compared to the image data transfer clock of the camera of about 30 MHz. The clock is about twice as fast. Therefore, the transfer clock of the image data transmitted from the readout circuit 25 can be set to 60 MHz. The communication circuit 3 is connected to the image processing apparatus via the control line 3 and transmits and receives commands.

  FIG. 4 is an operation flow of the entire robot according to the embodiment. The operations of the robot, the image processing apparatus, the image transmission apparatus, and the camera will be described according to this.

  The robot control device controls the robot arm according to a program created in advance, positions it at a position where an image of the workpiece 6 can be acquired by the camera, and controls the image data of the workpiece 6 captured by the image processing device 8 to perform image processing. The image processing command is issued through the process No. 2.

Here, the operation after the image processing command according to the present invention is issued will be described based on the flowchart. In the flowchart of the operation of FIG.
(Step 40)
The image processing apparatus receives the image processing command from the robot controller 9.
(Step 41)
A signal for turning on the illumination 7 is transmitted from the image processing apparatus, and the illumination 7 is turned on. At the same time, an image transfer command is transmitted to the image transmission apparatus via the control line 1.
(Step 42)
The image multiplexing transmission apparatus receives a trigger signal for transmitting the trigger signal 1 and the trigger signal 2 simultaneously to the camera 1 and the camera 2 (step 43) (step 44).
The camera 1 transmits image data, and the camera 2 also transmits image data at the same time.
(Step 45)
The image data of the camera 1 is transmitted to the single transmission path 12 by the transmission circuit 22 because the selection circuit 1 has selected the image data transmission path 1 by the control circuit 27 in advance.
(Step 46)
The selection circuit 2 receives a signal from the control circuit 27 in advance so as to select the image data of the camera 2. While the image data of the camera 1 is transmitted to the single transmission path 12, the image data of the camera 2 is written in the memory 24.
(Step 47)
The image processing apparatus receives the image data transmitted to the single transmission path 12.
(Step 48)
After transmission of the image data is completed, image processing is performed from the image data acquired by the image processing apparatus, and an image area necessary for stereo measurement is calculated. If one image data requires the entire image area, the entire area is calculated, and if it is partially necessary, only a part is calculated. (Identification of the required area has been described above for a simple example.)
(Step 49)
The calculated image area value is transmitted to the image multiplexing transmission apparatus via the control line 1.
(Step 50)
The control circuit 27 receives a required image area value in accordance with the operation of the robot arm 1 and instructs the reading circuit 25 to specify the image area value (via a control line (not shown)).
(Step 51)
The control circuit 27 instructs the selection circuit 1 to select the read image data transmission path 25a as the image data receiving port.
(Step 52)
The control circuit 27 activates the reading circuit 25 (via a control line (not shown)).
(Step 53)
The activated readout circuit 25 reads out the image data, reads it out in a predetermined signal format, and transmits it to the image data transmission path 25a. As described above, the image data to be transmitted may be the entire image data or only a necessary portion.
(Step 54)
The transmission circuit 22 transmits the read image data selected by the selection circuit 1 to a single transmission path.
(Step 55)
The image processing device 8 receives the transmitted image data (or partial image data).

The three-dimensional position information of the workpiece 6 is calculated by the CPU 33 by the stereo image method from the two images of the image data of the camera 1 transmitted in advance and the image data of the camera 2 received this time. Note that a method for calculating a three-dimensional position from two images by the stereo image method is a general method and will not be described.
(Step 56)
The image processing device 8 transmits information on the three-dimensional position of the workpiece 6 to the robot control device 9 via the control line 2 connected to the communication circuit 2.
(Step 57)
The robot controller 9 causes the robot arm 1 to perform a predetermined operation based on the information on the three-dimensional position of the workpiece 6.

  By taking such steps and transmitting only necessary image information among the captured image information via the single transmission path 12, the robot arm 1 can be precisely controlled without increasing the number of transmission paths. Become.

  FIG. 5 is a timing chart of image data (or partial image data) according to this embodiment. Here, it is assumed that a camera with 2 million pixels that can read out pixels simultaneously and that can be read out with a 30 MHz pixel clock is used for the camera 1 and the camera 2, respectively.

  The timing of each signal is denoted by reference numerals 60 to 68. However, two timing charts of image processing-1 and image processing-2 are described corresponding to two cases of memory reading [30 MHz] 64 and memory reading [60 MHz] 66 read at twice the speed. did.

  In addition, timing charts 69 and 70 are obtained according to a conventional image transmission method (a method of sequentially transmitting images obtained by the camera 1 and the camera 2).

  First, the trigger 1 and the trigger 2 are transmitted from the control circuit 27 of the image transmission apparatus 2 to the camera 1 and the camera 2, respectively (in FIG. 5, two triggers are combined into one).

  Upon receiving the trigger, the image data 1 and the image data 2 from the camera 1 and the camera 2 are transmitted through the image data transmission path 1 and the image transmission path 2 at the timing indicated by the image 1 [30 MHz] and the image 2 [30 MHz]. To the image transmission apparatus 2.

  Simultaneously with the start of transmission of image data 1 and image data 2, on the one hand, transmission of image data 1 of the selected camera 1 is started on the single transmission path indicated by 65, and on the other hand, at the timing of 63, camera 2 The frame 21 of the image data 2 is written in the memory.

  Thereafter, at the timing indicated by 67, first, an image region that requires measurement of a three-dimensional position is calculated by (1) of image processing-1. The calculation result is transmitted to the image transmission apparatus 2 via the control line 1 at the timing when (1) of the image processing-1 is completed, and an image area necessary for the memory read circuit is set at the timing 64 (not shown because the time is short). ) Immediately, the image data 2 is read out from the memory and transmitted to the single transmission line like the frame '21 in which the image frame '21 is 65.

  Upon receiving the image data 2, the image processing apparatus 8 calculates a three-dimensional position by the stereo method at the timing (2) of image processing-1, calculates a three-dimensional position of the workpiece, and transmits position information to the robot controller.

Here, the difference between the conventional image transmission method and the present invention will be described.
According to the conventional image multiplexing transmission apparatus (conventional transmission method) disclosed in Japanese Patent Application Laid-Open No. 2003-274374, the following problems occur.

  In the conventional example, the frame 11 required for the transfer of the image data of the camera 1 and the frame 22 required for the transfer of the image data of the camera 2 are processed by the image processing 1 and 2 after the image processing data is received as shown in the timing chart 69. Since it is executed, the time until the result of image processing is obtained is longer than that of this embodiment as shown in the time chart of 70.

If a 2 million pixel camera capable of simultaneous readout of two pixels can be read out with a 30 MHz pixel clock, the time for one frame is (200 × 10000) / 2) / (30 × 1000000) = 33 milliseconds (milliseconds).

Assuming that the partial image area is a quarter of the entire area, it takes about 8 milliseconds to read out, and a total of 33 milliseconds + 8 milliseconds = 41 milliseconds is required. (The camera image data transmission clock and the memory readout clock are the same.)
If the memory read clock is read out at 60 MHz (twice the speed of the camera), the above 8 msec at the time of partial reading can be further reduced to 4 msec, which is 1/2, 33 msec + 4 msec = 37 m from image acquisition. Calculation of three-dimensional position information can be started in seconds.

  On the other hand, in the case of the conventional example, 33 ms is twice and 66 ms is required. Further, if the necessary partial image area is small, the difference is further increased.

  In the present invention, the two images used for calculation in the stereo image method are partial images of the frame 11 and the frame 21 and are image data at the same time, but in the conventional method, the time at which the frame 11 and the frame 22 are used. Needless to say, if two images with a difference of 33 msec are used, and the camera position is affected by vibration or the like between the two images, an error occurs in the analysis result of the stereo method. That is, the conventional transmission method cannot simultaneously acquire a plurality of images.

  Furthermore, the readout clock can be faster than the image data transmission clock of the camera at the time of partial image readout.

According to Table 1, the method of the present invention can be transmitted in about half the time compared with the conventional transmission method (1) that does not use a memory.

  Table 2 shows a result of comparison between the image transmission method of the present invention and the conventional image transmission method.

  In the conventional image transmission method, although the circuit is simple, it is difficult to acquire images from two cameras at the same time without adding a transmission path. It took a lot to transmit to.

  The present invention is superior to the conventional image transmission technology because a plurality of images can be acquired simultaneously while simplifying the circuit and the transmission time is completed earlier. Further, since all the image data 1 is always transmitted to the image processing apparatus, it can be used immediately as a normal camera when the stereo method need not be used. Only when using the stereo method, image data acquired from two cameras may be used.

  The image transmission apparatus of the present invention can be suitably used in an industrial robot arm in which expansion of the image transmission path is limited particularly from the viewpoint of space saving and cost.

1 Robot arm 2 Image transmission device 3 Camera 1
4 Camera 1
5 Camera 2
8 Image processing device 9 Robot control device 10 Image data transmission path 1
11 Image data transmission path 2
12 Single transmission line 13 Control line 1
14 control line 221a selection circuit 1
21b Selection circuit 2
23 writing circuit 24 memory 25 reading circuit 26 clock oscillator 27 control circuit 22 transmitting circuit 29a trigger 129b trigger 2

Claims (6)

In an image transmission device that transmits image data acquired by two imaging devices through a single transmission path,
At least one selection circuit that selects one of the image data acquired by each of the two imaging devices;
A transmission circuit for transmitting image data to a single transmission line;
A memory for storing the other image data;
A readout circuit that reads out the other image data stored in the memory and transmits the image data to a transmission circuit,
All of the selected one image data is transmitted to a single transmission line, all the other image data is stored in a memory, and after the one image data has been transmitted, the other image data is transmitted. An image transmission device.   The image transmission apparatus according to claim 1, wherein the read circuit further includes a clock oscillation circuit for reading and transmitting image data from the memory.   A robot arm comprising the image transmission device according to claim 1.   4. The robot arm according to claim 3, wherein only a part of the other image data is transmitted in accordance with the operation of the robot arm. In an image transmission method for transmitting image data acquired by two imaging devices through a single transmission path,
One of the image data acquired by each of the two imaging devices is selected, all the selected one image data is transmitted to a single transmission line, and all the other image data is stored in the memory. An image transmission method comprising: reading out the other image data from a memory and transmitting the image data to a single transmission path after the transmission of is completed.   5. The image transmission method according to claim 4, wherein only a part of the other image information is transmitted.