JP2009218933A - Smart camera and robot vision system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a smart camera for achieving efficient data transfer with the high degree of freedom through a network in variously controlling a robot, and to provide a robot vision system equipped with the smart camera. <P>SOLUTION: A smart camera includes a first command interface CIF1(CIF1) and a second command interface CIF2(CIF2). In an image processing process DP, image processing is applied to image pickup data DT1 based on a command to be input from the CIF1, and result data DT3 configured of information beneficial for the control of a robot are generated, stored in a result data buffer DB3, and output to the CIF1. Meanwhile, either the image pickup data DT1 or the processed image data DT2 are selected as monitor image databased on a command to be input from the CIF2 by an image selector SW, and output from the CIF2. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a smart camera that extracts information from a captured image and a robot vision system having a visual function of the smart camera.

  In general, industrial robots move a functional part at the tip to a predetermined target position in order to grip and convey the object to be conveyed or perform various operations on the object to be processed. It is possible. Such movement of the functional unit of the robot is often performed based on a control program created in advance so that the functional unit sequentially moves the target position by teaching or the like. For example, in a robot in which a plurality of arms are connected in order to be able to rotate, the function unit is positioned at a predetermined position by controlling the rotation angle of the connecting part of each arm to the specified rotation angle sequentially by the control program. Is done.

  By the way, in order to cause the functional unit of the robot controlled based on the control program determined in advance to perform a predetermined operation, the relative positional relationship between the robot and the workpiece or the conveyed object is controlled. It must be similar to the positional relationship when the program was created. More specifically, for example, when placing a component on a substrate, the robot grips the component supplied in a predetermined position and orientation at a predetermined position of the functional unit in a predetermined direction, and the functional unit in a predetermined direction. By moving to a predetermined position in the direction, the components are arranged with respect to the board that should have been originally positioned in the predetermined position and direction. Therefore, conversely, any one of the supply position and supply direction of the component, the relative position between the component and the robot, and the arrangement position and orientation of the substrate, etc., may cause a deviation in the relationship between the component and the substrate. Placement accuracy cannot be secured.

Therefore, conventionally, in order to accurately place a component to be transported by a robot or the like on a substrate even if there is a slight deviation in the position or orientation of the component, for example, a system as described in Patent Document 1, for example, image processing There has also been proposed a system using a smart camera that performs the above in a processing unit integrated with the camera. This system basically includes a smart camera having an image sensor and a control module as a processing unit for processing data of a peripheral image captured by the image sensor, and a drive having a control module for driving and controlling a drive device (robot or the like). It is comprised by the control apparatus and the network used for communication between these smart cameras and drive control apparatuses. That is, the smart camera generates information necessary for controlling the driving device, such as a shift amount of the relative position of the component with respect to the substrate, by performing image processing on the image data of the component and the substrate captured through the image sensor by the control module. At the same time, the generated information is transmitted to the drive control device. Further, the drive control device performs position correction based on information transmitted from the smart camera via the network, and performs operations such as conveying parts to the corrected position. As a result, the parts conveyance position and the like are also corrected in real time, and the positioning accuracy of the parts by the driving device is naturally improved.
US Pat. No. 6,985,780

By the way, the smart camera including the system described in Patent Document 1 normally has a functional block as shown in FIG. 7 and is responsive to various commands input from the command interface CIFA through image processing or a network. Various processes including the data transfer performed are executed. That is, in such a smart camera, an “image processing command”, “result data acquisition command”, “
Necessary processes corresponding to various commands such as “monitor image acquisition command” are collectively executed through the image processing process DPa. For example, the image processing process DPa to which the “image processing command” is input acquires the processed image data DT2a in the processed image buffer DB2a to which the imaging data DT1a has been transferred from the imaging data buffer DB1a, and performs the required image processing. Examples of such image processing include the following processing. That is, the image processing process DPa that has acquired the processed image data DT2a extracts the contour of the imaging object based on the previous imaging data DT1a, and then uses this again as processed image data DT2a in the processed image buffer DB2a. save. Then, by comparing the stored processed image data DT2a with reference contour data, for example, result data DT3a generated from information indicating suitability is obtained and stored in the result data buffer DB3a. Thereafter, for example, when a “result data acquisition command” is input to the image processing process DPa via the command interface CIFA, the image processing process DPa sends the obtained result data DT3a from the result data buffer DB3a to the command interface CIFA. Output via.

  On the other hand, in the robot vision system having such a visual function, the captured image data DT1a captured by the camera is displayed on a monitor device of a computer as a development environment for creating and maintaining a control program. Sometimes. In such a case, a “monitor image acquisition command” is input to the image processing process DPa via the command interface CIFa. In this case, the command interface CIFa takes in the processed image data DT2a in the processed image data DT2a to which the imaging data DT1a has been transferred from the imaging data DT1a, and outputs it via the command interface CIFa. However, in this case, since the amount of image data is enormous compared to other types of data, it takes a relatively long time to transfer the image data through the network. That is, the processing for the “monitor image acquisition command” by the image processing process DPa usually requires a lot of time. In addition, since such an image processing process DPa itself normally ends processing based on one command and then executes processing based on the next command, when time-consuming processing is being executed. Will naturally delay processing for the next command. For example, when the “result data acquisition command” is input while the image processing process DPa is executing the process of transferring the image data based on the “monitor image acquisition command”, the “result data acquisition command” Is not obtained until the processing for the “monitor image acquisition command” being executed first is completed. For this reason, in particular, in a drive device that is feedback controlled based on the result data DT3a acquired by the processing for the “result data acquisition command”, the accuracy of positioning and the like may be reduced. .

  As a method of displaying the monitor image, it is possible to display the monitor image on a dedicated monitor device connected to the smart camera with a dedicated cable. However, to connect the dedicated cable to a smart camera that is required to be downsized. It is not easy to provide the connection terminals. In addition, even if a monitor-dedicated device is provided, an increase in cost is inevitable including securing an installation space for that purpose.

  The present invention has been made in view of such circumstances, and its purpose is a smart that enables more efficient and more efficient data exchange via a network for various control of a robot and the like. An object is to provide a camera and a robot vision system including the smart camera.

The smart camera of the present invention is a smart camera that sequentially outputs image data captured by an image sensor and outputs result data including information necessary for realizing various visual functions. And generating the result data from the imaging data buffer stored in the imaging data buffer and the imaging data held in the imaging data buffer based on an image processing command that is an image processing request command for the imaging data. Image processing means for generating processed image data processed as required, processed image data buffer for temporarily storing the processed image data, and access to both the imaging data buffer and the processed image data buffer Image selection that is configured to select the imaging data and the processed image data Image selection means for selectively outputting one of the imaged data and the processed image data based on a command, and the image processing instruction inputted from the outside is given to the image processing means, and the result data generated as a result A first command interface that outputs to the outside, and the image selection command that is input from the outside to the image selection means, and one of the imaging data and the processed image data that is selectively output from the image selection means And a second command interface for outputting to the outside as monitor image data.

  According to such a configuration as the smart camera, the result data is output independently from the first command interface, and the processed image data or the imaging data is output independently from the second command interface. Thus, for example, even when monitor image data such as processed image data and imaging data is being output from the second command interface, the result data used for controlling the robot or the like can be output from the first command interface in real time without delay. become able to.

  In addition, the captured image data is temporarily stored in the captured image data buffer and the processed image data is temporarily stored in the processed image buffer, and the captured image data or processed image data is selected and output as monitor image data directly from the image selection unit To be. That is, it is not necessary to output monitor image data from the second command interface via the image processing means, so that the image processing means can respond in real time to an image processing command or the like.

  In the smart camera of the present invention, in the smart camera, the result data output to the outside from the first command interface and the monitor image data output to the outside from the second command interface are bus-type. The gist is to output to the same communication path consisting of a network.

  According to such a configuration, even when two interfaces such as the first and second command interfaces are required, these interfaces can be realized by one physical network port or the like. This facilitates downsizing required for the smart camera, and for example, it is easy to realize a small smart camera mounted on a robot.

  In the smart camera of the present invention, in the smart camera, the processed image data buffer is configured to be able to hold a plurality of the processed image data, and the image selection unit includes the plurality of processed image data. The gist is to select and output one that conforms to the image selection command.

  According to such a configuration, it becomes possible to refer to arbitrary data of the plurality of processed image data through the second command interface. That is, the processing status can be easily verified, and the utility value as a smart camera can be further increased.

The smart camera of the present invention further includes a result data buffer that temporarily stores the result data generated by the image processing unit in the smart camera, and the first command interface stores the result data in the smart camera. The gist is that the data is read from the result data buffer and output to the outside.

  According to such a configuration, even if the image processing unit is performing image processing based on another image processing command, it is possible to immediately obtain result data before that, and increase the responsiveness as a smart camera. Will be able to.

  The robot vision system of the present invention includes a robot, a functional unit that is provided in the robot and performs a predetermined process on the workpiece, and sequentially captures the imaging data of the surrounding image of the robot including the workpiece, A smart camera that outputs result data including information required to realize the visual function of the robot, and a control program that controls the operation of the robot. The control program and the information output from the smart camera. A robot vision system including a robot controller that controls the operation of the robot including positioning of the functional unit based on result data, wherein the smart camera is composed of the above-described smart camera.

  According to such a configuration, the smart camera outputs the result data to the robot controller independently from the first command interface and the processed image data or imaging data from the second command interface. As a result, even if monitor image data such as processed image data and imaging data is being output from the second command interface, for example, the result data used for controlling the robot and the like is output from the first command interface to the robot controller in real time without delay. Will be able to.

  In addition, the smart camera temporarily stores the image data in the image data buffer and the processed image data in the processed image buffer, and directly selects the image data or the processed image data as monitor image data from the buffer. The output is selected from the means. That is, since it is not necessary to output the monitor image data from the second command interface via the image processing means, the image processing means can respond in real time to an image processing command or the like from the robot controller.

  Further, even when the smart camera requires two interfaces such as the first and second command interfaces, these interfaces can be realized by one physical network port or the like. This facilitates downsizing required for the smart camera, and also makes it easy to realize a robot vision system in which a small smart camera is mounted on the robot.

  In addition, it becomes possible to refer to arbitrary data of the plurality of processed image data through the second command interface. That is, the processing status can be easily verified, and the utility value as a robot vision system using this smart camera can be further increased.

  Furthermore, even when the smart camera is performing the above-described image processing, for example, it becomes possible to respond to a request for result data from the robot controller in real time. Accordingly, it is possible to provide a robot vision system with improved control accuracy that is not affected by the presence or absence of image data output by a smart camera.

In addition, monitor image data can be acquired from the smart camera at any time, and monitor image data from the smart camera can be displayed on a monitor device such as a computer through a robot controller without providing a dedicated monitor device. To be able to. That is, there is no concern that the robot vision system will increase unnecessary costs.

  In the robot vision system of the present invention, in the robot vision system, a development environment for creating a program for the robot or the smart camera is connected to the robot vision system, and the development environment includes the smart camera. The gist is that the monitor image data can be displayed.

  According to such a configuration, when creating a program for a robot or smart camera, monitor image data from the smart camera can be displayed in a development environment inevitably connected to the robot vision system. It is also easy to build an integrated environment.

Hereinafter, an embodiment in which a smart camera and a robot vision system according to the present invention are embodied will be described with reference to the drawings.
FIG. 1 is a block diagram showing an outline of the robot vision system according to the present embodiment.

  As shown in FIG. 1, this robot vision system is mainly configured by a robot system 2 and a smart camera 12 connected to the robot system 2. The robot system 2 includes a robot 10 and a robot controller 11. Here, the robot system 2 is a system having a multi-joint type robot 10 in which a plurality of arms are respectively connected at joints, and the robot 10 itself has a predetermined relationship between the arms connected at each joint. This is a part for processing and transporting the workpiece W by rotating it to a relative angle and moving the functional part T at its tip to the target position. Note that a motor and an encoder are provided at the joint of the robot 10. The robot controller 11 that controls the position of the robot 10 (functional unit T) captures a peripheral image Im of the robot 10 and processes the data of the captured peripheral image Im to control the position of the robot 10. A smart camera 12 that provides information necessary for realizing the visual function is connected. Further, the robot controller 11 has a development computer 13 as a development environment necessary for developing a control program for controlling the operation of the robot 10 and a teaching pendant 14 for inputting a teaching instruction when creating a control program by teaching the robot 10. Etc. can be connected as necessary.

  FIG. 2 shows a hardware configuration of the robot controller 11. As shown in FIG. 2, the robot controller 11 is provided with a CPU board 20, a robot control board 21, a digital input / output board 22, and a serial communication board 23 that are connected to a single ISA bus 24. ing.

As shown in FIG. 3, the CPU board 20 is a so-called small computer, and includes a CPU 30, ROM 31, RAM 32, SRAM 33, ISA bus interface 34, Ethernet (registered trademark) controller 35, and serial input / output (SIO) controller 36. Are respectively connected to one main bus 37. The CPU 30 is a so-called central processing unit, and the CPU 30 performs predetermined calculations based on an OS (operating system) and various application programs stored in the ROM 31 and the SRAM 33 while using the RAM 32 as a data memory. A position control command or the like is generated. As the ROM 31, it is desirable to adopt a rewritable ROM such as a flash ROM so that the stored OS, various application programs, and the like can be changed from the development computer 13 as necessary. The SRAM 33 is a memory that can store a larger amount of data than the ROM 31 by supplying power from a power source (not shown) even when the power of the robot controller 11 is cut off. An application program, various data, and the like are saved, and calculation results of the CPU 30 are also saved. The ISA bus interface 34 is connected to both the main bus 37 and the ISA bus 24 of the robot controller 11, and communicates between the CPU board 20 and the other boards 21 to 23 connected to the ISA bus 24 of the robot controller 11. It is an interface that enables That is, the data that needs to be transmitted to each of the boards 21 to 23 calculated by the CPU board 20, for example, a position control command for the robot 10 can be referred to from each of the other boards 21 to 23 via the ISA bus 24. It is like that. The Ethernet (registered trademark) controller 35 allows the robot controller 11 and other devices to communicate with each other via Ethernet (registered trademark) as a bus-type network. The Ethernet (registered trademark) is connected to the first port P1. ) Enable communication with other devices via cable. In the present embodiment, the smart camera 12 and the development computer 13 are connected to the first port P1. In particular, the smart camera 12 is always connected. On the other hand, the development computer 13 is connected as necessary, such as when a control program is created. In addition, communication using Ethernet (registered trademark) requires a unique address set for each device, and each Ethernet (registered trademark) controller 35, smart camera 12, and development computer 13 has a unique address. Is preset. The SIO controller 36 is for serial communication between the robot controller 11 and another device, and enables serial communication with the device connected to the second port P2. In the present embodiment, the teaching pendant 14 is connected to the second port P2, but the teaching pendant 14 is usually not connected except when performing teaching work on the robot 10.

Next, the robot control board 21 includes a so-called control controller. As shown in FIG. 4, the DSP (digital signal processing device) 40, ROM 41, RAM 42, ISA bus interface 43, motor control circuit unit 44, position counter 45 Each is provided in such a manner that it is connected to one main bus 46. In the robot control board 21, the DSP 40 uses the RAM 42 as a data memory and performs predetermined calculations based on the firmware and various programs stored in the ROM 41 and the position control command calculated by the CPU board 20. A control signal for controlling is generated. As the ROM 41, it is desirable to adopt a rewritable ROM such as a flash ROM so that stored firmware and the like can be changed from the development computer 13 or the like as necessary. The ISA bus interface 43 is connected to both the main bus 46 and the ISA bus 24 of the robot controller 11, and the robot control board 21 is connected to each of the other boards 20, 22, 23 connected to the ISA bus 24 of the robot controller 11. It is an interface that makes it possible to communicate with. That is, the robot control board 21 can refer to the position control command calculated by the CPU board 20, and can refer to the various information calculated by the robot control board 21 to the other boards 20, 22, and 23. . A power drive unit 47 that is also provided on the robot control board 21 is connected to the motor control circuit unit 44. The motor control circuit unit 44 calculates the power to be applied to the motor M provided in the robot 10 based on the control signal generated by the DSP 40 and supplies the supplied power to the motor M via the power drive unit 47. Control. Also, information such as the amount of current detected by a sensor (not shown) provided in the motor M is input to the motor control circuit unit 44, and feedback is performed for calculation of power to be given to the motor M. The position counter 45 is connected to an encoder EN that detects a rotation angle of a joint provided in the robot 10 and detects a rotation angle of the joint of the robot 10 based on a rotation angle signal input from the encoder EN. The robot control board 21 feeds back the detected rotation angle to the generation of the control signal, and the rotation angle detected by the position counter 45 is also sent to the CPU board 20 via the ISA bus 24 of the robot controller 11. Is used to generate the position control command. Here, for convenience, the robot 10 has been described as having one motor M and one encoder EN, but the robot 10 generally includes a plurality of these motors M and encoders EN.

  In the robot controller 11 shown in FIG. 1, the digital input / output board 22 outputs a digital signal corresponding to information instructed from the CPU board 20, and the CPU board 20 outputs information based on the input digital signal. It is a board that can be taken in. Further, the serial communication board 23 outputs serial communication data based on information instructed from the CPU board 20 from the port, and the CPU board 20 outputs information based on the serial communication data input to the port. It is a board that can be taken in.

  FIG. 5 shows a hardware configuration of the smart camera 12. The smart camera 12 is configured to include a control unit 12 </ b> C composed of a so-called small computer, and each of the CPU 50, ROM 51, RAM 52, Ethernet (registered trademark) controller 53, and video interface 54 is connected to one main bus 55. Are provided. The CPU 50 is also a so-called central processing unit. When the CPU 50 uses the RAM 52 as a data memory, the CPU 50 executes necessary processing, for example, an image processing process program based on the OS and various application programs stored in the ROM 51. Data processing for image data is sequentially performed. As the ROM 51, it is desirable to adopt a rewritable ROM such as a flash ROM so that the stored OS and various application programs can be changed from the development computer 13 or the like as necessary. . In the present embodiment, the RAM 52 has an area as the imaging data buffer DB1 for temporarily storing and storing predetermined image data and an area as the processing image buffer DB2. Further, the RAM 52 has an area as a result data buffer DB3 for storing data based on information calculated by data processing of image data. The Ethernet (registered trademark) controller 53 allows the smart camera 12 and other devices to communicate with each other via Ethernet (registered trademark), and enables communication with other devices via an Ethernet (registered trademark) cable. In this embodiment, the robot controller 11 is always connected to the Ethernet (registered trademark) controller 53. Note that the unique address used for communication in Ethernet (registered trademark) is preset in the Ethernet (registered trademark) controller 53 (smart camera 12) itself as described above. The Ethernet (registered trademark) controller 53 is provided with a first command interface CIF1 and a second command interface CIF2. Each of the command interfaces CIF1 and CIF2 is an interface unit virtually realized by the Ethernet (registered trademark) controller 53, and communication with each of the command interfaces CIF1 and CIF2 is performed by the Ethernet (registered trademark) controller 53 with a predetermined ID included in the communication data. This is done by determining the above. That is, although there is only one Ethernet (registered trademark) controller 53 as the smart camera 12, the above two interface units are virtually provided. An image sensor 56 that is also provided in the smart camera 12 is connected to the video interface 54. The image sensor 56 is, for example, a camera that uses a CCD as an imaging device, and captures a peripheral image Im of the robot 10, for example, an image of a workpiece W processed or gripped by the robot 10 or an image of the functional unit T at the tip of the robot 10. Then, the data Sim is output to the video interface 54. The video interface 54 is a part that transfers the data Sim imaged through the image sensor 56 to the RAM 52, and the CPU 50 performs the above-described required processing on the data thus transferred to the RAM 52, that is, the imaged data. It becomes.

  FIG. 6 is a functional block diagram showing an outline of functions for performing required processing on the imaged data mainly in the control unit 12 </ b> C of the smart camera 12. In the present embodiment, the control unit 12C is provided with an image processing process DP as an image processing unit and an image selector SW as an image selection unit. The image processing process DP and the image selector SW are functions realized by the image processing process program or the image selector program stored in the RAM 52 being executed by the CPU 50 of the control unit 12C. The command interfaces CIF1 and CIF2 of the smart camera 12 are connected to various commands (“image processing command”, “result data acquisition command”, “monitor image data”, and the like from the robot controller 11 (FIG. 1) via Ethernet (registered trademark). "Acquisition command" etc.) is input.

  Next, an example of image processing in the control unit 12C of the smart camera 12 will be described with reference to FIG. It is assumed that the imaging data buffer DB1 stores imaging data DT1 formed based on the data Sim input from the image sensor 56 while being appropriately updated.

  First, in the control unit 12C of the smart camera 12, an “image processing command” which is an image processing request command for the imaging data DT1 from the robot controller 11 (FIG. 1) is transmitted through the first command interface CIF1 to the image processing process DP. Is input. The image data DT1 stored in the image data buffer DB1 is taken into the image processing process DP to which the “image processing command” is input. When the imaging data DT1 is captured, the image processing process DP performs required data processing that is predetermined for the captured imaging data DT1, that is, so-called image processing. Through this image processing, processed image data DT2, which is processed image data, and result data DT3 including information necessary for realizing the visual function are generated. Here, the processed image data DT2 is image data generated based on the imaging data DT1, and is, for example, image data to which contour information, center position information, and the like of the imaging target imaged as the imaging data DT1 are added. The result data DT3 includes various types of information acquired from the imaging target imaged as the imaging data DT1. Specifically, coordinate information indicating the position and orientation of the workpiece W (FIG. 1), shape information indicating the shape of the workpiece W, color information indicating the hue and brightness of the workpiece W, overlap and contact between the plurality of workpieces W, and the like. Information including numerical data useful for position control of the robot 10 such as array information indicating the relationship between the robot 10 and the robot information indicating the orientation and coordinates of the functional unit T (FIG. 1) at the tip of the robot 10. The processed image data DT2 generated in the image processing process DP is stored in the processed image buffer DB2, and the result data DT3 is stored in the result data buffer DB3. A plurality of predetermined processed image data DT2 can be stored in the processed image buffer DB2. When new data is stored, the existing data is shifted to the rear position of the memory and the last data is discarded. Thus, new data, that is, the latest processed image data DT2 is stored at the leading position vacated by this shift operation.

  Thereafter, when a “result data acquisition command” is input from the external robot controller 11 (FIG. 1) to the image processing process DP via the first command interface CIF1, the image processing process DP stores the result data buffer DB3 in the result data buffer DB3. The stored result data DT3 is output from the result data buffer DB3 to the first command interface CIF1. Accordingly, the result data DT3 from the smart camera 12 is input to the robot controller 11, and positioning control of the functional unit T (FIG. 1) of the robot 10 is executed based on the result data DT3.

On the other hand, in this embodiment, the development computer 13 (FIG. 1) is connected to the robot controller 11 (FIG. 1), and the monitor image data from the smart camera 12 is displayed on the monitor device. ing. That is, the “monitor image data acquisition command” issued from the development computer 13 (FIG. 1) to the smart camera 12 is sent to the image selector via the robot controller 11 (FIG. 1) and the second command interface CIF 2 of the smart camera 12. It is given to SW. The “monitor image data acquisition command” includes mode information indicating which of the imaging data DT1 and the processed image data DT2 is to be acquired. In the “real image mode”, the imaging data DT1 is In the “processed image mode”, the processed image data DT2 is selected as monitor image data by the image selector SW. Further, in the “processed image mode”, it is possible to set information indicating what number of processed image data is to be selected, but if not set, the image selector SW reads from the processed image buffer DB2. The latest processed image data DT2 is selected and output.

  For this reason, for example, if the “monitor image data acquisition command” given to the image selector SW is “real image mode”, the image selector SW selects the imaging data DT1 stored in the imaging data buffer DB1, and selects it. The monitor image data is output from the second command interface CIF2. The monitor image data output from the second command interface CIF2 is transferred to the development computer 13 (FIG. 1) via the robot controller 11 (FIG. 1) and displayed on the monitor device. Incidentally, when such a “monitor image data acquisition command” in the “real image mode” is sequentially repeated from the development computer 13 (FIG. 1) to the smart camera 12, the monitor device of the development computer 13 The monitor image data composed of the imaging data DT1 imaged through the smart camera 12 is displayed in the form of a moving image.

  On the other hand, when the “monitor image data acquisition command” is “process image mode”, the image selector SW selects the process image data DT2 in the designated order stored in the process image buffer DB2, and selects this as the monitor image. Data is output from the second command interface CIF2. The monitor image data output from the second command interface CIF2 is also transferred to the development computer 13 via the robot controller 11 and displayed on the monitor device.

As described above, according to the smart camera and the robot vision system of the present embodiment, the effects listed below can be obtained.
(1) The result data DT3 is output independently from the first command interface CIF1, and the processed image data DT2 or the imaging data DT1 is output independently from the second command interface CIF2. Thus, for example, even when monitor image data such as processed image data DT2 and imaging data DT1 is being output from the second command interface CIF2, the first command interface CIF1 outputs the result data DT3 used for controlling the robot 10 and the like without delay. It becomes possible to output in real time.

  (2) Further, the imaging data DT1 is temporarily stored in the imaging data buffer DB1 and the processed image data DT2 is temporarily stored in the processing image buffer DB2, and the imaging data DT1 or the processed image data is directly stored in the buffers DB1 and DB2. DT2 is selectively output from the image selector SW as monitor image data. That is, it is not necessary to output the monitor image data from the second command interface CIF2 via the image processing process DP, so that the image processing process DP can respond to the “image processing command” or the like in real time.

  (3) A configuration requiring two interfaces such as the first and second command interfaces CIF1 and CIF2 is realized by one physical Ethernet (registered trademark) controller 53. As a result, downsizing required for the smart camera 12 is facilitated, and a small smart camera 12 mounted on the robot 10 can be easily realized.

  (4) By storing a plurality of processed image data DT2 in the processed image buffer DB2, any data among the plurality of processed image data DT2 can be referred to through the second command interface CIF2. That is, the processing status can be easily verified, and the utility value as the smart camera 12 can be further increased.

  (5) By storing the result data DT3 generated previously in the result data buffer DB3, even if the image processing process DP is in image processing based on another “image processing command”, the previous data DT3 is stored. The result data DT3 can be obtained immediately, and the responsiveness as the smart camera 12 can be improved.

  (6) The imaging data DT1 or the processed image data DT2 is output as monitor image data from the second command interface CIF2 of the Ethernet (registered trademark) controller 53. As a result, monitor image data can be obtained from the smart camera 12 at any time, and the monitor image data from the smart camera 12 can be developed through the robot controller 11 without a separate monitor dedicated device. 13 monitor devices can be displayed. That is, there is no concern that the robot vision system will increase unnecessary costs.

  (7) When creating a program for the robot 10 or the smart camera 12, monitor image data from the smart camera 12 can be displayed on the development computer 13 that is necessarily connected to the robot vision system. It is also easy to build a more efficient integrated environment.

In addition, the said embodiment can also be implemented in the following aspects, for example.
In the above embodiment, Ethernet (registered trademark) is used as the data transfer mechanism, but the data transfer mechanism is not limited to Ethernet (registered trademark), and other communication means such as serial communication, USB, IEEE 1394, etc. A transfer mechanism may be employed.

  In the above-described embodiment, the CCD is used as the image sensor of the image sensor 56. However, any image sensor may be used as long as it can input a captured image to the video interface.

  In the above-described embodiment, the smart camera 12 that includes the image sensor 56 as an integral unit is used. However, as such a smart camera, a housing unit that includes a CPU and the like and a camera unit that includes an image sensor are separated. A device in which the housing unit and the camera unit are separated from each other, for example, may be connected with a dedicated cable. By doing so, in a relatively small camera unit, the degree of freedom of the arrangement is increased, and for example, a configuration in which the camera unit is arranged at the tip of the robot becomes easy.

In the above embodiment, it is assumed that the robot controller 11 performs positioning control of the functional unit T of the robot 10 based on processing information obtained by image processing of the smart camera 12. However, the use as the robot vision system is not limited to this. That is, the position and orientation of the workpiece, the shape of the workpiece, the shade and brightness of the workpiece, the relationship between overlap and contact between a plurality of workpieces, the orientation and coordinates of the functional unit T at the tip of the robot 10, etc. When the process information is calculated, the robot controller 11 can perform various controls based on the process information. For example, it is possible to determine whether or not the workpiece is a target part from the information on the shape and color of the workpiece and reflect the result on the positioning control of the functional unit. Further, from the workpiece overlap information, it is possible to calculate a special trajectory for causing the functional unit to approach the workpiece, or to cause the functional unit to separate a plurality of workpieces. Further, even when various types of control are performed on the robot 10, control based on various information can be performed, so that control with a higher degree of freedom can be performed.

  In addition, the content of the image to be processed by the smart camera 12 is not limited to the above-described content and is arbitrary. For example, necessary processing is performed based on images related to workpiece transfer devices, workpiece processing devices, other robots and peripheral devices, etc., and information such as the orientation, coordinates, and shape of those objects is calculated. May be. By using these pieces of information for control, the robot controller 11 can control the function unit T of the robot 10 to prevent interference and contact with objects around it. That is, the operation of the robot constituting the robot vision system can be made more harmonious with the surrounding environment, that is, with other production facilities.

  In the above embodiment, the monitor image data from the smart camera 12 is displayed on the monitor device of the development computer 13. However, the present invention is not limited to this, and the monitor image data may be displayed on a monitor device or a monitor dedicated device of another computer.

  In the above embodiment, the result data buffer DB3 for holding the result data DT3 is provided. However, the result data buffer DB3 can be obtained by directly outputting the result data generated by the image processing process based on the image processing to the command interface. It can also be set as the structure which does not use.

  In the above embodiment, the processed image buffer DB2 holds a plurality of processed image data DT2, but only the latest processed image data DT2 may be stored in the processed image buffer DB2. This reduces the amount of memory required and makes it easier to implement the smart camera.

  In the above embodiment, two virtual first command interface CIF1 and virtual second command interface CIF2 are provided in one Ethernet (registered trademark) controller 35. However, the present invention is not limited to this, and one command interface may be provided for each of two Ethernet (registered trademark) controllers. That is, any configuration can be used as long as various commands can be independently transmitted to the image processing process and the image selector of the smart camera independently in real time.

  In the above embodiment, the image processing process DP performs image processing on the imaging data DT1 by giving an “image processing command” to generate result data DT3, which is stored in the result data buffer DB3. In addition, the result data DT3 generated earlier, which is stored in the result data buffer DB3 by the subsequent “result data acquisition command”, is output to the first command interface CIF1. However, the present invention is not limited to this, and the image processing process may perform image processing on the imaging data DT1 by giving an “image processing command”, and result data DT3 generated as a result may be output from the first command interface CIF1. . By doing so, the result data DT3 can be acquired with one command, and the result data DT3 generated based on the image processing applied to the latest imaging data DT1 is output.

1 is a block diagram showing a schematic configuration of an embodiment of a robot vision system according to the present invention. The block diagram which shows the structure of the robot controller of the embodiment. The block diagram which shows the structure of the CPU board of the embodiment. The block diagram which shows the structure of the robot control board of the embodiment. The block diagram which shows the structure of the smart camera of the embodiment. The functional block diagram which shows the function of the smart camera of the embodiment. The functional block diagram which shows the function of the conventional smart camera.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 2 ... Robot system, 10 ... Robot, 11 ... Robot controller, 12 ... Smart camera, 12C ... Control part, 13 ... Development computer, 14 ... Teaching pendant, 20 ... CPU board, 21 ... Robot control board, 22 ... Digital input Output board, 23 ... Serial communication board, 24 ... ISA bus, 30, 50 ... CPU, 31, 41, 51 ... ROM, 32, 42, 52 ... RAM, 33 ... SRAM, 34, 43 ... ISA bus interface, 35, 53 ... Ethernet (registered trademark) controller 36 ... Serial I / O controller 37, 46, 55 ... Main bus, 40 ... DSP, 44 ... Motor control circuit unit, 45 ... Position counter, 47 ... Power drive unit, 54 ... Video Interface, 56 ... Image sensor, CIF1 ... 1 command interface, CIF2 ... second command interface, DB1 ... imaging data buffer, DB2 ... processed image buffer, DB3 ... result data buffer, DT1 ... imaging data, DT2 ... processed image data, DT3 ... result data, M ... motor, T ... Function part, W ... Work, EN ... Encoder, P1 ... First port, P2 ... Second port.

Claims (6)

A smart camera that sequentially outputs image data captured by an image sensor and outputs result data including information necessary for realizing various visual functions,
An imaging data buffer in which the imaging data is temporarily stored and held;
The result data is generated from the imaging data held in the imaging data buffer based on an image processing command that is an image processing request command for the imaging data, and processed image data is generated by processing the imaging data as required. Image processing means,
A processed image data buffer in which the processed image data is temporarily stored;
The imaging data buffer and the processed image data buffer are configured to be accessible, and one of the imaging data and the processed image data is selected based on an image selection command that is a selection command for the imaging data and the processed image data. Image selection means for outputting automatically,
A first command interface for giving the image processing command input from the outside to the image processing means and outputting the result data generated as a result to the outside;
The image selection command input from the outside is given to the image selection means, and one of the imaging data and the processed image data selectively output from the image selection means is output to the outside as monitor image data A command interface;
A smart camera comprising: The result data output to the outside from the first command interface and the monitor image data output to the outside from the second command interface are output to the same communication path including a bus type network. The smart camera according to claim 1. The processed image data buffer is configured to be capable of holding a plurality of processed image data,
The smart camera according to claim 1 or 2, wherein the image selection means selects and outputs one of the plurality of processed image data that matches the image selection command. In the smart camera as described in any one of Claims 1-3,
A result data buffer for temporarily storing the result data generated by the image processing means;
The smart camera characterized in that the first command interface reads the result data from the result data buffer and outputs it to the outside. With robots,
A functional unit that is provided in the robot and performs predetermined processing on the workpiece;
A smart camera that captures a peripheral image of the robot including the workpiece while sequentially processing the imaging data and outputs result data including information necessary for realizing the visual function of the robot;
A robot controller having a control program for controlling the operation of the robot, and controlling the operation of the robot including positioning of the functional unit based on the control program and result data which is information output from the smart camera; A robot vision system comprising:
The said smart camera consists of a smart camera as described in any one of Claims 1-4. The robot vision system characterized by the above-mentioned. A development environment for creating a program for the robot or the smart camera is connected to the robot vision system,
The robot vision system according to claim 5, wherein the development environment is configured to be able to display monitor image data from the smart camera.

Images