JP4961405B2 - Image data generation apparatus for simulation apparatus - Google Patents

Description

  The present invention relates to an image data generation apparatus of a simulation apparatus that generates image data for use in a simulation apparatus used when designing a visual inspection system using an inspection camera.

When designing a visual inspection system to be built into a production facility of a factory, it will be designed by the requester's tailor-made, so it is possible to design a visual inspection system while actually testing at the factory of the requester who plans to incorporate it. If so, it can be designed relatively easily. However, it is often quite difficult to design at the requesting factory before delivering the visual inspection system. For this reason, the visual inspection system is designed by testing in the laboratory of the manufacturer that manufactures and sells, or by testing with a simulation device (simulator).
JP-A-4-217480 JP 2000-267719 A

  When the visual inspection system designed as described above is delivered to the requesting factory, installed, and operated, the image actually taken with the inspection camera and the image taken with the inspection camera in the laboratory or simulation device are taken. In some cases, the planned inspection function could not be performed due to the difference in the image. In such a case, it would be troublesome twice, but it was necessary to readjust at the actual factory.

  On the other hand, in the simulation apparatus, a countermeasure for reproducing the facility itself of the requesting factory in the virtual space can be considered. If this countermeasure is taken, the same environment as the factory can be reproduced in the simulation device, so that it is effective to some extent as the above-mentioned two-time countermeasure. However, in order to reproduce factory equipment with a simulation device, it is necessary to prepare and prepare 3D data (3D image data) of the factory equipment, and the work of generating 3D data of this factory equipment is very laborious. Since this is an operation, the design time may become very long, which is not a feasible measure. Even if factory equipment can be reproduced with a simulation device, it is difficult to reproduce fine contrast changes and pattern changes that occur due to aging in the actual factory. May need to be readjusted. In particular, the inability to reproduce fine contrast changes, pattern changes, and the like can be a major problem for visual inspection systems. Examples of simulation apparatuses are shown in Patent Documents 1 and 2.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image data generation device of a simulation device that can reproduce an environment similar to a factory at a delivery destination in a simulation device without lengthening the design time.

  According to the first aspect of the present invention, a plurality of two-dimensional photographed images obtained by photographing the background of the factory with a plurality of photographing angles from the set position of the inspection camera in the actual factory, respectively, are obtained as the photographing angle data. And means for inputting to the simulation apparatus, means for inputting the three-dimensional data of the work to be inspected to the simulation apparatus, means for setting the position of the inspection camera and the position of the work in the simulation apparatus, A two-dimensional photographed image having photographing angle data that matches the photographing angle of the inspection camera is selected from a plurality of two-dimensional photographed images, and means for setting the selected image as a background image; and the position of the background image The setting is made so that the plane of the background image is orthogonal to the optical axis of the inspection camera, and the photographing angle of the inspection camera Means for maintaining the orthogonal relationship of the plane of the background image with respect to the optical axis of the inspection camera when changing, and the inspection apparatus in which the workpiece is arranged on the background image in the simulation apparatus Since the image obtained by photographing with a camera is obtained by calculation, and the means for outputting the obtained image as an inspection camera image is provided, a simulation device that simulates an environment similar to the factory of the delivery destination without lengthening the design time Can be reproduced.

  Hereinafter, an embodiment in which the present invention is applied to a simulation apparatus for simulating a visual inspection system will be described with reference to the drawings. As shown in FIG. 1, the simulation apparatus 1 according to this embodiment includes a personal computer 2 and a simulation program (simulator) 3 installed in the personal computer 2.

  When the designed visual inspection system is simulated by the simulation apparatus 1, the visual inspection program 4 constituting the visual inspection system is taken into the simulation apparatus 1 (personal computer 2). And about the inspection camera 5, light source 6, etc. which comprise a visual inspection system, the virtual inspection camera 5, virtual light source 6, etc. which correspond virtually each using the program in the simulation apparatus 1 (personal computer 2) beforehand. Is provided. The position and orientation of the inspection camera 5 can be freely set and changed with the mouse or keyboard of the simulation apparatus 1. Further, conditions (illuminance, position, direction, color, etc.) of the light source 6 and the like can be freely set and changed with a mouse, a keyboard, or the like.

  Furthermore, for the inspection target workpiece 7, 3D data of the inspection target workpiece 7 is created in advance using 3D CAD, and the created 3D data (or 3D data provided by the supplier of the workpiece 7) is used as the simulation device 1 ( The data is taken in (input) to the storage unit in the personal computer 2). In addition, a PLC (programmable controller) 8 that outputs a trigger signal for instructing the timing of photographing by the inspection camera 5 is connected to the simulation apparatus 1 (personal computer 2). The PLC 8 has the same control function as that of a PLC incorporated in a factory facility that delivers a visual inspection system.

  Next, two-dimensional image data of the factory equipment to be delivered, specifically, an image (hereinafter referred to as a two-dimensional background image) that becomes the background of the inspection target workpiece 7 when the inspection target workpiece 7 is photographed by the inspection camera 5. ) Is prepared (created). In this case, as shown in FIG. 2A, it is assumed that two-dimensional background images 9a, 9b,... Are prepared, and these two-dimensional background images 9a, 9b,. In this case, as shown in FIG. 2B, the two-dimensional background images 9a and 9b are arranged so as to be orthogonal to the optical axes L1 and L2 of the inspection camera 5. It is assumed that the two-dimensional background image 9a corresponds to the photographing position (photographing angle) P1 of the inspection camera 5 along the optical axis L1, and the two-dimensional background image at the photographing position (photographing angle) P2 of the inspection camera 5 along the optical axis L2. 9b corresponds.

  Then, as shown in FIG. 2B, the 3D data of the inspection target workpiece 7 is arranged at a predetermined position (position where the workpiece 7 is arranged), and the inspection target workpiece 7 is detected by the inspection camera 5 at the imaging position (imaging angle) P1. Can be photographed as shown in FIG. 2C. The photographed image 10a (image with the two-dimensional background image 9a as the background) can be photographed. Similarly, when the inspection target workpiece 7 is photographed by the inspection camera 5 at the photographing position (photographing angle) P2, a photographed image 10b (an image with the background of the two-dimensional background image 9b) as shown in FIG. 2C is photographed. Can do.

  In the following, dimensional background images 9a and 9b are prepared in which the captured images 10a and 10b are similar images (that is, substantially equivalent images for image processing) similar to images captured by actual factory equipment. The procedure of (creating) will be specifically described. In order to facilitate understanding, the example will be described in a two-dimensional space, but in reality it is extended to a three-dimensional space and processed in the same way.

  First, as shown in FIG. 3, an image serving as the background of the work 7 to be inspected is taken with a digital camera 9 in an actual factory facility. In this case, the photographer 10 sequentially photographs the background of the work 7 to be inspected from the camera setting position P of the inspection camera 5 at, for example, five photographing positions 1 to 5, that is, five photographing angles B1 to B5. The photographed two-dimensional images A1 to A5 and the photographing angle data B1 to B5 are associated and recorded in pairs. Note that the imaging angle is set to increase (or decrease) in the order of the two-dimensional images A1B1, A2B2,... AnBn.

  Then, the two-dimensional background images A1B1, A2B2,... Prepared as described above are taken (input) together with the photographing angle data B1 to B5 into a storage unit (hard disk or the like) in the simulation apparatus 1 (personal computer 2). ). In this case, the two-dimensional background images A1B1, A2B2,... Are input to the simulation apparatus 1 through a memory card reader from the memory card or the like recorded, and the photographing angle data B1 to B5 are input by operating the keyboard or the like. It is the composition which makes it.

  When the processing for inputting the two-dimensional background images A1B1, A2B2,... And the 3D data of the inspection target workpiece 7 to the simulation apparatus 1 is completed, the position of the inspection camera 5 in the simulation apparatus 1 is determined. The position of the workpiece 7 to be inspected is set. Subsequently, in the simulation apparatus 1, a two-dimensional photographed image having photographing angle data matching the photographing angle of the inspection camera 5 is selected from the input two-dimensional background images A1B1, A2B2,. A process of setting the selected image as a background image is executed.

  Specifically, as shown in FIG. 4, when the shooting angle C of the inspection camera 5 matches the shooting angle data B1, the two-dimensional background image A1 is set as the background image. Similarly, when the shooting angle C of the inspection camera 5 matches the shooting angle data B2, the two-dimensional background image A2 is set as a background image, and when the shooting angle C of the inspection camera 5 matches the shooting angle data B3, it is two-dimensional. When the background image A3 is set as the background image and the shooting angle C of the inspection camera 5 matches the shooting angle data B4, the two-dimensional background image A4 is set as the background image, and the shooting angle C of the inspection camera 5 is set as the shooting angle data B5. If the two match, the two-dimensional background image A5 is set as the background image.

Even if the shooting angle C of the inspection camera 5 does not match the shooting angle data B1, the two-dimensional background image A1 is used as the background image until the intermediate angle between the shooting angle data B1 and the shooting angle data B2. If set and the intermediate angle is exceeded, the two-dimensional background image A2 is set as the background image. That is,
(B (i−1) + Bi) / 2 <C ≦ (Bi + B (i + 1)) / 2
Is set so that the two-dimensional background image Ai is set as the background image. An example of setting the background image is shown in FIG.

  That is, when the photographing angle C of the inspection camera 5 is within the angle C1 in FIG. 4, the two-dimensional background image A1 is set, and when the photographing angle C is within the angle C2 in FIG. 4, the two-dimensional background image A2 is set. When the shooting angle C is within the angle C3 in FIG. 4, the two-dimensional background image A3 is set. When the shooting angle C is within the angle C4 in FIG. 4, the two-dimensional background image A4 is set. When the shooting angle C is within the angle C5 in FIG. 4, a two-dimensional background image A5 is set.

  Then, as shown in FIG. 5, the position of the two-dimensional background image Ai is set so that the plane of the two-dimensional background image Ai is orthogonal to the optical axis 11 of the inspection camera 5. Furthermore, as shown in FIG. 6, when the imaging angle of the inspection camera 55 changes, the orthogonal relationship of the plane of the two-dimensional background image Ai with respect to the optical axis 11 of the inspection camera 5 is maintained. . For example, when the shooting angle C of the inspection camera 5 is within the angle C1 in FIG. 4, the same two-dimensional background image A1 is continuously set. At this time, as shown in FIG. It is set so that the orthogonal relationship of the plane of the two-dimensional background image A1 with respect to the optical axis 11 of the inspection camera 5 is maintained when C changes within the angle C1.

  The reason why the same two-dimensional background image Ai is continuously set when the imaging angle C of the inspection camera 5 is within the angle Ci in FIG. 4 is that the imaging angle C of the inspection camera 5 is within the angle Ci. Since the background of the work 7 to be inspected does not change much, there is almost no problem even if the same two-dimensional background image Ai is set continuously. In particular, since the object to be image-processed is the inspection object workpiece 7, if the image of the inspection object workpiece 7 is accurate, no problem will occur even if the background image is slightly inaccurate. In other words, the background is the background, and it doesn't have to be very accurate. In this case, since the prepared two-dimensional background images are only the five two-dimensional background images A1 to A5, the data amount of the prepared images can be greatly reduced, and the image processing is sufficiently accurate. Can be executed. Therefore, a sufficient effect can be obtained while the labor for preparing the image data can be greatly reduced.

  In addition, when the inspection camera 5 (camera viewpoint) is moved in parallel, the two-dimensional background image Ai is set to move in parallel as shown in FIG. In this case, it is configured such that it can be translated in three directions: the vertical direction, the horizontal direction, and the depth direction.

  Next, in the simulation apparatus 1, the control content of an example of processing for setting the two-dimensional background image Ai so that the plane of the two-dimensional background image Ai is orthogonal to the optical axis 11 of the inspection camera 5 is a flowchart of FIG. Will be described with reference to FIG. First, in step S10, the prepared two-dimensional background image Ai and the photographing angle data Bi are registered (input) in the storage unit in the simulation apparatus 1 (personal computer 2). In this case, the number of registered pairs of the two-dimensional background image Ai and the shooting angle data Bi is n (thus, i = 1,..., N).

  Subsequently, the process proceeds to step S20, where the two-dimensional background image Ai is sorted with reference to the shooting angle data Bi (A1B1, A2B2,..., AnBn). In this case, ascending order is used for explanation, but descending order may be used. In step S30, the position of the inspection camera 5 is set, and the imaging angle (camera viewpoint angle) C of the inspection camera 5 is acquired.

  Next, the process proceeds to step S40, and it is determined whether or not the shooting angle C is smaller than the shooting angle data B1. Here, when the shooting angle C is smaller than the shooting angle data B1, the process proceeds to “YES”, the process proceeds to step S130, and the two-dimensional background image A1 is set as the image G to be set.

On the other hand, when the shooting angle C is not smaller than the shooting angle data B1 in step S40, the process proceeds to “NO”, the process proceeds to step S50, and “0” is set to the variable i. Then, it progresses to step S60,
(B (i−1) + Bi) / 2 <C ≦ (Bi + B (i + 1)) / 2
Whether or not is satisfied is determined. Here, when the above equation is established, the process proceeds to “YES”, and the process proceeds to step S140, where the two-dimensional background image Ai is set as the image G to be set.

  In step S60, when the above equation is not satisfied, the process proceeds to “NO”, and the process proceeds to step S70, where the variable i is counted up (+1). Then, the process proceeds to step S80 to determine whether or not the variable i is smaller than n. If the variable i is smaller than n, the process proceeds to “YES”, returns to step S60, and repeats the same process. On the other hand, when the variable i is not smaller than n in step S80, the process proceeds to “NO”, and the process proceeds to step S90, where the two-dimensional background image An is set as the image G to be set.

  Thereafter, the process proceeds to step S100, and the reverse vector (V) of the optical axis 11 extending from the viewpoint of the inspection camera 5 is calculated as shown in FIG. And it progresses to step S110 and the plane P from which a normal line turns into a reverse vector (V) is produced. In step S120, the image G (that is, the two-dimensional background image Ai) is pasted on the plane P. Thereby, the process of setting the position of the two-dimensional background image Ai so that the plane of the two-dimensional background image Ai is orthogonal to the optical axis 11 of the inspection camera 5 is completed.

If adjustment of ambient light or the like is necessary on the screen of the simulation apparatus 1, the adjustment is performed.
In the next processing step, the equipment operation is described by the simulation apparatus 1.

Further, in the next processing step, the equipment operation simulation is started in the simulation apparatus 1.
Subsequently, in the next processing step, an image in the virtual space is captured by the inspection camera 5 at the timing of external trigger input.

In the next processing step, the captured image data is output to the visual inspection program.
Further, in the next processing step, a visual inspection program is executed to perform image processing.

  Now, an example of processing for creating the above-described two-dimensional background image Ai for the simulation apparatus 1 and an operation of the simulation apparatus 1 will be described with reference to FIGS.

  In this example, as shown in FIG. 9, in a target facility T of an actual factory, a work 7 to be inspected (virtual work made of 3D data) is placed at a position indicated by a circle, and a missing part of the work 7 is detected by image processing. Design a visual inspection system to inspect. In this case, processing for determining the position of the inspection camera 5 most suitable for image processing of the inspection target workpiece 7 from the result of actual image processing is performed.

  First, as shown to Fig.10 (a), about the target equipment T of the existing factory, it photographed from five positions (0)-(4) using the digital camera 9, and five two-dimensional background images A1- A5 (see FIG. 11) is acquired. Of the five positions (0) to (4), the position (0) is the position of the intersection of the XY axes perpendicular to the XY plane, and the positions (1) to (4) are respectively on the XY plane. The positions are 45 degrees, 135 degrees, −135 degrees, and −45 degrees (see FIG. 10B), and the height direction is a 45 degree position on the XZ plane (see FIG. 10C). .

  Next, the photographed two-dimensional background images A1 to A5 and photographing condition information (photographing angle data) are registered in pairs in the storage unit in the simulation apparatus 1 (see FIG. 11). Then, 3D data (see FIG. 12) of the design inspection target workpiece 7 is prepared, and this is taken in (registered) in the storage unit in the simulation apparatus 1.

  Thereafter, in the simulation apparatus 1, the two-dimensional background images A <b> 1 to A <b> 5 and the 3D data of the workpiece 7 to be inspected are overlaid in correspondence with the position and the imaging angle of the inspection camera 5, and the image captured by the inspection camera 5 is virtually Create pseudo (by calculation) in space. In FIG. 13, the example B1-B5 of creation of the picked-up image by the inspection camera 5 is shown. In FIG. 13, reference numerals (0) to (4) correspond to five positions (imaging angles) (0) to (4).

The photographed images B1 to B5 created as described above are given to the visual inspection program, and image processing is executed by the visual inspection program.
According to this embodiment having such a configuration, a plurality of two-dimensional photographed images Ai obtained by photographing the background of the factory at a plurality of photographing angles with a digital camera from the set position of the inspection camera 5 in an actual factory. , Input si to the simulation apparatus 1 together with the photographing angle data Bi, three-dimensional data of the work 7 to be inspected is input to the simulation apparatus 1, and the position of the inspection camera 5 and the position of the work 7 are set in the simulation apparatus 1. A two-dimensional photographed image Ai having photographing angle data Bi that matches the photographing angle of the inspection camera 5 is selected from a plurality of two-dimensional photographed images Ai. The selected image is used as a background image, and the position of the background image is determined. The plane of the background image is set to be orthogonal to the optical axis of the inspection camera 5, and the shooting angle of the inspection camera 5 is changed. Sometimes, the orthogonal relationship of the plane of the background image with respect to the optical axis of the inspection camera 5 is maintained, and in the simulation apparatus 1, the inspection camera 5 in which the 3D data of the work 7 is arranged on the background image is superimposed. Since the image obtained by shooting is obtained by calculation and the obtained image is output as an inspection camera image, an environment similar to the factory of the delivery destination can be obtained without increasing the design time of the visual inspection system. It can be easily reproduced in the simulation apparatus 1.

Further, in the above embodiment, even when the shooting angle C of the inspection camera 5 does not coincide with the shooting angle data B1, the two-dimensional background is obtained up to the intermediate angle between the shooting angle data B1 and the shooting angle data B2. When the image A1 is set as the background image and the intermediate angle is exceeded, the two-dimensional background image A2 is set as the background image. That is,
(B (i−1) + Bi) / 2 <C ≦ (Bi + B (i + 1)) / 2
Since the two-dimensional background image Ai is set as the background image, the two-dimensional background images to be prepared are only five two-dimensional background images A1 to A5. The image processing can be performed sufficiently accurately.

The perspective view of the simulation apparatus which shows one Example of this invention The figure which shows the arrangement | positioning relationship between a test | inspection camera, a test object workpiece | work, and a two-dimensional background image The figure which shows a mode that 2D background image of factory equipment is photographed with the digital camera The figure which shows the relationship which matches the imaging | photography angle of an inspection camera, and a two-dimensional background image The figure which shows the relationship where the optical axis of an inspection camera and the plane of a two-dimensional background image orthogonally cross The figure which shows that the relationship which an optical axis and the plane of a two-dimensional background image orthogonally cross is maintained when the imaging | photography angle of an inspection camera changes. The figure which shows that a two-dimensional background image is also translated when an inspection camera is translated The flowchart which shows the control content of the process which sets the position of a two-dimensional background image so that the plane of a two-dimensional background image may orthogonally cross with respect to the optical axis of an inspection camera Perspective view showing factory equipment and workpieces to be detected Diagram showing how factory equipment is photographed with a digital camera Figure showing a two-dimensional background image taken with a digital camera Perspective view of workpiece (3D data) to be detected The figure which shows the image which overlap | superposed the detection object work (3D data) on the two-dimensional background image

Explanation of symbols

  In the drawings, 1 is a simulation device, 2 is a personal computer, 3 is a simulation program, 4 is a visual inspection program, 5 is an inspection camera, 6 is a light source, 7 is a work to be inspected, 8 is a PLC, 9 is a digital camera, and 10 is a photographer. , 11 indicates the optical axis.

Claims (1)

Means for inputting a plurality of two-dimensional photographed images obtained by photographing a background of the factory at a plurality of photographing angles from a set position of an inspection camera in an actual factory together with the photographing angle data to a simulation device;
Means for inputting three-dimensional data of a workpiece to be inspected into the simulation apparatus;
In the simulation device, means for setting the position of the inspection camera and the position of the work,
Means for selecting a two-dimensional photographed image having photographing angle data matching a photographing angle of the inspection camera from the plurality of two-dimensional photographed images, and setting the selected image as a background image;
The position of the background image is set so that the plane of the background image is orthogonal to the optical axis of the inspection camera, and the inspection camera changes when the imaging angle of the inspection camera changes. Means for maintaining an orthogonal relationship of the plane of the background image to the optical axis of
In the simulation apparatus, there is provided means for obtaining an image obtained by taking an image obtained by photographing the work piece superimposed on the background image by the inspection camera and outputting the obtained image as an inspection camera image. An image data generation device of a simulation device characterized by the above.

Images