JP2005337870A - Detection system and labeled object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a detection system and a labeled object used for the detection system realizing improvement in the detection accuracy at low cost without depending on an installation environment. <P>SOLUTION: By being related to an identification code possessed by a pattern attached to a marker M1, types of electronic components housed in a component box 71 and relative positions of the electronic components with respect to the marker M1 are provided in advance. When the electronic components 71a to 71f are housed in the component box 71, they are housed therein according to the provisions. When detecting the marker M1, an information processor captures a photographed image photographed from above the component box 71. On the basis of the captured photographed image, analysis of the pattern on the marker M1 is performed to acquire the identification code. From the acquired identification code, information on the kind and position which is housed is acquired. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a detection system that can improve detection accuracy at low cost without depending on the environment, and a label used in the detection system.

  Conventionally, when detecting the position, shape, type, etc. of an object to be detected by image processing, in order to facilitate the detection, a label having a known size or pattern is attached to the object to be detected, and the position of the object is detected. A system that detects the position, shape, type, and the like of a detected object by recognizing the type has been proposed (for example, Patent Document 1).

  Since the marker used in such a system can be easily produced, a marker produced by printing a pattern on paper, a marker produced by plastic or vinyl resin, or the like may be used. It is common.

Further, in order to give the marker itself discriminability, a pattern as shown in the schematic diagram of FIG. 13 has been proposed as a pattern to be attached to the marker. In the pattern shown in FIG. 13A, the shape and position of the pattern are recognized by image processing, and the bit values of 5 × 5 cells inside the pattern are determined, whereby 2 ⁵ associated with the pattern. Get bit identification code. The pattern shown in FIG. 13B is also the same. The pattern shape and position are recognized by image processing, and the pattern attached to the marker is recognized by template matching with a pattern model prepared in advance. can do.
JP-A-9-21612

  However, when detecting the position, shape, type, etc. of a detected object using a label produced by printing a pattern on paper and a label produced by plastic or vinyl resin, etc., a light source, a label, an imaging device Depending on the positional relationship, the surface of the marker may strongly reflect the light from the light source. When this reflected light is directly incident on the image pickup device, it is overexposed and the image obtained by the image pickup device is white. Tobi phenomenon occurs. In an image in which such a white stripe phenomenon has occurred, the original color and shade of the sign cannot be reproduced, which makes it difficult to recognize the pattern itself, resulting in a problem that the recognition rate of the sign decreases. .

  In addition, in the system described in Patent Document 1 in order to increase the recognition accuracy of the sign object, a form using a sign object such as a light emitting diode is cited, but it cannot be used except in an environment where a power source can be secured. And has a problem of raising the cost of the entire system.

  Further, when recognizing the pattern shown in FIG. 13, first, a point sequence that forms a boundary between the background and the marker is extracted, and the four sides of the marker in the image are extracted using the least square method for the extracted point sequence. A line segment is applied to and the shape and position of the marker in the image are recognized. However, when the distance between the marker and the imaging device increases and the resolution of the marker as a whole decreases, there is a problem in that the side indicating the boundary of the marker cannot be accurately detected due to the influence of the quantization error. . FIG. 14 is a schematic diagram showing the influence of the quantization error at the time of edge detection. When a dot sequence obtained by extracting the boundary between the background and the marker is represented by a black circle, the actual marker edge is the broken line L0. On the other hand, the line segment calculated | required by the least squares method is extracted like the continuous line L1, for example. At this time, since the shape and position of the marker in the image cannot be accurately recognized, there is a problem that the recognition accuracy of the marker is lowered.

  The present invention has been made in view of such circumstances, and a part or all of the marker body disposed in the vicinity of the detected object when the position or type of the detected object is detected by the diffuse reflection member. By adopting a configuration that uses the formed marker, the recognition accuracy of the marker based on image analysis is constant even when the positional relationship between the marker and the imaging device that images the marker is arbitrarily set. It is an object of the present invention to provide a detection system capable of maintaining the level and improving the detection accuracy of the position or type of the detection target and a label used in the detection system.

  The detection system according to the first aspect of the present invention captures a label to be placed in the vicinity of one or a plurality of objects to be detected, and an image obtained by imaging the object to be detected together with the label, and within the captured image And a detection device that detects the position or type of the detected object based on the image of the label, wherein a part or all of the label is formed by a diffuse reflection member. To do.

  In the first invention, a part or all of the marker to be imaged together with the detected object is formed by the diffuse reflection member, so that the marker does not strongly reflect the light from the external light source in the specific direction. Therefore, even when the arrangement relationship of the light source, the sign body, and the imaging device is arbitrarily set, the so-called white stripe phenomenon is suppressed, and the original color and shade of the sign body cannot be captured due to overexposure. , The recognition accuracy of the marker based on the captured image is improved.

  A detection system according to a second aspect of the present invention is the detection system according to the first aspect of the present invention, comprising a plurality of labeled bodies with different patterns, and when detecting the position or type of the detected body, The labeling body is arranged in the vicinity of the object to be detected.

  In the second invention, a plurality of labeled bodies with different patterns are provided, and one labeled body is arranged in the vicinity of the detected body when detecting the position or type of the detected body. By preliminarily defining the correspondence between each pattern and information on the detected object, information on the detected object is obtained together with information on the position of the detected object when the pattern of the marker is detected by image analysis. It is done.

  A detection system according to a third invention is the detection system according to the second invention, wherein the detection device corresponds to identification information for identifying a plurality of patterns, and each of the detected information to be detected. Storage means for storing information defining the relative position between the body and the marker, and identification information acquisition means for acquiring identification information for identifying the pattern based on the pattern image included in the captured image, The position of each detected object is detected based on the acquired identification information and the identification information stored in the storage means.

  In the third aspect of the invention, since the information defining the relative position between each detected object and the labeled body is stored in association with the identification information for identifying each pattern, it is attached to the labeled body from the captured image. By analyzing the pattern thus obtained and acquiring identification information, the position of the detected object is detected.

  The detection system according to a fourth aspect of the present invention is the detection system according to the third aspect, wherein the detection device stores information relating to the detected object to be detected in association with each of the identification information, and the identification information acquisition Means for acquiring information relating to the detected object when the position of each detected object is detected based on the identification information acquired by the means.

  In the fourth aspect of the invention, since information related to the detected object is stored in association with identification information for identifying each pattern, the pattern attached to the marker is analyzed from the captured image to identify the information. For example, information such as the type, size, color, and shape of the detected object can be obtained.

  A detection system according to a fifth invention is the detection system according to any one of the second to fourth inventions, wherein the pattern includes a plurality of figures, and each figure is given one of at least two colors. It is characterized by that.

  In the fifth invention, the pattern attached to the marker includes a plurality of figures, and each figure is given at least one of two colors, so that the image analysis based on the captured image is performed. At this time, the identification information can be acquired by acquiring the color array.

  A detection system according to a sixth aspect of the present invention is the detection system according to any one of the first to fifth aspects, wherein the optical characteristic of the diffuse reflection member is a regular reflectance with respect to incident light having an arbitrary incident angle. And diffuse reflectance are substantially the same.

  In the sixth aspect of the invention, since the diffuse reflection member is used in which the regular reflectance and the diffuse reflectance are substantially the same for incident light having an arbitrary incident angle, the regular reflection component of the reflected light is reduced. It can be weakened and is not strongly reflected in a specific direction by the marker. Therefore, it is less likely that the original color and shade of the sign body cannot be captured due to overexposure, and the recognition accuracy of the sign body based on the captured image is improved.

  The detection system according to a seventh aspect is the detection system according to the sixth aspect, wherein the regular reflectance and the diffuse reflectance are 20% or less.

  In the seventh invention, since the regular reflectance and the diffuse reflectance are 20% or less, the sign body does not strongly reflect in a specific direction. Therefore, it is less likely that the original color and shade of the sign body cannot be captured due to overexposure, and the recognition accuracy of the sign body based on the captured image is improved.

  The detection system according to an eighth aspect of the present invention is the detection system according to any one of the first to seventh aspects, wherein the marker is circular.

  In the eighth invention, since the marker is circular, the influence of the quantization error is reduced when analyzing the captured image, and the marker recognition accuracy is improved.

  A detection system according to a ninth aspect of the invention is the detection system according to any one of the first to eighth aspects, wherein the marker is formed of felt.

  In the ninth invention, since the marker is formed of felt, diffuse reflection is caused by the marker, and it is not strongly reflected in a specific direction. As a result, it is less likely that the original color and shade of the sign body cannot be imaged due to overexposure, and the recognition accuracy of the sign body based on the captured image is improved.

  A labeling body according to a tenth aspect of the present invention is a labeling body that is disposed in the vicinity of the detected object when the position or type of the detected object is detected by a detection device. It is characterized by being.

  In the tenth invention, a part or all of the marker is formed by the diffuse reflection member, so that the marker does not strongly reflect the light from the external light source in a specific direction. Therefore, even when the arrangement relationship of the light source, the sign body, and the imaging device is arbitrarily set, the so-called white stripe phenomenon is suppressed, and the original color and shade of the sign body cannot be captured due to overexposure. , The recognition accuracy of the marker based on the captured image is improved.

  An indicator according to an eleventh aspect of the invention is the optical indicator according to the tenth aspect of the invention, wherein the diffuse reflection member has substantially the same regular reflectance and diffuse reflectance with respect to incident light having an arbitrary incident angle. It has the characteristics.

  In the eleventh aspect of the invention, since the diffuse reflection member having the same regular reflectance and diffuse reflectance with respect to incident light having an arbitrary incident angle is used, the regular reflection component of the reflected light is used. It can be weakened and is not strongly reflected in a specific direction by the marker. Therefore, it is less likely that the original color and shade of the sign body cannot be captured due to overexposure, and the recognition accuracy of the sign body based on the captured image is improved.

  The marker according to a twelfth aspect of the invention is characterized in that, in the marker according to the eleventh aspect of the invention, the regular reflectance and the diffuse reflectance are 20% or less.

  In the twelfth aspect, since the regular reflectance and the diffuse reflectance are 20% or less, the sign body does not strongly reflect in a specific direction. Therefore, it is less likely that the original color and shade of the sign body cannot be captured due to overexposure, and the recognition accuracy of the sign body based on the captured image is improved.

  A labeled body according to a thirteenth aspect of the present invention is the labeled body according to any of the tenth to twelfth aspects of the present invention, wherein the labeled body is formed of felt.

  In the thirteenth aspect, since the marker is formed of felt, diffuse reflection is caused by the marker, and it is not strongly reflected in a specific direction. As a result, it is less likely that the original color and shade of the sign body cannot be imaged due to overexposure, and the recognition accuracy of the sign body based on the captured image is improved.

  In the case of the first invention and the tenth invention, part or all of the marker imaged together with the detected object is formed by the diffuse reflection member, so that the marker light strongly reflects the light from the external light source in a specific direction. Disappears. That is, even when the arrangement relationship of the light source, the sign body, and the imaging device is arbitrarily set, the so-called white stripe phenomenon can be suppressed, and the original color and shade of the sign body cannot be imaged due to overexposure. As a result, the recognition accuracy of the marker based on the captured image can be improved. As a result, it is possible to improve the detection accuracy of the position or type of the detected object.

  In the case of the second invention, it is provided with a plurality of labeled bodies with different patterns, and when detecting the position or type of the detected object, one labeled object is arranged in the vicinity of the detected object. By predefining the correspondence between each pattern and information on the detected object, information on the detected object is acquired together with information on the position of the detected object when the pattern of the marker is detected by image analysis. be able to.

  In the case of the third invention, since the information defining the relative position between each detected object and the marker is stored in association with the identification information for identifying each pattern, the information is attached to the marker from the captured image. The position of the detected object can be detected by analyzing the pattern and acquiring the identification information.

  In the case of the fourth invention, since information related to the detected object is stored in association with the identification information for identifying each pattern, the identification information is obtained by analyzing the pattern attached to the marker from the captured image. By acquiring, for example, information such as the type, size, color, and shape of the detected object can be acquired.

  In the case of the fifth invention, the pattern attached to the marker includes a plurality of figures, and each figure is given at least one of two colors. Therefore, when performing image analysis based on the captured image The identification information can be acquired by acquiring the color array.

  In the case of the sixth invention and the eleventh invention, since the diffuse reflection member whose regular reflectance and diffuse reflectance are substantially the same for incident light having an arbitrary incident angle is used, regular reflection of reflected light is used. This component can be weakened and is not strongly reflected in a specific direction by the marker. Therefore, it is less likely that the original color and shade of the sign body cannot be captured due to overexposure, and the recognition accuracy of the sign body based on the captured image can be improved.

  In the case of the seventh and twelfth inventions, since the regular reflectance and the diffuse reflectance are 20% or less, the marker body does not strongly reflect in a specific direction. Therefore, it is less likely that the original color and shade of the sign body cannot be captured due to overexposure, and the recognition accuracy of the sign body based on the captured image can be improved.

  In the case of the eighth invention, since the marker is circular, the influence of the quantization error is reduced when analyzing the captured image, and the recognition accuracy of the marker can be improved.

  In the ninth and thirteenth inventions, since the marker is formed of felt, diffuse reflection is caused by the marker, and it is not strongly reflected in a specific direction. As a result, it is less likely that the original color and shade of the sign body cannot be captured due to overexposure, and the recognition accuracy of the sign body based on the captured image can be improved. In addition, since the marker can be formed of felt, there is an advantage that the cost of the entire system is not significantly increased even when a large number of markers are required.

Hereinafter, a form in which the detection system according to the present invention is applied to a work education system for educating electronic component mounting work will be specifically described with reference to the drawings.
FIG. 1 is a schematic diagram showing the overall configuration of the work education system according to the present embodiment. In the figure, reference numeral 50 denotes a work table on which work objects are placed. On the work table 50, component boxes 71 to 74 containing a printed circuit board 60 and various electronic components are placed as work objects. Yes. In the example shown in the figure, four component boxes 71 to 74 are arranged side by side on the front side of the work table 50, and the worker takes out electronic components from these component boxes 71 to 74 and mounts them on the printed circuit board 60. Do work.

  In order to monitor the work situation at this time, the printed circuit board 60 and the component boxes 71 to 74 are imaged from above the work table 50 using the imaging device 2. The imaging device 2 is specifically a CCD camera, and acquires, for example, 15 frames of 1024 × 768 pixels per second. The image acquired by the imaging apparatus 2 is sent to the information processing apparatus 1 such as a personal computer or a workstation. The information processing apparatus 1 recognizes the type and position of the components housed in the component boxes 71 to 74 and the mounting state on the printed circuit board 60 by analyzing the image sent from the imaging device 2, Properly give appropriate instructions to the staff.

  In this work education system, work information is presented to workers using the laser irradiation device 3, the monitor device 4, and the speaker 5. The monitor device 4 is a liquid crystal display device, a CRT display device or the like, and displays an image output from the information processing device 1 to present an electronic component mounting procedure, a correction instruction when there is an error in the mounting position, or the like. Do. In addition, the speaker 5 provides voice instructions for explanation of work contents and instructions for correction when there is an error in the mounting position. Further, the laser irradiation device 3 is installed above the work table 50 in the same manner as the imaging device 2, and irradiates the laser beam to the position indicated by the information processing device 1, and the components to be taken out from the component boxes 71 to 74. The work information is presented by notifying the worker of the position, the mounting position on the printed circuit board 60, and the like.

  FIG. 2 is a schematic diagram showing how work information is presented by the laser irradiation device 3. The laser irradiation device 3 includes a laser diode, a mirror unit, and a direction control device (see FIG. 6). The direction control device controls the mirror unit, and the irradiation direction of the laser light emitted from the laser diode is set to an arbitrary direction. An irradiation point by laser light is created at an arbitrary point on the work table 50. As shown in FIG. 2B, the mounting position of the electronic component can be presented to the worker by creating the irradiation point LP1 by the laser beam on the printed circuit board 60 placed on the work table 50. it can. Moreover, the position of the electronic component which should be mounted can be shown to an operator by making the irradiation point LP2 by a laser beam on the electronic component accommodated in the component boxes 71-74. In the example shown in FIG. 2C, a laser beam irradiation point LP <b> 2 appears inside the component box 72. The worker takes out the electronic component indicated by the irradiation point LP2 from the component box 72 and mounts the electronic component at the position indicated by the irradiation point LP1 on the printed circuit board 60, thereby completing the mounting operation of the electronic component.

  In order to accurately indicate the electronic component to be mounted by the laser beam irradiated by the laser irradiation device 3, it is necessary to obtain in advance information regarding the type, position, and number of components housed in the component boxes 71 to 74. is there. Therefore, in the present embodiment, markers M1 to M4 each having a different pattern are attached to the component boxes 71 to 74, respectively. The information processing apparatus 1 analyzes the image captured by the imaging apparatus 2 and reads the patterns attached to the markers M1 to M4, and acquires identification information for identifying each pattern. Information on the type and number of electronic components in the component box used for the mounting operation and the position of each electronic component is stored in the information processing apparatus 1 in association with the identification information, so that when the pattern is read, This information can be acquired, and an accurate instruction can be given to the laser irradiation device 3.

FIG. 3 is a schematic diagram illustrating a configuration example of the component box 71. The marker M1 is attached at an appropriate position in the component box 71. In the component box 71 to which the marker M1 is attached, the types and the number of electronic components 71a to 71f defined by the marker M1 are predetermined. Is housed in. The arrangement of the electronic components 71a to 71f is defined in advance by a relative position with respect to the marker M1, and the information processing apparatus 1 can detect the position of each of the electronic components 71a to 71f by detecting the position of the marker M1. . The same applies to the component boxes 72 to 74, and the positions of the electronic components accommodated in the respective component boxes 72 to 74 can be detected by detecting the positions of the markers M2 to M4.
In the following description, when it is not necessary to distinguish and explain the markers M1 to M4 attached to the component boxes 71 to 74, they are simply referred to as markers M.

  FIG. 4 is a schematic diagram illustrating details of the marker M. In the present embodiment, a circular marker M having a radius of about several centimeters is used as a marker to be placed in the vicinity of an electronic component that is a detection target. Since the component boxes 71 to 74 are placed at positions where workers can easily work, there is a situation in which the positional relationship between the light source and the component boxes 71 to 74 in the work site cannot be kept constant. Therefore, as a material of the marker M, even when light having an arbitrary incident angle is incident, the marker M does not reflect the light strongly toward the imaging device 2, that is, diffuses the light incident on the marker M. It is necessary to select one that can reflect and weaken the reflected light toward the imaging device 2. Therefore, in the present embodiment, felt cloth is used as the material of the marker M. Felt fabric is a combination of fine fibers that are not regular, and when light strikes, each of the fine fibers that are combined irregularly reflects light, so the reflection direction is fixed in a certain direction. Therefore, the reflected light toward the image pickup device 2 can be weakened as compared with a paper marker, and is therefore suitable as a material for the marker M.

  As a result of investigations by the inventors, it has been found that the marker M preferably has the following optical characteristics. The black portion of the marker M has a reflectance of about 20% or less with respect to all reflection angles regardless of the angle between the marker M and the light source with respect to the wavelengths of all visible light regions. Is required to be uniform in the visible light region regardless of the wavelength. The red portion of the marker M has a reflectivity of about all reflection angles with respect to light having a wavelength in the visible light region except for the red wavelength region, regardless of the angle between the marker M and the light source. 20% or less, the total reflectance is approximately 40% or more for the light in the red wavelength region, regardless of the angle between the marker M and the light source, and is large between the regular reflectance and the diffuse reflectance. It is required that there is no difference.

  Further, in the present embodiment, an octagonal pattern configured by combining eight triangles m0 to m7 is employed as a pattern to be attached to the marker M. Of the eight triangles m0 to m7, one is assigned white and the other seven are assigned either red or black. Further, the outer portion of the pattern is red. Hereinafter, the triangle m0 with white is referred to as a direction identifier m0, and the triangles m1 to m7 with red or black are referred to as cells m1 to m7. “0” is assigned to the red cells (m1 to m7), “1” is assigned to the black cells (m1 to m7), the array of “1” and “0” is obtained from the direction identifier, and the marker M is identified. To obtain identification information. Hereinafter, the arrangement is referred to as an identification code.

FIG. 5 is a schematic diagram showing an example of a pattern attached to the marker M. In the example shown in FIG. 5A, since cells of black, red, black, red, black, red, and black are arranged starting from the direction identifier m0, the information processing apparatus 1 displays the captured image. By analyzing, the identification code “1010101” is acquired. In the example shown in FIG. 5B, since the cells of black, black, red, black, black, red, and red are arranged starting from the direction identifier m0, the information processing apparatus 1 has taken in By analyzing the image, an identification code “0011011” is acquired. Thus, because it defines the identification code by a combination of colors subjected to cell M1 to M7, when using the marker M of the present embodiment, the kind of electronic components are two ⁷ (128 patterns) and The arrangement can be identified.

  FIG. 6 is a block diagram showing the configuration of the work education system according to the present embodiment. The information processing apparatus 1 includes a CPU 11, which includes a ROM 13, a RAM 14, an image memory 15, an operation unit 16, an HDD 17 (HDD: Hard Disk Drive), a monitor IF 18, a video capture IF 19, a DA / AD converter 20, A sound IF 23 is connected via the bus 12. Various control programs are stored in the ROM 13 in advance, and the CPU 11 causes the information processing apparatus 1 to function as a detection device by loading and executing the control program. The RAM 14 is configured by a flash memory or the like, and temporarily stores data generated when the control program is executed, data input through the operation unit 16, and the like.

  The image memory 15 temporarily stores image data captured through the video capture IF 19, image data generated during the marker M detection process, and the like. The operation unit 16 includes an input device such as a keyboard, a mouse, and a tablet, and accepts an instruction from a user such as an operation for selecting an education course.

  The HDD 17 has a magnetic recording medium, and a part of the storage area is used as an education course information DB 17a, a printed circuit board information DB 17b, and a marker identification information DB 17c (DB: database). A read process or a write process is executed accordingly, and necessary data is extracted or stored. Each database 17a-17c can be described by XML (eXtensible Markup Language), for example.

  The education course information DB 17a defines how to advance work education. Therefore, in the education course information DB 17a, the file path of the XML file describing the printed circuit board information DB 17b, the file path of the XML file describing the marker identification information DB 17c, the image information comment information to be output to the monitor device 4, and the sound IF 23 Are stored in association with each other, as well as audio data output through the terminal and information on the type of electronic component to be mounted.

  The printed circuit board information DB 17b defines the dimensions of the printed circuit board, the type of electronic component to be mounted, the dimensions of the electronic equipment to be mounted, the mounting position of the electronic component, and the like. Therefore, the name of the printed circuit board, the horizontal length and vertical length of the printed circuit board, the type of electronic component to be mounted, the x and y coordinates of the mounting position, the horizontal length and vertical length of the electronic component They are stored in association with each other.

  In the marker identification information DB 17c, the type of electronic component and the arrangement of each component with respect to the marker M are stored in association with the identification code of the marker M.

  The monitor IF 18 is an interface for connecting the monitor device 4 to the outside, and sends image data read from the HDD 17 to the monitor device 4 when presenting work information to a worker. The video capture IF 19 is an interface for connecting the imaging device 2 to the outside. In the present embodiment, an interface corresponding to the IEEE 1394 standard is used. As a result, an image composed of 1024 × 768 pixels output from the imaging device 2 can be captured in 15 frames per second without being compressed.

  The DA / AD converter 20 performs analog / digital output mainly for controlling the laser diode 31 and the direction control device 33 of the mirror unit 32 provided in the laser irradiation device 3. That is, digital output is performed through the digital output terminal 22 in order to control on / off of the laser diode 31, and analog output is performed through the analog output terminal 21 in order to irradiate the laser light in the target direction by controlling the direction control device 33. I do.

  The sound IF 23 is an interface for connecting the speaker 5 to the outside. The sound IF 23 converts digital audio data read from the HDD 17 into an analog audio signal and sends it to the speaker 5 for audio output.

  Hereinafter, the flow of processing by the work education system will be specifically described with reference to flowcharts. FIG. 7 is a flowchart showing a procedure of processing executed by the work education system. First, the information processing apparatus 1 displays information on an education course prepared in advance on the monitor device 4, and accepts selection of an education course desired by a worker through the operation unit 16 (step S11). The CPU 11 of the information processing apparatus 1 reads necessary information from the education course information DB 17a based on the accepted selection and sets the education course. Next, based on the read educational course information, information such as the contents of the work and how to proceed with the work is notified by the monitor device 4 and the speaker 5, and the work information is presented to the worker (step S12). Then, as a first step of the mounting operation, the storage position of the electronic component to be mounted and the mounting position of the electronic component are indicated by the laser beam from the laser irradiation device 3 to instruct the operation of the operation (step S13).

  Next, work status recognition processing is performed (step S14). The process for recognizing work status will be described in detail later, but here, by sequentially analyzing the images captured through the video capture board, it is determined whether the work by the worker has been completed, the parts The type of the boxes 71 to 74 is recognized, the number of parts in the parts boxes 71 to 74 is recognized, and the position of the parts on the printed circuit board 60 is recognized.

  As a result of the recognition processing of the work situation, the CPU 11 of the information processing apparatus 1 determines whether or not there is an erroneous work (step S15), and if it is determined that there is an erroneous work (S15: YES), an error is given to the worker. A correction is instructed (step S16). The error correction instruction is, for example, informing the monitor device 4 and the speaker 5 of an indication that an electronic component is mounted at an incorrect location, an indication that an incorrect type of electronic component is mounted, and the like. Presentation of the original mounting position, electronic components, and the like is performed by notifying the monitor device 4 and the speaker 5. After instructing error correction, or when it is determined in step S15 that there is no erroneous work (S15: NO), the CPU 11 of the information processing apparatus 1 determines whether or not work education has ended (step S17). If it is determined that the education has not ended (S17: NO), the process returns to step S12. If it is determined that work education has ended (S17: YES), work education by this routine is ended.

  FIG. 8 is a flowchart showing a processing procedure of work status recognition processing. The information processing apparatus 1 first recognizes the position of the printed circuit board 60 from the image captured through the video capture IF 19 (step S21). In general, since the printed board is green and has a quadrilateral shape, in this embodiment, the position of the printed board 60 is recognized using this feature. Specifically, first, threshold processing is performed on an image captured using color information of the printed circuit board 60, and only pixels corresponding to the printed circuit board 60 are extracted. An image composed of the extracted pixels (extracted image) is scanned in the vertical and horizontal directions, and only the edges are extracted from the extracted image. Hough transform is performed on the extracted edge image, and the inclination and intercept in the image coordinate system of the four sides of the printed circuit board 60 are obtained. From the four side parameters obtained by the Hough transform, the coordinates of the four corners of the printed circuit board 60 are obtained. The obtained coordinate value is temporarily stored in the RAM 14.

  Next, the CPU 11 of the information processing apparatus 1 determines whether or not the work by the worker has been completed (step S22). Whether or not the work by the worker has been completed can be determined by counting the number of skin-colored pixels in the printed circuit board area obtained in step S21 and whether or not the counted number of pixels exceeds a predetermined threshold. . That is, since the printed circuit board 60 is green as described above, when the number of skin-colored pixels is larger than the predetermined value in the printed circuit board region, it is determined that there is an operator's hand on the printed circuit board 60. Therefore, it can be determined that the work is in progress. On the contrary, when the image transitions from the state where the worker's hand is present on the printed circuit board 60 to the state where the worker's hand is not present, it can be determined that the work by the worker has been completed.

  When it is determined that the work by the worker has not been completed (S22: NO), the process waits until the work is completed, and when it is determined that the work by the worker has been completed (S22: YES), marker detection processing is performed. (Step S23). The marker detection process will be described in detail later. Here, the type of marker is identified by image processing, and the identification code assigned to the marker is acquired from the pattern attached to the marker. Since the marker identification information DB 17c in the HDD 17 is associated with the marker identification code and the type of the component box, information on the type of the component box associated with the acquired identification code is identified by the marker identification. The information is read out from the information DB 17c and the type of the parts box is recognized (step S24).

  Next, the CPU 11 of the information processing apparatus 1 recognizes the number of parts in the parts box (step S25). Specifically, a component box area is calculated from the position of the center of gravity of the marker, the area is binarized with a predetermined threshold value, and is divided into an area with a component and another area. Next, the binarized image is subjected to contraction / expansion processing to remove noise. Then, labeling is performed to count the number of component areas, and the number of areas where the size of the component area is equal to or greater than a certain value is counted as the number of parts. By comparing the counted number of parts with the number of parts specified by the identification code, it can be determined whether or not the parts are attached to the printed circuit board 60.

  Next, the CPU 11 of the information processing apparatus 1 recognizes the position of the component mounted by the worker by comparing images of the printed circuit board area before and after mounting the component (step S26). Since the printed circuit board 60 is green and generally has few green electronic components, the green component is reduced only in the region where the components are attached. In addition, when a white electronic component is attached, there is little change in the green component in the region where the component is attached. Therefore, the component mounting position is recognized by taking the difference between the two components of the green component and the blue component.

  FIG. 9 is a flowchart showing a processing procedure of marker M detection processing. In order to detect the image of the marker M from the image captured through the video capture IF 19 and acquire the identification code attached to the marker M, first, the marker area is extracted (step S31). Since the image output from the imaging device 2 is data in which each pixel is represented by a gray value in the RGB color space, the captured image is converted from the RGB color space to the YUV color space. In the YUV color space, the Y component is a luminance component, the U component is a hue component with a blue axis, and the V component is a hue component with a red axis, and can be uniquely converted using a known conversion formula. it can. After converting the RGB value of each pixel into a YUV value, the image is binarized using a preset threshold value for the value of the V component, and a pixel having a value of 0 (0-pixel) and a value of 1 are obtained. It classify | categorizes into a pixel (1-pixel). Then, by performing four adjacent labeling processes, an area where 1-pixels are connected is extracted, and the number of pixels in the extracted area is counted. If the counted number of pixels is within a preset range, it is determined that the area is a marker area.

  Next, the contour line of the marker image is extracted (step S32). The contour line of the marker image is on the boundary when at least one 0-pixel exists by examining 8 pixels around the 1-pixel among the pixels constituting the binary image generated in the process of step S31. It is determined that it is a pixel. When the boundary is extracted by this method, in addition to the contour line of the marker image, a boundary line by the direction identifier m0 and black cells (m1 to m7) appears. Therefore, after the boundary is extracted by the above-described method, the boundary line inside the marker image is removed and the contour line of the marker image is extracted. Further, after extracting the outline of the marker image, the coordinates of the center of gravity of the marker image are calculated.

  Next, the shape of the contour line of the marker image extracted in step S32 is calculated (step S33). In the present embodiment, the extracted shape of the contour line is regarded as an ellipse, and parameters that define an ellipse constituted by a point sequence on the contour line are calculated. Specifically, the coordinates of the point sequence constituting the contour line are expressed in a plane coordinate system with the centroid of the marker image as the origin, and parameters (major axis, The minor axis and the inclination of the ellipse are obtained. The parameters defining the ellipse are calculated a predetermined number of times (for example, 100 times) while changing the three selected points, and the average value and standard deviation of each parameter are obtained. Then, only a value in which the difference between the calculated value and the average value is within the standard deviation is taken out and used as an elliptical parameter for obtaining an average value.

  Next, the distortion of the marker image is corrected (step S34). Since the marker image is distorted unless it is an image obtained by imaging the marker M from the front, the distortion is corrected in this step. Using the ellipse parameter calculated in step S33, distortion can be easily corrected by the following equation.

  Here, (x, y) is the coordinates of each pixel before correction, and (X, Y) is the coordinates of each pixel after correction. Further, a, b, and θ represent the calculated major axis length, minor axis length, and ellipse inclination, respectively. The shape of the marker image whose distortion has been corrected is substantially circular.

  Next, the marker M is identified using the marker image whose distortion has been corrected (step S35), and the identification code attached to the marker M is acquired. FIG. 10 is an explanatory diagram for explaining a technique for acquiring an identification code from a marker image. In order to acquire an identification code using a marker image whose distortion has been corrected, first, the direction identifier m0 is extracted. Since the direction identifier m0 is set to white, a threshold value is set for the Y component value of the YUV value, and a pixel corresponding to the direction identifier m0 is extracted by extracting a pixel having a Y component value larger than the set threshold value. Can be extracted. The barycentric coordinates are obtained from the extracted pixels, and the obtained coordinates are stored as the position of the direction identifier m0.

  Next, scanning is performed counterclockwise once from the calculated position of the direction identifier m0 around the center of gravity of the marker image as the rotation center, and the value of the V component is stored for each angle. Further, the angle φ that is the boundary between the direction identifier m0 and the cell m1 is also stored. A graph with the scanning angle on the horizontal axis and the V component amount on the vertical axis is, for example, as shown in FIG. That is, the V component amount takes a value close to 0 at the angle corresponding to the direction identifier m0 and the pixels on the black cells (m1, m2, m4, m5), and the pixels on the red cells (m3, m6, m7). The V component amount takes a substantially constant value at an angle corresponding to.

  Among the obtained 360 V component data, the first φ and the last (360 / n) −φ (in this embodiment, n = 8 in this embodiment) data are pixels of the direction identifier m0. The remaining data is divided into n-1 groups in order of 360 / n, and the average value of the V component amount in each group is used as an identification index for each cell. Then, the maximum value and the minimum value are obtained from n−1 identification indices, and the intermediate value is set as the threshold Th (see FIG. 10B).

  When the identification index is larger than the threshold Th, the cell is identified as red, and a bit value of “0” is assigned. When the identification index is smaller than the threshold Th, the cell is identified as black, and the bit value “1” is given as (see FIG. 10C). Then, the bit values obtained in the order of cell m1, cell m2,..., Cell m7 are set in order from the least significant bit, and the identification code of marker M is acquired.

  FIG. 11 is a chart showing experimental results regarding the performance evaluation of the marker M. The brightness of the incandescent lamp is adjusted using an incandescent lamp installed so that the incident angle of light is 30 degrees with respect to the marker M and a camera installed so that the direction of regular reflection by the marker M is the optical axis. An evaluation experiment was performed while changing to 200 Lux, 400 Lux, 600 Lux, 800 Lux, and 1000 Lux. The distance between the marker M and the incandescent lamp was set to 35 cm, and the distance between the marker M and the camera was set to 55 cm. In addition, the same kind of experiment was performed using a marker in which a pattern was formed on a paper using a laser printer as a comparison target.

  According to this experimental result, it was possible to recognize in any case in the case of 200 Lux, but in other brightnesses, the marker M created by felt was correctly recognized, but it was correctly recognized by the marker printed on the paper. could not. That is, since the specular reflection component is strong in the paper marker, an image with white stripes is obtained as a whole, and the pattern recognition accuracy by image analysis is deteriorated. In particular, at work sites for electronic components, a certain level of brightness is required as a work environment, and the component boxes 71 to 74 are placed at positions where workers can easily work. It becomes difficult to use a strong paper marker in consideration of recognition accuracy, and it is understood that the felt marker M is more preferable.

The marker M used in the present embodiment has a pattern composed of one direction identifier m0 with white and cells m1 to m7 with either red or black. The configuration is not limited to this. FIG. 12 is a schematic diagram showing another configuration example of the pattern. In the pattern shown in FIG. 12A, each of the seven cells other than the direction identifier is given one color arbitrarily selected from red, black, or blue. Thus, it is possible to make a pattern of 3 ⁷ types (= 2187 combinations), the pattern by can increase the number of types and arrangement of electronic components that can be identified, thereby enabling more delicate information provided. Further, when the types and arrangements of electronic components to be identified are small, it is possible to use a pattern as shown in FIG. The pattern shown in FIG. 12 (b) is composed of one direction identifier with white and five cells with either red or black. With such a marker, only 32 patterns can be created. However, the direction identifier and the area of each cell can be increased, and the recognition rate of the marker itself is improved.

In the present embodiment, only one marker M is attached to each of the component boxes 71 to 74, but a plurality of markers M may be attached to each of the component boxes 71 to 74. Since 128 types of information can be described by the pattern attached to the marker M, it is possible to describe 128 ^N types of information by attaching N markers M, and the amount of information can be significantly increased. Become.

  In this embodiment, felt cloth is used as the material of the marker M. However, the black portion does not reflect light so much and the red portion reflects to some extent, but if the material does not reflect too much, paper is used. It is also possible to use wood, wool and the like.

  In the present embodiment, the relative position between the marker M and the electronic component is defined and the position of the electronic component is detected based on the detection position of the marker M. However, the position of the electronic component is considered. When there is no need, for example, it is specified that each type of electronic component is accommodated in each of the component boxes 71 to 74, and all the electronic components taken out from one component box (for example, the component box 71) are the same. In some cases, it is not always necessary to indicate individual electronic components by the laser irradiation device 3. In this case, since an electronic component to be mounted can be taught to an operator by pointing an arbitrary position in the component box to the laser irradiation device 3, an appropriate position in the component box can be indicated by the laser irradiation device 3. It is sufficient to set only one relative position to the marker M. That is, in the former case, the object to be detected when the position is detected by the work education system is an electronic component. In the latter case, the object to be detected is a component box itself.

  Further, in the present embodiment, the form in which the detection system of the present invention is applied to a work education system for educating electronic component mounting work has been described. However, the three-dimensional shape measurement for measuring the three-dimensional shape of the detected object The present invention can be applied to a system, a human motion recognition system used in motion capture, a tracking system for virtual reality, and the like.

It is a mimetic diagram showing the whole work education system composition concerning this embodiment. It is a schematic diagram which shows a mode that work information is shown by a laser irradiation apparatus. It is a schematic diagram which shows the structural example of a parts box. It is a schematic diagram explaining the detail of a marker. It is a schematic diagram which shows the example of the pattern attached | subjected to the marker. It is a block diagram which shows the structure of the work education system which concerns on this Embodiment. It is a flowchart which shows the procedure of the process performed by this work education system. It is a flowchart which shows the process sequence of the recognition process of a working condition. It is a flowchart which shows the process sequence of a marker detection process. It is explanatory drawing explaining the method for acquiring an identification code from a marker image. It is a graph which shows the experimental result regarding the performance evaluation of a marker. It is a schematic diagram which shows the other structural example of a pattern. It is a schematic diagram which shows the conventional pattern. It is a schematic diagram which shows the influence of the quantization error at the time of edge detection.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 2 Imaging apparatus 3 Laser irradiation apparatus 4 Monitor apparatus 5 Speaker 11 CPU
13 ROM
17 HDD
60 Printed circuit board 71, 72, 73, 74 Component box 71a, 71b, 71c, 71d, 71e, 71f Electronic component M marker

Claims (13)

A label to be placed in the vicinity of one or a plurality of detection objects, an image obtained by imaging the detection object together with the label, and an image of the label in the captured image In a detection system comprising a detection device for detecting the position or type of the detected object,
A detection system, wherein a part or all of the marker is formed of a diffuse reflection member.   It is provided with a plurality of labeled bodies with different patterns, and when detecting the position or type of the detected object, one of the labeled bodies should be arranged in the vicinity of the detected object. The detection system according to claim 1.   The detection device is associated with identification information for identifying a plurality of patterns, and storage means for storing information defining a relative position between each detected object to be detected and a label, corresponding to each of the identification information. An identification information acquisition unit that acquires identification information for identifying the pattern based on an image of the pattern included in the captured image, and based on the acquired identification information and the identification information stored in the storage unit, The detection system according to claim 2, wherein the position of each detected object is detected.   The detection device detects the position of each detected object based on the identification information acquired by the identification information acquired by the identification information acquired by the means for storing information relating to the detected object corresponding to each of the identification information. In addition, the detection system according to claim 3, further comprising means for acquiring information on the detected object.   The detection system according to any one of claims 2 to 4, wherein the pattern includes a plurality of figures, and each figure is given one of at least two colors.   6. The optical characteristic of the diffuse reflection member is such that a regular reflectance and a diffuse reflectance are substantially the same for incident light having an arbitrary incident angle. The detection system described in one.   The detection system according to claim 6, wherein the regular reflectance and the diffuse reflectance are 20% or less.   The detection system according to claim 1, wherein the marker is circular.   The detection system according to claim 1, wherein the marker is formed of felt. In the label placed in the vicinity of the detected object when the position or type of the detected object is detected by the detection device,
A sign body characterized in that a part or all thereof is formed of a diffuse reflection member.   The sign body according to claim 10, wherein the diffuse reflection member has an optical characteristic that the regular reflectance and the diffuse reflectance are substantially the same for incident light having an arbitrary incident angle.   The sign body according to claim 11, wherein the regular reflectance and the diffuse reflectance are 20% or less.   The marker according to any one of claims 10 to 12, wherein the marker is formed of felt.

Images