JP2009301464A - Image recognition processing method, image recognition processor and button image recognition processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition processing method for discriminating an imaging object regardless of the shape thereof. <P>SOLUTION: The image recognition processing method recognizes a button B from an image captured by a camera 3 by using a background member 2 having a plane with a regular pattern (background 2a) applied thereto, and the camera 3 for imaging the button B within the range of the background 2a together with the background 2a. The range not having the regular pattern of the background 2a in the captured image is discriminated as the button B. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image recognition processing method, an image recognition processing device, and a button image recognition processing device that capture an image of an imaging target with a camera and recognize the imaging target.

When a button with a button is sewn to a sewing object such as a cloth, the needle penetrates the hole of the button and sews the button. It is necessary to place it accurately. For this reason, a conveying member having a pin corresponding to the position of the hole portion of the button and the interval between the hole portions is provided, and the button hole portion is conveyed to the sewing machine stitching portion with the hole portion of the button inserted into the pin. There has been a method of positioning the position.
However, according to this method, when supplying a button with a pattern such as letters on the surface of the button so that the pattern faces a predetermined direction with respect to the cloth, the button cannot be identified because the pattern formation direction and position cannot be determined. There is a problem that it cannot be supplied in a desired direction.

For this reason, a method has been adopted in which a button having a pattern on the surface is imaged by a camera or the like, the direction of the button is determined from the captured image, and the position of the button is corrected (for example, Patent Document 1).
In addition, as a technique for clearly extracting the imaging target, the background is a striped pattern, and at least one of the two colors forming the striped pattern is preferably significantly different from the imaging target. A method for imaging an object is known (for example, Patent Document 2).
Japanese Patent Publication No. 2-23197 Japanese Patent Laid-Open No. 9-54892

However, in the method described in Patent Document 1, that is, in determining the direction of the button based on the captured image, if the button and the color of the upper surface of the base part on which the button is placed are very similar when the button is captured, There is a problem that it is difficult to distinguish between the button and the base portion, and the button cannot be suitably detected.
Further, there is a problem that even if the direction of the button can be determined, the position of the button hole cannot be detected correctly. For this reason, there is a problem that the button cannot be positioned accurately and cannot be detected when there is a defect in the formation of the button hole.
Furthermore, general buttons having no pattern could not be used, and the versatility was poor.
Further, even when the technique of Patent Document 2 is used, in the case of a button having transparency, there is a problem that the density change between the background and the button does not occur, and the shape of the outer periphery of the button cannot be reliably extracted. .

  An object of the present invention is to provide an image recognition processing method, an image recognition processing device, and a button image recognition processing device of a sewing machine capable of discriminating an imaging target regardless of the shape of the imaging target. To do.

  The invention according to claim 1 has a plane member with a known regular pattern, a background member whose background is the background, and an imaging object within the range of the plane with the pattern. In the image recognition processing method for recognizing the object to be picked up from an image picked up by the camera using the camera for picking up the whole background, the picked-up image picks up a range having no regular pattern of the background It includes a determination step of determining as an object.

  A second aspect of the present invention is the image recognition processing method according to the first aspect, wherein the camera captures the captured image while focusing on the background.

According to a third aspect of the present invention, in the image recognition processing method according to the first or second aspect, the imaging target in which the imaging target of the captured image is preset from the range of the image determined as the imaging target It is characterized by including the characteristic discrimination | determination process which discriminate | determines whether it has the characteristic of an object.
The “characteristics of the imaging object” in claim 3 includes the color of the imaging object, decorations such as characters and patterns applied to the imaging object, and a light / dark pattern of the image at the time of imaging by the three-dimensional shape of the imaging object. is there.

  According to a fourth aspect of the present invention, in the image recognition processing method according to the third aspect, in the feature determination step, an image that is a characteristic of the imaging target acquired from a previous captured image of the imaging target is used. It is characterized by doing.

  The invention according to claim 5 includes a plane member with a known regular pattern, a background member having the plane as a background, and an imaging object within a range of the plane with the pattern. An image recognition processing apparatus that recognizes the object to be imaged from an image captured by the camera, wherein the captured image includes a range that does not have a regular pattern of the background. It is characterized by comprising discrimination means for discriminating as an object.

  According to a sixth aspect of the present invention, in the image recognition processing apparatus according to the fifth aspect, the camera captures the captured image while focusing on the background.

According to a seventh aspect of the present invention, in the image recognition processing device according to the fifth or sixth aspect, the imaging target in which the imaging target of the captured image is set in advance from a range of the image determined as the imaging target. A feature discriminating means for discriminating whether or not the object has a feature is provided.
The “characteristics of the imaging object” in claim 7 are the color of the imaging object, decorations such as characters and patterns applied to the imaging object, and the shading pattern of the image at the time of imaging by the three-dimensional shape of the imaging object. is there.

  The invention according to claim 8 is the image recognition processing device according to claim 7, further comprising feature acquisition means for acquiring an image that is a characteristic of the imaging object from a captured image of the imaging object before the previous time. And

  The invention according to claim 9 has a plane member with a known regular pattern, a background member on which the plane is set as a background, and a button of clothing within the range of the plane with the pattern. A button image recognition processing device for recognizing the button from an image captured by the camera, wherein a range having no regular pattern of the background in the captured image is determined as the button. It is characterized by comprising a discriminating means.

  A tenth aspect of the present invention is the button image recognition processing apparatus according to the ninth aspect, wherein the camera captures the captured image while focusing on the background.

The invention described in claim 11 is the button image recognition processing device according to claim 9 or 10, wherein it is determined whether or not the button has a preset feature from the range of the image determined as the button. Means are provided.
The “features of the button” according to claim 11 includes the color of the button, decorations such as characters and patterns applied to the button, and the shading pattern of the image at the time of imaging by the three-dimensional shape of the button.

  According to a twelfth aspect of the present invention, in the button image recognition processing device according to the eleventh aspect of the present invention, the button image recognition processing device further includes a feature acquisition unit that acquires an image that is a characteristic of the button from a captured image of the button before the previous time.

  The invention according to claim 13 is the button image recognition processing device according to claim 11 or 12, wherein the feature determination means captures at least one of the front and back of the button set in advance. The front and back of the button is discriminated from the light and shade pattern data.

According to the first aspect of the present invention, a range having no regular background pattern is determined as an imaging object in the determination step. That is, a portion excluding a range having a regular background pattern from the captured image is determined as an imaging target. At this time, if there is a possibility that the shape, color, pattern and other features of the imaging target object may be confused with the regular pattern of the background, it can be dealt with by changing the regularity of the background pattern. For example, when the object to be imaged is a stripe pattern, a stripe pattern with a different stripe formation direction, a stripe pattern with a different stripe formation interval or different width, or a checkered pattern may be considered. Therefore, it is possible to determine the imaging object only by whether or not it has a regular background pattern regardless of the shape, color, pattern and other characteristics of the imaging object. Therefore, it is possible to more suitably determine the imaging object.
Conventionally, the density change is observed along the stripes in the background, and the location where the density change has occurred is recognized as the boundary between the background and the object to be imaged, and the density change is observed for each of the two color stripes. Even when the object has a density close to one of the colors of the stripes, the boundary can be detected by the other color. However, when there is a change in the color density of the surface, or when there is an imaging target that has a density change in the middle of the surface, such as when the surface has irregularities or holes, or a transparent material, etc. Cannot identify whether it is the background or part of the object to be imaged at a location that approximates the density of the stripe color. However, in the present invention, identification is performed based on whether the pattern follows a regular background pattern. For example, even if a captured image includes a portion having a density close to that of the background, it does not match the background pattern including the preceding and subsequent portions. As long as there is no misunderstanding.
For example, when a through-hole such as a hole is provided inside the object to be imaged, the inside of the through-hole is determined as a background by looking into the regular pattern of the background. No longer considered. That is, when the penetration part is provided inside the imaging target object, the penetration part is correctly determined. Therefore, it is possible to eliminate the problem that holes (for example, button holes) provided inside the imaging object cannot be extracted as in the prior art, and it is possible to more suitably determine the shape of the imaging object. It becomes. Therefore, it is possible to more appropriately discriminate the imaging object.
In addition, even if the imaging object has transparency, if the background is refracted by transmitting through the imaging object, the refraction part loses the regularity of the background, so it is determined as the background. It is discriminated as an object to be imaged. Therefore, in the case of an imaging object having transparency, the problem of the prior art that the density change between the background and the imaging object does not occur and the shape of the outer periphery of the imaging object may not be extracted can be solved. Therefore, it is possible to more appropriately discriminate the imaging object.

  According to the second aspect of the present invention, since the focus of the camera is adjusted to the background, the background can always be suitably imaged regardless of the thickness and height of the object to be imaged. Therefore, the regular pattern of the background can be discriminated more suitably, and the possibility of misidentification of the background and the object to be imaged can be greatly reduced. Therefore, it is possible to more appropriately discriminate the imaging object.

According to the third aspect of the present invention, in the feature determination step, it is determined whether or not the imaging target of the image has a preset characteristic of the imaging target from the range of the image determined as the imaging target. That is, first, after performing the process of excluding a range having a regular background pattern on the image to determine the range of the imaging target, the characteristics of the imaging target set in advance for the range of the imaging target are determined. Determine if you have. This makes it possible to limit the determination range of the presence / absence of the feature to the range of the imaging object in which the background is excluded from the image, as compared with the case where the range having the characteristics of the imaging object set in advance is searched from the entire image. Therefore, it is possible to reduce the processing load for determining the presence / absence of features and shorten the processing time.
In addition, regarding the characteristics of the imaging object, such as the color of the imaging object, decorations such as characters and patterns applied to the imaging object, and the shade pattern of the image at the time of imaging due to the three-dimensional shape of the imaging object, the imaging object of the captured image It is determined whether the object has the preset feature. Therefore, it is possible to discriminate between imaging objects with different colors, imaging objects with different decoration contents, imaging objects with different orientations such as characters and patterns, and imaging objects with different three-dimensional features. Therefore, in the prior art, the problem that the orientation of decorations such as characters and patterns applied to the imaging object cannot be determined can be solved, and the characteristics of the imaging object can be determined in more detail.

  According to the fourth aspect of the present invention, in the feature determination step, an image that is a feature of the imaging target acquired from the previous captured image of the imaging target is used. Therefore, it is not necessary to separately provide dedicated information, data, and the like for setting the characteristics of the imaging object. Therefore, it is possible to greatly reduce the labor for setting the characteristics of the imaging target.

According to the fifth aspect of the invention, the discriminating unit discriminates a range having no regular background pattern as an imaging object. That is, a portion excluding a range having a regular background pattern from the captured image is determined as an imaging target. At this time, if there is a possibility that the shape, color, pattern and other features of the imaging target object may be confused with the regular pattern of the background, it can be dealt with by changing the regularity of the background pattern. For example, when the object to be imaged is a stripe pattern, a stripe pattern with a different stripe formation direction, a stripe pattern with a different stripe formation interval or different width, or a checkered pattern may be considered. Therefore, it is possible to determine the imaging object only by whether or not it has a regular background pattern regardless of the shape, color, pattern and other characteristics of the imaging object. Therefore, it is possible to provide an image recognition processing device that can more suitably discriminate the imaging object.
Conventionally, the density change is observed along the stripes in the background, and the location where the density change has occurred is recognized as the boundary between the background and the object to be imaged, and the density change is observed for each of the two color stripes. Even when the object has a density close to one of the colors of the stripes, the boundary can be detected by the other color. However, when there is a change in the color density of the surface, or when there is an imaging target that has a density change in the middle of the surface, such as when the surface has irregularities or holes, or a transparent material, etc. Cannot identify whether it is the background or part of the object to be imaged at a location that approximates the density of the stripe color. However, in the present invention, identification is performed based on whether the pattern follows a regular background pattern. For example, even if a captured image includes a portion having a density close to that of the background, it does not match the background pattern including the preceding and subsequent portions. As long as there is no misunderstanding.
For example, when a through-hole such as a hole is provided inside the object to be imaged, the inside of the through-hole is determined as a background by looking into the regular pattern of the background. No longer considered. That is, when the penetration part is provided inside the imaging target object, the penetration part is correctly determined. Therefore, it is possible to eliminate the problem that holes (for example, button holes) provided inside the imaging object cannot be extracted as in the prior art, and it is possible to more suitably determine the shape of the imaging object. It becomes. Therefore, it is possible to provide an image recognition processing device that can more appropriately discriminate the imaging target.
In addition, even if the imaging object has transparency, if the background is refracted by transmitting through the imaging object, the refraction part loses the regularity of the background, so it is determined as the background. It is discriminated as an object to be imaged. Therefore, in the case of an imaging object having transparency, the problem of the prior art that the density change between the background and the imaging object does not occur and the shape of the outer periphery of the imaging object may not be extracted can be solved. Therefore, it is possible to provide an image recognition processing device that can more appropriately discriminate the imaging target.

  According to the sixth aspect of the present invention, since the focus of the camera is adjusted to the background, the background can always be suitably imaged regardless of the thickness and height of the imaging object. Therefore, the regular pattern of the background can be discriminated more suitably, and the possibility of misidentification of the background and the object to be imaged can be greatly reduced. Therefore, it is possible to provide an image recognition processing device that can more appropriately discriminate the imaging target.

According to the seventh aspect of the present invention, the feature determination unit determines whether or not the imaging target of the image has a preset characteristic of the imaging target from the range of the image determined as the imaging target. That is, first, after performing the process of excluding a range having a regular background pattern on the image to determine the range of the imaging target, the characteristics of the imaging target set in advance for the range of the imaging target are determined. Determine if you have. This makes it possible to limit the determination range of the presence / absence of the feature to the range of the imaging object in which the background is excluded from the image, as compared with the case where the range having the characteristics of the imaging object set in advance is searched from the entire image. Therefore, it is possible to reduce the processing load for determining the presence / absence of features and shorten the processing time.
In addition, regarding the characteristics of the imaging object, such as the color of the imaging object, decorations such as characters and patterns applied to the imaging object, and the shade pattern of the image at the time of imaging due to the three-dimensional shape of the imaging object, the imaging object of the captured image It is determined whether the object has the preset feature. Therefore, it is possible to discriminate between imaging objects with different colors, imaging objects with different decoration contents, imaging objects with different orientations such as characters and patterns, and imaging objects with different three-dimensional features. Therefore, in the prior art, it is possible to solve the problem that the orientation of decorations such as characters and patterns applied to the imaging object cannot be determined, and it is possible to provide an image recognition processing device capable of determining the characteristics of the imaging object in more detail. .

  According to the eighth aspect of the invention, the feature determination unit uses an image that is a feature of the imaging object acquired from a captured image of the imaging object before the previous time. Therefore, it is not necessary to separately provide dedicated information, data, and the like for setting the characteristics of the imaging object. Therefore, it is possible to greatly reduce the labor for setting the characteristics of the imaging target.

According to the ninth aspect of the present invention, the determining means determines a range having no regular background pattern as a button. In other words, a portion excluding a range having a regular background pattern from the captured image is determined as a button. At this time, if there is a possibility that the shape, color, pattern or other features of the button may be confused with the regular pattern of the background, it can be dealt with by changing the regularity of the background pattern. For example, when the button is a striped pattern, a striped pattern with a different stripe forming direction, a striped pattern with a different stripe formation interval or different width, or a checkered pattern may be considered. Therefore, the button can be determined only by whether or not it has a regular background pattern regardless of the shape, color, pattern and other characteristics of the button. Therefore, it is possible to provide an image recognition processing device that can more suitably discriminate buttons.
Conventionally, the density change is observed along the stripes in the background, and the location where the density change has occurred is recognized as the boundary between the background and the button, and the density change is observed for each of the two color stripes. Even when the density is close to one of the colors, the boundary can be detected by the other color. However, when there is a change in the color density of the surface, or when there is an imaging target that has a density change in the middle of the surface, such as when the surface has irregularities or holes, or a transparent material, etc. Cannot identify whether it is a background or part of a button where it approximates the density of the stripe color. However, in the present invention, identification is performed based on whether the pattern follows a regular background pattern. For example, even if a captured image includes a portion having a density close to that of the background, it does not match the background pattern including the preceding and subsequent portions. As long as there is no misunderstanding.
For example, the hole provided on the inner side of the button is correctly identified by the above-described feature. Therefore, it is possible to solve the problem of the prior art that the hole provided inside the button cannot be extracted, and it is possible to more suitably determine the shape and position of the button hole and the interval between the holes. Therefore, it is possible to provide a button image recognition processing apparatus that can suitably determine the direction of the button.
Furthermore, even if the button has transparency, if the background is refracted by passing through the button, the portion where the refraction has occurred will lose the regularity of the background and may be identified as the background. Not a button. Therefore, in the case of a button having transparency, there is no sudden change in density between the background and the button, and it is possible to solve the problems of the prior art that the shape of the outer periphery of the button may not be extracted. Therefore, it is possible to provide a button image recognition processing device that can more suitably discriminate buttons.

  According to the tenth aspect of the present invention, since the focus of the camera is adjusted to the background, the background can always be suitably imaged regardless of the thickness of the button. Therefore, the regular pattern of the background can be more suitably discriminated, and the possibility of misidentification of the background and the button can be greatly reduced. Therefore, it is possible to provide an image recognition processing apparatus that can discriminate buttons more preferably.

According to the eleventh aspect of the present invention, the feature determining means determines whether or not the button of the image has a preset button feature from the range of the image determined as the button. In other words, first, a process for excluding a range having a regular background pattern is performed on an image to determine a button range, and then it is determined whether the button range has a preset button feature. . This makes it possible to limit the determination range of the presence / absence of the feature to the range of the button from which the background is excluded from the image, as compared with the case where the range having the preset button feature is searched from the entire image. Therefore, it is possible to reduce the processing load for determining the presence / absence of features and shorten the processing time.
In addition, the button of the captured image has the preset characteristics, such as the color of the button, decorations such as characters and patterns applied to the button, and the shade characteristics of the image at the time of imaging due to the three-dimensional shape of the button. Determine whether or not. Accordingly, it is possible to determine buttons of different colors, buttons having different decoration contents, buttons having different directions such as characters and patterns, and buttons having different three-dimensional features. Therefore, in the prior art, it is possible to solve the problem that the orientation of decorations such as characters and patterns applied to the buttons cannot be determined, and it is possible to provide a button image recognition processing device for a sewing machine that can determine the features of the buttons in more detail.

  According to the twelfth aspect of the present invention, the feature discriminating unit uses the image that is the feature of the button acquired from the captured image of the button before the previous time. Therefore, it is not necessary to separately provide dedicated information or data for setting the button characteristics. Therefore, it is possible to greatly reduce the trouble of setting the button characteristics.

  According to a thirteenth aspect of the invention, the front and back of the button is discriminated from the shade pattern data of the image obtained when at least one of the front and back of the button is imaged by the feature acquisition unit. In this case, the shade pattern data of the image when the front of the button is imaged is different from the shade pattern data of the image when the back of the button is imaged. The front and back of the button is discriminated based on the difference in the light and shade pattern data. For example, in the case where the shade pattern data of the preset image is the shade pattern data of the image when the table of the button is captured, the feature determination unit can detect if the shade pattern data of the captured image matches the shade pattern data of the button of the captured image. The button is determined to be the front, and if it is different, it is determined to be the back. As a result, it is possible to determine whether or not the button is turned upside down with respect to the originally intended placement. Therefore, for example, by performing front / back determination of the button by the feature determination means before sewing, it is possible to eliminate a situation where the button is accidentally sewn with the button turned upside down, and the sewing quality is greatly improved.

(Outline of Button Image Recognition Processing Apparatus Using Image Recognition Processing Method According to the Present Invention)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
An image recognition processing method and an image recognition processing apparatus according to the present invention are an image recognition processing method and a button image recognition processing apparatus that capture an image of an imaging target with a camera and recognize the imaging target. Further, the button image recognition processing apparatus 1 according to the present invention is a button image recognition processing apparatus that recognizes various features such as the size, position, front and back, color, and orientation of a button by imaging the button with the background of the camera. The button image recognition processing device 1 is a button supplied to a button gripper in a well-known button supply device that supplies a button to a button gripper (not shown) that holds the button above a sewing product in a button sewing machine, for example. Is used for processing for recognizing various characteristics such as size, position, front and back, color, and orientation before supplying a button. The button image recognition processing device 1 outputs an error when there is a preset button feature, and when any of the various features described above recognizes a button different from the preset button feature. As a result, if the features of the buttons used for the sewing work are set in advance, buttons that do not match the preset features will not be accidentally supplied, and the button orientation will be prevented from being supplied incorrectly. To do. Therefore, the sewing quality of the button is improved. In particular, if the button itself is normal but there is a problem with the button position, orientation, front / back, etc., the recognized result is output to the position correction mechanism of the button supply device, etc. A button can be supplied.

(Whole structure of sewing machine button image recognition processing device)
A button image recognition processing device 1 (see FIG. 1) described below is one form of a button image recognition processing device for a sewing machine according to the present invention, and image recognition for discriminating the button B from an image obtained by capturing the button B together with the background 2a. It is a processing device.
FIG. 1 is a block diagram showing the configuration of the button image recognition processing apparatus 1.
The button image recognition processing device 1 has a plane with a known regular pattern, and the imaging target is within the range of the background member 2 having the plane 2a and the plane 2a having the background 2a. A camera 3 that captures an object (button B) together with the background 2a, an illumination 4 that illuminates the button B with light, a monitor 5 that displays various information of the button image recognition processing device 1, and each part of the button image recognition processing device 1 An image processing apparatus 10 that performs control and recognizes the button B from an image captured by the camera 3.

FIG. 2 is a top view showing an example of the relationship between the button B and the background 2 a of the background member 2. 2A shows a case where there is no malformation and the button B having a predetermined size is placed in the center of the background 2a in the correct orientation. FIG. 2B shows a case where there is no button. 2C shows a case where a button BA larger than a predetermined size is placed, FIG. 2D shows a case where the position of the button B is shifted from the center of the background 2a, and FIG. FIG. 2 (f) shows the position of the plurality of hole portions Ba of the button B. FIG. 2 (f) shows the position of the plurality of holes Ba. When the relationship is inclined with respect to the positional relationship shown in FIG. 2 (a), FIG. 2 (h) shows a case where a button BD having a part of the hole formed in the wrong position is placed. i) It is explanatory drawing when the button BE in which a part of hole is not formed is put.
As shown in FIGS. 1 and 2, the background member 2 has a plane with a known regular pattern, and the plane is the background 2a. The striped pattern is a black and white striped pattern having an equal width, and each striped pattern is provided with a certain width (5 pixels) when captured by the camera 3. Further, as shown in FIG. 2, when the background 2a and the button B are viewed from above with the button B placed on the upper surface of the background 2a, the outer portion of the button B not covered by the button B and the hole portion Ba You can see the background 2a. At this time, the background 2a can sufficiently cover the range when the button B is viewed from above, and when the button B is placed at the center of the background 2a, the background 2a can be seen around the button B over a predetermined width. It is provided to have a size.

The camera 3 is provided above the background 2a, and images the button B together with the background 2a. At this time, the camera 3 is focused on the background 2a. An image captured by the camera 3 is input to the image processing apparatus 10.
In addition, although the image by the camera 3 may be monochrome or color, in this embodiment, a monochrome image is used. An image input from the camera 3 to the image processing apparatus 10 is an analog image.

The illumination 4 irradiates the background 2a and the button B with light. The illumination 4 can adjust the amount of light, and can appropriately adjust the amount of light received by the camera 3 when the camera 3 captures the button B. At the time of imaging, irradiation is performed so that the direction, amount of light, and the like satisfy predetermined conditions.
The monitor 5 outputs and displays various information from the image processing apparatus 10. The contents displayed on the monitor 5 will be described later. However, as long as the captured image can recognize the background 2 and the button B by the camera 3 by the reflected light of ambient ambient light, it is not always necessary to provide illumination in the image processing apparatus 10. Further, in FIG. 1, the illumination 4 is oblique illumination, but it may be falling illumination.

(Image processing device)
The image processing device 10 controls each part of the button image recognition processing device 1 and performs processing for recognizing the button B from the digital image data D. The image processing apparatus 10 includes an A / D converter 11 that converts an analog image input from the camera 3 into digital image data D, a CPU 12 that performs various processes and control of the image recognition processing apparatus 1, and a CPU 12. The RAM 13 that functions as an area for storing various programs and data in the process, the ROM 14 that stores various programs and data called in the process performed by the CPU 12 in an unrewritable manner, and the various programs and data called in the process performed by the CPU 12 are rewritten. EEPROM 15 which stores the data as possible. Although not shown, the image processing apparatus 10 is connected to an operation panel for performing various settings for the image processing apparatus.

The A / D converter 11 converts an analog image input from the camera 3 into digital image data D with a plurality of pixels and outputs the digital image data D to the CPU 12. Each pixel of the digital image data D has one density value (luminance) (see FIG. 4).
The CPU 12 performs various processes and controls of the button image recognition processing apparatus 1 by calling and processing various programs and data from the ROM 14 or the EEPROM 15. In addition, the CPU 12 performs various processes for determining the button B from the digital image data D input from the A / D converter 11. The CPU 12 performs the above-described determination process of the button B by calling various programs from the ROM 14 and sequentially executing them. At this time, the digital image data D is stored in the RAM 13 or the EEPROM 15.

  The image processing apparatus 10 holds the stripe pattern of the background 2a as a known pattern, and the pattern data H of the stripe pattern is stored in the EEPROM 15. The EEPROM 15 has an area and capacity for storing button outer periphery check data 15o, hole check data 15p, density check data 15q, and direction check data 15r. Further, the EEPROM 15 may previously hold one or more of the outer periphery check data 15o, the hole check data 15p, the density check data 15q, and the direction check data 15r as a known pattern. Details of the outer peripheral check data 15o, the hole check data 15p, the density check data 15q, and the direction check data 15r of the button B will be described later.

FIG. 3 is an explanatory diagram showing various programs stored in the ROM 14 and various data stored in the EEPROM 15. FIG. 3A shows various programs stored in the ROM 14, and FIG. 3B shows various programs stored in the EEPROM 15.
The ROM 14 also includes an imaging program 14a, an image quality improvement program 14b, a background information determination program 14c, an object presence / absence check program 14d, a background contour check program 14e, an outline information acquisition program 14f, and an object error, which will be described later. A check program 14g, an in-object information acquisition program 14h, an in-object error check program 14i, a front / back determination program 14j, and a direction determination program 14l are stored. The CPU 12 performs recognition processing of the button B from the image by the camera 3 by calling and executing one of these programs according to the processing. Hereinafter, the processing contents when the CPU 12 executes each program will be described in detail.

First, the imaging program 14a will be described in detail. The CPU 12 executes the imaging program 14 a to control the camera 3 so that the camera 3 captures the button B together with the background 2 a, and the analog image obtained by the imaging is converted into digital image data via the A / D converter 11. A process of converting to D and storing in the RAM 13 is performed. At this time, the captured image may be displayed on the monitor 5. The digital image data D is rectangular or square multi-valued image data (gray scale).
In addition, the CPU 12 executes the image quality improvement program 14b to perform processing for removing noise of the digital image data D stored in the RAM 13 by the imaging program 14a. Note that the method of removing noise from the digital image data D is a well-known technique and will not be described in detail.

Next, the background information determination program 14c will be described in detail. The CPU 12 performs a process of recognizing a background portion from the digital image data D by executing the background information determination program 14c. Hereinafter, the processing will be described in detail.
FIG. 4 is an explanatory diagram in which a part of the digital image data D is enlarged. The numerical values attached around the digital image data D are indices for explaining the pixel positions of the digital image data D.
The CPU 12 determines whether or not the width of the black and white stripes of the background 2a are alternately arranged by a predetermined number of pixels in the direction orthogonal to the black and white stripes of the background 2a (vertical direction in FIG. 4). That is, the luminance of each pixel is read out along the vertical direction of the pixels in the imaging range, and it is determined for all the columns in the imaging range whether the pattern should be obtained by imaging black and white horizontal stripes. In the case of black and white horizontal stripes, if the background is 2a, a pattern showing a change in which white and black are switched every predetermined number of pixels should be obtained. At this time, the number of pixels in the vertical direction on the digital image data D corresponding to the width of the black and white stripes of the background 2a is preset as pattern data H and stored in the EEPROM 15. In FIG. Shows the case of 5 pixels. In addition, the luminance range determined as “black” or “white” is also set in advance and stored in the EEPROM 15. The CPU 12 acquires the luminance of each pixel from the top for each column, and determines whether all the five pixels have the luminance according to the striped pattern of the background 2a. That is, in the case of the digital image data D shown in FIG. 4, are the first to fifth pixels from the top all having a luminance indicating “black”, or are the sixth to tenth pixels all having a luminance indicating “white”? , And so on in order of 5 pixels. When all the five pixels have the luminance in accordance with the background, the CPU 12 determines that the five pixels are the background 2a. Otherwise, the portion is not the background, that is, the image of the button B that is the imaging target is shown. It is determined that it is a range. FIG. 4 shows that the luminance indicating “gray” (the spotted portion in FIG. 4) is acquired in the pixel portion corresponding to the shape of the button B imaged by the camera. Since the discrimination processing is performed in units of 5 pixels in the vertical direction, the minimum unit of the vertical pixels in which the background 2a and the button B are discriminated is 5 pixels, and the background is 5 times rougher than the resolution of the actual camera. 2a and button B are separated. For example, the boundary between the background 2a and the button B is recognized as a two-dot chain line P in FIG. In consideration of an error, an allowable range of ± α may be provided for 5 pixels (for example, when α = 1, if 4 out of 5 pixels have the same luminance as the background 2a, the allowable range is 1). Even if there are only two different colors, the background is considered).
In addition, since the unit pattern may be imaged from the middle of the edge portion of the imaging range, it is desirable to start the discrimination process from the position where the repeated unit pattern is recognized once.

FIG. 5 is an explanatory diagram showing a boundary line between the background 2 a and the button B. FIG. 5A shows the upper half boundary line L1 of the button B determined by the background information determination program 14c, and FIG. 5B shows the lower half boundary of the button B determined by the background information determination program 14c. FIG. 5C is a diagram illustrating the line L2, and FIG. 5C is an explanatory diagram illustrating the outer circumference data L of the button B synthesized by the outer shape information acquisition program 14f described later.
In the first column, the CPU 12 first determines the background 2a and the button B sequentially in units of 5 pixels from the top pixel downward, and determines the background 2a and the button B until a range determined as the button B is found. I do. If a range identified as button B is found, the range below it is assumed that button B exists, and at that time, the determination of that column ends and the next column is determined. If no is found, the background and the button B are sequentially determined in units of 5 pixels from the first to the bottom of the next column when the determination is completed up to the last pixel in the column, and then repeated. Thereafter, when the determination of the last column is completed, as shown in FIG. 5A, the range of the background 2a and the button B is determined on the boundary line L1 where the background 2a and the button B are determined. Done. After that, in the first column, the background and the button B are discriminated sequentially in units of 5 pixels from the lowest pixel in the vertical direction, and the background and the button B are discriminated until a range discriminated as the button B is found. Do. If a range identified as button B is found, the range above it is assumed that button B exists, and at that time, the determination of that column ends, and the next column is determined. Is not found, the background and the button B are discriminated in units of 5 pixels in order from the bottom to the top of the next column when the discrimination is finished up to the top pixel in the column, and then repeated. . At this time, for the process of sequentially determining the background and the button B from the lower side, the determination process is omitted for the columns that are all determined to be the background as a result of the determination of the background and the button B sequentially from the upper side. May be. As a result, as shown in FIG. 5B, the range between the background 2a and the button B is determined with the boundary line L2 as the boundary.
In the determination process, in order to shorten the determination time, if a range to be determined as the button B is found in the determination process for each column moving downward or upward, the subsequent range of the column is the button B. However, in this process, if the shape of the button B has a partial defect, there is a possibility that the defect cannot be identified. Because there is, the time cannot be shortened, but it is omitted from the top pixel to the bottom pixel, or from the bottom pixel to the top pixel, regardless of whether or not the button B pixel is detected. All pixels may be discriminated without any problem.

  When the button B has transparency, the striped pattern of the background 2a transmits the button B. However, when refraction occurs in the process of the background 2a transmitting the button B, the regularity of the background 2a is caused by the refraction. Is not determined as the background 2a, but is determined as the button B.

  Next, the object presence / absence check program 14d will be described in detail. The CPU 12 executes the object presence / absence check program 14d to determine whether the range determined as the background 2a by the execution of the background information determination program 14c is the entire digital image data D. Check if B is on board. When the entire digital image data D is determined as the background, that is, when there is no range determined as the button B on the digital image data D, the button B is not on the upper surface of the background 2a. Therefore, the CPU 12 outputs an error indicating that there is no button B. On the other hand, when the entire digital image data D has not been determined as the background, the background 2a is blocked by the button B on the upper surface of the background 2a. It is considered that Therefore, the CPU 12 performs an output indicating that the button B is present. As a result, as shown in FIG. 2B, the digital image data D when the button is not placed can be determined.

Next, the background contour check program 14e will be described in detail. By executing the background contour check program 14e, the CPU 12 is arranged so that the button B is surrounded by a continuous range 2a of the background 2a, that is, a predetermined number of pixels over all directions of the contour portion. It is determined whether there is a continuous background 2a. That is, the background contour check program 14e checks whether the button B is not placed so as to protrude from the background 2a, or whether the button B is placed at a position greatly deviated from the center of the background 2a. In the image data D shown in FIG. 4, the CPU 12 has a row of 10 pixels (horizontal arrangement of pixels) continuous from the upper end indicated by numerical values 1 to 10 and a column of 10 pixels continuous from the left end (vertical pixels). It is checked whether the range of 10 pixels continuous from the lower end and the right end of the digital image data D (not shown) are all determined as the background 2a. That is, the CPU 12 places the button B so that it protrudes from the background 2a depending on whether or not the range (predetermined range) of 10 pixels continuous from the top, bottom, left, and right edges of the image data D is all determined as the background 2a. It is checked whether the button B is not placed at a position far from the center of the background 2a. When all the ranges are not determined as the background, that is, when the range determined as the button B is included in the range, the button B is placed so as to protrude from the background 2a or the background does not protrude. It is determined that it is placed at a position greatly deviated from the center of 2a. Therefore, the background contour check program 14e outputs an error indicating that the button B protrudes from the background 2a or is placed at a position greatly deviated from the center of the background 2a. On the other hand, if the entire range has been determined as the background, the button B is placed in a roughly correct position without protruding from the background 2a or being greatly displaced from the center of the background 2a. Determined. Therefore, the CPU 12 performs an output indicating that the button B is placed at the correct position. As a result, the size of the button BA is large as shown in FIG. 2C, so that the outline of the background 2a is partially covered, or the button B is placed as shown in FIG. 2D. It is possible to determine the digital image data D when the position is shifted from the center of the background 2a.
In the above description, 10 pixels are processed as the contour of the digital image data D, but it goes without saying that a different number of pixels from the above can be set as the contour.
Further, in the above description, when pixels indicating the background 2a for 10 pixels continuous from the respective upper, lower, left and right side edges of the digital image data D are not determined, the protrusion of the button B from the background 2a and the background 2a of the button B Although a large deviation is checked from the center, the same determination is simply made when there is a portion that cannot be determined as the background 2a continuously for 10 pixels in each of the vertical and horizontal directions over the entire digital image data D. You can go.
The same determination may be made according to the position of the boundary line acquired by the background information determination program 14c in the image data D.

Next, the outline information acquisition program 14f will be described in detail. The CPU 12 synthesizes data (peripheral data) indicating the outer periphery of the range determined as the button B by the background information acquisition program 14c by executing the outer shape information acquisition program 14f. Specifically, as shown in FIG. 5 (c), the boundary line L1 and the boundary line L2 are combined and the range that is inside the boundary line L1 and inside the boundary line L2 is displayed on the button B. It is regarded as a range, and a part of the boundary lines L1 and L2 surrounding the range is connected as the outer peripheral data L of the button B.
When an automatic check mode, which will be described later, is ON, the CPU 12 stores the outer periphery data in the EEPROM 15.

Next, the object error check program 14g will be described in detail. The CPU 12 executes the object error check program 14g, and when the outer periphery check data of the button B is stored in advance, the outer periphery data and outer periphery check data of the button B acquired by executing the outer shape information acquisition program 14f. Compare 15o.
FIG. 6 is an explanatory diagram showing the relationship between the outer periphery check data 15o and the outer periphery data L. The outer periphery check data 15o is data indicating the pixel range F1 where the outer boundary of the button B shown in FIG. 6 is to be located, and is stored in the EEPROM 15 as described above. The CPU 12 determines whether all the positions of the outer periphery of the button B indicated by the outer periphery data L are included in the range F1. As a result, when a button BB in which a part of the button B is missing is placed as shown in FIG. 2 (e), or a button BC smaller than a predetermined size is placed as shown in FIG. 2 (f). In this case, the digital image data D when the button B is arranged with a large positional deviation with respect to the background 2a can be determined.

  By the above comparison processing, the CPU 12 determines whether the outer shape of the button B and the size of the button B shown in the digital image data D match the buttons set in the outer periphery check data 15o. At this time, if it is determined that the outer periphery data of the button B does not match the outer periphery check data 15o, the CPU 12 outputs an error indicating that the shape, arrangement, or size of the button B is abnormal. On the other hand, when it is determined that the outer periphery data of the button B matches the outer periphery check data 15o, the CPU 12 outputs an output indicating that there is no problem in the shape, arrangement, and size of the button B. Accordingly, it is possible to determine the digital image data D when the shape of the button B does not match the preset outer periphery check data 15o.

Next, the in-object information acquisition program 14h will be described in detail. The CPU 12 determines the background 2a and the button B within the range determined as the button B by executing the in-object information acquisition program 14h, and then data (hole portion) indicating the periphery of the hole portion of the button B. Data) and grayscale pattern data indicating the luminance of each pixel in the range excluding the hole Ba from the range determined as the button B by the background information determination program 14c.
FIG. 7 is an explanatory view showing a part of the digital image data D in which the periphery of one hole Ba of the button B is enlarged. The discrimination process between the background 2a and the button B within the range discriminated as the button B is performed by the same mechanism as the process of the background information discrimination program 14c described above. Specifically, the background 2a and the button B are discriminated in units of 5 pixels in order from the topmost pixel in each column in order from the lowest numerical value (leftmost) column in which the outer periphery of the button B is located. At this time, the range determined as the background 2 a within the range of the button B is determined as the hole Ba of the button B. That is, as shown in FIG. 7, the inner side of the two-dot chain line Q1 is determined as the hole Ba, and the two-dot chain line Q1 indicates one outline of the hole Ba.
Note that the determination processing range between the background 2a and the button B by the in-object information acquisition program 14h can be limited to a predetermined range set in advance. For example, when the approximate position where the hole Ba of the button B shown in FIG. 2 is provided can be specified in advance, the position of the pixel corresponding to the position where the hole Ba is provided and the surrounding area are within a predetermined range. By setting as, it is possible to reduce the processing time by reducing the range in which the background 2a and the button B are discriminated.
Therefore, the CPU 12 executes the background information determination program 14c, the outer shape information acquisition program 14f, and the in-object information acquisition program 14h described above, so that the regularity of the background 2a (for example, a stripe pattern of 5 pixels) is obtained. It functions as a “discriminating unit” that performs processing for discriminating the background 2a and determining the portion that is not discriminated as the background 2a as the button B.
When the discrimination process between the background 2a and the button B is completed, the CPU 12 acquires the positions of the outlines of all the holes Ba as hole data, and based on the button outer periphery data L and the hole data described above. Then, the range within the range of the button B and excluding the hole portion is extracted from the imaging data of the entire imaging range, and the shading pattern data on the front of the button B excluding the hole portion is acquired. Note that the number of independent holes is also acquired in the hole data.
In the subsequent processes related to the object, the direction of the object is corrected as necessary for the digital image data D or information acquired from the digital image data D based on the hole data. A direction conversion process for converting the image data D or corresponding information is performed.

Next, the in-object error check program 14i will be described in detail. When the hole check data 15p is stored in the EEPROM 15 in advance by executing the in-object error check program 14i, the CPU 12 acquires the hole data and the hole check data acquired by the in-object information acquisition program 14h. A process of comparing with 15p is performed.
FIG. 8 is an explanatory diagram showing the relationship between the hole check data 15p and the hole data. The hole check data 15p is data for indicating the range F2 in which the outline of the hole of the button B shown in FIG. 8 is to be located and checking the number of independent F2 ranges, that is, the number of button holes. Is stored in the EEPROM 15 as shown in FIG. The CPU 12 determines whether or not the outline Q of the hole Ba of the button B indicated by the hole data is all included in the range F2, and the number of independent holes included in the hole data and the hole check data. To the number of independent F2 ranges. Accordingly, when the positional relationship of the plurality of hole portions of the button B is inclined with respect to the correct positional relationship as shown in FIG. 2G, a part of the hole portion of the button as shown in FIG. It is possible to discriminate between the button BD formed in error and the digital image data D of the button BE in which a part of the hole of the button is not formed as shown in FIG.

  Through the above comparison process, the CPU 12 determines whether the size, position, and number of the hole Ba of the button B shown in the digital image data D match the hole check data 15p. When it is determined that the digital image data D hole data and the hole check data 15p do not match, the CPU 12 outputs an error indicating that the hole Ba of the button B is abnormal. On the other hand, when it is determined that the digital image data D hole data and the hole check data 15p match, the CPU 12 outputs an output indicating that there is no problem in the hole Ba of the button B.

Next, the front / back determination program 14j will be described in detail. The CPU 12 executes the front / back determination program 14j, so that the density of the image in the range determined as the button B acquired by the in-object information acquisition program 14h when the density check data 15q is stored in the EEPROM 15 in advance. The front and back of the button B are determined from the pattern data.
FIG. 9 is an explanatory diagram showing an example of the front and back of the button B. FIG. 9A shows a button table, and FIG. 9B shows the back of the button.
As shown in FIG. 9A, when a step Bb is provided on the table of the button B, an image density difference occurs between the step Bb and the outer peripheral portion Bc. On the other hand, since there is no step on the back of the button B as shown in FIG. The density check data 15q is data indicating the density of the image when the front and back of the button are imaged, and the data structure is similar to the density pattern data acquired by the in-object information acquisition program 14h. This is shade pattern data indicating the luminance of each pixel of the excluded button image. That is, the CPU 12 compares the light / dark pattern data obtained by the in-object information acquisition program 14h with the light / dark check data 15q set in advance as data (front pattern) representing the front side (back side) of the button B, and the digital image The front and back of the button B shown in the data D is determined.

In this determination, an approximate value determination process based on the density check data 15q is performed. Specifically, the density is numerically expressed for each pixel of the density check data 15q and each pixel of the density pattern, and the density value of the density pattern is compared with the density value of the density check data 15q for the density of the pixel at the corresponding position. Is checked if it is within a predetermined range of approximations. At this time, if a certain percentage of pixels are within the approximate value, they are considered to match, and otherwise, they are considered not to match.
Therefore, the CPU 12 performs an output indicating that there is no problem when the density pattern matches the table pattern of the density check data 15q in the approximate value determination process, that is, when it is determined to be face up. On the other hand, if the light and shade pattern does not match the front pattern, the CPU 12 further compares the light and shade pattern with data (back pattern) when the back side included in the light and shade check data 15q is captured. At this time, if the gray pattern matches the back pattern, that is, if it is determined that the button B shown in the digital image data D is face down, an error indicating that the button is face down is output. If the shade pattern does not match either the front pattern or the back pattern, the button B indicates a shade pattern that is neither the front nor the back of the button set in the shade check data 15q, that is, the button color, It can be considered that the buttons have different shade patterns at the time of imaging due to different features such as decorations such as letters and patterns or three-dimensional structures. Therefore, the CPU 12 determines that the button B is a different type of button, and outputs an error indicating that the button type is different. As a result, it is possible to distinguish buttons having different features such as button color, decoration such as characters and patterns, or three-dimensional structures.
Therefore, whether the CPU 12 executes the front / back determination program 14j, the density pattern data of the button B imaged in the digital image data D has the density pattern generated at the time of imaging according to the density check data 15q that is set in advance, that is, the button characteristics. It functions as a “feature discriminating means”. Of course, the image used in the front / back determination may be an image of a part of the front surface or the back surface of the button as long as the front / back determination is possible.

If the button has a surface feature such that it does not have a constant shape even if it rotates like a step Bb, the direction determination by the following direction determination program 14l is performed instead of the front / back determination program 14j. You can go.
Next, the direction determination program 141 will be described in detail. The CPU 12 executes the direction determination program 14l, and when the direction check data 15r is previously set in the EEPROM 15, the image in the range determined as the button B acquired by the execution of the in-object information acquisition program 14h. It is determined whether or not the button B is in the direction of the button B set in advance from the light and shade pattern data.

FIG. 10 is an explanatory diagram showing an example of the determination region of the button B by the direction determination program 14l. FIG. 10A is an explanatory diagram showing an example of the determination area B1 when the direction is determined when the character A is printed on the button B, and FIG. 10B is an example of b1 which is the direction check data 15r. It is explanatory drawing shown.
For example, as shown in FIG. 10A, when the character A is printed on the button B, the CPU 12 sets a determination region B1 for determining the orientation of the character A, and the density of the image in the region is determined. Get each pattern data. Next, the direction determination program 14l calls the direction check data 15r indicating the correct direction of the preset character A from the EEPROM 15 and compares it with the grayscale pattern data of the image acquired from B1 described above. As shown in FIG. 10B, the direction check data 15r is shading pattern data limited to only a part indicated by b1 for determining the direction of the button within the range determined as the button. In the comparison, the digital image data D is subjected to an approximate value determination process for the image of the button B at a position corresponding to the direction check data 15r with reference to the direction check data 15r. At this time, if B1 and b1 match as shown in the correspondence relationship in FIGS. 10A and 10B, the CPU 12 determines that the direction of the button B is the correct direction, and there is no problem with the button B. Outputs that effect. On the other hand, if B1 and b1 do not match, it is determined that the direction of the button B is different, and the digital image data D or the direction check data 15r is rotated. When the angle is output to the outside, and either one of the digital image data D and the direction check data 15r is rotated by 360 °, the error is output.

The position of the determination area B1, the area width, the shape, and the light / dark pattern b1 set in advance are merely examples, and can be individually set according to the pattern and other characteristics that serve as a reference for determining the direction of the button B. . The direction check data 15r is not limited to the shade pattern of the character A applied to the button B, and may be any feature as long as the direction of the button B can be determined by the shade pattern comparison process.
Therefore, the CPU 12 determines whether the shading pattern of the button B imaged in the digital image data D has a shading pattern generated at the time of imaging according to the preset button feature by executing the direction determination program 141. Functions as a means.

(Automatic check mode)
Next, the automatic check mode will be described. In the automatic check mode, the outer periphery data of the button B acquired by the outer shape information acquisition program 15f is stored in the EEPROM 15 as the outer periphery check data 15o, and the hole data acquired by the in-object information acquisition program 14h is stored in the hole check data 15p. Is the setting of the image processing apparatus 10 that stores the light and shade pattern data as the light and shade check data 15q. That is, by using the automatic check mode, it is possible to set the outer periphery check data 15o, the hole check data 15p, and the shading pattern data based on the image of the button B captured by the camera 3. When the outer periphery check data 15o is set from the outer periphery data L, it is necessary to separately set a threshold range based on the outer periphery position indicated by the outer periphery data L. That is, the CPU 12 stores the range data having an error in the set threshold range in the EEPROM 15 as the outer peripheral check data 15o with the outer peripheral shape of the button indicated by the outer peripheral data L as the center line. Similarly, when setting the hole check data 15p from the hole data, it is necessary to separately set a threshold range based on the outline of the hole indicated by the hole data. That is, the CPU 12 stores in the EEPROM 15 range data having a set threshold range error as the hole check data 15p with the shape of the hole of the button indicated by the hole data as the center line. At this time, the number of independent holes is also stored in the hole check data 15p. Further, when setting the shading pattern, it is necessary to separately set whether the acquired shading pattern is the shading of the front image (front pattern) or the shading of the back image (back pattern). The automatic check mode can be set ON / OFF by an operation panel (not shown).
Therefore, the CPU 12 acquires the shading pattern of the image of the button B indicating the feature of the button B from the digital image data D that is a captured image of the button previously captured by executing the in-object information acquisition program 14h. It functions as a “feature acquisition unit” to be set.

In the automatic check mode, the digital image data D of the latest button B may be sequentially stored, or the digital image data D of the button B stored first may be used continuously thereafter. . Further, not only one digital image data D but also an average value (pixel position or pixel density value) based on the digital image data D of the button B obtained by imaging several times is calculated and stored. A value may be used.
Even when the automatic check mode is executed, if an error determination is made when the background contour check program 14e, the object error check program 14g, or the in-object error check program 14i is executed, the digital image data D obtained by imaging at that time is used. It is desirable to perform processing so as not to set the outer periphery check data 15o, the hole check data 15p, and the shading pattern based on the above. The same applies to the case where the front / back determination program 14j determines a surface other than the front surface.
Alternatively, the direction check data 15r may be set using a part of the shading pattern acquired in the automatic check mode.

(Operation of sewing machine button image recognition processing device)
Next, the operation of the button image recognition processing apparatus 1 of the sewing machine will be described in detail. 11 and 12 are flowcharts showing the operation of the button image recognition processing apparatus 1 of the sewing machine. 11 shows step S1 to step S10, and FIG. 12 shows step S11 to step S21.
First, when the CPU 12 executes the imaging program 14a, the button 3 is imaged together with the background 2a by the camera 3, and the digital image data D is stored in the RAM 12 (step 1). Next, the CPU 12 executes the image quality improvement program 14b to perform noise removal processing on the digital image data D (step S2).

  Next, the CPU 12 executes the background information determination program 14c to determine the background 2a of the digital image data D and the button B (step S3 (determination step)). Next, the CPU 12 executes the object presence / absence check program 14d and determines whether or not the range determined as the background 2a in step S3 is the entire digital image data D (step S4). If all the digital image data D is the background 2a (step S4: YES), an error is output (step S20). When all the digital image data D is not the background 2a (step S4: NO), the CPU 12 executes the background outline check program 14e, and determines whether or not there is a background 2a corresponding to the predetermined number of pixels in the outline. It is determined whether or not B is arranged so as to be surrounded by a predetermined range of the background 2a (step S5). If the contour includes a portion that has not been identified as the continuous background 2a for a predetermined number of pixels (step S5: NO), an error is output (step S20). When all the contours are determined as the continuous background 2a corresponding to the predetermined number of pixels (step S5: YES), the CPU 12 executes the outer shape information acquisition program 14f and acquires the outer peripheral data of the button B (step S6).

  Next, the CPU 12 determines whether or not the outer periphery check data 15o is stored in the EEPROM 15 (step S7). If the outer periphery check data 15o is stored in advance (step S7: YES), the CPU 12 executes the object error check program 14g and whether the outer periphery data of the button B obtained in step S6 matches the outer periphery check data 15o. It is determined whether or not (step S8). If they do not match (step S8: NO), an error is output (step S20). If they match (step S8: YES), the CPU 12 determines whether or not the automatic check mode is ON (step S9). When the automatic check mode is OFF (step S9: NO), the CPU 12 executes the in-object information acquisition program 14h to discriminate between the background 2a and the button B within the range (discrimination step), and the hole of the button B Data and shading patterns are acquired (step S11). When the automatic check mode is ON (step S9: YES), the outer periphery data of the button B acquired in step S6 is stored in the EEPROM 15 as the outer periphery check data 15o used after the next button B is imaged (step S10). Thereafter, the process of step S11 described above is performed. If the outer periphery check data 15o is not stored in step S7 (step S7: NO), the process proceeds to step S9.

  Next, the CPU 12 determines whether or not the hole check data 15p is stored in the EEPROM 15 (step S12). When the hole check data 15p is stored (step S12: YES), the CPU 12 executes the in-object error check program 14i, and the hole data of the button B obtained in step S11 is stored in the hole check data 15p. It is determined whether or not they match (step S13). If they do not match (step S13: NO), an error is output (step S20). If they match (step S13: YES), the CPU 12 determines whether or not the automatic check mode is ON (step S14). When the automatic check mode is OFF (step S14: NO), the CPU 12 determines whether the density check data 15q is stored in the EEPROM 15 (step S16). When the automatic check mode is ON (step S14: YES), the hole data of the button B acquired in step S11 is stored as the hole check data 15p used after the next button B is imaged, and the shade pattern data is stored. Is stored in the EEPROM 15 as the density check data 15q used after the next button B is imaged (step S15 (feature acquisition step)), and then the process of step S16 described above is performed. In addition, when the hole check data 15p is not memorize | stored in step S12 (step S12: NO), it transfers to the process of step S14.

  When the density check data 15q is stored in step S16 (step S16: YES), the CPU 12 executes the front / back determination program 14j and uses the density pattern obtained in step S11 and the table pattern of the density check data 15q in the EEPROM 15 as a result. It is determined whether or not the button B is face up (step S17 (feature determination step)). If it is determined in step S17 that the button B is not front (step S17: NO), the CPU 12 determines whether or not the button B is face down from the light / dark pattern and the reverse pattern of the light / dark check data 15q of the EEPROM 15 (step S17). S18 (feature discrimination step)). If it is determined in step 18 that the button is back (step S18: YES), an error indicating that the button is back is output (step S20). If the button is not back in step S18, that is, if the front and back patterns do not match (step S18: NO), it is determined that the button is a different type (step S19), and an error indicating that the button type is different is output. (Step S20).

If any error output is performed in steps S4 to S18 (step S20), or if it is determined that the button B is face up in step S17, the determination result of the button B is displayed on the monitor 5 (step S21). Note that the determination result of the button B displayed on the monitor 5 may be information indicating the presence or absence of an error or information indicating the type of error. Further, the captured digital image data D may be displayed.
In the case where the button image recognition processing device is mounted on the button supply device, supply to the sewing machine is performed when the button is determined to be face up in step S17.

(Operational effect of the button image recognition processing device according to the present embodiment)
According to the above-described embodiment, the range in which the background 2a does not have the regular black and white stripe pattern in the determination process performed by the CPU 12 executing the background information determination program 14c and the in-object information acquisition program 14h. It is determined as button B. At this time, since identification is performed based on whether the pattern follows the background 2a (for example, a pattern having a black and white striped pattern of 5 pixels in the vertical direction), even if the portion close to the background 2a is present in the button B, for example, No misunderstanding will occur as long as it does not match the background pattern. In addition, it is possible to accurately identify the button pattern or unevenness and the button hole. Therefore, it is possible to determine the button B only by whether or not the button B has a regular pattern regardless of the shape, color, pattern and other characteristics of the button B. Therefore, the button B can be more suitably discriminated.
Further, the hole Ba of the button B is determined as the background 2a by looking into the regular pattern of the background 2a. Therefore, it is possible to solve the problem that the hole provided inside the imaging target object (for example, the hole of the button) cannot be extracted as in the prior art, and the shape of the button B can be more suitably determined. Become. Therefore, the button B can be identified more suitably.
Further, even if the button B has transparency, if the background 2a is refracted by passing through the button B, the portion where the refraction has occurred will lose the regularity of the background 2a. The button B is determined without being determined. Therefore, in the case of an imaging object having transparency, the problem of the prior art that the density change between the background and the imaging object does not occur and the shape of the outer periphery of the imaging object may not be extracted can be solved. Therefore, the button B can be identified more suitably.

  Furthermore, since the focus of the camera 3 is adjusted to the background 2a, the background 2a can always be suitably imaged regardless of the thickness of the button and the three-dimensional structure. Therefore, the regular pattern of the background 2a can be more preferably discriminated, and the possibility of misidentification of the background 2a and the button B can be greatly reduced. Therefore, the button B can be identified more suitably.

  Further, in the feature determination step performed by the CPU 12 executing the front / back determination program 14j and the direction determination program 14l, the light / dark check data 15q or direction check set in advance from the light / dark pattern of the range of the image determined as the button B. It is determined whether or not the data 15r matches. That is, first, after identifying the range identified as the button B by excluding the range identified as the background 2a from the digital image data D, whether the density pattern of the range of the button B matches the density check data 15q. Is determined. Thus, the range of comparison processing can be limited to the range determined as the button B, compared to the case where the entire range of the digital image data D is searched for a range that matches the density check data 15q. Therefore, the load of the comparison process can be reduced and the processing time can be shortened.

Further, the CPU 12 executes the front / back determination program 14j to determine the front / back of the button B from the density check data 15q of the image when the front / back of the button is imaged and the density pattern of the button B of the digital image data D. This makes it possible to determine whether the button B is not facing down. Therefore, for example, by performing the front / back determination of the button B before sewing, it is possible to eliminate a situation where the button B is accidentally sewn with the button B turned upside down, and the sewing quality is greatly improved.
Further, the CPU 12 executes the direction determination program 141 to determine the direction of the button B. As a result, it is possible to solve the problems of the prior art that the orientation of decorations such as characters and patterns applied to the imaging object cannot be determined, and the characteristics of the imaging object can be determined in more detail.

  Further, when the automatic check mode is ON, in the feature acquisition step performed by the CPU 12 executing the background information determination program 12c and the in-object information acquisition program 14h, the button B from the range of the button B of the digital image data D is displayed. Peripheral data, hole data, and shading pattern are acquired and set as outer circumference check data 15o, hole check data 15p, and shading check data 15q. Therefore, the digital image data D when the button B is correctly imaged can be used as the outer periphery check data 15o, the hole check data 15p, and the density check data 15q. Therefore, it is not necessary to separately provide dedicated information and data for setting the outer periphery check data 15o, the hole check data 15p, and the density check data 15q. Therefore, it is possible to greatly reduce the trouble of setting the outer periphery check data 15o, the hole check data 15p, and the density check data 15q.

(Other)
In the above-described embodiment, the button image recognition processing device for the sewing machine is described. However, the image recognition processing other than the buttons of the sewing machine is performed by the configuration having the same characteristics as the button image recognition processing device 1 and the operation process thereof. It is also possible to do this.

Further, the background 2a of the background member 2 in the above-described embodiment is not limited to a striped pattern.
FIG. 13 is an explanatory diagram illustrating an example of a background having regularity different from the background 2a in the embodiment. 13A is an explanatory diagram of a regular check pattern background 2b, and FIG. 13B is an explanatory diagram of a regular diagonal striped background 2c.
For example, as shown in FIGS. 13A and 13B, if the background has regularity, a range that cannot be determined as a background can be determined as an imaging target (for example, a button). In this case, it is necessary to store data corresponding to FIGS. 13A and 13B in the pattern data H as well. Needless to say, other patterns may be used as long as they have regularity.
The direction for determining the background 2a is the vertical direction in the above-described embodiment, but may be a horizontal direction or an oblique direction depending on the shape of the pattern to be used.

In the above-described embodiment, the image captured by the camera 3 is a black and white multi-value image (grayscale), but may be a color image. In this case, the comparison process is performed using the gray pattern as a color pattern and the gray check data 15q as preset color pattern data.
In the above-described embodiment, the analog image captured by the camera 3 is converted into the digital image data D by the A / D converter 11, but the camera 3 is a digital camera using a CCD image sensor, a CMOS image sensor, or the like. The digital image data D may be directly output to the CPU 12 and the A / D converter 11 may be omitted.
In addition, each numerical value setting such as the number of pixels (5 pixels) serving as a basis for the regularity of the striped pattern of the background 2a in the above-described embodiment and the number of pixels of the background outline (10 pixels) is merely an example. It goes without saying that numerical settings may be used. Further, the regularity threshold value of the striped pattern of the background 2a may be set within a certain numerical range (for example, 4 to 6 pixels) instead of a unique value (5 pixels).
In addition, the degree and shape of the pattern used for the background, the ratio of the shaded portion, and the like can be variously used as long as they are regular.
In the present invention, it is only necessary to be able to distinguish between a range having regularity of the background of the captured image and a range having no regularity in the discrimination of the imaging target, and the discrimination processing method for the range having regularity of the background is the above-described embodiment. It is not limited to. For example, the digital image data D is subjected to Fourier transform, the frequency component of the background pattern is differentiated from the frequency components of other parts with a specific value, and the background pattern and the symmetrical object image are respectively converted from the inverse Fourier transform image. It may be extracted as a binary image, and the position information may be acquired by an existing image recognition method such as edge detection or labeling.

It is a block diagram which shows the structure of a button image recognition processing apparatus. It is a top view which shows an example of the relationship between the button and the background 2a of a background member. 2 (a) shows no malformation, and a button of a predetermined size is placed in the center of the background in the correct orientation, FIG. 2 (b) shows a case where there is no button, and FIG. ) Shows a case where a button larger than a predetermined size is placed, FIG. 2D shows a case where the position of the button is shifted from the center of the background, and FIG. In FIG. 2 (f), when a button smaller than a predetermined size is placed, FIG. 2 (g) shows that the positional relationship between the plurality of hole portions of the button is relative to the positional relationship shown in FIG. 2 (a). When tilted, FIG. 2 (h) shows a button with a part of the hole formed in the wrong position. FIG. 2 (i) shows a button with no part of the hole formed. FIG. It is explanatory drawing which shows the various programs memorize | stored in the various programs memorize | stored in EEPROM and various programs. FIG. 3A shows various programs stored in the ROM 14, and FIG. 3B shows various programs stored in the EEPROM 15. It is explanatory drawing which expanded some digital image data. The numerical values attached to the periphery of the digital image data are indices for explaining the pixel positions of the digital image data. It is explanatory drawing which shows the boundary line of a background and a button. 5A shows a boundary line of the upper half of the button determined by the background information determination program, FIG. 5B shows a boundary line of the lower half of the button determined by the background information determination program, FIG. 5C is an explanatory view showing the outer periphery data of the button synthesized by the outer shape information acquisition program. It is explanatory drawing which shows the relationship between outer periphery check data and outer periphery data. It is explanatory drawing which shows a part of digital image data in which one of the hole parts of a button is contained. It is explanatory drawing which shows the relationship between hole part check data and hole part data. It is explanatory drawing which shows an example of the front and back of a button. FIG. 9A shows a button table, and FIG. 9B shows the back of the button. It is explanatory drawing which shows an example of the determination area | region of the button B by the direction determination program 141. FIG. 10A is an explanatory diagram showing an example of the determination area B1 when the direction is determined when the character A is printed on the button B, and FIG. 10B is an example of b1 which is the direction check data 15r. It is explanatory drawing shown. It is a flowchart which shows step S1 to step S11 among the flowcharts which show operation | movement of the button image recognition processing apparatus of a sewing machine. It is a flowchart which shows step S12 to step S21 among the flowcharts which show operation | movement of the button image recognition processing apparatus of a sewing machine. It is explanatory drawing which shows an example of the background which has the regularity different from the background in embodiment. 13A is an explanatory diagram of a background of a regular check pattern, and FIG. 13B is an explanatory diagram of a background of a regular diagonal stripe pattern.

Explanation of symbols

2 background member 2a background 3 camera 10 image processing apparatus 11 A / D converter 12 CPU
14 ROM
15 EEPROM
B button

Claims (13)

A plane member having a known regular pattern, a background member having the plane as a background, a camera for imaging an imaging object within the range of the plane with the pattern together with the background, In an image recognition processing method for recognizing the imaging object from an image captured by the camera using
An image recognition processing method comprising: a discrimination step of discriminating, as the imaging object, a range having no regular background pattern in the captured image.   The image recognition processing method according to claim 1, wherein the camera captures the captured image while focusing on the background.   The image recognition process according to claim 1, further comprising a feature determination step of determining whether or not the image pickup object has a preset characteristic from the range of the image determined as the image pickup object. Method.   The image recognition processing method according to claim 3, wherein in the feature determination step, an image that is a feature of the imaging object acquired from a captured image of the imaging object before the previous time is used. A background member having a plane with a known regular pattern, the background of which is a background, and a camera for imaging an imaging object within the range of the plane with the pattern together with the background An image recognition processing apparatus for recognizing the imaging object from an image captured by the camera,
An image recognition processing apparatus comprising: a determination unit configured to determine, as the imaging object, a range that does not have the regular pattern of the background in the captured image.   The image recognition processing apparatus according to claim 5, wherein the camera captures the captured image while focusing on the background.   The image recognition process according to claim 5, further comprising: a feature determination unit configured to determine whether or not the image pickup object has a preset characteristic from the range of the image determined as the image pickup object. apparatus.   The image recognition processing apparatus according to claim 7, further comprising a feature acquisition unit configured to acquire an image that is a characteristic of the imaging target from a previous captured image of the imaging target. A background member having a plane with a known regular pattern, the background of which is a background, and a camera for imaging a button of clothing within the range of the plane with the pattern together with the background A button image recognition processing device for recognizing the button from an image captured by the camera,
A button image recognition processing apparatus comprising: a determination unit configured to determine, as the button, a range having no regular background pattern in the captured image.   The button image recognition processing apparatus according to claim 9, wherein the camera captures the captured image while focusing on the background.   11. The button image recognition processing apparatus according to claim 9, further comprising a feature determining unit that determines whether or not the button has a preset feature from the range of the image determined as the button.   The button image recognition processing apparatus according to claim 11, further comprising a feature acquisition unit configured to acquire an image that is a feature of the button from a captured image of the button before the previous time.   The feature discriminating unit discriminates the front and back of the button from at least one of the front and back of the button when the image is obtained by capturing at least one of the front and back of the button. A button image recognition processing apparatus according to claim 1.

Images