JP6244787B2 - Image recognition apparatus, image recognition method, and image recognition program - Google Patents

Description

  The present invention relates to an image recognition technique.

  2. Description of the Related Art Conventionally, a technique for performing recognition by inputting a recognized image, reading the recognized template image from a database storing the recognized template image, and comparing the recognized template image with the recognized image (for example, Patent Document 1). ).

  In such image recognition, how to effectively remove a noise component contained in an image to be recognized is important for improving image recognition accuracy.

JP 2004-151771 A

  Here, when a display image to be subjected to image recognition is captured by a photographing apparatus such as a digital camera, color fringe noise caused by the display characteristics of the display image and the structure of the photographing apparatus may be a problem. In particular, a DLP (Digital Light Processing) equipped with a digital mirror device (DMD: Digital Mirror Device) ("DLP" is a registered trademark of Texas Instruments Incorporated, USA) is projected onto a projection surface (projector). When the captured image is taken with a camera with a built-in smartphone that performs automatic exposure control, color fringe noise appears prominently.

  In the DLP projection apparatus, the brightness of the display image projected on the projection surface is controlled by controlling the on / off time ratio of the mirror in the digital mirror device. Furthermore, in a single-plate projection method often used in a DLP projection apparatus, color is controlled by switching R, G, and B light at high speed. In a display image projected on the projection surface, when observed in a very short time, the moment when the mirror is on and the light is illuminated, the moment when the mirror is off and the light is not illuminated, and R, G, and B are displayed in a single color. When viewed with the human eye, the time ratio of the images is perceived by the brightness of the display image, and the color obtained by combining R, G, and B is felt as the color of the display image. On the other hand, in an automatic exposure control photographing apparatus, the exposure timing is controlled for each line on the display image in accordance with the brightness of the display image, and the photographed image is captured for each line. Therefore, at a certain line timing, each pixel value on the corresponding line of the display image is captured at the moment when the light hits the projection surface, and at another line timing, the corresponding line of the display image at the moment when the light does not hit the projection surface. There is a possibility that each pixel value above is captured. Further, there is a possibility that each pixel value on the corresponding line of the display image is captured at the moment when the R, G, B single colors are in contact with the projection surface. As a result, on the photographed image, there are lines that are photographed brightly and lines that are photographed darkly, and lines that are photographed in R, G, and B single colors, which appear as color fringe noise.

  Such color fringe noise may also appear due to the display characteristics of other display images and the structure of the photographing apparatus.

  And this color fringe noise reduces recognition accuracy, when performing image recognition based on a picked-up image.

  Such color fringe noise is generated almost randomly depending on the combination of the luminance control timing by the projection apparatus and the exposure control timing by the photographing apparatus. For this reason, conventionally, there is no known conventional technique for effectively removing the color fringe noise generated on the photographed image taken by the photographing apparatus, and there is a problem that the color fringe noise cannot be removed effectively. .

  The present invention is an image recognition apparatus that recognizes a display image in a captured image based on a captured image, and that effectively eliminates the color fringe noise caused by the display characteristics of the display image and the structure of the image capturing apparatus. The purpose is to improve the accuracy.

In an example of the aspect, the image recognition apparatus is an imaging unit that captures a display image that is an icon indicating predetermined information, and a display characteristic of the display image in a captured image obtained by capturing the display image by the imaging unit. Color fringe noise removing means for removing color fringe noise caused by the structure of the photographing means, and image recognition means for recognizing a display image in the fringe-removed image obtained by removing the color fringe noise.
In another example of the aspect, the image recognition apparatus includes a photographing unit that captures a display image indicating predetermined information, and a display characteristic and a photographing of the display image in a captured image obtained by the photographing unit capturing the display image. Color fringe noise removing means for removing color fringe noise caused by the structure of the means, and image recognition means for recognizing a display image in the fringe-removed image obtained by removing the color fringe noise, the display image Is an image projected on the projection plane by a projection apparatus equipped with a digital mirror device.
In another example of the aspect, the image recognition apparatus includes a photographing unit that captures a display image indicating predetermined information, and a display characteristic and a photographing of the display image in a captured image obtained by the photographing unit capturing the display image. Color fringe noise removing means for removing color fringe noise caused by the structure of the means, and image recognition means for recognizing a display image in the fringe-removed image obtained by removing the color fringe noise. The means compares the display image in the striped image with each registered image on an image database in which the same display image as the display image is registered, recognizes the captured display image, and recognizes the image The means evaluates the similarity using an edge-based feature amount in the collation between the display image in the captured image and each registered image in the image database, and the image recognition means The mask image created from the registered image registered in the base to calculate the edge-based feature value from an image area obtained by applying the fringe removed image.
In another example of the aspect, the image recognition apparatus includes a photographing unit that captures a display image indicating predetermined information, and a display characteristic and a photographing of the display image in a captured image obtained by the photographing unit capturing the display image. Color fringe noise removing means for removing color fringe noise caused by the structure of the means, and image recognition means for recognizing a display image in the fringe-removed image obtained by removing the color fringe noise. The means compares the display image in the striped image with each registered image on an image database in which the same display image as the display image is registered, recognizes the captured display image, and recognizes the image The means evaluates the similarity using an edge-based feature amount in the comparison between the display image in the photographed image and each registered image in the image database, and the image recognition means comprises the photographed image Other calculates the edge-based feature value from an image area obtained by applying a mask image created from the fringe removed image on the fringe removed image.
In another example of the aspect, the image recognition apparatus includes a photographing unit that captures a display image indicating predetermined information, and a display characteristic and a photographing of the display image in a captured image obtained by the photographing unit capturing the display image. Color fringe noise removing means for removing color fringe noise caused by the structure of the means, and image recognizing means for recognizing a display image in the striped image obtained by removing the color fringe noise, The noise removing unit removes the color component of the captured image.

  According to the present invention, in an image recognition device that recognizes a display image in a photographed image based on the photographed image, the color fringe noise caused by the display characteristics of the display image and the structure of the photographing device is effectively removed, It is possible to improve the accuracy of image recognition.

1 is a block diagram of an image recognition apparatus according to an embodiment of the present invention. It is a figure which shows the example of a mask frame. It is explanatory drawing of a mask image. It is explanatory drawing of a 1st image recognition process. It is explanatory drawing of the production | generation process of the mask image in a 2nd image recognition process. It is a block diagram which shows the hardware structural example of an image recognition apparatus. It is a flowchart which shows the process example of this embodiment including a 1st image recognition process. It is a flowchart which shows the process example of this embodiment including a 2nd image recognition process. It is a flowchart which shows the example of a HOG feature-value calculation process.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram of an image recognition apparatus 100 according to an embodiment of the present invention. The image recognition apparatus 100 according to this embodiment includes a photographing unit 101, a color fringe noise removing unit 102, an image recognition unit 103, and an alignment unit 107.

  The photographing unit 101 photographs a display image 104 indicating predetermined information. The display image 104 is an icon indicating predetermined information, for example. The icon is, for example, black and white data. The display image 104 is, for example, an image projected on a projection surface by a projection apparatus provided with a digital mirror device.

  The color fringe noise removing unit 102 removes color fringe noise caused by the display characteristics of the display image 104 and the structure of the photographing unit 101 in the photographed image 105 obtained by photographing the display image 104 by the photographing unit 101. The display characteristic of the display image 104 is, for example, a timing characteristic of turning on and off the mirror when the projection apparatus including the digital mirror device projects the display image 104 on the projection plane. The structure of the imaging unit 101 is, for example, a structure that controls the exposure timing during line scanning in the imaging unit 101. Here, the color fringe noise appearing in the captured image 105 appears as a color of a light source used for projection or a color obtained by mixing them. For this reason, the color fringe noise removing unit 102 removes the color fringe noise by removing the color component for each pixel of the captured image 105. More specifically, the color fringe noise removing unit 102 executes a process of setting the saturation value (S value) of the HSV color space to 0 for each pixel of the photographed image 105 to generate the fringe removed image 106. Alternatively, the color fringe noise removing unit 102 executes a process of replacing the R component value, the G component value, and the B component value with the maximum value among the component values for each pixel of the captured image 105, and the fringe removed image 106. Is generated. As a result, a fringe-removed image 106 as illustrated in FIG. 1 in which the influence of the color fringe noise is reduced is generated from the captured image 105 affected by the color fringe noise as illustrated in FIG. Can do.

  The image recognition means 103 recognizes the display image 104 in the fringe-removed image 106 obtained by removing the color fringe noise. For example, the image recognizing unit 103 collates the display image 104 in the striped image 106 with each registered image on the image database 108 in which the same display image 104 as the display image 104 is registered, and displays the captured display image 104. recognize. For example, the image database 108 is built in the image recognition apparatus 100. Alternatively, the image database 108 is built in a server device (not shown) that is different from the image recognition device 100. In this case, the image recognition unit 103 executes processing for recognizing the display image 104 by communicating with the image database 108 in the server device.

  Here, for example, an alignment unit 107 may be provided between the color fringe noise removing unit 102 and the image recognition unit 103 (or between the photographing unit 101 and the color fringe noise removing unit 102). In this case, the display image 104 includes an icon and an icon frame that is configured by a line having a direction different from the direction of the color fringe noise and is arranged around the icon, and the alignment unit 107 includes the stripe removal image 106 (or The icon is aligned by detecting the icon frame from the photographed image 105). This alignment process is a process for preventing the feature amount calculated for image recognition by the image recognition unit 103 from being changed depending on the position of the display image 104 or the like. If the feature quantity calculated by the image recognition means 103 is a feature quantity that does not change with the position or size of the display image 104, the alignment means 107 may not be provided. For example, a diamond-shaped frame shown in FIG. 2A is used as the icon frame. Since this shape does not include a component parallel to the fringe of the color fringe noise, it is not easily affected by the color fringe noise when detecting the icon frame. In the alignment process, first, straight lines on the four sides of the rhombus frame are detected by the Hough transform, and the intersection of the detected straight lines (the apex of the rhombus) is calculated. Then, alignment is performed by projective transformation using the calculated four intersections. As the shape of the icon frame, in addition to the rhombus shown in FIG. 2 (a), a shape that does not include a component parallel to the color fringe noise stripe (such as a circle shown in FIG. 2 (b)) (not used for detection). If it is.

  In the configuration of FIG. 1, the image recognition unit 103, for example, compares the display image 104 in the captured image 105 with each registered image in the image database 108, such as HOG (Histogram of Oriented Gradients) feature amount. Evaluate the degree of similarity using edge-based features.

As a specific operation example of the image recognition unit 103 related to the similarity evaluation using the edge-based feature amount, an operation when the image recognition unit 103 executes the first image recognition process will be described. In this case, the image recognition unit 103 calculates each edge-based feature amount from each image region obtained by applying each mask image created from the registered image registered in the image database 108 to the fringe-removed image 106.
FIG. 3 is an explanatory diagram of the mask image, and FIG. 4 is an explanatory diagram of the first image recognition process.

  FIG. 3 is a diagram showing the relationship between the display image 104 and the mask image 301 corresponding thereto. When the mask image 301 is applied to the display image 104, the white color portion illustrated in FIG. 3B outputs the pixels on the display image 104 as they are, and the black color illustrated in FIG. In this part, the pixels on the display image 104 are masked and not output.

  Here, in the image database 108, the same icons as those of various patterns of the display image 104 are registered. For example, the number of types of registered icons is N. That is, N types of registration icons are registered in the image database 108.

  When the image recognition unit 103 executes the first image recognition process, the image database 108 is configured to mask portions other than the outline portion of each icon corresponding to the N types of registered icons. For example, N types of mask images 401 (# 1) to 401 (#N) shown in FIG. 4 are registered.

  The image recognizing unit 103 uses the feature values 402 (# 1) to 402 (from the image regions obtained by applying the mask images 401 (# 1) to 401 (#N) of FIG. #N) is calculated. If this feature quantity is, for example, an HOG feature quantity, each of the feature quantities 402 (# 1) to 402 (#N) is a luminance gradient histogram.

  Here, for example, in FIG. 4, when a mask image that closely matches the icon on the fringe removal image 106 is applied to the fringe removal image 106 as in the mask image 401 (# 1), the calculation is performed based thereon. The feature amount 402 (# 1) is a feature amount that well indicates the feature of the icon in the striped image 106. When distance calculation is performed between the feature amount 402 (# 1) and the feature amount of the registered icon corresponding to the mask image 401 (# 1) stored in the image database 108, the calculation result Indicates a very close distance.

  On the other hand, when a mask image that does not match the icon on the striped image 106 is applied to the striped image 106 as in the mask images 401 (# 2) to 401 (#N) in FIG. The feature amounts 402 (# 2) to 402 (#N) calculated from the values hardly show the feature of the icon in the fringe-removed image 106. These feature amounts 402 (# 2) to 402 (#N) are the feature amounts of the registered icons corresponding to the mask images 401 (# 1) to 401 (#N) stored in the image database 108, respectively. If each distance calculation is executed between the two, any calculation result indicates a far distance.

  As described above, when the image recognition unit 103 executes the first image recognition process, the mask images 401 (# 1) to 401 (#N) make the presence of the icon on the striped image 106 more prominent. This has the effect of further removing the influence of the color fringe noise, thereby making it possible to improve the image recognition rate.

  Next, an operation when the image recognition unit 103 executes the second image recognition process will be described as another specific operation example of the image recognition unit 103 related to the similarity evaluation using the edge-based feature amount. In this case, the image recognizing unit 103 calculates an edge-based feature amount from an image area obtained by applying the mask image created from the captured image 105 or the striped image 106 itself to the striped image 106.

  The mask image in this case is generated as follows. FIG. 5 is an explanatory diagram of a mask image generation process in the second image recognition process. First, the color fringe noise removing unit 102 in FIG. 1 removes the color fringe noise component from the captured image 105 to obtain a fringe-removed image 106. Next, an edge component 501 is extracted from the fringe-removed image 106 and used as first data 501. On the other hand, a hue component 502 is extracted from the photographed image 105, an edge component 503 is extracted from the hue component 502, and each edge portion in the edge component 503 is further expanded to create an edge expansion component 504. Data 504. By masking the edge component 501 of the first data with the edge expansion component 504 of the second data, the mask image 505 of the icon and the icon frame portion from which the influence of the color fringe noise has been removed is calculated.

  The image recognizing unit 103 calculates the edge base feature amount from the image region obtained by applying the mask image 505 created from the captured image 105 or the striped image 106 itself to the striped image 106 as described above. If this feature amount is, for example, an HOG feature amount, the feature amount 506 is a luminance gradient histogram.

  In FIG. 5, since the mask image 505 is created from the captured image 105 or the striped image 106 itself, it is a mask image that closely matches the icon on the striped image 106. Therefore, when the mask image 505 is applied to the stripe-removed image 106, the feature amount 506 calculated from the mask image 505 is a feature amount that well indicates the feature of the icon in the stripe-removed image 106. The feature amount 506 includes an icon most similar to the icon in the stripe-removed image 106 by performing each distance calculation between each feature amount of each registered icon stored in the image database 108. The distance from the feature amount of the registered icon can be minimized. As a result, the image recognition unit 103 can obtain a correct image recognition result.

  As described above, the image recognition apparatus 100 shown in FIG. 1 effectively removes the color fringe noise caused by the display characteristics of the display image 104 and the structure of the photographing unit 101, thereby improving the accuracy of image recognition. It becomes possible to make it.

  FIG. 6 is a block diagram illustrating a hardware configuration example of the image recognition apparatus 100 in FIG.

  The image recognition apparatus 100 is realized on a computer system that is a portable information terminal such as a so-called smartphone.

  The image recognition apparatus 100 includes a CPU (Central Processing Unit) 602, a ROM (Read Only Memory) 603, and a RAM (Random Access Memory) 604. The image recognition apparatus 100 includes an external storage device 605 such as a solid storage device, a communication interface 606, an input device 607 such as a touch panel display device, and a display device 608. Further, the image recognition apparatus 100 includes a portable recording medium driving device 609 capable of setting a portable recording medium 160 such as a micro SD memory card or a USB (Universal Serial Bus) memory card. The imaging device 612 is a digital camera mechanism that can capture still images and video images, and includes a lens, an autofocus drive control device, an exposure control device, an imaging sensor, and the like. The devices 602 to 609 and 612 described above are connected to each other by a bus 611.

  In addition to a program for controlling general operations of the entire smartphone, the ROM 603 stores a control program for image recognition processing shown by flowcharts in FIGS. The CPU 602 reads this control program from the ROM 603 and executes the RAM 604 as a work memory, thereby realizing the image recognition function shown by the block in FIG. As a result, for example, an image recognition process for recognizing an icon from the captured image 105 obtained by the user capturing the display image 104 including the icon with the imaging device 612 is executed. The image database 108 in FIG. 1 may be built in the external storage device 605 in FIG. 6, but may be built in, for example, a server device on the Internet (not shown). In this case, the CPU 602 communicates with the image database 108 in the server device via the communication interface 606 via the Internet, and executes image recognition processing. The CPU 602 causes the display device 608 to display an image recognition result such as an icon.

  FIG. 7 is a flowchart showing a processing example of the present embodiment including the first image recognition processing (see FIG. 4) by the image recognition unit 103 described above, which is executed by the CPU 602 of FIG. This process is realized as an operation in which the CPU 602 executes a control program stored in the ROM 603 or the RAM 604.

  First, the CPU 602 executes preprocessing (step S701). Here, the CPU 602 obtains the captured image 105 described above with reference to FIG. 1 (function of the imaging unit 101) and generates a fringe-removed image 106 by removing the color fringe noise from the captured image 105 (of the color fringe noise removing unit 102). Function) and alignment processing (function of the alignment means 107).

  Specifically, the CPU 602 acquires the captured image 105 in, for example, the RAM 604 by controlling the imaging device 612.

  In removing the color fringe noise, as described above, the CPU 602 performs processing for setting the saturation value (S value) of the HSV color space to 0 for each pixel of the acquired captured image 105, or R component value and G component value. , And a process of replacing the B component value with the maximum value among the component values. The CPU 602 stores the striped image 106 obtained as a result in, for example, the RAM 604.

  In the alignment process, as described above, the CPU 602 detects, for example, the straight lines on the four sides of the frame of the rhombus (see FIG. 2A) by the Hough transform, and calculates the intersection (the apex of the rhombus) of the detected line. Then, alignment is performed by projective transformation using the four intersections.

  Next, the CPU 602 executes the above-described first image recognition process (steps S702 → S703 → S704 repetition process → S705).

  First, the CPU 602 sequentially acquires mask images 401 (# 1 to #N) (see FIG. 4) corresponding to each registered icon from the image database 108 stored in the external storage device 605 or accessible from the communication interface 606. Then, the acquired one mask image 401 is applied to the fringe-removed image 106, and feature amount extraction is executed on the resulting image (step S702). Details of the feature amount extraction will be described later with reference to FIG.

  The CPU 602 determines the feature amount 402 (any one of # 1 to #N) (see FIG. 4) obtained corresponding to the mask image 401 and the feature amount of each registered icon in the image database 108. A feature distance is calculated (step S703).

  The CPU 602 determines whether or not processing has been completed for all registered icons (step S704).

  If the determination in step S704 is NO, the CPU 602 repeatedly executes the processes in steps S702 and S703 using the mask image 401 of the next registered icon.

  If the determination in step S704 is YES, the CPU 602 selects the registered icon on the image database 108 when the result of the feature distance calculation in step S703 indicates the closest distance as a recognition result, and displays it on the display device 608. Output (step S705).

  FIG. 8 is a flowchart showing a processing example of this embodiment including the second image recognition processing (see FIG. 5) by the above-described image recognition means 103, which is executed by the CPU 602 of FIG. This process is realized as an operation in which the CPU 602 executes the control program stored in the ROM 603 or the RAM 604, as in the case of FIG.

  First, the CPU 602 executes preprocessing (step S801). The processing here is the same as that in step S701 in FIG. 7, and the CPU 602 performs processing of acquiring the captured image 105, generating the fringe-removed image 106 by removing color fringe noise from the captured image 105, and alignment processing. Run.

  Next, the CPU 602 executes the second image recognition process described above (repeating process of steps S802 → S803 → S804 and S805 → S806).

  First, the CPU 602 executes processing for generating a mask image 505 (see FIG. 5). As described with reference to FIG. 5, the CPU 602 extracts the edge component 501 as the first data from the striped image 106 and stores it in the RAM 604. On the other hand, the CPU 602 extracts the hue component 502 from the captured image 105, extracts the edge component 503 from the hue component 502, and further expands each edge portion in the edge component 503 to create an edge expansion component 504. Second data 504 is stored in the RAM 604. The CPU 602 masks the edge component 501 of the first data on the RAM 604 by masking the edge component 501 of the first data on the RAM 604 with the edge expansion component 504 of the second data on the RAM 604, thereby masking the icons and icon frame portions. An image 505 is calculated and stored in the RAM 604.

  The CPU 602 applies the mask image 505 generated in the RAM 604 to the fringe-removed image 106, and executes feature amount extraction on the resulting image (step S803). This process need only be executed once, unlike step S702 in FIG. Details of the feature amount extraction will be described later with reference to FIG.

  The CPU 602 calculates the feature amount distance between the feature amount 506 (see FIG. 5) obtained corresponding to the mask image 505 and the feature amount of each registered icon in the image database 108 (step S804).

  The CPU 602 determines whether or not processing has been completed for all registered icons (step S805).

  If the determination in step S805 is NO, the CPU 602 repeatedly executes the feature amount distance calculation in step S804 for the next registered icon.

  If the determination in step S805 is YES, the CPU 602 selects the registered icon on the image database 108 when the result of the feature amount distance calculation in step S804 indicates the closest distance, and displays it on the display device 608. Output (step S806).

  FIG. 9 is a flowchart illustrating an example of a HOG feature amount calculation process that is a feature amount extraction process in step S702 of FIG. 7 or step S803 of FIG.

  The HOG feature amount is a feature amount obtained by calculating a luminance gradient / luminance intensity from a local region of an image, and is a feature amount that can represent a rough shape of an icon.

  First, the CPU 602 divides the image after the mask application into blocks (step S901).

  Next, the CPU 602 calculates a luminance gradient and luminance intensity for each cell obtained by further dividing the block while moving the block in the image (step S902).

  For each cell in the block, the CPU 602 accumulates the luminance intensity calculated in step S902 on the element corresponding to the luminance gradient calculated in step S902 of the luminance gradient histogram (step S903).

  Next, the CPU 602 normalizes the luminance gradient histogram for each block (step S904). Since the shape of the luminance gradient histogram can be adjusted by normalization processing, it is possible to cope with changes in brightness.

  The CPU 602 determines whether the processing has been completed for all blocks in the image (step S905).

  If the determination in step S905 is NO, the CPU 602 executes a series of processing in steps S902 to S904 for the next block.

  If the determination in step S905 is YES, the CPU 602 ends the HOG feature amount extraction process.

As described above, in the present embodiment, it is possible to reduce the influence of the color fringe noise in the feature amount calculation by removing the color component as the preprocessing.
Further, by calculating the feature amount using the mask, it is possible to further reduce the influence of the color fringe noise on the feature amount.
Here, by creating a mask from a registered icon, it is possible to further reduce the influence of color fringe noise.
Alternatively, by creating the mask from the captured image or the striped image, the feature amount needs to be calculated only once, so that the recognition process can be executed at high speed.
In the present embodiment, the icon frame that is not affected by the color fringe noise can be detected by performing alignment using an icon frame having a shape that does not include a component parallel to the stripes of the color fringe noise.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
Photographing means for photographing a display image showing predetermined information;
In the photographed image obtained by photographing the display image by the photographing means, color fringe noise removing means for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
Image recognition means for recognizing the display image in a striped image obtained by removing the color fringe noise;
An image recognition apparatus comprising:

(Appendix 2)
The display image is an icon indicating the predetermined information.
The image recognizing device according to Supplementary Note 1, wherein

(Appendix 3)
The icon is black and white data.
The image recognizing device according to Supplementary Note 2, wherein

(Appendix 4)
The display image is an image projected on a projection plane by a projection apparatus including a digital mirror device.
The image recognition device according to any one of appendices 1 to 3, wherein

(Appendix 5)
The image recognition means compares the display image in the striped image with each registered image on an image database in which the same display image as the display image is registered, and recognizes the captured display image.
The image recognition device according to any one of appendices 1 to 3, wherein

(Appendix 6)
The image database is built in the image recognition device.
The image recognizing device according to appendix 5, characterized in that:

(Appendix 7)
The image database is built in a server device different from the image recognition device,
The image recognition means executes a process of recognizing the display image by communicating with the image database in the server device.
The image recognizing device according to appendix 5, characterized in that:

(Appendix 8)
The display image is composed of the icon and an icon frame configured by a line having a direction different from the direction of the color fringe noise and arranged around the icon.
Alignment means for aligning the icon by detecting the icon frame from the captured image,
The image recognizing device according to appendix 2 or 3, characterized by the above.

(Appendix 9)
The icon frame is a diamond frame,
The image recognizing device according to appendix 8, characterized by:

(Appendix 10)
The icon frame is a circular frame.
The image recognizing device according to appendix 8, characterized by:

(Appendix 11)
The color fringe noise removing means removes a color component of the captured image;
The image recognition apparatus according to any one of appendices 1 to 10, wherein

(Appendix 12)
The color fringe noise removing unit executes a process of setting the saturation value of the HSV color space to 0 for each pixel of the captured image, and generates the fringe removed image.
The image recognition apparatus according to appendix 11, wherein

(Appendix 13)
The color fringe noise removing unit performs a process of replacing the R component value, the G component value, and the B component value with the maximum value among the component values for each pixel of the captured image, and Generate,
The projection image recognition system according to appendix 11, wherein

(Appendix 14)
The image recognition means evaluates similarity using an edge-based feature amount in collation between the display image in the captured image and each registered image in the image database;
The image recognition device according to any one of appendices 5 to 7, characterized in that:

(Appendix 15)
The image recognition means calculates the edge-based feature amount from an image region obtained by applying a mask image created from the registered image registered in the image database to the fringe-removed image;
15. The image recognition device according to appendix 14, wherein

(Appendix 16)
The image recognition means calculates the edge-based feature amount from an image region obtained by applying a mask image created from the captured image or the fringe-removed image to the fringe-removed image.
15. The image recognition device according to appendix 14, wherein

(Appendix 17)
The mask image is created from first data obtained by edge extraction after removing the color component of the photographed image and second data obtained by expanding the edge extracted from the hue component of the photographed image.
Item 17. The image recognition device according to appendix 16, wherein

(Appendix 18)
The edge-based feature value is a HOG feature value.
18. The image recognition device according to any one of appendices 14 to 17, characterized in that:

(Appendix 19)
Take a display image showing the predetermined information,
In the photographed image obtained by photographing the display image, color fringe noise caused by the display characteristics of the display image and the structure of the photographing means is removed,
Recognizing the display image in a striped image obtained by removing the color fringe noise;
An image recognition method characterized by the above.

(Appendix 20)
On the computer,
A shooting step of shooting a display image showing predetermined information;
In a photographed image obtained by photographing the display image in the photographing step, a color fringe noise removing step for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
An image recognition step for recognizing the display image in the striped image obtained by removing the color fringe noise;
An image recognition program for executing

(Appendix 21)
In a projection image recognition system comprising: a projection device that projects an image showing predetermined information; and a photographing device that captures and captures the image captured on the projection screen projected by the projection device;
Color fringe noise removing means for removing color fringe noise of a photographed image caused by the structure of the projection device and the photographing device;
A projected image recognition system characterized by that.

100 Image recognition device 602 CPU
603 ROM
604 RAM
605 External storage device 606 Communication interface 607 Input device 608 Display device 609 Portable recording medium driving device 610 Portable recording medium 611 Bus 612 Imaging device

Claims (19)

Photographing means for photographing a display image which is an icon indicating predetermined information;
In the photographed image obtained by photographing the display image by the photographing means, color fringe noise removing means for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
Image recognition means for recognizing the display image in a striped image obtained by removing the color fringe noise;
An image recognition apparatus comprising: The icon is black and white data.
The image recognition apparatus according to claim 1 . The display image is composed of the icon and an icon frame configured by a line having a direction different from the direction of the color fringe noise and arranged around the icon.
Alignment means for aligning the icon by detecting the icon frame from the striped image or the captured image,
The image recognition apparatus according to claim 1 , wherein the image recognition apparatus is an image recognition apparatus. The icon frame is a diamond frame,
The image recognition apparatus according to claim 3 . The icon frame is a circular frame.
The image recognition apparatus according to claim 3 . Photographing means for photographing a display image showing predetermined information;
In the photographed image obtained by photographing the display image by the photographing means, color fringe noise removing means for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
Image recognition means for recognizing the display image in a striped image obtained by removing the color fringe noise;
Equipped with a,
The display image is an image projected on a projection plane by a projection apparatus including a digital mirror device.
An image recognition apparatus characterized by that. Photographing means for photographing a display image showing predetermined information;
In the photographed image obtained by photographing the display image by the photographing means, color fringe noise removing means for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
Image recognition means for recognizing the display image in a striped image obtained by removing the color fringe noise;
Equipped with a,
The image recognition means compares the display image in the striped image with each registered image on an image database in which the same display image as the display image is registered, and recognizes the captured display image,
The image recognition means evaluates the similarity using an edge-based feature amount in collation between the display image in the captured image and each registered image in the image database,
The image recognition means calculates the edge-based feature amount from an image region obtained by applying a mask image created from the registered image registered in the image database to the fringe-removed image;
An image recognition apparatus characterized by that. Photographing means for photographing a display image showing predetermined information;
In the photographed image obtained by photographing the display image by the photographing means, color fringe noise removing means for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
Image recognition means for recognizing the display image in a striped image obtained by removing the color fringe noise;
Equipped with a,
The image recognition means compares the display image in the striped image with each registered image on an image database in which the same display image as the display image is registered, and recognizes the captured display image,
The image recognition means evaluates the similarity using an edge-based feature amount in collation between the display image in the captured image and each registered image in the image database,
The image recognition means calculates the edge-based feature amount from an image region obtained by applying a mask image created from the captured image or the fringe-removed image to the fringe-removed image.
An image recognition apparatus characterized by that. The image database is built in the image recognition device.
The image recognition apparatus according to claim 7 or 8 , wherein The image database is built in a server device different from the image recognition device,
The image recognition means executes a process of recognizing the display image by communicating with the image database in the server device.
The image recognition apparatus according to claim 7 or 8 , wherein The mask image is created from first data obtained by edge extraction after removing the color component of the photographed image and second data obtained by expanding the edge extracted from the hue component of the photographed image.
The image recognition apparatus according to claim 7 or 8 , wherein The edge-based feature value is a HOG feature value.
The image recognition apparatus according to claim 7 , wherein the image recognition apparatus is an image recognition apparatus. Photographing means for photographing a display image showing predetermined information;
In the photographed image obtained by photographing the display image by the photographing means, color fringe noise removing means for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
Image recognition means for recognizing the display image in a striped image obtained by removing the color fringe noise;
Equipped with a,
The color fringe noise removing means removes a color component of the captured image;
An image recognition apparatus characterized by that. The color fringe noise removing unit executes a process of setting the saturation value of the HSV color space to 0 for each pixel of the captured image, and generates the fringe removed image.
The image recognition apparatus according to claim 13 . The color fringe noise removing unit performs a process of replacing the R component value, the G component value, and the B component value with the maximum value among the component values for each pixel of the captured image, and Generate,
The image recognition apparatus according to claim 13 . Take a display image, which is an icon that shows certain information,
In the photographed image obtained by photographing the display image, color fringe noise caused by the display characteristics of the display image and the structure of the photographing means is removed,
Recognizing the display image in a striped image obtained by removing the color fringe noise;
An image recognition method characterized by the above. In the computer of the image recognition device ,
A shooting step of shooting a display image that is an icon indicating predetermined information;
In a photographed image obtained by photographing the display image in the photographing step, a color fringe noise removing step for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
An image recognition step for recognizing the display image in the striped image obtained by removing the color fringe noise;
An image recognition program for executing Take a display image showing the predetermined information,
In the photographed image obtained by photographing the display image, color fringe noise caused by the display characteristics of the display image and the structure of the photographing means is removed,
Recognizing the display image in the striped image obtained by removing the color fringe noise ;
The removal of the color fringe noise removes a color component of the captured image.
An image recognition method characterized by the above. In the computer of the image recognition device ,
A shooting step of shooting a display image showing predetermined information;
In a photographed image obtained by photographing the display image in the photographing step, a color fringe noise removing step for removing color fringe noise caused by the display characteristics of the display image and the structure of the photographing means;
An image recognition step for recognizing the display image in the striped image obtained by removing the color fringe noise;
Was executed,
The color fringe noise removing step removes a color component of the captured image;
Images recognition program.