JP2010238164A - Information correlation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image correlation device which enables to suitable execute of image retrieval paying attention to color information. <P>SOLUTION: With respect to an objective image input from an objective image input part 11, and a compared image stored in a comparison object database 12, an object chromaticity image and a comparison chromaticity image where chromaticity distribution within the images are histogrammed on a chromaticity diagram are prepared respectively. Using a retrieval image in which the object chromaticity image and the comparison chromaticity image are arranged side by side, an optical correlation operation device 30 performs optical correlation operation by a JTC method, and correlation data to which operation by a differential operation part 62 is added are acquired. An image retrieval processing part 15 performs image retrieval processing by determining the similarity of the images from the acquired correlation data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an information correlation apparatus that searches for information by paying attention to color information.

  In today's society where image technology and communication technology have been remarkably developed, fields that require strict color management, such as telemedicine / diagnosis, electronic commerce, and various designs incorporating colors, are growing. Further, in such a field, there is a method for obtaining information about a target image by creating a database of a plurality of comparison images and performing an image search by comparing the target image to be searched with each of the plurality of comparison images. It is being considered.

  As a processing method for comparing the target image with the comparative image and determining the similarity, etc., a joint transform correlation (JTC) method using a search image in which the target image and the comparative image are juxtaposed is known. ing. This is a method for determining similarity between images using optical calculation processing. In this method, it is possible to execute image comparison and image search using the image comparison at high speed by optical correlation calculation (see, for example, Patent Document 1). It is an object of the present invention to provide an image correlation device and an image search method that can suitably execute an image search focusing on color information.

JP 2008-234327 A

  However, since the invention described in Patent Document 1 is a combination of a device that electrically reads data with writing light and a device that reads data with writing light, there is a problem that the optical system becomes complicated and large. It was. In addition, autocorrelation data and cross-correlation data peculiar to the JTC method are generated. A relatively large autocorrelation data appears between two relatively small cross-correlation data, and this autocorrelation data becomes the cross-correlation data. There is a problem with the accuracy of the image, such as being difficult to read due to overlap, and analysis of the image is difficult. Therefore, there is a problem that it is difficult to determine the similarity of the shape and the like between the target image and the comparison image.

  The present invention has been made to solve the above-described problems. The information search apparatus is made compact, the accuracy of the cross-correlation data of the JTC method is improved, and the similarity of shapes and the like can be easily determined. The purpose is to do.

  In order to achieve such an object, the invention according to claim 1 includes a target information input unit that inputs target information to be searched for information, a comparison information database that stores a plurality of comparison information as a database, An information search processing unit that prepares target chromaticity information of the target information and comparative chromaticity information of the comparison information, and performs information search processing by determining similarity from their correlation data by JTC; An optical correlation calculation unit that performs optical correlation calculation between the target chromaticity information and the comparative chromaticity information and outputs the obtained correlation data, and the optical correlation calculation unit includes the target chromaticity Optical image data acquisition that Fourier transforms the target optical image based on the information to form a target Fourier transformed optical image for the target chromaticity information, and Fourier transforms the target Fourier transformed optical image to obtain optical image data Department and A search light image based on search information in which the target chromaticity information and the comparative chromaticity information are juxtaposed is output, and the search light image is subjected to Fourier transform to obtain the target chromaticity information and the comparative chromaticity information. Correlation data acquisition that forms a combined Fourier transform optical image of the image, and Fourier transforms the combined Fourier transform optical image to acquire correlation basic data indicating a correlation between the target chromaticity information and the comparative chromaticity information A difference calculation unit that creates correlation data by taking a difference between the optical image data acquired by the optical image data acquisition unit and the correlation basic data acquired by the correlation data acquisition unit. An information correlator characterized by the above.

  The comparison preprocessing unit is provided, and the target information or the comparison information is histogrammed on a chromaticity diagram of the chromaticity distribution, and the histogram is binarized, and the corresponding target chromaticity information or It is preferable to generate the comparative chromaticity information.

  Preferably, the correlation data acquisition means acquires correlation information including an autocorrelation peak and a cross-correlation peak for the target chromaticity information and the comparative chromaticity information as the correlation data.

  The optical correlation calculation unit includes a spatial light modulator, one Fourier transform lens that receives light from the spatial light modulator and performs Fourier transform, a sensor that detects light from the Fourier transform lens, and the sensor A correlation data acquisition device for receiving the signal, and in the optical image data acquisition unit and the correlation data acquisition unit, the optical image first Fourier transformed is stored in the memory of the correlation data acquisition device, and the optical image is stored in the optical image. A corresponding electrical signal is fed back to the spatial light modulator, a Fourier transform is performed based on the electrical signal, and an optical image generated thereby is stored in a memory of the correlation data acquisition device. preferable.

  Examples of the “information” include general digital data such as binary patterns of 0 and 1, and image data. Examples of the image data include a chromaticity histogram, texture, and shape data.

  As for binarization of the chromaticity histogram, it is preferable to make a determination only by the density of the chromaticity distribution or only by the contour.

  The calculation of the feature amount of the texture is preferably performed using a density co-occurrence matrix.

  The shape data is preferably contour data defined by data contour extraction.

  In the specific aspect of the information correlator described above, the chromaticity distribution in the information is histogrammed and binarized on the chromaticity diagram for each of the target information of the information search target and the comparison information of the comparison / search target. The target chromaticity information and the comparative chromaticity information, which are the chromaticity value histogram information, are prepared. Then, for these target chromaticity information and comparative chromaticity information, an optical correlation calculation using search information in which the target chromaticity information and the comparative chromaticity information are juxtaposed is performed, and the difference calculation described above is performed. The similarity of information is judged from the correlation data obtained by the above. According to such a configuration, it is possible to execute an information search focusing on color information with sufficient accuracy and at high speed.

  Here, the information correlator performs histogramming and binarization on the chromaticity diagram of the chromaticity distribution for the target information or the comparison information, and generates the corresponding target chromaticity information or comparative chromaticity information. It is preferable to include a calculation unit.

  Thus, by generating the chromaticity information from the original information, it is possible to suitably execute the information search using the target chromaticity information and the comparative chromaticity information. As the chromaticity value histogram image used as the chromaticity image, specifically, for example, a two-dimensional image in which the chromaticity distribution on the chromaticity diagram is represented by an outline on the image can be used. Specifically, a two-dimensional image representing contour information by binarizing the chromaticity distribution on the chromaticity diagram with the threshold value of the intensity distribution (shading) on the image is preferable. The target image may be configured to directly input the target chromaticity image instead of the target image. Similarly, with respect to the comparative image, a comparative chromaticity image may be directly stored in a database instead of the comparative image.

  In the information correlation apparatus, it is preferable that the correlation data acquisition unit acquires correlation information including auto-correlation peaks and cross-correlation peaks for target chromaticity information and comparative chromaticity information as correlation data. Similarly, in the correlation data acquisition step, the information search method preferably acquires correlation information including autocorrelation peaks and cross-correlation peaks for target chromaticity information and comparative chromaticity information as correlation data.

  In addition, regarding the specific configuration of the optical correlation calculation means, the information correlation apparatus performs the second Fourier transform on the basis of the electrical signal by a feedback method, and thus the optical system can be miniaturized. The spatial light modulator is preferably a luminance modulation type.

  According to the information correlation apparatus of the present invention, the correlation data is created by taking the difference between the optical image data acquired by the optical image data acquisition unit and the basic correlation data acquired by the correlation data acquisition unit. Since the difference calculation unit is provided, it is possible to execute with high accuracy and high speed. Moreover, the analysis of the image becomes easy. Furthermore, the calculation speed can be increased by binarizing data in the pre-comparison processing unit.

It is a block diagram which shows roughly the structure of Embodiment 1 of an image correlation apparatus. It is a figure which shows xy chromaticity diagram used for a chromaticity image. It is a figure which shows typically an example of target image (a) and the target chromaticity image (b) corresponding to it. (A)-(c) is a figure which shows typically about the image search using the chromaticity image P1. (A)-(c) is a figure which shows typically about the image search using chromaticity image P1, P2. It is an image which shows the result of JTC by Embodiment 1 of an image correlation apparatus. It is a flowchart which shows the process by the structure of Embodiment 1-3 of an image correlation apparatus. (A) (b) is explanatory drawing which shows the mode of the binarization in this Embodiment 1-3. It is explanatory drawing which shows the calculation of the texture feature-value in this Embodiment 1-3. It is explanatory drawing which shows the calculation of the texture feature-value extraction engine. It is a block diagram which shows roughly the structure of Embodiment 2 of an image correlation apparatus. It is an image which shows the result of JTC by Embodiment 2 of an image correlation apparatus. It is a block diagram which shows roughly the structure of Embodiment 3 of an image correlation apparatus.

  Hereinafter, a preferred embodiment of an information correlation apparatus according to the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram schematically showing a configuration of a first embodiment of an image correlation apparatus 1 as an information correlation apparatus according to the present invention. The image correlation device 1 according to the present embodiment includes an image search control device 10 and an optical correlation calculation device 30, and performs image search focusing on color information between a target image and a comparison image. It is configured to be able to do.

  The image search control device 10 includes a target image input unit 11, a comparison target database 12, a chromaticity image generation unit 13, a search area setting unit 14, an image search processing unit 15, a pre-comparison processing generation unit 16, It is comprised. The target image input unit 11 is a target image input unit that inputs a target image to be subjected to image search or a target chromaticity image corresponding to the target image. The comparison target database 12 stores a plurality of comparison images to be compared with the target image or a plurality of comparison chromaticity images corresponding thereto as a database.

  A chromaticity image generation unit 13 is provided for the target image input unit 11 and the comparison target database 12. The chromaticity image generation unit 13 performs predetermined data processing on the target image input from the target image input unit 11 or the comparison image read from the comparison target database 12, and the corresponding target chromaticity image, or A comparative chromaticity image is generated.

  The target chromaticity image used for the image search is a color obtained by forming a histogram of the chromaticity distribution (distribution of chromaticity values) in the original target image on the chromaticity diagram (xy chromaticity diagram) shown in FIG. It is a degree value histogram image. That is, the chromaticity image is an image obtained by mapping x, y chromaticity values on chromaticity coordinates and stacking the counts to form a histogram. Here, the x and y chromaticity values in the chromaticity diagram are obtained from the tristimulus values X, Y, and Z in the CIE-XYZ color system by x = X / (X + Y + Z), y = Y / (X + Y + Z). be able to.

  Further, in the xy chromaticity diagram of FIG. 2, a region inside the horseshoe-shaped curve indicated by a solid line is a human visible region. In the chromaticity image shown below, a horseshoe-shaped curve indicating the visible region is indicated by a broken line for reference. However, in a chromaticity image actually used for image search, such a curve is indicated in the image. It is not necessary. In addition, the chromaticity diagram used for the chromaticity image in the present invention is not limited to the above xy chromaticity diagram, but other chromaticity such as a UCS chromaticity diagram and a chromaticity diagram in the CIE-LAB color system. Similar processing is possible using a chromaticity diagram in space.

  Specifically, for such a chromaticity value histogram image that is a chromaticity image, for example, the chromaticity distribution on the chromaticity diagram is binarized with the intensity distribution (shading) on the image as a threshold value. A two-dimensional image representing contour information can be used. Further, a two-dimensional image showing the chromaticity distribution by other methods may be used as the chromaticity image. FIG. 3 is a diagram schematically illustrating an example of the target image (a) and the corresponding target chromaticity image (b). Here, a flower image is shown as an example of the target image (a). Further, in the target chromaticity image (b), a region where the chromaticity value data is distributed with a certain degree of intensity is schematically shown by hatching. As described above, the target chromaticity image is an intensity distribution image reflecting the color information of the target image, that is, the distribution amount of each color component in the image.

  As the target chromaticity image, when the target chromaticity image is input by the target image input unit 11, it is used as it is for the image search. When the target image that is the original image is input in the target image input unit 11, the chromaticity image generation unit 13 forms a histogram on the chromaticity diagram of the chromaticity distribution for the target image. A target chromaticity image to be generated is generated.

  Further, the comparison chromaticity image to be compared with the target chromaticity image is a chromaticity value histogram image in which the chromaticity distribution in the comparison image that is the original image is histogrammed on the chromaticity diagram in the same manner as the target chromaticity image. It is. As a comparative chromaticity image, when a comparative chromaticity image is stored in the comparison target database 12, it is read as it is and used for image retrieval. When the comparison image which is the original image is stored in the database 12, the chromaticity image generation unit 13 forms a histogram on the chromaticity diagram of the chromaticity distribution for the comparison image, and the corresponding comparison is performed. A chromaticity image is generated.

  The image search processing unit 15 prepares a target chromaticity image and a comparative chromaticity image, and performs image search processing by determining similarity from their correlation data by JTC. Specifically, the image search processing unit 15 includes the image data of the target image or the target chromaticity image input from the target image input unit 11 and the comparison image or the comparison chromaticity image read from the comparison target database 12. Based on the image data, the chromaticity image generation unit 13 performs chromaticity image generation processing as necessary, and prepares a target chromaticity image and a comparative chromaticity image used for image search. Then, correlation data between the target chromaticity image and the comparative chromaticity image is acquired using the optical correlation calculation device 30, and image search processing is performed by determining the similarity of the images from the obtained correlation data.

  The search area setting unit 14 is for setting a search area in advance when a method of applying a search area to a correlation image that is correlation data is used in the image search executed by the image search processing unit 15. . This search area and the like will be described later together with specific image search processing.

  The comparison preprocessing generation unit 16 binarizes an image obtained by histogram-izing the target image or the comparison image on the chromaticity diagram of the chromaticity distribution, and generates a corresponding target chromaticity image or comparative chromaticity image. To do.

  In the image search control device 10, an image output unit 18 and a correlation data input unit 19 are provided for the image search processing unit 15. In response to an instruction from the image search processing unit 15, the image output unit 18 generates a search image that is a target chromaticity image in the first stage and an image in which the target chromaticity image and the comparative chromaticity image are juxtaposed in the second stage. And output to the optical correlation calculation device 30. Further, the correlation data input unit 19 inputs correlation data between the target chromaticity image and the comparative chromaticity image acquired by the optical correlation calculation device 30 and sends them to the image search processing unit 15.

  Specifically, the image search control device 10 is used for, for example, a CPU that executes various processes such as an image search process, a ROM that stores software programs necessary for processing operations, and the comparison target database 12. And a storage device such as an internal memory or an external storage device. In the configuration illustrated in FIG. 1, an input device 21, a display device 22, and an external I / F 23 are connected to the image search control device 10.

  The input device 21 is input means used for inputting information and instructions necessary for processing operations such as image search processing in the image correlation device 1. Examples of such an input device 21 include a keyboard, a mouse, and various buttons and switches. The display device 22 is configured by, for example, a liquid crystal display or the like, and is used for displaying data and information necessary for the image search process and displaying the obtained image search result as necessary.

  The external I / F 23 is used when an external device such as another storage device or processing device is connected to the image search control device 10. The input device 21, display device 22, and external I / F 23 may be provided as necessary. For the input of the target image or the target chromaticity image by the target image input unit 11, for example, a configuration in which image data is input via the input device 21 or the external I / F 23 in the above configuration can be used.

  The optical correlation calculation device 30 receives light from a laser diode 31 (for example, a wavelength of 532 nm), a pinhole portion 32, a collimating lens 33, a transmissive spatial light modulator 34, and the spatial light modulator 34. One Fourier transform lens 35 that performs Fourier transform, a surface sensor 36 that detects an image based on light from the Fourier transform lens 35, and a correlation data acquisition device 37 are configured. The correlation data acquisition device 37 includes an optical image data acquisition unit 60 that acquires a target image, a correlation data acquisition unit 61 that acquires a correlation basic image, and a difference that creates a correlation image by taking the difference between the target image and the correlation basic image. A computing unit 62; a first memory 63a for storing the target image; a second memory 63b for storing the correlation basic image; a video card 64 for transmitting the first electric signal and the second electric signal to the spatial light modulator 34; And an image capturing card 65 composed of a CCD or CMOS sensor.

  The laser diode 31 outputs laser light. The pinhole part 32 uses laser light as a point light source. The collimating lens 33 is a parallel light beam having a uniform wavefront phase. The spatial light modulator 34 inputs a target image and a search image in which the target chromaticity image and the comparative chromaticity image are juxtaposed from the image output unit 18, performs spatial light modulation based on these images, This is an optical image generation device that generates a target optical image and a search optical image and outputs them to a Fourier transform lens 35. The Fourier transform lens 35 optically Fourier transforms the target light image and the search light image to form a target Fourier transform light image for the target chromaticity image, and the target chromaticity image and the comparison chromaticity image, respectively. And a combined Fourier-transform optical image for. The second target light image is again subjected to Fourier transform to form a difference target light image, and the second search light image is subjected to Fourier transform to form a correlated light image. Details will be described later. The surface sensor 36 is composed of a CCD or CMOS, and detects a Fourier-transformed optical image as image data. The image capturing card 65 captures an image detected by the surface sensor 36. The laser diode 31 is a reading light source such as a laser light source, and can constitute a light guide optical system that guides the reading light supplied from the reading light source to the collimating lens 33 through the pinhole portion 32. Thus, by configuring the optical correlation calculation device 30 using the spatial light modulator 34, the optical correlation calculation for the target chromaticity image and the comparative chromaticity image can be suitably executed.

  Further, the correlation data acquisition device 37 performs optical correlation calculation on the target light image to acquire a target image for difference, and performs optical correlation calculation between the target chromaticity image and the comparison chromaticity image to perform correlation. A basic image is acquired, these difference calculations are performed, and an output value is output to the correlation data input unit 19 of the image search control device 10.

  The optical image data acquisition unit 60 generates a difference target image from an image based on a target light image based on a target chromaticity image obtained from the surface sensor 36 via the image capture card 65.

  The correlation data acquisition unit 61 acquires a correlation basic image based on a search image obtained by juxtaposing the target chromaticity image and the comparative chromaticity image, obtained from the surface sensor 36 via the image capturing card 65. The correlation data acquisition unit 61 acquires a correlation basic image including autocorrelation peaks and cross-correlation peaks for the target chromaticity image and the comparative chromaticity image as correlation data.

  In the optical correlation calculation device 30, the spatial light modulator 34 generates a target light image from the target chromaticity image, and the Fourier transform lens 35 first Fourier-transforms the target light image to generate a target Fourier transform light image. The image data is received by the surface sensor 36, converted into a target Fourier transform image, passed through an image capture card 65, and the target Fourier transform image is stored in the first memory 63a in the optical image data acquisition unit 60. A target Fourier transform image is read from the memory 63a, a first electrical signal is created from this image, and fed back to the spatial light modulator 34. Based on the first electrical signal, the spatial light modulator 34 receives the second target light image. , The second Fourier transform is performed by the Fourier transform lens 35, the target light image generated thereby is received by the surface sensor 36, and a difference target image is obtained. Stored in the first memory 63a and transmits the image data acquisition unit 60, and transmits the difference calculation section 62.

  The optical correlation calculation device 30 generates a search light image from the search image by the spatial light modulator 34, and a combined Fourier transform light image in which the search image is first Fourier transformed by the Fourier transform lens 35 is generated. 36, converted into an image of a combined Fourier transform optical image, and through the image capture card 65, the correlation data acquisition unit 61 stores this combined Fourier transform image in the second memory 63b, and the video card 64 stores the second image. An image of the combined Fourier transform optical image is read from the memory 63b, a second electrical signal is created from this image, and fed back to the spatial light modulator 34. Based on the second electrical signal, the spatial light modulator 34 The image is modulated into a search light image, a second Fourier transform is performed by the Fourier transform lens 35, and the correlated light image generated thereby is received by the surface sensor 36, and the image capture function is received. Via de 65, and transmits to the correlation data acquisition unit 61 as the correlation image, stored in the second memory 63 b, is intended to be sent to the difference calculation section 62.

  The difference calculation unit 62 creates a correlation image by taking the difference between the difference target image acquired by the optical image data acquisition unit 60 and the correlation image acquired by the correlation data acquisition unit 61.

  In the image correlation apparatus 1 shown in FIG. 1, for each of the target image to be searched and the comparative image to be compared (see the image (a) in FIG. 3), the image search processing unit 15 selects the color in the image. A target chromaticity image and a comparative chromaticity image, which are chromaticity value histogram images obtained by histogramating the chromaticity distribution on a chromaticity diagram, are prepared (see image (b) in FIG. 3). Then, with respect to the target chromaticity image and the comparative chromaticity image, the optical correlation calculation device 30 performs the optical correlation calculation described above, and determines the similarity of the images from the obtained correlation data.

  In this way, the original image is not used as it is, but the binarized image generated by the comparison preprocessing generation unit 16 is compared as necessary using the corresponding chromaticity value histogram image, and the light according to the JTC method is used. According to the configuration in which the correlation is used to measure the degree of correlation of the chromaticity image and the image search is performed, the image search focusing on the color information can be executed with sufficient accuracy and at high speed by the light calculation. It becomes.

  As an example of the application of such an image search, there is a search for a flower image shown in FIG. For example, the search target flower image input from the target image input unit 11 is compared with each of the national flower images registered in the comparison target database 12 using the corresponding chromaticity image. Then, an image search method of selecting a production area determined to have the highest similarity among the production areas registered in the database 12 can be considered.

  Alternatively, another example of application of image search is search for human skin images. Such retrieval of human skin images is considered to be effective in fields such as diagnosis of skin diseases and advice on cosmetics. Further, if information such as diagnosis contents and cosmetic numbers is added to the human skin image registered in the comparison target database 12, the additional information can be used effectively based on the result of the image search. It becomes.

  Here, in the image correlation apparatus 1 shown in FIG. 1, the image search control apparatus 10 forms a histogram on the chromaticity diagram of the chromaticity distribution for the target image or the comparison image, and the target chromaticity image or the comparison color. A chromaticity image generation unit 13 for generating a chromaticity image is provided. As described above, the image correlation apparatus 1 is configured to be able to generate a chromaticity image from the original image, so that the image search using the target chromaticity image and the comparative chromaticity image is preferably executed. It becomes possible to do.

  As described above, the target image input unit 11 may directly input the target chromaticity image instead of the target image, as described above. Similarly, with respect to the comparison image, the comparison chromaticity image may be directly stored in the comparison target database 12 instead of the comparison image. In such a case, the chromaticity image generation unit 13 may be omitted if unnecessary.

  In addition, the correlation data acquisition device 37 used in the optical correlation calculation device 30 uses, as correlation data between the target chromaticity image and the comparative chromaticity image, an autocorrelation peak for the target chromaticity image and the comparative chromaticity image, A correlation basic image including a correlation peak is acquired and a difference calculation is performed. In this way, a correlation image including information on the correlation between the target chromaticity image and the comparative chromaticity image is acquired, and the image search processing unit 15 determines the similarity of the images using the correlation image, and By performing the image search process based on the image search, it is possible to accurately execute the image search using the chromaticity image. As such a correlation image acquisition device, for example, an image sensor such as a CCD can be used.

  When the correlation image is acquired in the optical correlation calculation device 30 as described above, the image search processing unit 15 applies a search area set in advance corresponding to the position of the cross-correlation peak to the correlation image. Then, it is preferable to determine the similarity between the target chromaticity image and the comparative chromaticity image with reference to the correlation strength acquired in the search region. In this way, by applying a predetermined search region to the correlation image acquired by the optical correlation calculation and acquiring information about the correlation degree between the target chromaticity image and the comparative chromaticity image, the chromaticity image The image search using the correlation data can be executed reliably.

  For such a search area, in the configuration shown in FIG. 1, the image search control apparatus 10 includes a search area setting unit 14 for setting a search area. As a setting method of the search area in this case, for example, the optical correlation calculation device 30 performs optical correlation calculation between target chromaticity images in advance, and sets the search area with reference to the obtained correlation image. A method is mentioned. According to such a method, it is possible to set the search area for the correlation image so as to reliably correspond to the position of the cross-correlation peak.

  A processing flow of the image correlation apparatus 1 according to the first embodiment will be described with reference to FIGS.

  The image search method using the image correlation apparatus 1 shown in FIG. 1 will be described more specifically with reference to FIGS. In the following, a case where not the original image but the target chromaticity image and the comparative chromaticity image are prepared as the images used for the image search will be described, and generation of the chromaticity image from the original image (FIG. 3). Description of reference) is omitted.

  In the image search method of the present embodiment, first, in the image correlation device 1 in FIG. 1, the target chromaticity image P <b> 1 that is the target of the image search is input from the target image input unit 11 of the image search control device 10. In the target chromaticity image P1, as schematically shown in the chromaticity image (a) of FIG. 4, one or a plurality of data areas A1 in which chromaticity value data is distributed with a certain intensity exist. .

  For the target chromaticity image P1, the search area setting unit 14 performs a process for setting a search area for the correlation image. Specifically, as shown in a search image (b) in FIG. 4, an image in which two identical target chromaticity images P1 are juxtaposed is generated as a search image for setting a search area. Here, the width of the target chromaticity image P1 is 2d, and the image P1a obtained by moving the target chromaticity image P1 by + d and the image P1b obtained by moving by −d are juxtaposed to form a search image. Further, these images P1a and P1b include areas A1a and A1b corresponding to the data area A1 in the target chromaticity image P1, respectively.

Here, a correlation image obtained by optical correlation calculation using the JTC method will be briefly described. Two images juxtaposed in the search image as comparison targets are set as a (x, y) and b (x, y), and images obtained by juxtaposing these images by + d and −d are arranged in the search image f (x, y). y). x and y are coordinates. This search image f (x, y) is expressed by the following equation (1).
Formula 1: f (x, y) = a (x−d, y) + b (x + d, y)
Represented by If a function obtained by Fourier transform of the image f (x, y) is F (Vx, Vy), this F (k) is expressed by the following equation (2).
Sought by.

For this function F (Vx, Vy), | F (Vx, Vy) | ² is obtained by using a complex conjugate function, and further converted into a power spectrum P in the real space. (3) is obtained.
Here, in the expression (3), the superscript ★ indicates a conjugate complex number.

  In the right side of the equation (3), the first term and the second term correspond to the autocorrelation peak B1 in the correlation image shown in FIG. The third term and the fourth term correspond to the cross-correlation peaks F1a and F1b that are separated by ± 2d from the autocorrelation peak, respectively.

Equation (4) shows the Fourier transform of the power spectrum P. In the formula, * indicates convolution and superscript ★ indicates a conjugate complex number. Therefore, the third term and the fourth term of the equation (4) are correlation data of a and b.

  Here, when the optical correlation calculation is performed between the target chromaticity images as described above, since A (Vx, Vy) = B (Vx, Vy), two cross-correlation peaks F1a , F1b peak intensities (correlation intensities) are equal to each other. In addition, by referring to the correlation image between the target chromaticity images and setting a search area based on the two cross-correlation peaks F1a and F1b appearing in the correlation image, an image search process for the target chromaticity image Can be executed with high accuracy.

  For a specific method of setting the search area with reference to the correlation image for the target chromaticity images, for example, the operator configures the input device 21 with reference to the correlation image displayed on the display device 22. A method of manually setting a search area by operating a mouse or the like can be used. Alternatively, a method may be used in which a threshold for peak identification or the like is set in advance for the correlation image, and the search area is automatically set based on the setting conditions. As the shape of the search area, for example, a rectangular area, a circular area, or the like can be used.

  Next, an image search for the target chromaticity image P1 is executed using the search area set as described above. The comparison chromaticity image P2 to be compared is selected and read from the plurality of comparison chromaticity images stored in the comparison target database 12, and the target chromaticity image P1 and the comparison chromaticity image P2 are prepared. Then, as shown in the search image (a) of FIG. 5, a search image in which the target chromaticity image P1 and the comparative chromaticity image P2 are juxtaposed is generated. In the example of the image (a), the target chromaticity image P1 is arranged on the left side of the search image, and the comparative chromaticity image P2 is arranged on the right side. Further, in these images P1 and P2, the positions, ranges, and the like of the data areas A1 and A2 where the chromaticity value data are distributed with a certain intensity are different from each other.

  Next, binarization is performed by the comparison preprocessing generation unit 16. Specific examples are shown below.

  As shown in FIG. 8A, the chromaticity histogram is binarized. Here, when N% is lowered from the vertex, the chromaticity histogram is sliced horizontally, a threshold value is taken, and this is converted into a two-dimensional data pattern. Further, as shown in FIG. 8B, binarization may be performed by extracting the contour of the shape data. By performing binarization, similarity of color distribution can be detected by comparing binarized patterns. Thereby, a sufficient processing speed can be obtained even with an ON / OFF control type or liquid crystal type spatial light modulator.

  Further, as shown in FIG. 9, a case will be described in which image data is based on a space pattern (texture), that is, a texture feature amount. A concentration co-occurrence matrix (method based on evaluation of a two-dimensional joint probability density function) will be described. This two-dimensional combined probability density function is a probability density function indicating that a pixel that is a distance d in the θ direction from a pixel having a density value i may have a density value j. The density co-occurrence matrix has a large value near the diagonal pixel when the texture is coarse, and is relatively uniformly distributed when the texture is fine.

  As shown in FIG. 10, the texture feature quantity extraction engine grayscales and normalizes the source image, generates a density co-occurrence matrix of θ = 0, generates a density co-occurrence matrix of θ = 45 °, and θ = 90 A density co-occurrence matrix of 0 ° is generated, a density co-occurrence matrix of θ = 135 ° is generated, and loop processing thereof is performed. This processing is performed by the pre-comparison processing generator 16 shown in FIGS.

  Next, a first stage process is performed on the binarized target light image generated by the comparison preprocessing generation unit 16. That is, the target light image is output from the image output unit 18 to the spatial light modulator 34 of the optical correlation calculation device 30. The readout light from the pinhole part 32 is input from the laser diode 31 to the readout part 34 b of the spatial light modulator 34 via the collimator lens 33. Then, the target light image is output from the readout unit 34b by the readout light. This target light image is optical Fourier transformed by the Fourier transform lens 35 to become a target Fourier transform light image, and the target Fourier transform image received by the surface sensor 36 is detected. This is transmitted to the optical image data acquisition unit 60 via the image capturing card 65, the target Fourier transform image is stored in the first memory 63a, and the first electric signal corresponding to the target Fourier transform image is spatially modulated from the video card 64. The data is transmitted to the writing unit 33a of the device 34. Then, the second target light image is read from the reading unit 34 b of the spatial light modulator 34, optical Fourier transformed by the Fourier transform unit lens 34 to become a difference target light image, and the difference target image received by the surface sensor 36 is obtained. read out. This is transmitted to the optical image data acquisition unit 60 via the image capturing card 65. The obtained optical image for difference is transmitted to the difference calculation unit 62 through the first memory 63a.

  Next, the second stage of the search light image generated by the comparison preprocessing generation unit 16 and binarized will be described. The search image is output from the image output unit 18 to the spatial light modulator 34 of the optical correlation calculation device 30. The readout light from the pinhole part 32 is input from the laser diode 31 to the readout part 34 b of the spatial light modulator 34 via the collimator lens 33. Then, an optical image generated by the readout light corresponding to the search optical image is output from the readout unit 34b. This search light image is optical Fourier transformed by the Fourier transform lens 35 to generate a combined Fourier transform light image. A combined Fourier transform image received by the surface sensor 36 is detected. This is transmitted to the correlation data acquisition unit 61 via the image capture card 65, the combined Fourier transform image is stored in the second memory 63b, and the second electrical signal corresponding to the combined Fourier transform optical image is spatially modulated from the video card 64. The data is transmitted to the writing unit 33a of the device 34. Then, the second search light image is read from the reading unit 34 b of the spatial light modulator 34, optical Fourier transformed by the Fourier transform unit lens 34 to become a correlated light image, and a correlation image that has been Fourier transformed by the surface sensor 36 is detected. . This is transmitted to the correlation data acquisition unit 61 via the image capture card 65. The obtained correlation basic image is transmitted to the difference calculation unit 62.

  FIG. 5B includes information about the difference target image formed by the joint transform correlation method (JTC method) and the degree of correlation between the target target chromaticity image and the comparative chromaticity image. The correlation basic image (b) shown is acquired.

  The correlation basic image (b) includes an autocorrelation peak B1 and two cross-correlation peaks F1a and F1b that are 2d apart from the peak B1 in the ± direction.

  The difference calculation unit 62 calculates the difference between the difference target image and the correlation basic image (b). As a result, the correlation image (c) in which the autocorrelation peak of the difference target image and the autocorrelation peak B1 of the correlation basic image (b) are canceled and the cross correlation peaks F1a and F1b are left is obtained. Then, the correlation image after the difference is transmitted to the correlation data input unit 19. Note that the autocorrelation peak B1 may be canceled and the remaining portion may appear somewhat, but there is no problem in processing.

  The search region setting unit 14 refers to the correlation image (c) and sets a search region based on the positions and ranges of the cross-correlation peaks F1a and F1b.

  As described above, the retrieval image is output from the image output unit 18 to the optical correlation calculation device 30, and optical correlation calculation is performed between the target chromaticity image and the comparative chromaticity image juxtaposed in the optical correlation calculation device 30. To obtain a correlation image (c) in which the cross-correlation peaks F1a and F1b are left as shown in FIG. The image search processing unit 15 performs image search processing by determining image similarity from the correlation image (c) thus obtained.

  Specifically, the autocorrelation peak B2 does not appear or hardly appears in the correlation image of FIG. 5C or FIG. In addition, two cross-correlation peaks F2a and F2b that are 2d apart from the autocorrelation peak B2 in the ± direction appear at an intensity corresponding to the degree of correlation between the target chromaticity image and the comparative chromaticity image. On the other hand, rectangular search areas Ra and Rb set based on the correlation image are applied. Then, the image search processing unit 15 determines the similarity between the target chromaticity image and the comparative chromaticity image with reference to the correlation strength acquired in the search regions Ra and Rb. If the first stage processing is performed in advance, the autocorrelation data is considered not to change greatly in most cases, such as when the image sizes are almost the same, so the second stage of other types of target images. It can also be used for processing, eliminating the need to repeat the first stage processing and simplifying the processing.

  Such image comparison processing between the target chromaticity image and the comparative chromaticity image and similarity determination processing are sequentially executed for each of the plurality of comparative chromaticity images stored in the comparison target database 12. Accordingly, for example, an image search such as extracting an image having the highest similarity with respect to the target image and the color information from among the plurality of comparison images can be performed. Further, according to the image correlation device 1, a correlation image is created by taking the difference between the difference target image acquired by the optical image data acquisition unit 60 and the correlation basic image acquired by the correlation data acquisition unit 61. Since the difference calculation unit 62 is provided, it can be executed with sufficient accuracy and at high speed. Moreover, the analysis of the image becomes easy. Further, by performing binarization of data by the comparison preprocessing unit 16, the calculation speed can be increased. Furthermore, since JTC is used, the optical system can be miniaturized. For example, since the pixel pitch of the photodetector is fine (about 1.5 μm), the f value of the lens can be suppressed. However, it is necessary to increase the laser output to some extent. In the matched filter, the f value of the lens is high and the size is increased. Optical adjustment is easy. The lens f value is small.

  Here, in the configuration in which the search region is applied to the correlation image as described above and the image search is performed with reference to the correlation strength acquired in the search region, the correlation strength acquired from the correlation image is the target color. This is a quantitative index of the similarity between the degree image and the comparative chromaticity image. That is, in the above method, when the chromaticity distributions of the target chromaticity image and the comparative chromaticity image are somewhat different, a cross-correlation peak appears, but the intensity is smaller than that of the same chromaticity distribution. Therefore, by referring to the value of the correlation strength acquired from the correlation image, for example, it is possible to acquire various information using image search such as arranging a plurality of comparison images in descending order of similarity with the target image. Become.

  The image correlation apparatus 110 according to the second embodiment illustrated in FIG. 11 will be described. In addition, since operation | movement is the same as that of Embodiment 1, a part number is set to 100s, description is used and a difference is demonstrated. The image correlator 110 is different in that a reflective half mirror 139 is provided. FIG. 12 shows a processing result by the image correlation device 110. According to the image correlator 110, the same effects as the image correlator 1 can be obtained.

  The image correlation apparatus 210 of Embodiment 3 shown in FIG. 13 will be described. In addition, since operation | movement is the same as that of Embodiment 1, a part number is set to 200 series, description is used and a difference is demonstrated. This image correlator 210 is a tandem image correlator 210, and is different in that the spatial light modulator 244, the Fourier transform lens 245, and the surface sensor 246 are again connected in a column, and the card 64 is unnecessary. ing. According to the image correlator 210, the same effects as the image correlator 1 can be obtained. However, compared with the first and second embodiments, the apparatus is larger, but there is an effect that the processing speed is increased. The spatial light modulators 234 and 244 are preferably magneto-optical effect type. In addition, by unitizing the unit tandem configuration of the tandem connection, a plurality (two in this case) of these can be connected.

  The image correlation apparatus according to the present invention is not limited to the above-described embodiments and examples, and various modifications are possible. For example, in the above-described embodiment, an example in which an image acquisition device such as a CCD is used as the correlation data acquisition device 37 of the optical correlation calculation device 30 has been described. However, for example, the cross correlation peak is arranged to be detected selectively. You may use the photodetector of other structures, such as a photodetector. The spatial light modulator can be implemented in various modes such as a liquid crystal type, a magneto-optical effect type, and an optical address type PAL-SLM.

  INDUSTRIAL APPLICABILITY The present invention can be used as an image correlation device that can suitably execute an image search focusing on color information.

DESCRIPTION OF SYMBOLS 1 ... Image correlation apparatus 10 ... Image search control apparatus 30 ... Optical correlation calculation apparatus 11 ... Target image input part 12 ... Comparison object database 13 ... Chromaticity image generation part 14 ... Search area setting part 15 ... Image search process part 16 ... Comparison Pre-processing generation unit 18 ... image output unit 19 ... correlation data input unit 21 ... input device 22 ... display device 23 ... external I / F 31 ... laser diode 32 ... pinhole 33 ... collimator lens 34 ... spatial light modulator 34a ... Write unit 34b ... Read unit 35 ... Fourier transform lens 36 ... Surface sensor 37 ... Correlation data acquisition device 60 Optical image data acquisition unit 61 ... Correlation data acquisition unit 62 ... Difference calculation unit 63a ... First memory 63b ... Second memory 64 ... Video card 65 ... Image capture card

Claims (4)

A target information input unit for inputting target information to be searched for information;
A comparison information database that stores a plurality of comparison information in a database, and
An information search processing unit that prepares target chromaticity information of the target information and comparative chromaticity information of the comparison information, and performs information search processing by judging similarity from their correlation data by JTC;
An optical correlation calculation unit that performs optical correlation calculation between the target chromaticity information and the comparative chromaticity information and outputs the obtained correlation data;
The optical correlation calculation unit includes:
A target optical image based on the target chromaticity information is Fourier transformed to form a target Fourier transformed optical image for the target chromaticity information, and the target Fourier transformed optical image is Fourier transformed to obtain optical image data. An optical image data acquisition unit;
A search light image based on search information in which the target chromaticity information and the comparative chromaticity information are juxtaposed is output, and the search light image is Fourier transformed to obtain the target chromaticity information and the comparative chromaticity information. Correlation data acquisition that forms a combined Fourier transform optical image of the image, and Fourier transforms the combined Fourier transform optical image to acquire correlation basic data indicating a correlation between the target chromaticity information and the comparative chromaticity information And
A difference calculation unit that creates correlation data by taking a difference between the optical image data acquired by the optical image data acquisition unit and the correlation basic data acquired by the correlation data acquisition unit;
An information correlation apparatus comprising:   The comparison preprocessing unit is provided, and the target information or the comparison information is histogrammed on a chromaticity diagram of a chromaticity distribution, and the histogram is binarized, and the corresponding target chromaticity information or The information correlation apparatus according to claim 1, wherein the comparison chromaticity information is generated.   The information according to claim 1, wherein the correlation data acquisition unit acquires correlation information including an autocorrelation peak and a cross-correlation peak for the target chromaticity information and the comparative chromaticity information as the correlation data. Correlator.   The optical correlation calculation unit includes a spatial light modulator, one Fourier transform lens that receives light from the spatial light modulator and performs Fourier transform, a sensor that detects light from the Fourier transform lens, and the sensor A correlation data acquisition device for receiving the signal, and in the optical image data acquisition unit and the correlation data acquisition unit, the optical image first Fourier transformed is stored in the memory of the correlation data acquisition device, and the optical image is stored in the optical image. A corresponding electrical signal is fed back to the spatial light modulator, a second Fourier transform is performed based on the electrical signal, and an optical image generated thereby is stored in a memory of the correlation data acquisition device. The information correlation apparatus according to claim 1.