JP2003308515A - Feature extraction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a feature extraction device used for accumulating an optical characteristic of a sample as one-dimensional data in a wide wavelength band, and for automatically extracting and analyzing an optical feature from the data. <P>SOLUTION: Multiple kinds of inspection light rays each having a different wavelength stepwise and a narrow band are emitted from an inspection light generator 30 to the sample held to a sample holder 23 of a body part 2; light rays reflected from the sample are imaged in different wavelength bands by an imaging device 50 to obtain a plurality of images; and an arithmetic unit 3 execute calculation as to the plurality of images by a predetermined rule to extract a feature. An image clearly expressing the feature is stored in a representative data storage part 88 as a representative image. The imaging device 50 includes a CCD camera 51 and an infrared camera 52, and the two kinds of cameras are used by switching them over to a predetermined wavelength. Corrections of the cameras are carried out so that the degree of approximation of the images taken by both the cameras at the switching point is set to a predetermined level or above. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for inspecting and classifying a sample to be inspected, in particular, a paper surface of a paper sheet on which characters or picture patterns are printed, by optically analyzing its characteristics.

[0002]

2. Description of the Related Art In paper currency, securities, stamps, etc., which embody the exchange value using paper sheets as a medium, the authenticity discrimination is always a big problem. When distinguishing between genuine and false, it is necessary to properly understand what the characteristics of the paper sheet are.

In recent years, the performance of color copiers, color printers, scanners, etc. has improved and prices have declined, making it easier to induce counterfeiting of paper sheets. Therefore, it is possible to make markings that are not visible to the naked eye under normal lighting conditions, using substances that are not included in commercially available color copying toners and printer inks, such as fluorescent substances and other functional materials. . These markings naturally constitute the characteristics of paper sheets.

In the paper sheet authenticity discriminating apparatus described in Japanese Unexamined Patent Publication No. 2001-522232, a specific portion of a sheet to be discriminated is irradiated with ultraviolet rays, and transmitted light or reflected light from the ultraviolet ray irradiated portion is wavelength-selective filter. Only the light of a specific wavelength band is received by the photosensor array over this period, the authenticity of the paper quality of the paper to be identified is identified based on the optical output signal of the photosensor array, and the authenticity of the fluorescent substance that emits fluorescence when irradiated with ultraviolet rays. When both of the discrimination results are true, the paper to be discriminated is true.

The above-mentioned paper sheet authenticity discriminating device discriminates whether or not the paper sheet used for the paper sheet is genuine by the light output signal of the photo sensor. However, this device confirms already recognized optical characteristics, and does not check even printed patterns on the paper, so it cannot be said that it is a perfect method for verifying authenticity of paper sheets.

In the document inspection apparatus disclosed in Japanese Patent Laid-Open No. 4-288697, at least two light portions having different wavelengths are directed onto a common area of a document, and reflected light or transmitted light is detected by a linear detector. .

The above document inspection apparatus can use a printed figure as an inspection reference. Further, a plurality of detectors detect reflected light or transmitted light, and the signals coming from the detectors are evaluated individually or in an appropriate combination, and multifaceted inspection is possible. However, it is hard to say that the technical pursuit has been sufficiently made on how to process the data obtained by a plurality of detectors in association with each other to improve the function of the inspection device.

Although a device such as a spectrophotometer for measuring the optical characteristics of a sample in a wide wavelength band is well known, only spot-shaped measurement areas are set, and it is difficult to measure the characteristics of the entire paper surface. Further, it merely shows the measured value under the measurement condition at the time of measurement. Although there is a device that captures an optical image of the sample paper surface at a plurality of wavelengths with a CCD camera or the like, the search wavelength band is narrow (400 to 1000 nm). Moreover, only the sampled surface image is obtained, and the feature extraction is not performed from the image corresponding to each search wavelength.

Therefore, in order to search the optical properties of the sample paper surface over a wide wavelength band and extract the features, an image pickup apparatus composed of a plurality of types of cameras having different image pickup wavelength bands, and a large number of sheets obtained for each wavelength from the plurality of types of cameras. It is necessary to have an arithmetic unit for extracting features from the captured images of and further classifying these features. For example, if 7200 images of about 1 million pixels are picked up from a wide wavelength band and the features of all the images are extracted, enormous labor and time are required. Furthermore,
When these captured images have different camera characteristics such as brightness and density gradient, feature extraction becomes difficult. In addition, even if an attempt is made to determine whether the optical properties of a reference sample obtained from a large number of samples and a new sample are the same, under these circumstances, the comparison and determination work itself is difficult, and a sufficiently satisfactory result cannot be obtained. I couldn't do it.

[0010]

DISCLOSURE OF THE INVENTION The present invention is capable of extracting optical characteristics of a sample in a wide wavelength band not limited to the visible light region, capable of centrally processing the obtained data, and a system of an arithmetic unit. An object of the present invention is to provide a feature extraction device and a feature extraction system that can efficiently use resources.

[0011]

(1) The feature extraction apparatus of the present invention captures an image of a sample when performing an operation on a sample image according to a predetermined rule to capture an image of the sample to extract features. The imaging device switches and uses a plurality of cameras having different image-capable wavelength bands at a predetermined wavelength in a wavelength overlapping region, and the cameras are controlled so that the degree of approximation of the images captured by both cameras at the switching point becomes a predetermined level or higher. It was decided to make a correction.

According to this structure, it is possible to search for optical characteristics in a wide wavelength band by a plurality of cameras. Further, even if the image is switched from one camera to the other camera, the difference in images is small, and the optical characteristics of these images can be extracted by the same image processing method. Therefore, even if there are a plurality of cameras, the image processing software and the data storage unit can be integrated to save system resources and speed up the processing.

(2) Further, in the present invention, the sample is held in the sample holder, the sample is irradiated with the inspection light generated by the inspection light generator, and the sample surface is irradiated by the image pickup device in different predetermined wavelength bands. It was decided to take an image and acquire an image corresponding to each of these wavelength bands.

According to this structure, it is possible to reliably acquire an image for each wavelength band necessary for extracting the optical characteristics of the sample.

(3) Further, in the present invention, a plurality of types of light having different wavelengths in stages are applied to the sample as inspection light.

According to this structure, it is possible to accurately search and analyze the optical characteristics of the sample surface corresponding to each of these plural kinds of irradiation light, and to accurately grasp the characteristics.

(4) In the present invention, a white reference point indicating the brightness level is provided in the picked-up image, and the gray level value of the reference point of the picked-up images sequentially obtained is set to a preset white level value. The image pickup means is controlled to have a brightness that matches within the allowable error value of.

With this structure, it is possible to compensate for variations in the sensitivity of the camera.

(5) In the present invention, the sample holder for holding the sample, the irradiation means for irradiating the sample with a plurality of kinds of light having different wavelengths stepwise as the inspection light, and the sample surface in different wavelength bands. In a feature extraction device that has an image capturing unit that captures an image corresponding to each wavelength band and performs a calculation on a captured image of the sample according to a predetermined rule to extract a feature, the image capturing of the sample holder is performed. A white reference point indicating a brightness level and a black reference point indicating a darkness level are provided in the image, and each of the white level value and the black level value of these black and white reference points becomes a predetermined gradation level value. Thus, the camera image data is sequentially obtained while performing the scaling correction calculation on the grayscale value of the captured image at the same rate.

According to this structure, the dynamic range of each picked-up image is made uniform, and variations in the sensitivity of the image pickup means (in brightness and density gradient) can be eliminated.

(6) Further, in the present invention, when the characteristic image of the sample surface is taken, the preceding captured image is displayed under the predetermined capturing condition and the captured image is displayed, and the captured image is displayed on the preceding captured image display unit. The image is provided with reference point coordinate setting input means for setting the coordinates of the white reference point and the coordinates of the black reference point, and the level values of the black and white reference points whose coordinates have been set by the setting input means are obtained later. It was decided to use it as an index for the same ratio scaling correction calculation of the image gray value.

According to this structure, the white reference point and the black reference point can be set at appropriate coordinate positions within the image while confirming the preceding photographed image, so that the same-value scaling correction of the gray value of the captured image obtained thereafter is performed. It is possible to set both black and white reference points having appropriate level values as indices for calculation.

(7) Further, in the present invention, regarding the arithmetic imaging means, the difference between the black and white level values of the above-mentioned black and white reference point is obtained for the imaged image obtained from the sample surface, and the difference is not more than a preset value. In this case, the scaling correction calculation is performed on the second predetermined level value obtained by adding and subtracting a predetermined set value to both the black and white level values, and the excessive gradation correction calculation of the captured image with the slight gray level difference is performed. I tried to prevent.

According to this structure, when an image with a slight difference in gray level is obtained, that is, an image in which almost all white or black is obtained, the minute gray feature is not abnormally enlarged and corrected, and is corrected abnormally. It is possible to prevent meaningless feature extraction and / or similar classification calculation of these features due to mixing of different images.

(8) In the present invention, the sample holder for holding the sample, the irradiation means for selectively irradiating the sample with plural kinds of light as the inspection light, and the reflected light from the sample in different wavelength bands. In a feature extraction system, which has an image capturing unit that captures an image corresponding to each wavelength band and performs a calculation on a captured image of a sample according to a predetermined rule to extract a feature, the sample holder, irradiation Means and an image pickup means obtained by connecting the main body portion including the image pickup means to a calculation device that performs a calculation on the image of the sampled image according to a predetermined rule to extract a feature, through a communication line. Were collected in the arithmetic unit to extract the features.

According to this structure, the arithmetic unit can be installed at an arbitrary location apart from the main body.

[0027]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. First, the structure will be described with reference to FIGS.

The feature extracting device 1 comprises a main body 2 and a computing device 3. The main body 2 has a housing 10, and the components inside thereof are divided into an optical section 11 and a control section 12 as shown in FIG. 1 to 6 depict the components contained exclusively in the optical section 11.

The housing 10 of the main body 2 is movably supported on the floor by a plurality of casters 21. A door 22 is provided in a part of the housing 10, and the sample holder 23 is put in and taken out by opening the door 22. When the door 22 is closed, the inside of the housing 10 becomes a dark room state. Sample holder 2
Reference numeral 3 holds paper sheets such as banknotes, securities and stamps so that the surface to be inspected is vertical. The surface to be inspected is irradiated with inspection light, and reflected light (including excitation light of the fluorescent substance, if any) is imaged. Then, the inspection light generator 30 that functions as an irradiation unit that irradiates the sample with a plurality of types of light having different wavelengths stepwise, and the sample surface is imaged in different wavelength bands to correspond to each wavelength band. The configuration of the image capturing device 50 that functions as an image capturing unit that acquires an image will be described.

The light source section 31 of the inspection light generator 30 is 250
It emits light in the wavelength range of up to 2,000 nm. Since it is not possible to generate light of such a wide band with a single lamp, two types of lamps, a xenon lamp 32 and a halogen lamp 33, are used (see FIGS. 4 and 5). The xenon lamp 32 is used to obtain inspection light of 250 to 380 nm,
The halogen lamp 33 is used to obtain inspection light of 381 to 2,000 nm.

Xenon lamp 32 and halogen lamp 33
Are arranged facing each other in the lamp house 34. Since it takes time for the lamp to emit the light of the specified spectrum after the power is turned on, both lamps are turned on continuously while the equipment is operating, instead of turning on and off the lamp each time the lamp is switched. The movable side mirror reflects the light on the required side each time and takes it out. Reference numeral 35 denotes the movable mirror, which has a reflecting surface facing the xenon lamp 32 at a 45 ° angle and a reflecting surface facing the halogen lamp 33 at a 45 ° angle shown in FIG. It is switched by a motor or a solenoid (not shown). The light striking the movable mirror 35 turns at a right angle and enters the lens house 37 through the emission window 36 on one side of the lamp house 34.

The light whose divergence angle is adjusted by the lens group in the lens house 37 enters the spectroscope 38. Spectroscope 38
Is an aberration correction type zero dispersion double monochromator,
Driving wavelength range is 250 to 2,000 nm, full width at half maximum is 30 n
The resolution is set to m or 60 nm, and the resolution is set to 15 to 30 nm, but can be set to "white light including the entire wavelength range of the light source" when necessary. Light in a predetermined wavelength band emitted from the spectroscope 38 becomes inspection light.

A lens house 39 is provided at the exit window of the spectroscope 38.
Are connected. A lens (not shown) arranged in the lens house 39 is for adjusting the divergence angle of the inspection light, and is 180 mm × 18 in the sample holder 23.
Same as when irradiating 0 mm area, 50 mm x 5
The divergence angle is switched between when irradiating an area of 0 mm.

The inspection light emitted from the lens house 39 travels in the space while being repeatedly reflected by the fixed mirrors 40 and 41, and illuminates the sample held by the sample holder 23. The light reflected and absorbed by the surface to be inspected of the sample enters the imaging device 50.

As described above, the band of the inspection light is 250-2.
Although it covers 000 nm, there is no imaging means having such a wide sensitive area, so the imaging device 50 is configured with two types of cameras. One of them is a CCD camera 51, which is arranged directly in front of the sample holder 23 as seen in FIGS. The other is an infrared camera 52, which is arranged in parallel with the CCD camera 51 at a position horizontally displaced from the front of the sample holder 23.

The CCD camera 51 including a silicon image pickup device is used to pick up an image of reflected light having a wavelength of 250 to 1,000 nm. The infrared camera 52 having an imaging element made of platinum indium or indium antimony is 1,0
It is used to image reflected light of 00 to 2,000 nm.
These cameras are about 65,000 (≈ 256 x 256)
The number of pixels is about 1 million (≈1024 × 1024) pixels or more, and the sample surface to be analyzed is input as a grayscale detailed image with 256 gradations or more.

The cameras are switched by the movable mirror 53 shown in FIGS. The movable mirror 53 is moved in the horizontal direction by a drive mechanism (not shown), and when the movable mirror 53 reaches the position shown in FIG. 2, the reflected light from the sample holder 23 is straight.
It reaches the camera 51. When the movable mirror 53 comes to the position shown in FIG. 3, the reflected light is bent at a right angle by the movable mirror 53 and again at a right angle by the fixed mirror 54 so that the infrared camera 5 can receive the reflected light.
Reach 2.

55a, 55b, 56a and 56b are a sample holder 23 and a CCD camera 5 by a driving mechanism (not shown).
1 and a converging lens that appears in and out of each optical path between the fixed mirror 54 and the infrared camera 52. Converging lens 55a,
56a is used for converging the reflected light when the irradiation area is 180 mm × 180 mm, and the converging lens 55
b and 56b are used to converge the reflected light when the irradiation area is 50 mm × 50 mm.

In FIGS. 1, 2 and 3, the converging lens 55 is used.
The combination of a and 55b is the same or the converging lens 56
Although the combination of a and 56b is depicted as existing in the optical path at the same time, this is merely for convenience of illustration, and in fact only one of each combination will advance into the optical path and the other will One is retracted outside the optical path.

CCD when the converging lenses 55a and 56a are used and when the converging lenses 55b and 56b are used
Alternatively, since the light receiving area and / or the image forming position on the infrared imaging element slightly changes, the driving mechanism (not shown) moves these cameras 51 and 52 along the optical axis to perform compensation corresponding to the change. There is. A zoom lens mechanism may be used instead of moving the camera.

A CCD camera 51 is combined with a spectroscope (ultraviolet visible image spectroscope) 57 in order to narrow down the imaged light to a predetermined wavelength range, and an infrared camera 52 is equipped with a bandpass filter (near infrared bandpass filter). 58 are combined. The spectroscope 57 has a half width of 30 nm and a resolution of 15 nm. The bandpass filter 58 has a half width of 3
The resolution is 0 to 60 nm and the resolution is 30 nm. The spectroscope 57 and the bandpass filter 58 also have a function of selecting light in the entire wavelength range, like the spectroscope 38 on the light source side.

FIG. 6 shows an example of the structure of the bandpass filter 58. These are three turrets 59a, 59b, 5 which are index-rotated by motors not shown respectively.
It is a combination of 9c. Each turret 59
a, 59b, and 59c are 13 filter holders 60 each.
Are held radially, and each one filter holder 60 is aligned with the optical path. In FIG. 6, a hatched circle (a circle represents a filter) indicates the position of the optical path.

Of the 13-piece filter holders 60 held by the turrets 59a, 59b, 59c, 12 pieces each have a filter 61 with a slightly different pass wavelength band, and the remaining 1 piece is for all wavelength bands. Let it go through. Filter holder 60 that is transparent through the turrets 59b and 59c
By selecting, the 12 types of filters of the turret 59a can be used independently. If the filter holder 60 that is transparent to the turrets 59a and 59c is selected, the 12 types of filters of the turret 59b can be similarly used alone. If the filter holder 60 that is transparent to the turrets 59a and 59b is selected, the 12 types of filters of the turret 59c can be similarly used alone. Further, if the filter holder 60 that is transparent to all of the turrets 59a, 59b, and 59c is selected, light in the entire wavelength range is selected.

Next, the block configuration of the feature extraction apparatus 1 will be described with reference to FIG.

The control section 12 of the main body section 10 includes a lamp control section 70, a mirror control section 71, a lens control section 72, an irradiation side spectroscope control section 73, an imaging side spectroscope control section 74, and a camera control section 75. The lamp control unit 70 controls turning on / off of the xenon lamp 32 and the halogen lamp 33. The mirror controller 71 controls the operations of the movable mirrors 35 and 53. The lens controller 72 controls the operation of the lenses in the lens house 39 and the converging lenses 55a, 55b, 56a, 56b. The irradiation-side spectroscope control unit 73 includes the spectroscope 38.
Control the behavior of. The imaging-side spectroscope control unit 74 includes the spectroscope 5
7 and the operation of the bandpass filter 58. The camera control unit 75 controls the setting of the exposure time of the CCD camera 51 and the infrared camera 52 and other shooting conditions and the shooting.

The block configuration of the arithmetic unit 3 is as follows. A central processing unit 80 includes a microprocessor and a storage device, controls the arithmetic unit 3, and processes and outputs data by a program. Reference numeral 81 denotes an operation unit, which is composed of input means such as a keyboard and various switches provided in the main body casing of the arithmetic unit 3.
A display unit 82 is composed of a monitor such as a CRT and a liquid crystal display, and a display unit such as a light emitting diode provided in the main body housing of the arithmetic unit 3. The arithmetic unit 3 and the main body 2 are connected by a cable (not shown) and exchange signals with each other.

Reference numerals 83 to 88 are data processing block groups formed by using the system resources of the central processing unit 80 as they are or by adding hardware elements to the system resources of the central processing unit 80. Reference numeral 83 denotes an A / D conversion unit, which converts analog data of an image transmitted from the image pickup device 50 into digital data. Reference numeral 84 properly controls the CCD camera 51 and the infrared camera 52, unifies the image data obtained by the CCD camera 51 and the image data obtained by the infrared camera 52, and cuts it out as an input image of a required size that is separately selected and set. In addition, it is a correction calculation unit that compresses the image size. Reference numeral 85 denotes a correction data storage unit that stores parameters such as a correction coefficient for the calculation in the correction calculation unit 84. An image buffer unit 86 holds an image cut out to have the required size, a compressed image, and a temporary intermediate image that is processed and operated to extract the features of the image. Reference numeral 87 is an image analysis unit for extracting a feature, and 88 is a representative data storage unit for storing a representative image selected as having a clear feature and related data.

Next, the operation of the feature extraction device 1 will be described. First, the door 22 of the main body 2 is opened, the sample holder 23 is taken out, and the sample of the paper sheet is attached. The sample holder 23 having the sample attached thereto is set at a predetermined position in the housing 10, and the door 22 is closed. Then, the sample is irradiated with the inspection light, and the light reflected from the sample is imaged by the imaging device 50.

The inspection light generator 30 is 250 to 2,000.
Light having a wavelength range of nm, which is a wide wavelength range from the ultraviolet region to the near infrared region, is selectively and sequentially generated as all-wavelength light (white light) or inspection light in a predetermined band while shifting the center wavelength at a predetermined pitch. The surface of the sample looks like the inspection light. For example, when a fluorescent substance is contained in the substance forming the paper or in the printing ink, the fluorescent substance emits excitation light in response to the inspection light having a specific wavelength. An optical image obtained by combining the excitation light and the reflected light is analyzed by the spectroscope 57 or the bandpass filter 58.
Through the CCD camera 51 as a grayscale image of specific wavelength light.
Alternatively, the infrared camera 52 takes an image.

FIG. 8 exemplifies the transition of the image of the paper sheet when the wavelength of the inspection light to be irradiated is shifted and the spectroscope and the bandpass filter on the camera side are passed through, that is, when the image is taken as light of the entire wavelength range. In (a), the original pattern of the paper sheet is not visible at all, and the pattern due to the excitation of the fluorescent material is only visible in the lower right. (B) → (c) → (d) →
(E) → (f) As the wavelength of the inspection light becomes longer, the visual visible light pattern originally observed in the paper sheet gradually appears and the excitation light pattern disappears. In (g), a clear and bright image with the original pattern of the paper sheet is obtained. After that, (h) → (i) →
(J) → (k) As the wavelength of the inspection light becomes longer, the image loses sharpness and brightness. In (l), only the pattern sensitive to near infrared rays can be seen.

FIG. 8 is merely a simplified drawing example for the purpose of explanation, and the actual image does not follow the same transition as this. Actual paper sheets from 250 nm to 2,000
Images taken over a wavelength of 0 nm show much more complex changes than this.

As described above, the inspection light generator 30 sequentially irradiates a plurality of types of light having different wavelengths and narrow bands as inspection light, or all-wavelength light (white light) as inspection light. The center wavelength of the inspection light is set to 86 steps (one step of which is all wavelengths) between 250 and 2,000 nm, and the spectroscope 57 and the bandpass filter 58 are added together on the imaging side to obtain 86 steps (of which). Assuming that the passing wavelength band of all wavelengths is set in one step, 86 × 86 = 7,396 images are captured.

Now, it is possible to obtain a large number of images having different image pickup wavelength bands by the feature extraction device 1. For example, fine image data in which the number of pixels is about 1 million and the gray scale of each pixel is tens of thousands. When all of the above are treated as the features of the paper sheet, a huge amount of memory is required, and in the later discrimination of the authenticity or the difference of the same type of paper sheet, such a large amount of data is stored as the features. If so, matching will take too long. First, even if samples of the same type are printed, there is some misalignment or blurring in printing, and they are not always in the same printing state. Also,
It is quite possible that when used, fading occurs and dirt components such as sebum adhere, resulting in differences in captured images, and even samples of the same type may be rejected as counterfeit tickets. Therefore, except when performing a very fine and highly accurate feature search, the features to be extracted are compressed to some extent by compressing the image size and / or the gray scale while retaining the fine and fine optical characteristics. It is better to have a tolerance and allow it.

Therefore, the arithmetic unit 3 performs a similar classification operation based on the feature extraction according to a predetermined rule with respect to a large number of cut-out images and / or compressed images cut out to a required size and input to the image buffer unit 86, and An image that particularly clearly shows the characteristics of the paper sheet is stored in the representative data storage unit 88 as a representative image. The calculation method for automatically extracting the features will be described below.

Differences between images are used for feature extraction and similar classification. FIG. 9 shows the concept of difference analysis. That is, attention is paid to the difference in data between one image and another image. The first image has "ABCDEF" data,
If the second image has only "ABDEF" data, the difference between the two is "C". This difference "C"
Is used as a material for analysis.

As the difference data, the difference value (difference value) about the "brightness" of each pixel corresponding to each image and the frequency of each difference value (the number of data for each difference value) are obtained. In the present embodiment, “brightness” is “bright” means that the amount of reflected light is large in the imaged wavelength band.
That is, the characteristic pattern that should appear in the imaging wavelength band is more clearly shown. "bright"
Can be paraphrased as “deep”. Also "frequency"
Indicates the number of pixels having the difference value.

Subsequent calculations are performed using the AD value of 256 gradations, which is obtained by converting the brightness of the actual pixel into a digital value.
Therefore, the difference value of "brightness" between the two images is replaced with any of the AD values of -255 to +255 as shown in FIG.

In the calculation, the product of the AD value of each difference value and the frequency is calculated, and then the total sum of the products is calculated. At this time, when the product is calculated while the positive and negative signs are added to the AD value (Fig. The example shown in FIG. 10 corresponds to this) and the case where the product regarding the “index” which is the absolute value of the AD value is obtained (the example shown in FIG. 11 corresponds to this). Here, the sum of products in the former case is called "sum of products", and the sum of products in the latter case is called "exponential sum".

When the calculation is performed using the exponent, the difference between the differences is further emphasized in order to facilitate the calculation.
FIG. 11 shows an example of the method. In FIG. 11, the absolute value of the difference value is obtained, and a constant value is further subtracted from the absolute value to obtain a “difference value index”. If a negative value appears, the difference value index is set to "zero".

In FIG. 11, the "constant value" is separately set to "30", so that the section from -30 to +30 in the difference value is a noise component and all the difference value exponents are "zero". Become. That is, the difference value is -30
All the pixels (pixels) from to +30 are treated as the same pixel within the tolerance range. Therefore, the influence of noise between the approximate images is eliminated, and only pixels having a true difference with a difference value of ± 30 or more are elements that are recognized as another image. FIG. 12 shows a graph of the relationship between the difference value and the difference value index.

Features are extracted based on the difference value, and
A method to select an image that is clearly represented as a representative image
This will be described with reference to FIG. Eight images are shown in FIG.
ing. These images have different center wavelengths at a predetermined pitch.
The images are taken in each wavelength band and arranged in order of wavelength.
is there. Image P_i-4Is dark, and from there P_i-3, P
_i-2And P_iThe brightness gradually increases as the_i
Peaks at. Image P_iAfter passing, image P_{i + 3}Until
The brightness gradually decreases. That is, the person from the black image
The image gradually emerges and then gradually fades out
Then, an example is shown of returning to a black image. The brightness of the image is opposite
Even if it is rolling, the operation will be the same as above,
Detailed description is omitted.

The sum (product sum) of the products of the difference value between adjacent images and the frequency of each difference value index is represented by D _i . The change from "low brightness" to "high brightness" is expressed as "plus" and vice versa as "minus". The sum of products D _i indicates the amount of difference between the two compared images.

Described below is the procedure for selecting a representative image when the above-described sum of products is smaller than a separately determined threshold value and the images are judged to be similar images.

Image P_i-4And P_i-3, P_i-3And P_i-2, P_i-2
And P_i-1, P_i-1And P_iSum of products between_iBoth are
In other words, it is in the area where the image of a person stands out. Toko
Roga P _iAnd P_{i + 1}D between_iIs a "minus"
Then, switch to the fade-out area of the person image, P_{i + 1}When
P_{i + 2}, P_{i + 2}And P_{i + 3}Maintain "minus" during the period.
An image of the trend that has shifted from "small brightness" to "brightness"
Image P_iAnd image P_{i + 1}Such a result due to reversal between
Image P_iAnd P_{i + 1}Is the point of change
It will happen.

The image P _i located immediately before the change point is the brightest and sharpest in the preceding and following image groups. That representative data from the feature common to the image group in which appeared the most easy identification form, the _{P i-4, P i-} 3 ... P i + 4 as well as employing the same as the representative image as a similar image Store and register in the storage unit 88.

Described next is the procedure for selecting a representative image when it is determined that the above-mentioned sum of exponents is larger than a separately determined threshold value and the images are not similar images.

In FIG. 14, the image groups of the group 1 that have changed from "low brightness" to "high brightness" are the images P _i and P.
_The case where the image is suddenly converted to a group 2 image group which is not similar between _{i + 1} is shown. In this case, the sum of the exponents becomes larger than the separately determined threshold value. At this time, as described above, the last image P _i of the image group of group 1 is registered as the representative image. The images P _i-4 , P _i-3 , P _i-2 , and P _i-1 are similar images, and only information other than image data is stored and registered in the representative data storage unit 88.

The above work is repeated to obtain 250 to 2,000.
Several types of representative images and similar images thereof are extracted from a large number of image groups acquired over the wavelength range of nm, and stored and registered in the representative data storage unit 88 together with similar image information.

In this way, the optical characteristics are automatically extracted from one sample, and an image showing the characteristics well is sorted and stored as a representative image. This series of image analysis is shown in FIG. It is performed according to the flow chart of.

In step S101 of FIG. 15, the difference value for each pixel between the "current image" and the "previous image" is calculated over all pixels. The “image of this time” is the analysis image that is currently undergoing analysis (the image to be analyzed), and the “previous image” is the analysis image that was analyzed immediately before. It is assumed that the "previous image" has already been found to be similar to any of the representative images. In step S102, the absolute value sum of the products of the difference value exponent emphasizing the value of the difference value obtained by the method of FIG. 11 and the frequency of each difference value exponent is obtained as the exponent sum. In step S103, it is determined whether the sum of exponents is equal to or greater than a predetermined threshold. If the predetermined threshold value has not been reached, it is determined that the current image and the previous image are similar, and the process proceeds to step S104.

In step S104, it is determined whether the current image is "brighter" than the previous image. As described above, “bright” means that the amount of reflected light is large in the imaged wavelength band. That is, the characteristic pattern that should appear in the imaging wavelength band is more clearly shown. In the “brightness” comparison, not the exponential sum, but the sum of products of difference values with positive and negative signs and frequencies (sum of products in FIG. 10) is used (also in steps S105 and S106).

If the current image is brighter than the previous image, the process proceeds to step S105. Here, a difference value for each pixel between a representative image having a similar relationship to the previous image and the current image is obtained over all pixels, and the brightness is compared. As a result, if it is determined in step S106 that the current image is brighter than the representative image by the predetermined value or more, the process proceeds to step S107.

In step S107, the representative image of the similar group is updated, and the current image is set as a new representative image,
An image that has been a representative image up to now is registered as a new representative image.

If the current image is not brighter than the previous image in step S104, the process proceeds to step S108. Then, it is registered as being similar to the previous representative image.

If it is determined in step S106 that the current image is not brighter than the representative image by the predetermined value or more, the process proceeds to step S108. Then, it is registered as being similar to the previous representative image.

If it is determined in step S103 that the sum of indexes is greater than or equal to the predetermined threshold value, it means that the current image and the previous image are dissimilar, and the process proceeds to step S109.

In step S109, all representative images registered so far and / or this time in the feature extraction step and the current image are compared with the product sum to find the minimum difference representative image as the closest approximation image in step S109. Proceed to S110.

In step S110, the most approximate representative image is temporarily set as the representative image of the current image, and the difference value for each pixel between this temporarily set representative image and the current image is calculated over all pixels. In step S111, the sum of the products of the difference value exponent emphasizing the difference value obtained by the method of FIG. 11 and the frequency of each exponent is obtained as the exponent sum. Step S
At 112, it is determined whether the sum of exponents is greater than or equal to a predetermined determination value. If the predetermined determination value has not been reached, it is determined that the minimum difference representative image as the temporary representative and the current image are similar, the process proceeds to step S108, and the same process as described above is executed.

If the exponent sum is greater than or equal to the predetermined judgment value in step S112, it is determined that the minimum difference representative image as the temporary representative and the current image are dissimilar, and the process proceeds to step S113. Then, the image of this time is registered as a new representative image.

As the image data, all the detailed image data used for the feature extraction calculation for the representative image are left. For images other than the representative image, only the collation data (classification information such as the characteristic extraction conditions of the optical characteristics) with the representative image is left, and the image data itself can be erased. This makes it possible to save the amount of memory used for one sample.

In the feature extraction device 1, the read image from the camera corresponding to the representative image is also stored and held as the representative real image. Even if the difference value calculation between the front and rear images and other classification and sorting operations are simply performed at high speed based on the separately set compressed image data, detailed camera image data is still stored as the representative real image, When the re-sorting calculation is performed according to the calculation specification, it is possible to perform highly accurate feature extraction from each representative actual image data and the collation data.

In the feature extraction apparatus 1, when it is necessary to distinguish the new sample by using the extracted feature, the central waves of the light projected and / or the light received at a predetermined pitch are changed, Imaging is performed by emphasizing the optical characteristics of the paper surface of the new sample for each wavelength band. Then, a representative image of a reference sample specimen is acquired, and it is checked whether or not an image approximate to this representative image in a predetermined wavelength range has been acquired. Furthermore, the expression bands of the similar images of the respective representative images are checked at the required locations for the same and different, and the collation determination is performed. In addition, even when a large number of similar samples with unique characteristics are sporadically divided and inspected, the characteristics of the sample over a wide wavelength band are centrally extracted and classified, as described above, It is easy to match and match the characteristics of, and it is possible to make a correct matching determination in a short time.

Here, the collation data will be described with reference to the case of the imaging log shown in FIG. The imaging log is composed of a search specification section and an imaging data index section. The “search number” is a number assigned to each sample or each search unit. "Sample size" indicates the size of the sample to be imaged,
In this example, the size is 50 mm × 50 mm. The description of "irradiation wavelength" has a wavelength range of 250 to 970 nm and is 3
It shows that it changes at a pitch of 0 nm. "camera"
Indicates that, of the CCD camera and the infrared camera, the CCD camera was used. "CCD image size = 4 x 4 (25
6 × 256) ”means 4 pixels × 4 pixels as one unit and 2 pixels vertically and horizontally.
It is shown that it is handled as an area of 56 pixels. "C
The “CD exposure time” is 30 mS. “CCD exposure time automatic adjustment = 0” means that the automatic adjustment is off. Although the numerical values of the sweep start wavelength, the sweep end wavelength, and the imaging wavelength pitch are entered in the “CCD detection wavelength”, they are all 0 in this example. This indicates that the setting is for all wavelengths (white light).

Any character can be entered in the "MEMO" field for later reference. The “number of images” is the number of captured images calculated by the setting contents such as the irradiation wavelength. In this example, the number of captured images is 25.
It indicates that there are five types. "CCD cooler temperature = -39" means that the cooling temperature of the CCD image sensor is -3.
The temperature of 9 ° C. and “IR cooler temperature = 0” indicate that the IR image sensor is not used.
The “cutout position” indicates the coordinates within the effective area of the captured image, and is set to “10, 33, 229, 233” in this example. The “white reference position” indicates a white reference image area, which is a rectangular area represented by “119, 39, 127, 47” in this example. Similarly, the “black reference position” is “123, 23
It is a rectangular area represented by "8, 131, 246". “Folder” is the folder name in the hard disk that stores the captured image data, and uses the search number.

In the index portion of the latter half of the image pickup data, each image from "search start" to "search end" is given a serial number and the image pickup condition is recorded. For example, 000000
The description of line 03 is as follows. "17: 2
8:50 ”is the image capturing time, which indicates that the image was captured at 17:28:50. "F = MCX0250-0000.BM
"P" means that the image file is in BMP format, and the actual image data is stored under this file name.
The image pickup means is a CCD image pickup element, the lamp is a xenon lamp,
The wavelength of the irradiation light is 250 nm, and the light receiving side shows that light of the entire wavelength region is transmitted.

How to read the data following the file name is as follows. "T = 30.0" means an exposure time of 30.0 m
Indicates S. “W = 636” indicates that the read value of the white reference at the white reference position under the above conditions is the 636 AD value. “B = 621” also indicates that the black reference read value at the black reference position is the 621AD value.
“S = 735 >> 601” is the value of the effective dynamic range to be corrected. Corrected from 735 AD to 60
A dynamic range of 256 gradations is obtained between 1 AD values. "OK" at the end indicates that the image is a normal captured image.

Of the data shown in FIG. 25, only the file name data and the image pickup condition data attached thereto are left as the collation data of the characteristic extraction conditions of the optical characteristics, and the actual image data is erased to save the memory. be able to.

Now, the CCD camera 51 and the infrared camera 5
2 have their own light-sensing characteristics. FIG. 16 shows it conceptually. In FIG. 16, the wavelength band is plotted on the horizontal axis and the sensitivity is plotted on the vertical axis. The wavelength bands that can be captured by both cameras overlap in the vicinity of 1,000 nm.
Both cameras are switched and used with 0 nm as the switching point. However, as it is, at the switching point, the brightness of the image and the density gradient are completely different between the image captured by the CCD camera 51 and the image captured by the infrared camera 52. Under this condition, there are two types of image data under the same conditions, and the image processing software and the data storage unit are dualized, and the system resources of the arithmetic unit 3 are consumed extra.

Therefore, in the present invention, C at the switching point
One or both of the cameras are corrected so that the degree of approximation between the images captured by the CD camera 51 and the infrared camera 52 is equal to or higher than a predetermined level, that is, as if a single camera was used to capture an image across a switching point. Specifically, the exposure time of the camera is adjusted to correct the brightness of the image, and the sensitivity and gain are adjusted to correct the density gradient. Only one of the brightness and the density gradient may be corrected, or both of them may be corrected. Since it is desirable that the imaging characteristic is flat even if the switching point is separated, correction is performed in each wavelength band. FIG. 17 conceptually shows the sensitivity characteristic after the correction.

To correct the image brightness and density gradient,
A bright (white) reference point and a dark (black) reference point are set in advance on a part of the image pickup surface, and the readings of both cameras are bright when the readings are bright and dark when the readings of both cameras are the same level. Need to be corrected. To determine the reference point, use one of the following two methods.

In the method shown in FIG. 18, the pre-shooting is performed under the preset shooting conditions for the feature shooting of the sample to display this captured image, and the coordinates of the reference point are set in the image plane of the sample. The display unit 82 of the arithmetic device 3 serves as a preceding captured image display unit that displays a captured image of the preceding capturing. An input means such as a keyboard included in the operation unit 81 serves as a setting input means for setting the coordinates of the reference point.

Reference numeral 90 is an image of the sample, in which the white reference point 91 is in the bright part, the black reference point 92 is in the dark part,
Set each. When the black and white reference points 91 and 92 of the image captured by the CCD camera 51 and the black and white reference points 91 and 92 of the image captured by the infrared camera 52 are compared, the brightness of both reference points by both cameras and the reference points 91 and 92 are compared. The camera control value is corrected so that the density gradient (grayscale difference) between the two becomes a degree of approximation equal to or higher than a predetermined level.

In the method shown in FIG. 19, a black and white reference marker separately arranged on the sample holder 23 so as to be located at a predetermined position outside the sample surface is used. That is, the white reference marker 93 and the black reference marker 94 are attached to the imaging area and are not covered with the sample of the paper sheet, and the images of the black and white reference markers 93 and 94 are captured together with the image of the sample. Based on the images of the black and white reference markers 93 and 94, the camera control value is corrected in the same manner as above.

Regarding the correction of the camera control value, a predetermined degree of approximation can be obtained by the same correction between the picked-up images of each camera. Further, when the sensitivity difference and / or the light source lamp for each wavelength band of the camera is switched, an approximate image is similarly obtained.

The light accumulation time adjustment can be realized by using a CCD having a so-called time variable shutter function. C in the present embodiment using FIGS. 20 and 21.
The procedure for automatically setting the exposure time of both the CD and infrared cameras will be described. In the present embodiment, as shown in FIG. 21, the white level value (wn) of the obtained image within the separately set allowable maximum exposure time (Tm) is the white level value (W) of the reference image separately set as a target. Predetermined tolerance (standard white level value ± 3
The exposure time is automatically selected and set so that it falls within 0%).

First, a white level value (w1) for photographing at a predetermined exposure time (T1) is obtained (step S201) and compared with a target reference white level value (W) (step S20).
2). In this comparison, if the difference between the two levels is within the predetermined allowable value, this image is determined to be a proper image, and the automatic exposure time setting ends (step S203). On the other hand, if it is out of the allowable value, the corresponding proper exposure time (Tf) is predicted and set from the preset proper exposure time table for each value of the difference between the two levels in the comparison (step S20).
4) Obtain the white level value (wf) of this shooting (step S205). The appropriate exposure time (Tf) can also be determined by using a mathematical formula described later.

Next, the new white level value (wf) is compared again with the reference white level value (W) (step S206), and if it is within the allowable value, it is determined as a proper image and the exposure time is automatically adjusted. The setting ends (step S207). In the present embodiment, when the white level value (wf) of the captured image at the exposure time (Tf) is not appropriate, the more appropriate exposure time (Te) is re-predicted and imaged as described above (step S20).
8) This is set as the final exposure time, and the same exposure time optimization is not performed beyond this. That is, the irradiation light source light,
It prevents the re-predictive photography of optimized exposure time from continuing indefinitely due to an abnormal variation of the gray scale value of the captured image caused by a defect of the optical optical path and / or the camera. In the present embodiment, it is determined whether the difference between the white level value (we) of the captured image at the final exposure time (Te) and the target reference white level value (W) is less than or equal to a separately set predetermined approximate value. It is confirmed (steps S209 to S212).

Further, even if the white level value of each captured image does not change correspondingly even if the exposure time is changed, this is detected and an alarm is activated. Further, the predicted proper exposure time is the maximum allowable exposure time (T
When m) is exceeded, this is also detected and an alarm is activated, but a detailed description is omitted.

The proper exposure time (Tf) can be calculated from the following equation. Tf = T1 * (W-w0) / (w1-w0) Tf <Tm Here, it is assumed that the exposure time and the imaging white level are directly proportional. FIG. 21 shows an example in which the white level value of the captured image at the camera exposure time “0” is approximately calculated as (w0).

Further, in the feature extraction apparatus 1, it is possible to obtain a picked-up image in a state where the photographing exposure time is fixed to an arbitrary set value by the selection operation, and to perform the feature extraction and the classification calculation from these images. From these captured image values, it is also possible to confirm and compare the absorption / reflectance of irradiation light under each photographing condition of the sample.

Further, regarding the adjustment of changing the white level value of the picked-up image, the physical properties of forming a film of a transition metal oxide (IrOx, Ta ₂ O ₅ , WO _3, etc.) on the glass plate surface in front of the CCD camera or the like. The element may be placed and a voltage may be applied to the physical element to adjust the light transmittance or the light transmission amount of the film. In this adjustment, similarly to the above, an applied voltage to be used for each imaging wavelength, a shutter open time setting table, or the like is prepared in advance.

An example of the method of correcting the density gradient using the above-mentioned black and white reference correction will be described with reference to FIG. Here, an “arithmetic imaging unit” in which the arithmetic unit 3 and the imaging unit 50 are combined is assumed. In the example of FIG. 23, both black and white reference parts are set for a camera-captured image with 16,384 gradations and approximately 1 million pixels. The white level value W of the white reference part is set so that the white part w has a gradation value of 200 in the camera image data of 256 gradations. Similarly, the black level value B of the black reference portion is set so that the black portion b has the gradation value 30 in the camera image data of 256 gradations.

The gray values of all the pixels are set to the same ratio (here, ratio α = (WB) / (w-
Scaling correction calculation is performed according to (b)) to sequentially obtain camera image data. As a result, the camera image data in which the characteristics of the sample to be measured are expressed in a predetermined grayscale even from a camera-captured image in which the offset (brightness level) value and / or the black-and-white reference value varies in scale due to changes in the measurement environment Is obtained.

Further, in the present embodiment, a measure for preventing enlargement calculation is taken so that an excessive enlargement calculation is not performed in the process of the enlargement / reduction correction calculation. This will be described with reference to the flowchart of FIG. In converting from the scale of 16,384 tones shown in FIG. 23 to the scale of 256 tones, FIG.
The conversion area indicated by X to MIN is determined. If the difference between the white reference light amount level W and the black reference light amount level B at 16,384 gradations is extremely small, conversion to 256 gradations w and b will result in extremely widening. To prevent this, the allowable ranges DW and DB are set. MAX-MIN of 16,384 gradations finally decided
The interval is converted to a scale of 255 to 0 with 256 gradations.

In the flow of FIG. 22, first, image pickup is performed (step S301), and the difference value (WB) between the white level value W and the black level value B of the black and white reference portions of the image picked up by the camera is a lower limit value set separately. It is checked whether or not it is X (154) or more (step S
302), if the lower limit value is X or more, the MAX value is W + α *
(255-w) = W + 55α (step s30
3). On the other hand, the MAX value is set to W + DW = W + 50 so that an excessive expansion operation is not performed when the value is less than the lower limit value X (step S304).

Next, the lower limit MIN is determined. It is checked whether or not the difference value (WB) between the white level value W and the black level value B is at least the lower limit value Y (113) set separately (step S305).
When the lower limit value is Y or more, the MIN value is W + α (0-w) = W
It is set to −200α (step S306). On the other hand, when the value is less than the lower limit value Y, M is set so that excessive enlargement calculation is not performed.
The IN value is set to B-DB = B-20 (step S30).
7).

MAX to MIN at 16,384 gradations
CC so that the range of values is 0 to 255 with 256 gradations
Each pixel value of the sample imaged by the D camera is converted.

In this way, the white level value W and the black level value B
If the difference value is less than or equal to a preset value, the white level value W and the black level value B are not directly scaled, but the set values are added and subtracted to the black and white level values to obtain the second predetermined level. By obtaining the value and performing the scaling correction calculation on the second predetermined level value, an image pickup screen with a small difference in gray level, in other words, an image in which the level values of the black and white reference parts are almost the same, the difference is excessive. It is prevented from being converted into a grayscale image which has been subjected to gradation correction calculation.

The values of the lower limit values (X, Y) and the operation constants (DW, DB) used in the above description are merely examples, and the present invention is not limited to these. Further, although the calculation is performed using both the black and white level values (W, B) of the black and white reference part of the image captured by the camera, the same scaling correction calculation is performed by using the maximum and minimum grayscale values in the sample image. It can be performed.

In this embodiment, the following functions are provided for the purpose of picking up a fine grayscale image by an infrared camera as shown in FIG. In FIG. 24, the numbers of the camera image pickup elements (same as pixels) are represented by 65,535 (= 25) on the horizontal axis.
6 × 256), the vertical axis represents the received light level 16,3
By taking 84 gradations and setting black and white reference values for each element, it is possible to perform a gradation gradation correction calculation for each of these image pickup elements.
In this case, as shown in the lower diagram of FIG. 24, each image value of the camera image pickup element in the state where the light emitted from the light source is shielded is used as each black reference value as shown in the lower diagram of FIG. The image value of each of the camera image pickup devices in the state where the light source light is applied to the reference paper as the white reference is stored as each white reference value. The black-and-white reference values for each image sensor stored in this way are set to predetermined black-and-white level values, for example, 25
In a 6-gradation image, the white part w has a gradation value of 200, and the black part b has a gradation value of 30, and the camera imaged image value of the sample is corrected and calculated for each imaging device. The scaling correction calculation can be executed, and the features can be grasped in detail and precisely.

Further, by setting a plurality of black-and-white reference values corresponding to the light emitted from the light source and / or each of the light emitting and receiving wavelength bands, it is possible to perform appropriate correction under each measurement condition.

By correcting the camera control value in this way, the image processing software and the data storage unit can be unified, and the system resources can be saved and the processing speed can be increased. Also, if necessary, brightness and /
Alternatively, even when the density gradient is additionally corrected with different accuracy by calculation processing, since the original captured images are close to each other, accurate calculation processing can be performed and image characteristics can be clarified. Data for performing these correction calculations are stored in the correction data storage unit 8
Store it in 5.

When the number of pixels (pixel size) between the cameras is different, the matching process can be performed by image compression correction or the like (detailed description is omitted).

FIG. 26 shows a feature extraction system according to the second embodiment of the present invention. In this system, the image analysis function is separated from the feature extraction device 1A, and the image analysis is performed by the communication line 4
Arithmetic unit 50 connected to the feature extraction apparatus 1A via 00
I decided to do it at 0.

The structure of the main body 2 of the feature extraction apparatus 1A is the first
It is the same as the configuration of the main body 2 of the feature extraction apparatus 1 of the embodiment. That is, although not shown in FIG. 26, the optical section 11 includes the sample holder 23, the inspection light generating device 30, and the imaging device 50. In this case as well, the sample holder 23 holds samples of paper sheets such as banknotes, securities and stamps. The inspection light generation device 30 functions as an irradiation unit that selectively irradiates a plurality of types of light as inspection light onto the sample. The image pickup device 50 functions as an image pickup means for picking up reflected light from the sample (including excitation light of the fluorescent substance if there is excitation light) in different wavelength bands to obtain images corresponding to the respective wavelength bands. .

The structure of the control section 12 of the main body 2 is the same as that of the feature extracting apparatus 1 of the first embodiment. However, the configuration of the arithmetic unit 3 is different from that of the first embodiment. That is, the image analysis unit 87 and the representative data storage unit 88 disappear from the arithmetic unit 3 of the first embodiment, and instead the communication unit 8
9 is provided. The communication unit 89 connects to the communication line 400. The communication line 400 may take any form. It may be a private line or the Internet.

The arithmetic unit 500 is connected to the communication line 400. The calculation device 500 is separate from the feature extraction device 1 and can be installed in a facility such as a data center.

The block configuration of the arithmetic unit 500 is as follows. A central processing unit 501 includes a microprocessor and a storage device, controls the arithmetic unit 500, and processes and outputs data by a program. 5
Reference numeral 02 denotes an operation unit, which is composed of input means such as a keyboard and various switches provided on the main body housing of the arithmetic device 500. A display unit 503 includes a monitor such as a CRT and a liquid crystal display, and a display unit such as a light emitting diode provided in the main body of the arithmetic unit 500. A communication unit 504 exchanges data with the communication unit 89 of the feature extraction device 1A. The communication units 89 and 504 can be configured by a communication interface such as a modem.

An image analysis unit 505 is a feature extraction device 1A.
A feature is extracted by performing an operation on the image data transmitted from the device according to a predetermined rule. A representative data storage unit 506 stores the representative image and related data selected as having clear characteristics. The image analysis unit 505 and the representative data storage unit 506 are configured by using the system resources of the central processing unit 501 as they are or by adding hardware elements to the system resources of the central processing unit 501. The image analysis unit 505 and the representative data storage unit 506 perform the same processing as the image analysis unit 87 and the representative data storage unit 88 of the first embodiment, but can handle a larger amount of data.

As described above, the feature extraction device 1A installed on the site where the sample is handled and the feature extraction device 500 installed in the remote data center and connected to the feature extraction device 1A via the communication line 400 are used for feature extraction. When the system is configured, the image data collected by the plurality of feature extracting devices 1A can be collected in the arithmetic unit 500 of the data center, and the feature extraction and aggregation and the feature matching with the reference sample can be collectively processed.

As a result, centralized characteristic data covering a wide wavelength band can be collectively managed in a large-scale data processing environment such as a data center for each sample. By re-examining the same sample by the same feature extraction device 1A and checking whether the inspection data obtained as a result matches the previous data, the inspection capability of the feature extraction device 1A concerned changes the environment or changes over time. It is also easy to diagnose whether or not the change has occurred and whether there is a problem in reproducibility.

When performing the characteristic inspection on the first sample, the inspection light wavelength band, control set values of various cameras, various calculation constants for correction, etc. of the conventional inspection example for the same sample are read out, and the It can be set to the same value as the value.
Therefore, it is possible to easily perform the feature extraction and the collation determination process under the same conditions as those of the same type of sample.

Furthermore, regarding a plurality of separately selected samples, a plurality of feature data corresponding to each sample, which are feature-extracted and classified to form a part of the above-mentioned unified feature data, and these feature data. It is possible to search for and display the variation and the average characteristic of. Further, when the characteristic data of the first-appearing sample is obtained, it is possible to perform the sameness / difference calculation based on the Euclidean distance between the characteristic data and the selected plurality of samples.

When a plurality of feature extracting devices 1A are connected to one computing device 500, each feature extracting device 1
It is possible to perform individual control or integrated control using the above-described one control calculation method while considering the variation of A, but detailed description thereof is omitted.

Although various embodiments of the present invention have been described above, various modifications may be made without departing from the scope of the invention.

[0126]

The present invention has the following effects.

(1) When an image of a sample is picked up in order to extract features by performing a calculation on the image of the sample according to a predetermined rule, the image pickup device for picking up the image of the sample uses a plurality of cameras having different imageable wavelength bands. It is used by switching at a predetermined wavelength in the wavelength overlapping region, and since it is decided to correct the cameras so that the degree of approximation of the images captured by both cameras at the switching point becomes a predetermined level or more, it is possible to use a wide range of multiple cameras. It is possible to search for optical characteristics in the wavelength band.
Further, even if the image is switched from one camera to the other camera, the difference in the images is small, and the optical characteristics of these images can be extracted by the same image processing method. Therefore, even if there are a plurality of cameras, the image processing software and the data storage unit can be integrated to save system resources and speed up the processing.

The cameras and / or the input images are also corrected so that the brightness of all the captured images and / or the approximation degree of the gray level gain of the captured images of both cameras at the switching point are equal to or higher than a predetermined level. This makes it possible to unify image data and easily perform data processing while covering a wide wavelength band with a plurality of cameras having different sensitivity bands.

(2) The sample is held in the sample holder, the sample is irradiated with the inspection light generated by the inspection light generator, the sample surface is imaged by the image pickup device in different predetermined wavelength bands, and Since the image corresponding to the wavelength band is acquired, the image for each wavelength band necessary for extracting the optical characteristics of the sample can be surely acquired.

(3) Since the sample is irradiated with a plurality of kinds of light having different wavelengths stepwise as the inspection light, the optical characteristics of the sample surface corresponding to each of the plurality of kinds of irradiation light are accurately measured. It is possible to accurately analyze the characteristics by conducting a search analysis.

(4) A white reference point indicating a brightness level is provided in the picked-up image, and the gradation level value of the reference point of the picked-up images sequentially obtained is within a predetermined allowable error value with respect to a preset white level value. Since it is equipped with an image pickup means that is controlled to have a brightness that matches the above, compensation for variations in camera sensitivity,
It is possible to eliminate the adverse effects caused by variations in the sensitivity of the camera. In addition, it becomes possible to search for optical characteristics in a wide wavelength band by a plurality of cameras. In addition, even if the camera is switched from one camera to the other, the difference in the images is small, and the optical characteristics can be extracted by the same image processing method. Therefore, even if there are a plurality of cameras, the image processing software and the data storage unit can be integrated to save system resources and speed up the processing.

(5) A sample holder for holding a sample, an irradiating means for irradiating the sample with a plurality of kinds of light having different wavelengths stepwise as inspection light, and an image of the sample surface in different wavelength bands, and In a feature extraction device that has an image pickup unit that obtains an image corresponding to a wavelength band, and that extracts a feature by performing an operation on a captured image of the sample according to a predetermined rule, the brightness in the image captured by the sample holder A white reference point indicating a level and a black reference point indicating a darkness level are provided, and the picked-up image is obtained such that each of the white level value and the black level value of these black and white reference points becomes a predetermined grayscale level value. Since the camera image data is sequentially acquired while performing the scaling correction calculation for the grayscale values of the same, the dynamic range of each captured image is made uniform, and variations in the sensitivity of the imaging means (brightness and density gradient Oite) can be eliminated.

Even when precise coincidence correction that cannot be performed by the correction of the brightness level, such as the correction of the exposure time of the camera, is required,
A white reference point indicating a brightness level and a black reference point indicating a darkness level are provided in the picked-up image so as to be used as an index for uniformization correction of the picked-up image. By performing the scaling correction calculation of the grayscale values of the picked-up images at the same rate so that the predetermined grayscale level value becomes, the respective images can be approximately corrected so as to have substantially the same level. Therefore, the clarification correction of the characteristics in the arithmetic processing can be performed more easily and accurately.

In order to make the image characteristics clearer, the grayscale value of the image is adjusted so that the black and white levels of both the black and white reference points obtained by the image pickup means become preset black and white levels (grayscale gradation values). Is obtained as the camera image data while performing the scaling correction calculation, the brightness of the image pickup signal level and / or the grayscale gain is constant even when the image pickup means, the light source light amount, and / or other measurement environment changes. The characteristics are corrected and unified, and it is possible to accurately grasp the characteristics of the captured image data.

Further, each of the black and white reference points is composed of a plurality of pixels which are close to each other, and the average value of these pixels is used as each reference value, and an error due to an incorrect reference value due to cosmic ray noise or other point noise is caused. The calculation is prevented. Further, the grayscale values of both black and white reference points of the corrected image are set to stay inside with a predetermined width so as not to fall below the minimum value or above the maximum value of the camera image data. As a result, it is possible to accurately correct and display an image feature that is slightly over or under the black and white reference points without overcurrent or undercurrent.

By arranging a white reference marker and a black reference marker separately in the image pickup area and inputting the coordinates of these, it is possible to correct the accurate target level.

The black-and-white reference point provided in the image picked up by the camera can be set at any position in the picked-up image of the sample surface, and is set at the black-and-white reference marker provided at a predetermined position outside the sample surface. By also being able to
It is possible to perform correction so as to clarify the characteristics of the light and shade on the sample surface, and it is also possible to clarify the characteristics between the captured images using the black and white reference marker as a reference.

(6) When the characteristic image of the sample surface is taken, the preceding captured image is displayed under the predetermined capturing condition and the captured image is displayed, and the image displayed on the preceding captured image display unit is white-referenced. A reference point coordinate setting input means for setting the coordinates of the point and the coordinates of the black reference point is provided, and the level values of the black and white reference points whose coordinates are set by the setting input means have the same ratio of the gradation value of the captured image obtained thereafter. Since it was decided to use it as an index for the scaling correction calculation, it is possible to set the white reference point and the black reference point at appropriate coordinate positions in the image while checking the preceding captured image. It is possible to set both black and white reference points having appropriate level values as indices for the scaling correction calculation.

(7) With respect to the arithmetic image pickup means, the difference between the black and white level values of the above black and white reference point is obtained for the picked up image obtained from the sample surface, and if this difference is less than a preset value, both black and white are obtained. By performing a scaling correction calculation on a second predetermined level value obtained by adding and subtracting a predetermined set value to the level value, it is possible to prevent an excessive gradation correction calculation of a captured image with a small gray level difference. Therefore, when an image with a slight difference in gray level, that is, an image with almost all white or black is obtained, it is possible to eliminate the abnormal enlargement correction of the minute gray features, and to correct the abnormally confusing corrected image. It is possible to prevent meaningless feature extraction and / or similar classification calculation of these features due to mixing.

In the case of an image of almost all white or black as described above, the maximum and / or minimum value of black and white is set to a value far from the signal level of the black and white reference point and corrected as a peculiar image different from other images, and By displaying it, it is possible to reliably prevent meaningless feature extraction and / or a similar classification operation of these features.

When capturing an image with a slight difference in gray level, a black and white reference value is set individually for each image sensor on the light-receiving surface of the camera so that gray scale correction can be selected for each image sensor. The camera image value when lost is the black reference value for each pixel, while the camera image value when the light source light is radiated to the separately provided white reference paper is the white reference value for each pixel,
By performing the scaling correction calculation so that the image value of these black and white reference values becomes a predetermined grayscale gradation value, it is possible to accurately grasp the characteristics of the minute grayscale level image. Further, it is also possible to set a plurality of black-and-white reference values corresponding to the light source light to be measured and / or the wavelength band of light emission and reception, and select and use these for each measurement condition to correct.

Each of the images captured by the camera is corrected by the type of camera that captured each image, its exposure time, the coordinate values and reference values of both black and white reference points, the type of light source lamp, the irradiation wavelength and the imaging wavelength. By storing together with the black and white reference values and the measurement conditions such as the pixel size (resolution) of the acquired image, the characteristics of the captured images obtained under different imaging conditions, for example, the images corrected with different black and white reference values It becomes possible to perform extraction, comparison of these images, confirmation of reproduction of the image before correction, and the like.

(8) A sample holder for holding a sample, irradiation means for selectively irradiating the sample with a plurality of types of light as inspection light, and reflected light from the sample is imaged in different wavelength bands to obtain respective images. In a feature extraction system that has an image pickup unit that obtains an image corresponding to a wavelength band, and that extracts a feature by performing an arithmetic operation on a captured image of a sample according to a predetermined rule,
The main body unit including the sample holder, the irradiation unit, and the imaging unit is connected to an arithmetic unit that performs a calculation on a captured image of a sample according to a predetermined rule to extract a feature through a communication line, and the main unit unit is connected. Since the obtained picked-up images are collected in the arithmetic device to extract the features, the arithmetic device can be installed at an arbitrary location apart from the main body.

Since the arithmetic unit can be installed without being physically connected to the main body, the arithmetic unit is positioned as a data center to provide a large data processing capability, and even if the hardware becomes large in size, an environment for accepting it can be provided. It is possible to give. As a result, even if enormous amount of feature data is obtained from the sample, the data can be collectively managed for each sample.

The inspection ability of the main body can be diagnosed by re-inspecting the same sample and collating and confirming the degree of coincidence with the previous inspection data. Further, by checking and confirming the degree of coincidence of the inspection data of a large number of samples belonging to the same type, it is possible to grasp the variation in the characteristics between these samples. Also, since the average characteristics of many samples can be obtained,
When it is necessary to extract the characteristics of a new sample and determine the difference,
The collation determination of the characteristics can be performed more accurately.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a feature extraction device according to a first embodiment of the present invention, in which an optical mechanism portion is a cross-sectional view.

FIG. 2 is a horizontal sectional view of the main body of the feature extraction device.

FIG. 3 is a horizontal sectional view similar to FIG. 2, showing a different state.

FIG. 4 is a horizontal sectional view of a light source section.

FIG. 5 is a horizontal sectional view similar to FIG. 4, showing a different state.

FIG. 6 is a front view of a bandpass filter.

FIG. 7 is a block configuration diagram of a feature extraction device.

FIG. 8 is a diagram showing a transition of an image when an image is taken while shifting a wavelength.

FIG. 9 is a diagram for explaining a feature extraction method and shows an image of a difference between images.

FIG. 10 is a table of AD values that express the difference values of images as digital values.

FIG. 11 is a table of indexes that are absolute values of difference values.

Fig. 12 Graph of index

FIG. 13 is a first explanatory diagram of a method of selecting a representative image based on a difference value.

FIG. 14 is a second explanatory diagram of a method of selecting a representative image based on a difference value.

FIG. 15 is a flowchart of image classification work.

FIG. 16 is a view conceptually showing a situation in which captured images of a plurality of cameras are used without correction.

FIG. 17 is a view conceptually showing a situation in which captured images of a plurality of cameras are used after being corrected.

FIG. 18 is a diagram showing a situation in which a black and white reference point for camera correction is set in a sample image.

FIG. 19 is a diagram of a black and white reference marker for camera correction.

FIG. 20: Flowchart of exposure time setting

FIG. 21 is a graph illustrating automatic exposure time setting.

FIG. 22 is a flowchart of black-and-white reference scaling correction.

FIG. 23 is a diagram illustrating black-and-white reference scaling correction.

FIG. 24 is a diagram showing an example in which grayscale correction is performed for each image sensor.

FIG. 25 is a diagram showing an example of an imaging log.

FIG. 26 is a configuration diagram of a feature extraction system according to a second embodiment of the present invention.

[Explanation of symbols]

1 Feature extraction device 2 body 3 arithmetic unit 10 housing 11 Optics section 12 Control section 23 Sample holder 30 Inspection light generator 31 light source 38 Spectrometer 50 Imaging device 51 CCD camera 52 infrared camera 57 Spectrometer 58 bandpass filter 87 Image analysis unit 91 White reference point 92 Black reference point 93 White reference marker 94 Black reference marker 1A Feature extraction device 400 communication line 500 arithmetic unit

Front page continuation (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 1/04 101 H04N 1/04 101 1/19 G07D 7/12 1/407 7/20 // G07D 7/12 H04N 1/04 103E 7/20 1/40 101E (72) Inventor Toshinori Tanaka 2-4 Toranomon, Minato-ku, Tokyo Inside the Printing Bureau, Ministry of Finance (72) Inventor Jun Kuribayashi 2-3-15 Nishigahara, Kita-ku, Tokyo No. Ministry of Finance Printing Bureau Takinogawa Factory (72) Inventor Shigeru Omatsu 5-19-1 Hokushicho, Sakai City, Osaka Prefecture (Reference) 3E041 AA01 AA03 BB03 BB06 EA03 5B047 AA04 AB02 BA08 BB04 BC05 BC07 BC09 BC11 BC14 CA19 CB22 DC01 DC07 5B057 AA11 BA02 BA08 BA12 BA15 CA01 CA08 CA12 CA16 CB08 CE11 DC22 DC30 DC32 DC36 5C072 AA01 BA05 CA02 CA11 DA02 DA04 DA09 DA21 DA30 EA05 EA08 FB18 RA15 UA01 UA17 VA10 5C077 LL04 MP01 PP15 PP44 PP45 PQ PQ PPQ PPQ

Claims (8)

[Claims] 1. A feature extraction device for extracting features by performing a calculation on a sample image according to a predetermined rule, wherein the image capturing device for capturing the sample image includes a plurality of cameras having different image-capable wavelength bands in a wavelength overlapping region. The feature extraction apparatus is characterized in that the camera is corrected so that the degree of approximation of images captured by both cameras at the switching point becomes equal to or higher than a predetermined level. 2. An image corresponding to each wavelength obtained by holding a sample on a sample holder, irradiating the sample with inspection light generated by an inspection light generator, and imaging the sample surface in different wavelength bands by the imaging device. The feature extraction device according to claim 1, wherein 3. The sample is irradiated with a plurality of types of light having different wavelengths stepwise as inspection light.
Alternatively, the feature extraction device according to item 2. 4. A white reference point indicating a brightness level is provided in a captured image, and a gray level value of the reference point of sequentially obtained captured images is within a predetermined allowable error value with respect to a preset white level value. The feature extraction device according to claim 1, further comprising an image pickup unit controlled to have the same brightness. 5. A sample holder for holding a sample, an irradiation unit for irradiating the sample with a plurality of types of light having different wavelengths stepwise as inspection light, and the sample surface is imaged in different wavelength bands to obtain respective wavelengths. In a feature extraction device that has an image pickup unit that obtains an image corresponding to a band, and that extracts a feature by performing an arithmetic operation on a captured image of the sample according to a predetermined rule, a brightness level in the image captured by the sample holder. In addition to providing a white reference point indicating the and a black reference point indicating the darkness level,
Operational image capturing means for sequentially obtaining camera image data while performing expansion / reduction correction operation on the grayscale value of the captured image at the same ratio so that each of the white level value and the black level value of these black and white reference points becomes a predetermined grayscale level value. A feature extraction device comprising: 6. A preceding picked-up image display means for displaying the picked-up image by performing the preceding pick-up under a predetermined shooting condition in the characteristic pick-up of the sample surface, and a white reference point on the image displayed on the preceding picked-up image display means. And a reference point coordinate setting input means for setting the coordinates of the black reference point, and the level values of the black and white reference points whose coordinates are set by the setting input means are scaled by the same ratio of the captured image gray value obtained thereafter. The feature extraction device according to claim 5, wherein the feature extraction device is used as an index for correction calculation. 7. The calculation image pickup means obtains the difference between the black and white level values of the black and white reference points in the imaged image obtained from the sample surface, and when the difference value is less than or equal to a preset value, the black and white level value. In addition, the scaling correction calculation is performed on the second predetermined level value obtained by adding and subtracting the predetermined setting value to prevent excessive gradation correction calculation of a captured image with a small difference in gray level. The feature extraction device according to claim 5. 8. A sample holder for holding a sample, an irradiation means for selectively irradiating the sample with a plurality of types of light as inspection light, and reflected light from the sample are imaged in different wavelength bands to obtain respective wavelengths. In a feature extraction system that has an image capturing unit that acquires an image corresponding to a band, and that extracts a feature by performing an operation on a captured image of a sample according to a predetermined rule, the sample holder, the irradiation unit, and the image capturing unit. The main body provided is connected to a computing device that performs a calculation on the image of the sampled image according to a predetermined rule to extract features, and the captured images obtained by the main body are collected in the computing device. The feature extraction system is characterized in that the feature extraction is performed by using the feature extraction system.