JP2009076012A - Object identification device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object identification device improved in object identification accuracy. <P>SOLUTION: The object identification device has imaging means for generating image data obtained by imaging an object and indicated by the light receiving intensity of each of a plurality of wavelength bands of near infrared light and visible light; area information extracting means for extracting area information indicated by information on dimensionality corresponding to the number of wavelength bands while indicating information of every wavelength band of every area; conversion means converting each area information extracted by the area information extracting means into feature information indicated by information on dimensionality not more than the above dimensionality while indicating the feature of the object corresponding to the area; and object identifying means for identifying an object shown by the object corresponding to each area based on the feature information converted by the conversion means and sample feature information which associates the object to be identified with a range where feature information corresponding to the object exists. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an object identification device, and more particularly, to an object identification device that identifies an object existing around a vehicle.

  2. Description of the Related Art Conventionally, various devices that support the vision of a driver driving a vehicle by identifying an object from captured image data have been proposed. The object extraction device disclosed in Patent Document 1 calculates feature amounts of each image from a far-infrared image and a visible image, and classifies a multidimensional space of feature amounts into several clusters by clustering. The region is divided according to the distribution of the feature amount in the feature amount space.

In the multicolor infrared imaging device disclosed in Patent Document 2, infrared radiant energy from a subject is incident on a sensor of the multicolor infrared imaging device via an optical filter in a plurality of wavelength band ranges, and stored in a storage unit in advance. The true temperature T0 and the background temperature Ta of the subject are obtained based on the emissivity values of the plurality of substances, and the materials or states of the plurality of substances are discriminated.
JP 2000-306191 A JP 2004-117309 A

  However, since the above prior art uses far-infrared images, an expensive infrared camera different from the camera that captures a visible image is required. In addition, it is technically impossible to acquire a far-infrared image and a visible image with one camera at present, and two cameras are required, resulting in a large apparatus.

  In addition, when a far-infrared image is used, since the thermal radiation amount of an imaging target is measured, a person or a vehicle can be detected from its temperature characteristics. However, there are problems that it is difficult for humans to detect in summer when the temperature rises to the equivalent of body temperature, and that effective features cannot be obtained from road surfaces and roadside artifacts without heat radiation.

  As described above, in the conventional technique, it is difficult to say that the object identification accuracy is high for the reason described above.

  In view of the above problems, an object of the present invention is to provide an object identification device with improved object identification accuracy.

  In order to achieve the above object, the invention of claim 1 is image data obtained by imaging a subject, and is indicated by received light intensities of light in a plurality of wavelength bands of near infrared light and visible light. An imaging unit that generates image data, a subject image indicated by the image data generated by the imaging unit is divided into a plurality of regions, information for each wavelength band for each region is shown, and the wavelength band Area information extracting means for extracting area information indicated by information of the number of dimensions according to the number of areas, and each area information extracted by the area information extracting means indicates the characteristics of the subject corresponding to the area, A sampler in which conversion means for converting to feature information indicated by information of a dimension number equal to or less than the number of dimensions, an object to be identified, and a range in which the feature information corresponding to the object exists are associated Based on the storage means in which feature information is stored in advance, the feature information converted by the conversion means, and the sample feature information stored by the storage means, an object indicated by the subject corresponding to each of the regions is displayed. Object identifying means for identifying.

  According to the first aspect of the present invention, the imaging means is image data obtained by imaging the subject, and an image indicated by the received light intensity of each of light in a plurality of wavelength bands of near infrared light and visible light. The region information extraction unit divides the subject image indicated by the image data generated by the imaging unit into a plurality of regions, shows information for each wavelength band for each region, and generates the data. The area information indicated by the information of the number of dimensions according to the number of bands is extracted, and the conversion means indicates each area information extracted by the area information extraction means, indicating the characteristics of the subject corresponding to the area, and Sample feature information in which the object to be identified is associated with the range in which the feature information corresponding to the object exists is converted to feature information represented by information of the number of dimensions less than or equal to the number of dimensions. Are stored in advance, and the object identifying means identifies the object indicated by the corresponding subject for each region based on the feature information converted by the converting means and the sample feature information stored by the storage means . In this way, by using near-infrared light having different reflection characteristics depending on the material of the object and its state, an object identification device with improved object identification accuracy can be provided.

  The invention of claim 2 further comprises edge extraction means for extracting an edge from the subject image indicated by the image data in the invention of claim 1, wherein the object identification means is extracted by the edge extraction means. An object is identified for each region surrounded by edges.

  According to the second aspect of the present invention, since the region surrounded by the edge is likely to be the same object, the object identification device that improves the object identification accuracy by identifying the object for each region surrounded by the edge Can be provided.

  The invention according to claim 3 is the invention according to claim 1 or 2, wherein the sample feature information is obtained from image data generated by photographing an object to be identified in advance by the imaging means. Further, higher-order local autocorrelation feature information is further included, and the object identifying means further uses the higher-order local autocorrelation feature information to identify an object indicated by the subject corresponding to the region.

  According to the third aspect of the present invention, it is possible to provide an object identification device with improved object identification accuracy by using higher-order local autocorrelation feature information.

  According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, the object identifying means is distributed among elements of each dimension in a plurality of feature information included in the sample feature information. The object indicated by the subject is identified without using a dimension including an element whose value is smaller than a predetermined value.

  According to the invention of claim 4, the processing time required for identification can be shortened by reducing the number of dimensions.

  The invention according to claim 5 is the invention according to any one of claims 1 to 3, wherein the converting means converts the feature information obtained by converting the region information into two or more dimensions of the feature information. Is further converted into compressed feature information having a one-dimensional value obtained by linearly transforming an element corresponding to the object, and the object identifying means includes the compressed feature information and the sample feature information stored in the storage means. Based on the above, the object indicated by the subject corresponding to each of the areas is identified.

  According to the invention of claim 5, the processing time required for identification can be shortened by reducing the number of dimensions.

  The invention of claim 6 is the invention of any one of claims 1 to 5, wherein the region information is information indicating the received light intensity of red light, green light, blue light, and near infrared light. is there.

  According to the invention of claim 6, the received light intensity of red light, green light, blue light, and near infrared light can be used as the region information.

  According to the present invention, it is possible to provide an object identification device with improved object identification accuracy.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of an object identification device according to the present embodiment. The imaging unit 42 of the object identification device 10 includes an optical unit 22 configured to include a filter capable of receiving light in a plurality of wavelength bands including near infrared light for each wavelength band, and an optical axis of the optical unit 22. A CCD image sensor (hereinafter referred to as “CCD”) 24 disposed at the rear, an analog signal processing unit 26 for performing various analog signal processing on the input analog signal, and the input analog signal as digital An analog / digital converter (hereinafter referred to as “ADC”) 28 that converts data into data and a digital signal processing unit 30 that performs various types of digital signal processing on the input digital data are included. . The imaging unit 42 may be a CMOS camera.

  The digital signal processing unit 30 also has a built-in line buffer having a predetermined capacity, and performs control for directly storing the input image data in a predetermined area of the memory 48.

  The output terminal of the CCD 24 is connected to the input terminal of the analog signal processing unit 26, the output terminal of the analog signal processing unit 26 is connected to the input terminal of the ADC 28, and the output terminal of the ADC 28 is connected to the input terminal of the digital signal processing unit 30. . Accordingly, the analog signal indicating the subject image output from the CCD 24 is subjected to predetermined analog signal processing by the analog signal processing unit 26, converted into image data as digital data by the ADC 28, and then input to the digital signal processing unit 30. The

  On the other hand, the object identification device 10 controls a central processing unit (CPU) 40 that controls the overall operation of the object identification device 10, a memory 48 that stores digital image data obtained by photographing, and access to the memory 48. A memory interface 46, an LCD (Liquid Crystal Display) 38, and an LCD interface 36 that controls access to the LCD 38 are included.

  In the object identification device 10 of the present embodiment, a VRAM (Video RAM) and a ROM are used as the memory 48. In the ROM, for example, a flash memory is used, and an OS, a program, and sample feature information described later are stored.

  The digital signal processing unit 30, the CPU 40, the LCD interface 36, and the memory interface 46 are connected to each other via a system bus BUS. Therefore, the CPU 40 can control the operation of the digital signal processing unit 30, access the memory 48 via the memory interface 46, and control the LCD 38 via the LCD interface 36. By accessing the memory 48, image data for each RGBI is generated from the image data.

  Next, the filter will be described with reference to FIG. As shown in FIG. 2, the filter used in the present embodiment transmits a red transmission filter 50 that transmits R (red light), a green transmission filter 52 that transmits G (green light), and B (blue light). The blue transmission filter 54 and the near-infrared light transmission filter 60 that transmits I (near-infrared light) are alternately arranged.

  As shown by the bold rectangle, the filter 58 composed of the four filters corresponds to one pixel.

  The imaging unit 42 and an image generation unit (to be described later) are image data obtained by imaging a subject, and image data indicated by received light intensities of light in a plurality of wavelength bands of near infrared light and visible light. It is designed to generate.

  As described above, in this embodiment, a filter that transmits near-infrared light is used. However, near-infrared light depends on the molecular structure of the irradiated object and its motion, and light in a specific wavelength band is emitted. It has the property of being absorbed. In other words, near-infrared light reflection characteristics vary depending on the material of the object and its state. By utilizing this property, it is possible to know the material and state of the object irradiated with light by analyzing the near-infrared spectrum from the object.

  The CCD 24 that receives visible light by the above-described filter needs sensitivity necessary for imaging even in a part of the near-infrared region. Therefore, a single camera using the filter can emit both visible light and near-infrared light. It can also be acquired and can be easily downsized. Furthermore, near-infrared light is not thermal radiation from an object, but is reflected light on the surface of the object like visible light, but its reflectivity shows different characteristics from visible light depending on the material of the object. It is very effective for identification.

  As described above, in this embodiment, not only visible light but also near infrared reflection characteristics can be used, so that the object identification performance can be improved. In particular, plants (leaves) and human skin (substances containing water) are characterized by near-infrared reflection characteristics, so that the discrimination performance can be greatly improved compared to the case of using only visible light.

  Furthermore, by using a filter in which the above-described four filters are alternately arranged, RGBI images having the same viewing angle can be acquired without causing a time shift. In addition, since they have the same viewing angle, pixel correspondence between images is facilitated.

  In addition to the above-described filter, a configuration using a prism such as a 3CCD configuration may be used. Alternatively, an imaging element in which a plurality of types of photodiodes capable of receiving different wavelength bands are stacked in the depth direction may be used.

  Next, the program configuration of the object identification device 10 will be described with reference to FIG. The object identification device 10 includes an image generation unit 62, a region information extraction unit 64, a conversion unit 66, and an object identification unit 68.

  The image generation unit 62 and the imaging unit 42 image a subject through the above-described filters that can receive light in a plurality of wavelength bands including near-infrared light for each wavelength, and obtain image data indicating the subject. Generate.

  The area information extraction unit 64 divides the subject image indicated by the image data generated by the image generation unit 62 into a plurality of areas, shows information for each wavelength band for each area, and according to the number of wavelength bands. The area information indicated by the information on the number of dimensions is extracted.

  The above-mentioned plurality of regions indicates, for example, one region with one pixel or one region with a plurality of pixels. In addition, in the case of the present embodiment, the region information when one pixel is one region is four-dimensional information because there are four wavelength bands of RGBI, and the element is the received light intensity of each RGBI. It is. In addition, a case where a plurality of pixels form one region is also four-dimensional information, and the feature information of the region may be an average value of RGBI received light intensity in each pixel.

  The conversion unit 66 indicates each region information extracted by the region information extraction unit 64 with the feature of the subject corresponding to the region and information on the number of dimensions below the number of dimensions (four dimensions in the present embodiment). To feature information. Details of this conversion will be described later.

  The object identification unit 68 identifies the object indicated by the corresponding subject for each area based on the feature information converted by the conversion unit 66 and sample feature information described later stored in the memory 48.

  The above is the program configuration of the object identification device 10. Next, details of the conversion and sample feature information will be described.

  First, details of the conversion will be described. FIG. 4 is a diagram illustrating a conversion formula used in the conversion unit 66. FIG. 6A shows a conversion formula from (I, R, G, B) indicating region information to feature information (P1, P2, P3, P4) in the feature space. The feature space is a main space generated by principal component analysis from the received light intensity (I, R, G, B) of sample points extracted in advance as shown in FIG. 5 from image data obtained by photographing an object to be identified. A space composed of components. Note that the space shown in FIG. 5 is drawn in two dimensions for easy understanding.

  In this feature space, the feature information obtained from the object to be identified is distributed as a whole as shown in FIG. The statistical information obtained by organizing the feature information distributed in this way is the sample feature information described above.

  This sample feature information is stored in the memory 48 in a database format as shown in FIG. As shown in the figure, the sample feature information is obtained by associating an object to be identified with a range in which feature information corresponding to the object exists.

  Specifically, for example, in the case of a road, a1 ≦ P1 ≦ b1, c1 ≦ P2 ≦ d1, e1 ≦ P3 ≦ f1, and g1 ≦ P4 ≦ h1 are ranges in which feature information exists. When the feature information obtained by converting the region information is included in this range, the object indicated by the subject corresponding to the region is identified as a road. The sample feature information of the road is shown by an inequality, but it may be a set of points obtained by converting sample points.

  Returning to FIG. 4, the 4 × 4 square matrix shown in FIG. 4A is a matrix in which four eigenvectors obtained by the principal component analysis are arranged. This square matrix is normalized so that the sum of norms of four eigenvectors is 1.

  The conversion unit 66 converts the region information into the feature information using the formula shown in FIG. 4A, but converts the region information (I, R, G, B) as shown in FIG. The value obtained by linearly transforming the feature information (P1, P2, P3, P4) into elements corresponding to two or more dimensions of the feature information (P1, P2, P3, P4) It may be further converted into the compressed feature information.

  In the case of (B) in the figure, the value obtained by linearly converting P2 and P3 to 0.5 × P2 + 0.5 × P3 is one dimension value (R1, R2, R3) as the compression feature information. It is shown.

  Further, the object identifying unit 68 does not use a dimension including an element whose variance value is smaller than a predetermined value in each dimension element in the feature information included in the sample feature information obtained by converting the sample point. You may make it identify the object which a to-be-photographed object shows. The predetermined value may be appropriately determined by performing an experiment or the like, or the value may be arbitrarily set. By reducing the number of dimensions in this way, the processing time required for identification can be shortened.

  Hereinafter, the object identification process will be described with reference to FIG. 8 using a flowchart. In the following flowchart, the procedure for identifying each pixel is described. However, as described above, the screen may be divided into arbitrary regions and identification may be performed for each region composed of a plurality of pixels. . In this case, as described above, if the pixel values included in the region are averaged and the received light intensity (R, G, B, I) of the region is calculated, the identification result can be obtained in the same procedure.

  FIG. 8 shows an RGB image, an I image, an edge image, an identification image (part 1), and an identification image (part 2).

  Among these, the RGB image is an image obtained by the light transmitted through the red transmission filter 50, the green transmission filter 52, and the blue transmission filter 54 among the filters described in FIG. 2. The I image is an image obtained by the light transmitted through the near infrared light transmission filter 60.

  The edge image is an RGB image or an image obtained by edge extraction processing conventionally used from an I image. The identification image (No. 1) shows the identification result for each pixel. The identification image (No. 2) shows the identification result obtained by correcting the identification result inappropriately determined by noise or the like for the identification image (No. 1) for each region surrounded by the edges and identifying the object.

  In this embodiment, the identification result is “road”, “plant”, “building”, “sky”, “other”, but is not limited to this, and a part of the object or more The object may be used as the identification result.

  Based on these, first, the flowchart of FIG. 9 will be described. The flowchart shown in the figure is a flowchart showing the object identification process.

  First, in step 101, the image generation unit 62 generates image data indicating the RGB image and the I image shown in FIG. In the next step 102, the region information extraction unit 64 performs region information extraction processing. In step 103, the conversion unit 66 performs a conversion process. In the next step 104, the object identification unit 68 performs an object identification process and ends the process.

  Next, the region information extraction processing will be described using the flowchart of FIG. First, in step 201, the loop counter k is initialized with zero. In the next step 202, R received light intensity r, G received light intensity g, B received light intensity b, I received light intensity i, that is, region information in the pixel D (k) is calculated. In the next step 203, the calculated region information (r, g, b, i) is substituted into the array F (k), and in step 204, k is incremented by one.

  In the next step 205, it is determined whether k <M (M is the total number of pixels). If the determination is affirmative, there is pixel region information that has not yet been assigned to the array F (k). If the process of 202 is performed and a negative determination is made, the area information of all pixels is substituted for F (k), and the process ends.

  By this process, the area information is substituted into F (k), and thus the process of extracting the area information is performed.

  Next, the conversion process by the conversion unit 66 will be described using the flowchart of FIG. First, at step 301, the loop counter k is initialized with zero. In the next step 302, AF (k) is substituted for P (k). Here, A is a 4 × 4 square matrix shown in FIG. 4 (A), F (k) is the above-mentioned array regarded as a 4 × 1 matrix, and P (k) is characteristic information. Show.

  In the next step 303, k is incremented by one. In the next step 304, it is determined whether or not k <M (M is the total number of pixels). If the determination is affirmative, it is still substituted into the feature information P (k). Since there is feature information of pixels that are not present, the process of step 302 is performed again. When a negative determination is made, the feature information of all the pixels is substituted into P (k), and the process ends.

  Next, the object identification processing by the object identification unit 68 will be described using the flowchart of FIG. In this flowchart, it is assumed that the sample feature information further includes higher-order local autocorrelation feature information obtained from image data generated by photographing an object to be identified. This high-order local autocorrelation feature information is a feature amount indicating surface information (texture information) calculated from images in each RGBI wavelength band, and this information can be used to more accurately determine what each divided region is. It becomes possible to identify.

  First, in step 401, the loop counter k is initialized with zero. In the next step 402, P (k) is compared with the sample feature information. The comparison here indicates finding sample feature information that includes P (k) in the range.

  In the next step 403, it is determined whether P (k) is within the range of two or more pieces of sample feature information. Since the ranges of the sample feature information described above may overlap, in this flowchart, the comparison is further performed using the higher-order local autocorrelation feature information in step 404, and the process proceeds to step 405. On the other hand, if a negative determination is made in step 403, the process proceeds to step 405.

  Note that, when the higher-order local autocorrelation feature information is not used as in step 404, any one of the sample feature information including P (k) is sample feature information including P (k). The sample feature information of a range having the closest center of P (k) among the centers of two or more ranges that are fixedly selected as sample feature information including P (k) You may make it do.

  In the next step 405, the comparison result in step 403 or 404 is set as the object indicated by the kth pixel.

  In the next step 406, k is incremented by one. In the next step 407, it is determined whether or not k <M (M is the total number of pixels). If the determination is affirmative, there is a pixel that has not yet been identified. When the process of step 402 is performed again and a negative determination is made, since all the pixels have been identified, an edge extraction process is performed at step 408.

  The identification image at that time is the identification image (No. 1) shown in FIG. Since there are objects that are identified as objects different from the actual object due to noise or the like, there are some cases where the identification results are different from the identification results for the surrounding pixels.

  In the next step 409, the most frequent identification result in the area surrounded by each edge is set as an object indicated by the area. An identification image (part 2) shown in FIG. 8 shows the identification result thus obtained. An image in which an object is identified with higher accuracy can be obtained from this identification image (part 2).

  As described above, in the present embodiment, not only people and vehicles but also roads, sky, buildings, plants, and the like shown in the image based on images of a plurality of wavelength bands for the visible to near-infrared region. Even the background can be identified.

  In addition, the identification result (identification image (part 2) in FIG. 8) obtained by the present embodiment may be provided to the user through the LCD 38, but input from another detection device such as a pedestrian detection device or a vehicle detection device. It is also effective to use it as information.

  For example, in the pedestrian detection device, when the identification result is obtained, the search range of the pedestrian can be limited to only the region identified as “others”. This is because the other areas are identified as any one of “road”, “plant”, “building”, and “sky”, and there is no pedestrian among them. When used in this manner, the search range is limited, which is very effective in terms of reducing the calculation cost and the false detection rate in the pedestrian detection device.

  In the embodiment described above, RGB is used, but other color spaces such as HSV may be used.

It is a figure which shows the structure of the object identification apparatus which concerns on this Embodiment. It is a figure which shows a filter. It is a figure which shows the program structure of an object identification apparatus. It is a figure which shows the conversion formula used in a conversion part. It is a figure which shows the example of a space comprised by the main component produced | generated by the principal component analysis. It is a figure which shows the example of distribution of feature information. It is a figure which shows sample characteristic information. It is a figure which shows the image regarding an object identification process. It is a flowchart which shows an object identification process. It is a flowchart which shows area | region information extraction processing. It is a flowchart which shows a conversion process. It is a flowchart which shows an object identification process.

Explanation of symbols

10 Object Identification Device 22 Optical Unit 24 CCD
26 Analog signal processing unit 30 Digital signal processing unit 36 LCD interface 38 LCD
40 CPU
42 Imaging unit 46 Memory interface 48 Memory 50 Red transmission filter 52 Green transmission filter 54 Blue transmission filter 58 Filter 60 Near infrared light transmission filter 62 Image generation unit 64 Area information extraction unit 66 Conversion unit 68 Object identification unit

Claims (6)

Imaging means for generating image data that is image data obtained by imaging a subject and that is indicated by the received light intensity of each of light in a plurality of wavelength bands of near-infrared light and visible light; and
The subject image indicated by the image data generated by the imaging unit is divided into a plurality of regions, information for each wavelength band for each region is shown, and information on the number of dimensions according to the number of wavelength bands Area information extraction means for extracting area information indicated by:
Conversion means for converting each area information extracted by the area information extraction means into feature information indicated by information of the number of dimensions equal to or less than the number of dimensions, indicating the characteristics of the subject corresponding to the area;
Storage means in which sample feature information in which an object to be identified is associated with a range in which the feature information corresponding to the object exists is stored in advance;
Object identifying means for identifying an object indicated by a corresponding object for each of the regions based on the feature information converted by the converting means and the sample feature information stored by the storage means;
An object identification device. An edge extracting means for extracting an edge from the subject image indicated by the image data;
The object identification device according to claim 1, wherein the object identification unit identifies an object for each region surrounded by the edge extracted by the edge extraction unit. The sample feature information further includes higher-order local autocorrelation feature information obtained from image data generated by previously capturing an object to be identified by the imaging unit.
The object identification device according to claim 1, wherein the object identification unit further identifies the object indicated by the subject corresponding to the region using the higher-order local autocorrelation feature information.   The object identifying means identifies an object indicated by the subject without using a dimension including an element whose variance value is smaller than a predetermined value among the elements of each dimension in the plurality of feature information included in the sample feature information. The object identification device according to any one of claims 1 to 3. The converting means further converts the feature information obtained by converting the region information into compressed feature information in which a value obtained by linearly converting elements corresponding to two or more dimensions of the feature information is a one-dimensional value. Converted,
4. The object identification unit according to claim 1, wherein the object identification unit identifies an object indicated by a corresponding subject for each of the regions based on the compression feature information and the sample feature information stored by the storage unit. 5. The object identification device according to claim 1.   The object identification device according to claim 1, wherein the region information is information indicating received light intensity of red light, green light, blue light, and near infrared light.