JP5111152B2 - Object recognition device - Google Patents

Description

  The present invention relates to an object recognition apparatus that increases the resolution of a photographed image of a photographing means by image processing so that an obstacle such as a distant preceding vehicle can be well recognized from the photographed image after image processing.

  Conventionally, in order to provide driving support such as follow-up driving, collision damage reduction, automatic steering control, etc., a camera is mounted on the vehicle as a photographing means, and the front and rear of the vehicle are monitored by images taken by this camera. Various object recognition devices for recognizing obstacles on the road including moving objects such as passing vehicles from the rear and stationary objects are being researched and developed and put into practical use.

  By the way, the resolution of an object farther than, for example, 50 m or more in the photographed image of the camera decreases. Therefore, it is difficult to recognize an object such as a vehicle that is a distant obstacle from a captured image (original image) of the camera.

  And since there are physical limitations and economic limitations in increasing the resolution of the camera, this kind of object recognition device increases the resolution of the captured image by image processing called super-resolution processing, and becomes a distant obstacle. It is considered to recognize an object such as a vehicle.

  In the proposed super-resolution processing, the pixel position of the object movement between frames is specified by the image processing of the correlation method for the captured images of a plurality of consecutive frames of the camera, and the pixel values before and after the movement are added and averaged. Thus, each pixel value after movement is obtained in units of pixels smaller than one pixel before movement, and the resolution of the captured image is increased (for example, see Patent Document 1).

Then, it becomes possible to detect and recognize an object that becomes a distant obstacle from the captured image that has been increased in resolution by the super-resolution processing by an edge histogram method described later.
Japanese Patent Laying-Open No. 2006-318272 (paragraphs [0007]-[0015], FIGS. 1-4, etc.)

  In the case of the conventional super-resolution processing disclosed in Patent Document 1, a plurality of frames of captured images are required to form a high-resolution image of one frame, and the amount of images is large, and the processing takes time. In addition, the processing is complicated and the memory consumption increases, and there is a problem that the resolution of a captured image cannot be easily increased in a short time by simple image processing.

  In addition, complicated calculation processing of correlation between frames is necessary, and errors in high resolution may occur due to calculation errors and the like, leading to erroneous recognition.

  The present invention provides an unprecedented breakthrough super-resolution process to easily and rapidly increase the resolution of a captured image of a photographing means mounted on the own vehicle in a short time by simple image processing. An object of the present invention is to make it possible to recognize an object such as a vehicle that is a distant obstacle from an image.

In order to achieve the above-described object, an object recognition apparatus of the present invention is equipped with a photographing unit mounted on a host vehicle for photographing the outside of the vehicle, and a wavelet inverse transform is performed on the photographed image of the photographing unit so that the photographed image has a high resolution. High-resolution means for recognizing, recognition processing means for recognizing an object in a high-resolution image obtained by the high-resolution of the high-resolution means, and a vertical direction and a horizontal direction about the outline portion of the object in the captured image. Extraction means for extracting a high-frequency component in at least one of the directions, and the wavelet inverse transform of the high-resolution means is performed by adding the high-frequency component extracted by the extraction means to the photographed image, For the high frequency component, the ratio of the resolution of the high resolution image to the resolution of the captured image is K, and the line width of the outline of the high frequency component is narrowed to 1 / K. Is characterized by use of the inverse wavelet transform (claim 1).

Further, the object recognition device of the present invention includes a photographing unit mounted on the own vehicle for photographing the outside of the vehicle, and a high resolution unit for performing a wavelet inverse transform on the photographed image of the photographing unit to increase the resolution of the photographed image. Recognizing processing means for recognizing an object in a high-resolution image obtained by increasing the resolution of the high-resolution means, and at least one of a vertical direction and a horizontal direction with respect to an outline portion of the object in the captured image. Extraction means for extracting a high-frequency component of the image, and the wavelet inverse transform of the high-resolution means is performed by adding the high-frequency component extracted by the extraction means to the photographed image, and the extraction means comprises the photographing means vertical captured image 2 frames before and after may be the one for extracting at least one direction of the difference in the horizontal direction as the high-frequency component (claim 2 ).

  In the case of the invention of claim 1, assuming that the captured image of each frame of the imaging means mounted on the host vehicle is a low-resolution image obtained by wavelet transformation, the captured image is subjected to the superposition of wavelet inverse transformation of the high-resolution means. Resolution processing is performed, and the resolution of the captured image can be increased by image processing of inverse wavelet transform.

  In this case, a high-resolution one-frame image can be formed from a single-frame captured image, the amount of images to be processed is small, and the high-resolution of the captured image can be easily achieved in a short time with simple image processing. it can. In addition, complicated calculation of correlation between frames is unnecessary, and the recognition processing unit can recognize an object such as a distant vehicle in a high-resolution captured image.

  Therefore, by using an unprecedented revolutionary super-resolution process that performs inverse wavelet transform, the captured image of the camera mounted on the vehicle can be improved in a short time with simple image processing, and it is far from the captured image It is possible to detect and recognize an object such as a vehicle that becomes an obstacle to the vehicle without error.

The high frequency component extracted by the extracting unit is because the ratio of the resolution of the high resolution image for the resolution of the captured image as K, was Ru frequency component der the narrow image the line width of the contour line to 1 / K Further, a clearer and more practical high-resolution image can be restored by the wavelet inverse transform of the high-resolution means.

In the second aspect of the invention, the specific extraction configuration of the high-frequency component by the extraction unit requires two frames of the captured image before and after the imaging unit, but it is complicated as in the image processing of the conventional correlation method. The high-frequency component can be extracted by a simple method for obtaining a difference in at least one of the vertical direction and the horizontal direction of the captured images of the two frames before and after the photographing unit, instead of a simple process.

  Next, in order to describe the present invention in more detail, an embodiment will be described in detail with reference to FIGS. Note that the object to be recognized may be various distant moving objects or stationary objects that may become obstacles to traveling such as in front of or behind the host vehicle, but in the following embodiments, the description will be simplified. For this reason, the object to be recognized is assumed to be a preceding vehicle (front vehicle) that travels in front of the host vehicle, for example, on the same lane more than 50 m away.

(First embodiment)
First , a first embodiment will be described with reference to FIGS.

  FIG. 1 shows a block configuration of an object recognition device 2 mounted on the own vehicle 1, and FIG. 2 is a flowchart for explaining its operation. FIG. 3 is an explanatory diagram of basic wavelet transformation, and FIG. 4 is an explanatory diagram of basic wavelet inverse transformation.

  FIG. 5 and FIG. 6 are explanatory diagrams of recognition processing of the edge histogram method of the recognition processing means, where (a) is a captured image, (b) is an edge histogram image, (c) is an edge image, and (d) is an edge image. It is a divided image obtained by dividing a captured image into a plurality of vehicle regions.

  FIG. 7 shows the relationship between the contour component in the vertical direction (longitudinal direction) of the image and the frequency component of the wavelet transform, (a) shows an example of an input image of wavelet transform, and (b) shows an example of the frequency component.

  8 to 11 are explanatory diagrams of examples of inverse wavelet transform of the resolution enhancement means. FIG. 12 shows each image for explaining the conditions of the experimental example, (a) is a reference high-resolution image, (b) is a degraded image with degraded resolution, and (c) is an HL component input to the wavelet inverse transform. It is an image. FIG. 13 shows experimental results, where (a) shows the result of inverse wavelet transformation by inputting the HL component, and (b) shows the result of inverse wavelet transformation without inputting the HL component.

[Constitution]
As shown in FIG. 1, the object recognition device 2 mounted on the host vehicle 1 is formed by a photographing means 3, an image processing means 4 for processing the photographed image, and a recognition processing means 5 at the subsequent stage.

  The photographing means 3 is, for example, a monocular camera having a semiconductor image sensor mounted so as to photograph the front of the own vehicle 1 at the base of a rearview mirror of the own vehicle 1 and can be continuously photographed in time. While the car 1 is traveling, for example, every 1/30 second frame, a color or monochrome photographed image (original image) of the latest front frame is output to the image processing means 4.

  The image processing means 4 is formed by a microcomputer together with the recognition processing means 5 and forms the high resolution means of the present invention.

  Then, the high resolution means performs image processing of wavelet inverse transformation on the photographed image of the photographing means 3 to increase the resolution of the photographed image of each frame.

  Next, image processing for wavelet inverse transformation of the high resolution means will be described.

  In general, an image can be decomposed into a lower resolution image and a plurality of frequency components. This means that a high-resolution original image can be restored from the low-resolution image and the frequency component, and the decomposition and restoration of the image is performed by wavelet transform (WT) and inverse wavelet transform (IWT). realizable.

  The wavelet transform is basically a transform that attempts to express the waveform of a given input by adding and scalinging the wavelet function on the time axis, translating, and adding discrete data such as image data. The wavelet transform for data is called discrete wavelet transform.

  The wavelet transform of the image data is a two-dimensional discrete wavelet transform. This two-dimensional discrete wavelet transform uses a wavelet function and a scaling function, and uses the input image's vertical (vertical) and horizontal (horizontal) directions. ), A differential high-pass filter (HPF) process for extracting high-frequency components and an integral low-pass filter (LPF) process for extracting low-frequency components.

  Specifically, in the two-dimensional discrete wavelet transform of image data, the input image is subjected to horizontal HPF processing and LPF processing to generate two high and low frequency components (high frequency component H and low frequency component L). Further, the two frequency components are subjected to vertical HPF processing and LPF processing, respectively.

When wavelet transform is performed by performing HPF processing and LPF processing in the vertical direction and horizontal direction of the input image in this way, as shown in FIG. 3, the input image (original image) OGa has frequency components LL and HL. , LH, and HH.
The frequency components LL, HL, LH, and HH are hereinafter referred to as an LL component, an HL component, an LH component, and an HH component in consideration of the generation process and the like.

  The HL component is a high-frequency component in the vertical direction of the input image OGa generated by performing HPF processing in the horizontal direction of the input image OGa and LHP processing in the vertical direction. Similarly, the LH component is in the horizontal direction of the input image OGa. The HH component is a high-frequency component in the vertical and horizontal directions of the input image OGa. The LL component is image data of the low-resolution image SI indicating pixel values at half the resolution of the input image OGa.

  On the other hand, as shown in FIG. 4, the wavelet inverse transform is basically doubled vertically and horizontally by a low resolution image OGb that is an LL component and three frequency components (HL component, LH component, and HH component). This is a conversion for generating a resolution image EI.

  The wavelet transform is a reversible transform. Therefore, data of a low resolution image (corresponding to the low resolution image OGb) generated by wavelet transform of an image I (corresponding to the input image OGa) and data of three frequency components (HL component, LH component, HH component) are obtained. When the wavelet inverse transform is performed using the low resolution image OGb, the resolution is increased and the image I is restored (generated).

  That is, the wavelet transform is a transform that decomposes the target input image OGa into a total of four pieces of data, that is, the LL component of the half-resolution image SI and three frequency components (HL component, LH component, HH component). The inverse wavelet transform, which is the inverse transform, is a super-resolution processing transform that increases the resolution of the target input image OGb.

  Then, when a frequency component (HL component, LH component, HH component) appropriate for wavelet inverse transformation is used, in which the photographed image of the photographing means 3 is used as a low resolution input image OGb, a high resolution image is obtained from the low resolution input image OGb. The EI is generated (restored), and the captured image is increased in resolution.

  At this time, the wavelet transform and the wavelet inverse transform can be performed using one frame of the input images OGa and OGb, respectively. Therefore, if there is a captured image of one frame as the input image OGb, the super-resolution processing for increasing the resolution can be performed by inverse wavelet transform.

Therefore, the high resolution means is a photographic image of each frame of imaging means has captured a low resolution input image OGb, an input image OGb the low resolution, the wavelet inverse transformation with the appropriate frequency components to be described later By enlarging by (WZP method), a super-resolution conversion is performed to generate (restore) an image EI of each frame in which the captured image is increased in resolution.

  Incidentally, the recognition processing means 5 subsequent to the image processing means 4 recognizes an object such as a preceding vehicle from the high-resolution image EI of the high-resolution means obtained by the wavelet inverse transformation. As a technique for this, in the present embodiment, focusing on horizontal and vertical edge components that are included in many areas of the vehicle in the image, a vehicle recognition technique (edge histogram) using an edge histogram that is strong against noise and has a short calculation time. Method).

  The vehicle recognition by the edge histogram method will be described with reference to FIGS.

  For example, FIG. 5A and FIG. 6A show one frame of captured images (actual images) P1 and P2, respectively, and these captured images P1 and P2 respectively show the preceding vehicles α1 and α2 in front.

  In the edge histogram method, attention is paid to horizontal and vertical edge components that are included in the vehicle areas of the captured images P1 and P2, and the respective pixel luminances of the captured images P1 and P2 are spatially processed in the vertical and horizontal directions. Then, the image edges are extracted, and the extracted image edges are projected in the vertical and horizontal directions and accumulated, and the horizontal edge histograms Heg and vertical edges shown in the histogram images P11 and P12 in FIGS. A histogram Veg is obtained.

  The histogram images P11 and P12 are images obtained by superimposing the horizontal edge histogram Heg and the vertical edge histogram Veg on the captured images P1 and P2, respectively. Further, the extracted horizontal and vertical edge images are as shown in edge images P12 and P22 in FIGS. 5 and 6C, and a large number of horizontal and vertical edges are displayed on the contour of the vehicle such as the preceding vehicles α1 and α2. It can be seen that an edge exists.

  Then, the intersections of the horizontal and vertical edges at which the horizontal edge histogram Heg and the vertical edge histogram Veg both increase are connected, and as shown in the divided images P13 and P23 of FIGS. 5 and 6D, the captured images P1 and P2 are connected. A plurality of rectangular vehicle regions β are created as candidate regions for the preceding vehicles α1 and α2 to be recognized.

  Furthermore, for example, the vehicle quality of each candidate region β is evaluated using four of red, symmetry, smoothness, and shape as evaluation functions of elements that characterize the vehicle, and each evaluation function in each vehicle region β is large. The vehicle is selected as the region of the preceding vehicles α1, α2, and the preceding vehicles α1, α2 are recognized by the edge histogram method.

  Incidentally, the captured images P1 and P2 are actually images EI generated by the super-resolution processing of the high resolution means.

  When the captured images P1 and P2 are increased in resolution by the super-resolution processing of wavelet inverse transform of the resolution increasing means, the preceding vehicles α1 and α2 are positioned, for example, 50 m or more ahead of the host vehicle, and the captured images P1 and P2 Even if the vehicle area β of the preceding vehicles α1 and α2 in the middle is originally of low resolution, the super-resolution processing increases the resolution of the portion of the vehicle area β, and many horizontal lines generated by the preceding vehicles α1 and α2 The vertical edge components can be detected, and the preceding vehicles α1 and α2 can be recognized by the edge histogram method.

  At this time, since the vehicles such as the preceding vehicles α1 and α2 to be recognized include a large number of vertical (vertical) and horizontal (horizontal) edge components in the contour, the photographed image is displayed in the vertical (vertical) and horizontal directions. When the resolution is increased in at least one of the directions (horizontal direction), at least one of the vertical and horizontal contours is enlarged and emphasized to detect at least one of the vertical and horizontal edges Vehicles such as the preceding vehicles α1, α2 can be recognized.

  In the case of vehicles such as the preceding vehicles α1, α2, etc., the lateral width is constant regardless of the vehicle type, etc., and the edge of the vertical contour (vertical edge), that is, the vehicle side edge is easy to detect from the edge interval etc., but the height is Since it is different for each vehicle, it is not easy to detect the edge of the horizontal contour (horizontal edge). In addition, the vehicle side edge often appears clearly. From this point, the vehicle side edge is easy to detect, but the edge (horizontal edge) of the horizontal contour of the upper part (roof part) is easily covered with the background (landscape). It is difficult to detect.

  For this reason, it is generally preferable to recognize a far-off vehicle with an emphasis on the vertical edge, and in this case, the super-resolution processing of the high-resolution means may be processing for increasing the resolution at least in the vertical direction. desirable. Depending on the driving environment, it may be preferable to focus on the horizontal edge. In such a case, the super-resolution processing of the high-resolution means is a process for increasing the resolution at least in the horizontal direction. Is preferred.

When the resolution of the captured image (input image OGb) of the imaging unit 3 is increased by wavelet inverse transformation of the resolution increasing unit to generate the image EI, basically, as described above, the input image OGb that becomes the LL component. Data (pixel data) and three frequency components (LH component, HL component, HH)
Component side data is required, but the vehicle side edge (vertical edge) required for vehicle recognition appears in the high frequency component (HL component) in the vertical direction. Of the frequency components (LH component, HL component, HH component), at least an HL component that is a high frequency component H in the vertical direction may be input to perform inverse wavelet transform. The fact that the vehicle side edge (vertical edge) appears in the HL component is also apparent from the contour image P31 of the HL component of FIG. 7B extracted from the wavelet transform input image P3 of FIG. is there. In the HL component in FIG. 7B, since the upper and lower portions of the contour of the image P31 are dark, the edges of the horizontal contour (horizontal edges) are few, and the left and right portions of the contour of the image P31 are brightened in the vertical direction. Therefore, it includes many vertical contour edges (vertical edges).

  Further, since the horizontal edge of the vehicle appears in the horizontal high-frequency component (LH component), at least the horizontal direction is required when the resolution of the captured image (input image OGb) of the imaging means 3 is increased by inverse wavelet transformation that places importance on the horizontal edge. The wavelet inverse transform may be performed by inputting the LH component which is the high frequency component H in the direction.

  Therefore, in the super-resolution processing of the high-resolution means, the three frequency components (HL component, LH component, and HH component) that are optimal for increasing the resolution in both the horizontal and vertical directions of the captured image (input image OGb). ) Data is not required, and at least one of the HL component and the LH component is selected so as to increase the resolution of the captured image (input image OGb) in at least one of the vertical direction and the horizontal direction. Input and perform inverse wavelet transform. In this case, even if the captured image (input image OGb) is inversely converted to an image (enlarged image) EI different from the original enlarged image by wavelet inverse transformation, the image EI at least either of the input image OGb in the vertical direction or the horizontal direction Since the image is increased in resolution in one direction, the subsequent recognition processing means 5 can detect and recognize vehicles such as the preceding vehicles α1, α2 far from the image EI.

  More specifically, the processing of the high resolution means will be described. At least the data of the HL component is input, and the captured image (input image OGb) of the photographing means 3 is increased in resolution in the vertical direction by wavelet inverse transformation that emphasizes the vertical edge. In this case, the resolution enhancement means, most simply, performs wavelet inverse transform using the LL component composed of the captured image (input image OGb) and the high frequency component (HL component) in the vertical direction as shown in FIG. Do. In addition, when at least LH component data is input and the captured image (input image OGb) of the imaging unit 3 is increased in resolution in the horizontal direction by inverse wavelet transformation with emphasis on the horizontal edge, the resolution increasing unit is the simplest. As shown in FIG. 9, wavelet inverse transformation is performed using an LL component composed of a captured image (input image OGb) of each frame and a high-frequency component (LH component) in the horizontal direction. These wavelet inverse transforms are the simplest super-resolution processing, and the amount of data handled is extremely small, and the resolution in the vertical and horizontal directions of the captured image (input image OGb) of each frame is easily increased in a short time. be able to.

  In the wavelet inverse transform of the high resolution means, instead of the wavelet inverse transform of FIG. 8 and FIG. 9, as shown in FIG. 10, the LL component composed of the captured image (input image OGb) of each frame and the vertical direction Wavelet inverse transform may be performed by inputting data of a high frequency component (HL component) and a horizontal high frequency component (LH component). In this case, high resolution in both horizontal and vertical directions can be achieved. Furthermore, as shown in FIG. 11, if wavelet inverse transform is performed by inputting data of an LL component and three frequency components (HL component, LH component, HH component) composed of a captured image (input image OGb) of each frame, Of course, complete image restoration can be performed, and higher resolution in both horizontal and vertical directions can be achieved.

  Next, data acquisition of frequency components (at least one of the HL component and the LH component) necessary for the wavelet inverse transform of the high resolution means will be described.

  In the data of these frequency components, for example, the preceding vehicles α1 and α2 are located relatively close to the own vehicle 1, and the portions of the preceding vehicles α1 and α2 in the captured image (input image OGb) have a high resolution that allows vehicle recognition. Sometimes, high-resolution image differentiation processing or wavelet transformation may be performed in advance to obtain the image in advance. However, for simplicity, the captured image (input image OGb) is sometimes subjected to differentiation processing or wavelet transformation. The obtained frequency component data may be used, and in this case, it has been proved through experiments and the like that the resolution can be increased without any practical problem.

  Therefore, in the case of the present embodiment, the high resolution means performs differential processing or wavelet transformation on the captured image (input image OGb) of each frame for which high resolution is achieved, and data of frequency components necessary for wavelet inverse transformation, that is, vertical Data of HL component, LH component, etc. in at least one of the direction and the horizontal direction is obtained.

  In this case, frequency component data necessary for wavelet inverse transformation is acquired from the captured image (input image OGb) of each frame, so that the processing is further simplified and practical.

[Operation]
Focusing on the high frequency component H in the vertical direction, the resolution of the captured image (input image OGb) of each frame is increased in the vertical direction (longitudinal direction) by the resolution increasing means, and the preceding vehicle α1, for example, 50 m or more ahead of the host vehicle 1, When recognizing a vehicle that is a distant obstacle to be recognized, such as α2, based on the above-described configuration, the object recognition device 2 performs the processing of steps S1 to S3 in FIG.

  That is, when the photographing unit 3 outputs a new one-frame photographed image in step S1 of FIG. 2, the latest one-frame photographed image (input image OGb) is taken into the image processing unit 4.

  Then, in step S2 of FIG. 2, the high resolution means of the image processing means 4 obtains the HL component by a simple method of wavelet transforming the captured image (input image OGb), for example, and consists of the captured image (input image OGb). Based on the LL component and the HL component obtained by the wavelet transform, the captured image (input image OGb) is subjected to the super-resolution processing of the wavelet inverse transform of FIG. 8 to increase the resolution of the captured image (input image OGb). A high-resolution image EI in which the vertical resolution is double that of the captured image (input image OGb) is generated.

  Further, in step S3 of FIG. 2, the recognition processing means 5 detects the vertical edge of the high-resolution image EI by the above-described edge histogram method, and the vehicle region β of the far-off vehicle to be recognized such as the preceding vehicles α1, α2 and the like. Is selected, and the far-off vehicle to be recognized is recognized based on this selection.

  Then, at least during traveling of the vehicle 1, the process returns from step S3 to step S1, and the next one frame of the captured image is subjected to super-resolution processing to repeatedly recognize a far-off vehicle to be recognized.

  In this case, it is possible to increase the resolution of the captured image (input image OGi) for each frame from the captured image (input image OGi) for each frame by an unprecedented innovative image processing using inverse wavelet transform. it can. In addition, since the wavelet inverse transformation is performed by a simple method of adding only the HL component of the three frequency components LL, HL, LH, and HH, the captured image (input image OGi) can be shortened with extremely simple image processing. The resolution can be increased in time and is extremely practical.

[Experimental result]
Next, the result of performing wavelet inverse transform by inputting only the HL component as described above will be described.

  In this experiment, first, a reference captured image (front vehicle image) P4 shown in FIG. This photographed image P4 has 256 × 256 pixels (pixels), 8 bits / pixel (pixels) in which the preceding vehicle α (corresponding to the preceding vehicles α1, α2) is located relatively close and is photographed at a high resolution in the center. pixel), and there are few backgrounds that hinder vehicle recognition, and it is possible to recognize the preceding vehicle α stably by performing edge histogram processing or the like as it is.

  In addition, assuming that an actual inter-vehicle distance is assumed as a captured image in which the preceding vehicle α is located far away and the vertical and horizontal resolutions thereof are reduced, the portion of the preceding vehicle α is 32 × 32 pixels (pixels), 8 bits (bits) / bit. A degraded image P41 with a reduced resolution is prepared for a pixel. The degraded image P41 is an image in which the recognition of the preceding vehicle α becomes unstable if the edge histogram processing or the like is performed as it is.

  Then, wavelet inverse transformation was performed on the images P4 and P41, and the pixel values after the inverse transformation and the degree of restoration in the contour portion were evaluated.

  The wavelet inverse transformation was performed by adding the HL component shown by the image P42 in FIG. This HL component is an image of the preceding vehicle α having a resolution (64 × 64 pixels (pixels), 8 bits (pixels) / pixels (pixels)) double that of the deteriorated image P41 from the captured image P4 in FIG. It is a component obtained by cutting out and subjecting it to wavelet transform.

  The degree of restoration of the image is evaluated using the mean square error (MSE), and the contour strength of how clear the contour is is obtained by using the difference in brightness before and after the contour portion. It was evaluated in numerical units. Further, the stability of the recognition after the wavelet inverse transformation was evaluated from the increase in resolution after the wavelet inverse transformation, whether or not the contour can be enhanced by combining with the frequency, and the like.

  When the wavelet inverse transform of FIG. 8 was performed using the deteriorated image P41 and the HL component under the above conditions, an image P43 of FIG. 13A was obtained. For comparison, FIG. 13B shows an image P44 obtained as a result of inverse wavelet transform using only the degraded image P41 without inputting the HL component.

  From the comparison of the images P43 and P44, it can be seen that the image P43, which is the result of inverse conversion of the HL component input, is enhanced by restoring the vertical contour by the HL component. Further, the difference between the deteriorated image P41 and the image P44 of the inverse transformation result without HL component indicates the restoration accuracy of the wavelet inverse transformation. Further, the difference between the inverse transformation result image P43 of the HL component input and the inverse transformation result image P44 of the HL component non-input indicates an improvement effect by the HL component input.

  As is clear from the comparison of the images P4, P43, and P44, it is confirmed that if the HL component is input and the wavelet inverse transform is performed, a high-resolution image P43 equivalent to the captured image P4 can be obtained from the degraded image P41. It was.

  Degraded image P41, image P44 obtained by wavelet inverse transformation without HL component input, image P43 obtained by inputting HL component and inverse wavelet transformation, double resolution (64 × 64 pixels, 8 bits) When the MSE indicating the degree of restoration of the image and the numerical evaluation of the outline enhancement were obtained for the image of / pixel), the results are as shown in Table 1 below.

  In Table 1, it can be seen that the closer to the evaluation result of the image having the double resolution, the better the recognition result. Also from the results of Table 1, the image P43 obtained by inputting only the HL component and inversely transforming the wavelet is improved in both restoration accuracy and contour strength as a whole image, and is a high-resolution image sufficient for vehicle recognition. Was confirmed.

  From the above, it was confirmed by experiments that the wavelet inverse transform of the high resolution means is performed by inputting only the HL component, thereby enhancing the vehicle vertical contour and improving the vehicle recognition accuracy.

  Note that the same result is obtained when focusing on the high-frequency component H in the horizontal direction and performing only the LH component and performing the wavelet inverse transform of FIG. Further, the wavelet inverse transform of FIG. 10 may be performed by inputting the HL component and the LH component, or the wavelet inverse transform of FIG. 11 may be performed by inputting the HL component, the LH component, and the HH component.

(Second Embodiment)
Next , a second embodiment will be described with reference to FIGS.

  In the present embodiment, the image processing means 4 in FIG. 1 further forms the extraction means of the present invention.

  That is, in the wavelet inverse transform of the high resolution means, the data of the frequency components such as the HL component and the LH component may be data for the entire captured image to be inversely transformed, but the preceding vehicles α, α1, It may be partial high-frequency component data in at least one of the vertical direction and the horizontal direction of the substantially contour portion of the recognition target vehicle such as α2. The reason why the resolution of the contour portion of the recognition target is increased is that the vehicle recognition by the recognition processing means 5 can be performed without any problem.

  When the resolution of a captured image is partially increased by inverse wavelet transform, the amount of data can be further reduced and processing can be performed more easily and quickly, and only the contour portion to be recognized is increased in resolution to be clear. Therefore, there is an advantage that the image edge becomes clearer and the vehicles such as the preceding vehicles α, α1, and α2 can be easily recognized.

  Therefore, in the case of this embodiment, before performing the super-resolution processing of wavelet inverse transform by the high resolution means, the extraction means of the image processing means 4 causes the preceding vehicles α1, α2, etc. in the photographed image of the photographing means 3 to Only the high-frequency component (HL component, LH component) in at least one of the vertical direction and the horizontal direction is extracted only for the contour portion of the vehicle that is substantially recognized. This partial high-frequency component extraction may be performed in any way, but some examples will be described below.

  For example, when a vehicle to be recognized such as a preceding vehicle α, α1, or α2 to be recognized gradually moves forward, a photographed image (high resolution image) when the vehicle to be recognized is close by the extraction means. Then, the contour of the vehicle to be recognized is detected by a known image recognition process using a differentiation method, an edge histogram method, etc., and the vehicle is captured, and the vehicle is captured even if the vehicle to be recognized moves away. Subsequently, when the vehicle to be recognized is located far away, at least one of the vertical direction and the horizontal direction is applied to the contour portion of the vehicle being captured by the photographing means 3 by the differential method or wavelet transform. The high frequency components (HL component, LH component) are extracted, and the partial high frequency components are extracted.

  Further, when considering the case where the vehicle to be recognized gradually approaches from a distance, the extracted image of the image captured by the image capturing device 3 of each frame is arranged in the left-right direction at an appropriate interval from the differential image or the like. A pair of edge components in the vertical direction is detected, and a vehicle portion to be substantially recognized in the photographed image is identified as such a pair of edge components in the vertical direction is a side edge of the vehicle to be recognized. For the specified part of the photographed image, the high-frequency component (HL component, LH component) in at least one of the vertical direction and the horizontal direction is extracted by the differential method or wavelet transform to extract the partial high-frequency component. It is preferable.

  Further, when the vehicle 1 also includes a radar forward exploration means together with the photographing means 3, the vehicle to be recognized is captured by the exploration, and the forward exploration means in the photographed image of the photographing means 3 is captured by the extraction means. If the high frequency component (HL component, LH component) in at least one of the vertical direction and the horizontal direction is extracted by the differential method or wavelet transform and the partial high frequency component is extracted from the captured portion. Good.

  Then, for example, the HL component of only the vertical contours on the left and right sides of the preceding vehicle α to be substantially recognized shown in the image P5 of FIG. 14A is extracted and extracted as a partial vertical high-frequency component by the extraction means, for example. Partial HL component data is input to the wavelet inverse transform of FIG. 8 of the resolution increasing means to increase the resolution of the captured image of each frame.

  In this case, an image P51 of FIG. 14B was obtained as the image EI after the wavelet inverse transformation. In the case where HL component data is not input, an image P52 of FIG. 14C is obtained as the image EI after the wavelet inverse transformation.

  Comparing the images P51 and P52, the image P51 is sharpened by enhancing only the vertical contour portion of the vehicle α with high resolution by the HL component partially input from the image P52.

  Furthermore, a partial HL component is defined as a partial HL component extracted from the vertical contour portion of the vehicle α in the HL component shown by the image P42 in FIG. The same wavelet inverse transformation is performed for the case of input and the case of no input, the input image (degraded image) as the LL component of the wavelet inverse transformation, and the images P52 and HL obtained by the wavelet inverse transformation with no HL component input. For the image P51 obtained by inverse wavelet transform by inputting the components, and for the image of the double resolution (64 × 64 pixels, 8 bits / pixel), the MSE indicating the degree of image restoration and the numerical evaluation of the contour enhancement are obtained. As a result, the results are shown in Table 2 below.

  From Table 2, since the HL component is partially input, the images P51 and P52 do not have a large difference in MSE. However, with respect to edge enhancement, the image P51 is greatly improved, and the vertical contour of the vehicle α is improved by the HL component. It was confirmed that only the part was enhanced and emphasized with high resolution. Then, from the numerical evaluation of the edge emphasis of the image P43 in Table 1 and the image P51 in Table 2, the same result as in the case of the image P43 inputted as a whole can be obtained also in the case of the image P51 in which the HL component is partially inputted. It has been found.

  Therefore, in the case of the present embodiment, the HL component is extracted only for the contour portion of the substantially vehicle α that is the recognition target of the captured image by the extraction means of the image processing means 4, and the wavelet of the high resolution means that inputs this partial HL component. By reverse conversion, only the contour portion of the vehicle α necessary for recognition can be made high-resolution and clear, and the vehicle α can be recognized by the recognition processing means 5 at the subsequent stage. In this case, the data amount of the inverse wavelet transform is smaller than in the case of the embodiment, and the distant vehicle α can be recognized more easily and quickly.

  In the present embodiment, the same result can be obtained when focusing on the high frequency component H in the horizontal direction and performing only the partial LH component and performing the wavelet inverse transform of FIG. Alternatively, the wavelet inverse transform of FIG. 10 may be performed by partially inputting the HL component and the LH component, or the wavelet inverse transform of FIG. 11 may be performed by partially inputting the HL component, the LH component, and the HH component. Also good.

(Third embodiment)
Next, a third embodiment corresponding to claim 1 will be described with reference to FIGS.

  As shown in FIG. 15 which is a schematic diagram of the processing of the present embodiment, the captured image (low-resolution input image OGb) is subjected to wavelet transform (WT in FIG. 15) to the high-resolution input image (original image OGa). This is an image of the LL component obtained by applying, and the resolution in the vertical direction and the horizontal direction is ½ of the original image OGa.

  Then, wavelet transform is performed on the photographed image by the extraction means formed by the image processing means 4, and its LL component (LL * in FIG. 15), HL component (HL * in FIG. 15), LH component (in FIG. 15). LH *) and HH components (HH * in FIG. 15), and the captured image (LL component) and at least the partial HL component, LH component, and HH component based on the HL *, LH *, and HH * If wavelet inverse transformation (inverse WT in FIG. 15) is performed with any one of the inputs as input, the high-resolution image EI corresponding to the original image OGa is restored as described above.

  By the way, the extraction state of each of the high-frequency components HL, LH, and HH depends on the resolution of the image. For example, an example of the HL component extracted from an image having a resolution of vertical × horizontal = 256 × 256 pixels, 128 × 128 pixels, and 64 × 64 pixels is as shown in FIG. 16, FIG. 17, and FIG. Thus, when they are binarized, as shown in FIG. 16, FIG. 17 and FIG. 18B, it can be seen that the higher the pixels, the better the extraction of HL components without unnecessary dispersion.

  The HL *, LH *, and HH * (partial HL component, LH component, and HH component) are contour lines with the vertical and horizontal resolutions of the captured image being reduced to ½ of the original image OGa. The line width is doubled. Further, the contour line is discontinuous accordingly.

  Therefore, in the present embodiment, first, when the ratio of the resolution of the high-resolution image (desired restoration resolution) to the resolution of the photographed image is K (= desired restoration resolution / photographed image resolution), the extraction is performed. The line width of the contour line of the HL *, LH *, HH * (partial HL component, LH component, HH component) extracted by the means is reduced to 1 / K. Specifically, since K = 2, the line width of the contour line is reduced to ½. Second, for the discontinuous portion of the contour line, pixels are interpolated to make it smooth.

  That is, for HL * (HL component), the line width in the vertical direction is reduced to ½, and pixels are interpolated for discontinuities in the vertical direction. For LH * (LH component), the line width in the horizontal direction is reduced to ½, and pixels are interpolated for discontinuities in the horizontal direction. For HH * (HH component), the line width in the vertical and horizontal directions is narrowed to ½, and the pixels are interpolated for discontinuities in the vertical and horizontal directions.

  Then, for example, the processing of the line width for the HL component and the interpolation of the pixels will be specifically described with reference to FIGS. FIG. 19 shows an example of the process of interpolating pixels with respect to the vertical discontinuity, and the line width in the vertical direction is halved. (A) is the process, and (b) is the post-process. Indicates. FIG. 20 shows an example of the contour line of the HL component before and after the processing.

  When the contour line of the image of the HL component of HL * extracted by the extraction unit is formed by each pixel G which is shaded in at least one direction in FIG. 19A, the line segment in the vertical direction is The processed contour line is reduced to a half width (line width), and further, the discontinuity in the vertical direction is interpolated by the pixels marked with a circle in FIG. It is formed by cross-hatched pixels G.

  As a result, as shown in FIG. 20, the HL component after processing on the right side of FIG. The resolution becomes narrower to / 2, resulting in a higher resolution, and smoothed by pixel interpolation.

  Then, if the LL component of the photographed image and the processed HL component are input and the wavelet inverse transformation (inverse WT in the figure) is performed by the high-resolution means, the resolution can be further improved in the vertical direction. The restored image EI can be restored.

  If the LL component of the photographed image and the LH component after pixel interpolation are input by the high resolution means and wavelet inverse transformation is performed, the image EI with higher resolution in the horizontal direction can be restored. it can. Furthermore, when the LL component of the photographed image, the processed HL component, and the LH component are input to perform inverse wavelet transformation, and the LL component of the photographed image, the processed HL component, the LH component, and the HH component When the wavelet inverse transform is performed by inputting the above, it is possible to restore the image with higher resolution even better.

(Fourth embodiment)
Next, a fourth embodiment corresponding to claim 2 will be described with reference to FIGS.

  In the present embodiment, attention is paid to the fact that the photographed images (moving images) photographed continuously in time series by the photographing means 3 are used, and the photographed images are displaced between the frames (movement of pixel positions). Here, “displacement” occurs when an object (for example, a preceding vehicle) in the frame moves or the photographing posture of the photographing means 3 changes. Note that the captured images in FIGS. 21 and 22 are those obtained by capturing the preceding vehicle later.

  Then, as shown in FIG. 21 which is a schematic diagram of the processing of the extraction unit in the present embodiment, if the difference in luminance in the horizontal direction is taken for the captured images of the two frames F1 and F2 which are temporally different from the imaging unit 3. , HL component can be extracted. Further, the LH component can be extracted by calculating the vertical luminance difference between the captured images of the two frames F1 and F2. Further, the HH component can be extracted by calculating the difference in luminance in the diagonal direction by taking the difference in luminance in both the horizontal and vertical directions of the captured images of the two frames F1 and F2. Note that the two frames before and after for obtaining the difference may be two consecutive frames, or may be two frames that move back and forth a certain number of frames apart.

  As described above, in the present embodiment, the extraction unit converts the difference in at least one of the vertical and horizontal directions of the captured images of the two frames F1 and F2 before and after the photographing unit 3 into a high frequency component (HL component, LH component, HH component).

  Furthermore, as shown in FIG. 22 which is a schematic diagram of the processing of the high resolution means in the present embodiment, from the difference in luminance between the photographed image (LL component) and the photographed images of the two frames F1 and F2 by the extracting means. Wavelet inverse transformation (inverse WT in FIG. 15) is performed with at least one of the extracted HL component, LH component, and HH component as input, and a high-resolution image EI corresponding to the original image OGa is restored.

  In this case, a high-resolution image EI can be restored by extracting a high-frequency component (HL component, LH component, HH component) from a low-resolution captured image by a simple process of obtaining a difference between the captured images of two frames.

  The present invention is not limited to the above-described embodiments, and various modifications other than those described above can be made without departing from the spirit of the present invention. The acquisition method of (HL component, LH component, HH component) may be any.

  Further, the object to be recognized is not limited to the front vehicle (including the opposite vehicle) such as the preceding vehicles α, α1, and α2, but is a vehicle such as an overtaking vehicle in various directions such as behind the own vehicle 1. In this case, the photographing direction and range of the photographing means 3 may be set in accordance with the recognition countermeasure.

  Furthermore, the object to be recognized is not limited to a vehicle, and may be various moving objects and stationary objects on the road that are likely to be obstacles to the traveling of the host vehicle 1. It can be preset according to the size (mainly width and height).

  Next, it goes without saying that the configuration of the photographing means 3 and the recognition method of the recognition processing means 5 may be whatever.

  In the fourth embodiment, pixel interpolation is not necessarily performed and may be omitted depending on circumstances.

  The present invention can be applied to recognition of various objects from captured images of imaging means mounted on a vehicle.

It is a block diagram of a 1st embodiment of the present invention. 2 is a partial flowchart for explaining the operation of FIG. 1. It is explanatory drawing of basic wavelet transformation. It is explanatory drawing of basic wavelet inverse transformation. It is explanatory drawing of the process of the recognition process means of FIG. It is another explanatory view of the processing of the recognition processing means of FIG. It is explanatory drawing of the relationship between the contour component of the orthogonal | vertical direction (vertical direction) of an image, and the frequency component of the wavelet transform. It is explanatory drawing of an example of the wavelet inverse transformation of the high resolution means of FIG. It is explanatory drawing of the other example of the wavelet inverse transformation of the high resolution means of FIG. It is explanatory drawing of the further another example of the wavelet inverse transformation of the high resolution means of FIG. It is explanatory drawing of the other example of the wavelet inverse transformation of the high resolution means of FIG. It is explanatory drawing of the conditions of an experiment example. It is explanatory drawing of an experiment example result. It is explanatory drawing of the wavelet inverse transformation of the 2nd Embodiment of this invention. It is a schematic diagram of the process of the 3rd Embodiment of this invention. It is explanatory drawing of the example of an image of the high frequency component of FIG. It is explanatory drawing of the other example of a high frequency component of FIG. FIG. 16 is an explanatory diagram of still another image example of the high frequency component of FIG. 15. It is explanatory drawing of the process of the line width of the high frequency component of FIG. It is explanatory drawing of an example of the line width before and after the process of the high frequency component of FIG. It is a schematic diagram of the high frequency component extraction of the 4th Embodiment of this invention. It is explanatory drawing of the wavelet reverse transformation of the 4th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Own vehicle 2 Object recognition apparatus 3 Imaging | photography means 4 Image processing means 5 Recognition processing means α, α1, α2 A preceding vehicle to be recognized

Claims (2)

A shooting means mounted on the vehicle for shooting outside the vehicle;
High resolution means for applying a wavelet inverse transform to the captured image of the imaging means to increase the resolution of the captured image;
Recognition processing means for recognizing an object in a high-resolution image obtained by increasing the resolution of the high-resolution means ;
Extraction means for extracting a high-frequency component in at least one of a vertical direction and a horizontal direction with respect to a contour portion of the object substantially in the photographed image ,
The wavelet inverse transform of the high resolution means is performed by adding the high frequency component extracted by the extraction means to the photographed image,
The high-frequency component is used for inverse wavelet transformation of the high-resolution means by reducing the line width of the outline of the high-frequency component to 1 / K, where K is the ratio of the resolution of the high-resolution image to the resolution of the captured image. object recognition apparatus, characterized by being. A shooting means mounted on the vehicle for shooting outside the vehicle;
High resolution means for applying a wavelet inverse transform to the captured image of the imaging means to increase the resolution of the captured image;
Recognition processing means for recognizing an object in a high-resolution image obtained by increasing the resolution of the high-resolution means;
Extraction means for extracting a high-frequency component in at least one of a vertical direction and a horizontal direction with respect to a contour portion of the object in the captured image;
With
The wavelet inverse transform of the high resolution means is performed by adding the high frequency component extracted by the extraction means to the photographed image,
The extraction means, the vertical direction, the object-recognition device you and extracting at least one direction of the difference in the horizontal direction as the high-frequency components of the photographed images of two frames before and after the photographing means.