JP2011022072A - Position detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide high position detection precision even when image correction is made while attaining high-speed processing and cost reduction. <P>SOLUTION: This position detector includes: correction means 12L, 13L, 12R, 13R for correcting an optical distortion generated at a stereo image comprised of a pair of images with a partially different correction amount and outputting the corrected stereo image; a search means 14 of setting one side of the corrected stereo image as a reference image and the other side as a comparison image and of searching for a matching region in the comparison image matching a detection target region set in the reference image; a size change means 15 of changing a size of a detection target region which the search means 14 uses for searching according to a correction amount by the correction means 12L, 13L at the position for setting the detection target region in the reference image; and distance calculation means 16 of calculating a distance from a photographing point of a photographed object, based on a parallax determined from the position of the detection target region in the reference image and the position of the matching region in the comparison image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a position detection device that detects the position of an object based on a stereo image captured by a stereo camera having a pair of cameras.

  The principle of triangulation is based on the position (distance) of an object present in the camera field of view based on a stereo image taken simultaneously with a stereo camera having a pair of cameras (left camera, right camera) arranged with a predetermined baseline length There is known a position detection device that detects the above (for example, see Patent Document 1 below). Such a position detection device using a stereo camera is different from active ranging such as radar, and has the advantage that a wide range of distance information and shape can be obtained by a single shooting. Used for surveillance cameras.

When distance measurement is performed by the stereo method, the distance is obtained based on a positional shift (parallax) in the left and right camera images (stereo images) of the position detection target image captured by the left and right cameras.
When obtaining the parallax, it is necessary to search for the position detection target image in one of the stereo images (referred to as a reference image) in the other image (referred to as a comparison image). In the search, correlation processing such as SAD (Sum of Absolute Difference) method is generally used.

  In the correlation process, the reference image is divided into a plurality of blocks, each block is used as a template, a window having the same size as the template is moved within the search range on the comparison image set on the epipolar line, and the most correlated with the template. Is a process for determining a certain block (for example, in the SAD method, a block having the smallest integrated value of absolute values of differences for each pixel). This type of processing requires a large amount of calculation, and in applications where it is necessary to determine the shape and distance instantly, high-speed hardware is required and the cost is high, or sufficient performance cannot be achieved in terms of detection speed. There is.

  With respect to such a problem, Patent Document 1 reduces the calculation amount by changing the window size of the corresponding point search for parallax calculation in the high resolution image based on the distance information obtained from the low resolution image. The processing burden is reduced.

JP 2008-309637 A

  By the way, due to optical distortion of the left and right cameras, optical axis shift, etc., or when measuring distances over a wide range or when the object is at a short distance, a lens such as a wide angle or fisheye is used. Since it is used, the captured image is distorted. For this reason, after correcting the captured stereo image, matching processing using a correlation calculation or the like for distance calculation is performed.

  However, the technique described in Patent Document 1 does not consider the influence of correction of the optical distortion occurring in the image as described above. Therefore, when the image is corrected, the matching accuracy in the matching process is corrected. May be low, and there is a problem that sufficient position detection accuracy may not be obtained.

  The present invention has been made in view of such points, and position detection that can achieve high position detection accuracy even when image correction is performed while achieving high-speed processing and low cost of the apparatus. An object is to provide an apparatus.

  The position detection apparatus according to the present invention includes a correction unit that corrects optical distortion generated in a stereo image including a pair of images with partially different correction amounts and outputs a corrected stereo image; Search means for searching for a matching area in the comparison image that matches a detection target area set in the reference image, one of which is a reference image and the other is a comparison image, and the detection used by the search means for the search A size changing unit that changes a size of the target region according to a correction amount by the correcting unit at a position where the detection target region in the reference image is to be set; a position of the detection target region in the reference image; and Based on the parallax obtained from the position of the matching area in the comparison image searched by the search means, the distance from the shooting point of the shot object is calculated. Constructed and a that the distance calculation means.

  According to the present invention, there is an effect that high position detection accuracy can be realized even when image correction is performed while increasing the processing speed and reducing the cost of the apparatus.

1 is a block diagram illustrating an overall configuration of a stereo camera system according to an embodiment of the present invention. It is a block diagram which shows the function structure of the position detection apparatus of embodiment of this invention. It is a figure which shows the image of equidistant projection of embodiment of this invention. It is a figure which shows the image correct | amended by center projection of embodiment of this invention. It is a figure which shows the reference | standard image for demonstrating the example of a setting of the window size according to the correction amount of embodiment of this invention. It is a figure which shows the comparative image for demonstrating the example of a setting of the window size according to the correction amount of embodiment of this invention. It is a figure which shows the reference | standard image for demonstrating the other example of the setting of the window size according to the correction amount of embodiment of this invention. It is a figure which shows the comparative image for demonstrating the other example of a window size according to the corrected amount of embodiment of this invention. It is a flowchart which shows the process by the position detection apparatus of embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the direction along the optical axis of a stereo camera (cameras 2L and 2R), which will be described later, is defined as the Z direction (Z axis), and the base line length direction of the stereo camera is defined as the X direction (X axis). Description will be made with reference to an orthonormal coordinate system in which the orthogonal direction is the Y direction (Y axis). In the present embodiment, the X direction is set in the horizontal direction, and the Y direction is set in the vertical upward direction.

  FIG. 1 shows the overall configuration of a stereo camera system according to an embodiment of the present invention. This stereo camera system 1 includes a stereo camera body having a pair of cameras (left camera 2L, right camera 2R) and a stereo image (a pair of images photographed simultaneously with a predetermined baseline length) captured by the stereo camera body. And a position detection device 3 for detecting the position of the object including the distance from the shooting point (stereo camera body) of the object (including the object part) existing in the shooting field of view of the stereo camera body. . By detecting positions including distances at a plurality of locations on the object, the shape of the object can also be obtained.

  The left camera 2L and the right camera 2R included in the stereo camera body are arranged on the left and right (X direction) with a predetermined baseline length, and these cameras 2L and 2R are image sensors (imaging devices) made of CCD, CMOS, or the like. 4L and 4R and photographing lenses (shooting optical systems) 5L and 5R for forming images of subjects in the photographing field of view on the imaging surfaces of the image sensors 4L and 4R, respectively.

  Stereo images captured by the left camera 2L and the right camera 2R are sequentially output at a predetermined frame interval (frame rate), and sequentially output to the position detection device 3 via the left image buffer 6L and the right image buffer 6R. Is output. The position detection device 3 detects a position including a distance to an object existing in the image, and the detected information is output to the application 8 via the distance image memory 7, and according to the function of the application 8. It is processed.

  The functional configuration of the position detection device 3 is shown in FIG. The position detection device 3 includes an image acquisition unit (left image acquisition unit 11L, right image acquisition unit 11R), correction unit (left distortion correction unit 12L, left distortion correction table 13L, right distortion correction unit 12R, right distortion correction table 13R). , A search unit (search unit 14), a size change unit (window size change unit 15), and a distance calculation unit (distance calculation unit 16).

  The left image acquisition unit 11L sequentially acquires the left images taken by the left camera 2L via the left image buffer 6L. The right image acquisition unit 11R sequentially acquires right images taken by the right camera 2R via the right image buffer 6R.

  The left distortion correction unit 12L corrects the optical distortion generated in the left image acquired by the left image acquisition unit 11L, and outputs a correction image (left correction image) having a partially different correction amount. Optical distortion occurring in the left image photographed by the left camera 2L (distortion caused by a manufacturing error including an axial deviation of the photographing lens 5L with respect to a predetermined reference, etc., and specifications of the photographing lens 5L (wide angle lens, fisheye lens) The distortion generated based on the left distortion correction table 13L is measured in advance and stored in the left distortion correction table 13L as distortion correction data, and the left distortion correction unit 12L generates a partial left image based on the left distortion correction table 13L. The image is corrected (reduced, enlarged, shifted, rotated, etc.) with a different correction amount, and the corrected image (left corrected image) is output to the search unit 14.

  The right distortion correction unit 12R corrects the optical distortion generated in the right image acquired by the right image acquisition unit 11R, and outputs a correction image (right correction image) having a partially different correction amount. Optical distortion occurring in the right image photographed by the right camera 2R (distortion caused by a manufacturing error including an axial deviation of the photographing lens 5R with respect to the predetermined reference and the specifications of the photographing lens 5R (wide-angle lens, The distortion generated based on the fisheye lens etc.) is measured in advance and stored as distortion correction data in the right distortion correction table 13R. The right distortion correction unit 12R converts the right image based on the right distortion correction table 13R. The image is corrected (reduced, enlarged, shifted, rotated, etc.) with a partially different correction amount, and the corrected image (right corrected image) is output to the search unit 14.

For example, when fisheye lenses are used as the photographing lenses 5L and 5R, the images are equidistant projection images as shown in FIG. In this case, the image is corrected to a central projection image as shown in FIG. As described above, when the correction is performed from the equidistant projection to the central projection, in the corrected image, the peripheral resolution is deteriorated according to the correction amount. In each projection method, if the angle from the optical axis is θ and the focal length is f, the imaging height r can be obtained by the following equation.
Equidistant projection: r = fθ
Central projection: r = ftanθ

  The image corrected from the equidistant projection to the central projection is an image that extends radially from the center to the periphery, and is not corrected in the center or has a relatively high resolution because the correction amount is small. Since it is large, the image has a relatively low resolution.

  Here, for convenience, the left and right image acquired from the left and right image buffers 6L and 6R via the left and right image acquisition units 11L and 11R are corrected by the left and right distortion correction units 12L and 12R. The left and right images temporarily stored in 6L and 6R are corrected by the left and right distortion correction units 12L and 12R, respectively, and the corrected images are acquired via the left and right image acquisition units 11L and 11R. May be output.

  The search unit 14 is set by the window size changing unit 15 described later using the left corrected image corrected by the left distortion correcting unit 12L as a reference image and the right corrected image corrected by the right distortion correcting unit 12R as a comparative image. According to the window size, a matching area in the comparison image that matches a detection target area that is an area that is a source of matching determined in the reference image is searched. In this search process, a block according to a window size set by the window size changing unit 15 described later is set as a detection target area, and this is used as a template, and a block in a comparative image having a high correlation (matching) with the template is set as a matching area. Matching process.

  Various known processes can be used as the matching process, but in this embodiment, a SAD (Sum of Absolute Difference) method is used. The SAD method calculates and evaluates an integrated value of absolute values of differences in luminance values of pixels corresponding to a template set in a reference image and a block in a comparison image to be correlated with the template. And a block in the comparison image related to the smallest evaluation value is used as a matching area.

  Based on the left distortion correction table 13L and the right distortion correction table 13R, the window size changing unit 15 obtains a correction amount in each part in the corrected stereo image (left correction image, right correction image), and according to the correction amount. The search unit 14 changes the size (window size) of the detection target region used for matching (search). In this embodiment, the correction amount at each part in the image is obtained based on the left and right distortion correction tables 13L and 13R. However, a separate left and right correction amount table is provided, and the image is obtained from these left and right correction amount tables. You may make it acquire the correction amount in each inside part.

  In the present application, the term “correction amount” includes not only the correction amount but also an amount (distortion amount, resolution, etc.) corresponding to the correction amount. That is, since the correction amount is substantially synonymous with the distortion amount or resolution in each portion in the image, the window size may be changed according to the distortion amount or resolution.

  Here, “change the window size” means that the window size set for the uncorrected portion in the image is set as a reference size according to partial enlargement or reduction accompanying image distortion correction ( That is, depending on the amount of correction, this means that the window size is enlarged or reduced, and includes not only changing the area but also changing the shape.

  For example, in the reference image (left correction image) shown in FIG. 5, since the center portion has a small correction amount and high resolution, the detection target region is set with a small window size as shown in the block BL1 in the center portion. In the comparison image (right-corrected image) shown in FIG. 6, a search (matching) is performed as indicated by an arrow A1 in the figure from the block BL1 ′ corresponding to the block BL1 (or a block in the vicinity thereof) BL1 ′.

  Further, in the reference image (left corrected image) shown in FIG. 5, the four corners (for example, the upper left corner) have a large correction amount and low resolution, so that the four corners are detected with a large window size as shown in the block BL2. A target area is set, and a search (matching) is performed as indicated by an arrow A2 in the figure from the block BL2 ′ corresponding to the block BL2 (or a block in the vicinity thereof) BL2 ′ in the comparison image (right correction image) shown in FIG.

  In addition, for example, in the reference image (left correction image) shown in FIG. 7, the correction amount is small and the resolution is high in the central portion, so that the detection target region is set with a small window size as shown in the block BL3 in the central portion. Then, in the comparison image (right correction image) shown in FIG. 8, a search (matching) is performed from the block BL3 ′ corresponding to the block BL3 (or a block in the vicinity thereof) BL3 ′ as indicated by an arrow A3 in the figure.

  Further, in the reference image (left correction image) shown in FIG. 7, since the correction amount is large in the horizontal direction but small in the vertical direction at the center part of the upper and lower sides (for example, the center part of the upper side), A detection target area is set with a window size whose dimension is expanded in the horizontal direction as shown in BL4, and in the comparison image (right correction image) shown in FIG. 8, a block corresponding to the block BL4 (or a block in the vicinity thereof) BL4 ′ The search (matching) is performed as indicated by an arrow A4 in FIG.

  Further, in the reference image (left correction image) shown in FIG. 7, the center of the left and right sides (for example, the center of the left side) has a large correction amount in the vertical direction but is small in the horizontal direction. A detection target region is set with a window size whose size is expanded in the vertical direction as shown in BL5, and in the comparison image (right correction image) shown in FIG. 8, a block corresponding to the block BL5 (or a block in the vicinity thereof) BL5 ′ The search (matching) is performed as indicated by an arrow A5 in FIG.

  In the above-described search, matching may be executed using the entire range on the epipolar line in the comparison image as a search range, or for example, the distance (position in the Z direction) obtained for the most recently processed frame, Matching may be executed in accordance with a search range that is changed and set in a range narrower than the entire range on the epipolar line according to a distance measured using an active distance measuring device or the like.

  The distance calculation unit 16 determines the distance from the shooting point of the portion based on the parallax (positional deviation) obtained from the position of the detection target region in the reference image and the position of the matching region in the comparison image specified by the search unit 14. Is calculated. That is, assuming that the left and right cameras 2L and 2R are arranged in parallel with each other, the distance z to an object (target object) that is a position detection target is L, the base length is f, the focal length is f, and the parallax is D can be calculated by z = L × f / D. The distance (z1, z2, z3,...) Calculated for each detection target region, that is, the position in the Z direction can specify the position related to the calculation of the distance (the position in the X direction and the position in the Y direction of the block). The state is stored in the distance image memory 7.

  Next, processing by the position detection device 3 of this embodiment will be described with reference to a flowchart shown in FIG.

  When the process is started, shooting by the stereo camera body is started (step S1). Thereby, image data (frame data) obtained by digitally converting the imaging signals from the image sensors 4L and 4R of the left and right cameras 2L and 2R are sequentially supplied to the position detection device 3 via the left image buffer 6L and the right image buffer 6R, respectively. Is output.

  Next, in the position detection device 3, the left distortion correction unit 12L and the right distortion correction unit 12R respectively read distortion correction data from the left distortion correction table 13L and the right distortion correction table 13R (step S2), and these distortion correction data. Accordingly, the optical distortion generated in the left image and the right image is corrected (step S3). Here, in order to simplify the description, for example, the equidistant projection image shown in FIG. 3 is corrected to the central projection image shown in FIG. 4.

  Next, the correction amount at the position to be set as the detection target area in the left image is compared with a predetermined threshold value (step S4), and when it is determined that the correction amount is smaller than the predetermined threshold value (Yes) In step S5, a small window (small window size) as shown in block BL1 in FIG. 5 is selected as the window size to be set as the detection target area.

  In step S4, when it is determined that the correction amount at the position to be set as the detection target region in the left image is larger than the predetermined threshold value (in the case of No), the process proceeds to step S9 to detect the detection target region. As a window to be set as, a large window (large window size) is selected.

  Next, the position to be set as the detection target region in the left image is the top and bottom (upper center or lower center), right and left (left center or right center), or four corners (upper left corner). Part, lower left corner, upper right corner, or lower right corner) (step S10), and when it is determined to be up and down, it is greatly enlarged in the horizontal direction as shown in block BL4 of FIG. When a window (horizontal window) is selected and determined to be left and right, a window (vertically long window) greatly enlarged in the vertical direction as shown in block BL5 of FIG. 7 is selected, and when it is determined that there are four corners. A window size (four corner window) enlarged in both the vertical and horizontal directions as shown in the block BL2 in FIG. 5 is selected.

  Next, in step S6, detection of corresponding points in the stereo image with the window size selected in step S5, S11, S12, or S13, that is, corresponding to the detection target region set in the reference image according to the selected window size. A matching region in the comparison image to be searched is searched.

  Next, it is determined whether or not all the matching processes have been completed (step S7). If it is determined that the matching process has not been completed (in the case of No), the process returns to step S4, and detection to be set in the reference image When it is determined that the same process is repeated while changing the position of the target area and the process is finished (in the case of Yes), each detection target area in the reference image and each of the corresponding comparison images in the reference image The distance for each region is calculated from the matching region (step S8), and is output as a distance image, and this process ends. The distance image is an image using the position of each region where the distance is calculated and the distance as parameters.

  According to the above-described embodiment, a relatively small size window is used for a portion with a small correction amount (that is, a high-resolution portion) in an image, and a portion with a large correction amount (that is, a low-resolution portion) is relatively. Since the matching process is performed using a large-sized window, the calculation amount of the matching process can be reduced by using a small window in the high-resolution part, reducing the processing load and increasing the processing speed. Can be planned. On the other hand, since a large window can be used in the low-resolution part, many feature parts of the image can be obtained, so high matching accuracy can be realized even in the low-resolution part, and the position (distance) can be detected with high precision. can do. Therefore, high position detection accuracy can be realized even when image correction is performed while increasing the processing speed and reducing the cost of the apparatus.

  In the above-described embodiment, the window size is set according to the distortion correction amount for the left image as the reference image. However, the distortion correction amount for the right image as the comparison image may be used, or both are used. May be. In the above-described embodiment, the case of correcting the equidistant projection image captured using the fisheye lens to the central projection image has been described, but this is merely an example for simplifying the description. This is applicable even when different corrections are performed.

  The embodiment described above is described for facilitating understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment includes all design changes and equivalents belonging to the technical scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Stereo camera system, 2L ... Left camera, 2R ... Right camera, 3 ... Position detection apparatus, 11L ... Left image acquisition part, 11R ... Right image acquisition part, 12L ... Left distortion correction part, 12R ... Right distortion correction part, 13L ... Left distortion correction table, 13R ... Right distortion correction table, 14 ... Search unit, 15 ... Window size change unit, 16 ... Distance calculation unit, BL1 to BL5 ... Block (detection target area).

Claims (2)

Correction means for correcting optical distortion occurring in a stereo image composed of a pair of images with partially different correction amounts and outputting a corrected stereo image;
Search means for searching for a matching area in the comparison image that matches a detection target area set in the reference image, using one of the corrected stereo images as a reference image and the other as a comparison image;
A size changing means for changing a size of the detection target area used by the search means for searching according to a correction amount by the correction means at a position where the detection target area in the reference image is to be set;
A distance for calculating the distance from the photographing point of the photographed object based on the parallax obtained from the position of the detection target region in the reference image and the position of the matching region in the comparison image searched by the search means A position detecting device comprising: a calculating means.   The position detection apparatus according to claim 1, wherein the size changing unit changes the size of the detection target region to a larger size as a correction amount by the correction unit increases.

Images