JP2013115559A - Bird's -eye video generation apparatus, bird's-eye video generation method, video display system and navigation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a bird's-eye video generation apparatus, a bird's-eye video generation method, a video display system and a navigation device, capable of easily adjusting conversion parameters for converting a captured image into a bird's-eye image.SOLUTION: A bird's-eye video generation apparatus includes: a bird's-eye conversion section 23 for using bird's-eye parameters read from a bird's-eye parameter storage section 25 for storing the bird's-eye parameters for converting an image into a bird's-eye image by shading conversion to convert a captured image captured by a camera 10 into a bird's-eye image; an interval index calculation section 24b for calculating an interval index value indicating a uniform degree of intervals of a pattern of a sign on the bird's-eye image obtained by converting the captured image in which the sign of the pattern at equal intervals is imaged; and a bird's-eye parameter calculation section 24c for, on the basis of the calculated interval index value, calculating the bird's-eye parameters by which the intervals of the pattern of the sign on the bird's-eye image are uniformed, and storing the calculated bird's-eye parameters in the bird's-eye parameter storage section 25.

Description

  The present invention is an overhead video generation apparatus, an overhead video generation method, and a video display system using the same, which are mounted on a mobile body such as a vehicle and convert an image captured around the mobile body into an overhead image and present it to a user. And a navigation device.

  2. Description of the Related Art Conventionally, there has been known an image processing apparatus that converts a bird's-eye view of a camera image mounted on a vehicle and presents it to a user. In such an apparatus, when a camera is set, a posture parameter indicating the posture state is stored, and the camera image is viewed based on the posture parameter on the assumption that the camera is installed in the posture state. It was converted.

  As a conventional technique for appropriately installing a camera in a vehicle, for example, there is a method disclosed in Patent Document 1. In this method, a test pattern (reference pattern, test chart) serving as an index is provided at a position away from the vehicle, and the camera is installed in the posture state indicated by the posture pattern based on the imaging state of the test pattern captured by the camera. Check whether there is any.

  Patent Document 2 proposes an image processing apparatus that adjusts the camera mounting state with reference to a parallel line such as a white line drawn on the ground or an infinite distance obtained from the parallel line. In this apparatus, an adjustment mechanism that adjusts the imaging direction of the camera is provided, and the imaging direction can be adjusted even if the imaging direction is shifted after the camera is attached to the vehicle.

Non-Patent Document 1 discloses a method for estimating a posture parameter itself. In this method, a dedicated test pattern is imaged by a camera, and posture parameters are estimated from the test pattern imaging state.
Patent Documents 3 and 4 propose a method for estimating a camera posture parameter based on a parallel line such as a white line drawn on the ground or an infinite distance obtained from the parallel line.
Furthermore, Patent Document 5 discloses posture parameter estimation for estimating posture parameters by performing a bird's-eye view conversion on a captured image of parallel lines drawn on the ground based on appropriate posture parameters and calculating parallelism of the parallel lines. A device has been proposed. With this apparatus, it is possible to estimate the posture parameter only with parallel lines.

Japanese Patent Laid-Open No. 2001-91984 JP 2000-142221 A JP-A-7-77431 JP 7-147000 A JP 2008-182652 A

RYTsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses”, Transactions on Pattern Analysis and Machine Intelligence 22 (11), IEEE, 1987, p.323-344 .

The conventional techniques represented by Patent Documents 1 and 2 flexibly cope with a temporary posture deviation because the camera posture needs to be adjusted when the camera is not installed in the posture indicated by the posture parameter. There was a problem that it was not possible.
For example, in a camera mounted on a vehicle, when the vehicle is tilted due to loading of passengers or luggage, the camera posture is shifted. In this case, if the number of passengers, the weight of luggage, etc. change, the camera posture may also change, and it is necessary to reestimate the camera posture each time.
In addition, the conventional technique needs to prepare a dedicated test pattern. Therefore, a storage place for the test pattern and a place for adjusting the camera posture are required, and the cost for preparing the test pattern is high. For this reason, there existed a subject that posture estimation of a camera could not be performed simply.

  Further, the conventional techniques represented by Patent Documents 3 and 4 require an infinite distance obtained from parallel lines for camera posture estimation. For this reason, if the road that captures the white line is curved, or if there are obstacles such as vehicles or buildings, infinity cannot be obtained (or it becomes difficult), and camera posture estimation is not easy There was a problem.

  Furthermore, in the conventional technique represented by Patent Document 5, since the posture of the camera is estimated using the parallel line of the traveling lane, in the side camera attached to the side of the vehicle, only one of the parallel white lines of the traveling lane is present. It cannot be imaged and cannot be easily applied to posture estimation of the side camera. Further, in Patent Document 5, since the camera mounting height and focal length are used for estimation of posture parameters, it is necessary to measure these values in advance and input them to the posture parameter estimation device.

  Furthermore, in Non-Patent Document 1 and Patent Documents 1 to 5 described above, since the camera posture is adjusted by viewpoint conversion based on the posture parameter, setting of the virtual viewpoint and the conversion processing based on this are complicated, and simple parameter adjustment is performed. I can't do it.

  The present invention has been made to solve the above-described problems. An overhead image generation apparatus, an overhead image generation method, and an overhead conversion using the overhead image generation apparatus that can easily adjust conversion parameters for converting a captured image into an overhead image. It is an object of the present invention to obtain a video display system for displaying a video and a navigation device capable of displaying a bird's-eye converted image using the video display system.

  A bird's-eye view video generation device according to the present invention includes a parameter storage unit that stores a conversion parameter for converting an image into a bird's-eye view image by projective transformation, and a captured image captured by the imaging unit using the conversion parameter read from the parameter storage unit. To obtain a bird's-eye view image obtained by converting a bird's-eye view image obtained by converting a captured image obtained by capturing an image of an equally spaced pattern by the bird's-eye view conversion unit. Based on the interval index value calculated by the interval index calculation unit and the interval index calculation unit that calculates the interval index value indicating the conversion parameter, the conversion parameter in which the interval of the marking pattern is aligned within a predetermined range on the overhead image is calculated. And a parameter calculation unit stored in the parameter storage unit.

  According to the present invention, there is an effect that the conversion parameter for converting the captured image into the overhead image can be easily adjusted.

It is a block diagram which shows the structure of the video display system provided with the bird's-eye view video production | generation apparatus which concerns on Embodiment 1 of this invention. 1 is a block diagram illustrating a configuration of a navigation device including an overhead video generation device according to Embodiment 1. FIG. It is a block diagram which shows the structural example of a labeled imaging data acquisition part. 4 is a flowchart showing an operation of the overhead view video generation device according to the first embodiment. It is a figure which shows the overhead view conversion which concerns on Embodiment 1. FIG. FIG. 10 is a diagram illustrating an overhead parameter estimation process based on an interval index according to the first embodiment. It is a block diagram which shows the structure of the video display system provided with the bird's-eye view video production | generation apparatus which concerns on Embodiment 2 of this invention. FIG. 10 is a block diagram illustrating a configuration of a navigation device including the overhead view video generation device according to the second embodiment. 6 is a flowchart illustrating an operation of the overhead view video generation device according to the second embodiment. It is a figure which shows the estimation process of the overhead parameter based on the space | interval parameter | index which concerns on Embodiment 2. FIG.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a video display system including an overhead view video generation device according to Embodiment 1 of the present invention, and shows a vehicle surroundings display system for a vehicle equipped with the overhead view video generation device according to the present invention. Show. In FIG. 1, a vehicle surroundings display system 1 according to Embodiment 1 includes a camera 10, a bird's-eye view image generation device 20, a monitor 30, and an operation unit 40.
The camera 10 is an imaging unit that is installed on the vehicle body and images the surroundings of the vehicle. The bird's-eye view video generation device 20 is a device that converts an image of image data captured by the camera 10 into a bird's-eye view image as seen from directly above. The monitor 30 is a display unit that displays an overhead image generated by the overhead image generation device 20. A driver of the vehicle can recognize an obstacle or the like by viewing the overhead image displayed on the monitor 30 from an image as if the vehicle was viewed from directly above.
The operation unit 40 is an operation unit that accepts an operation input from the outside. In addition to hardware such as a keyboard and buttons, the operation unit 40 may be software keys or buttons displayed on the screen of the monitor 30 that performs input using a touch panel. Good.

The overhead view video generation device 20 has a function of estimating an overhead view parameter for overhead conversion of an image captured by the camera 10. The bird's-eye view parameter is a conversion parameter for converting the coordinate system of the image captured by the camera 10 into the coordinate system of the bird's-eye image viewed from directly above the same object to be captured based on the two-dimensional projective transformation.
In addition, since the overhead image generation device 20 generates an overhead image using the ground (road surface) as a reference plane, the pedestrian crossing drawn on the road has little distortion and the like, as if the vehicle was actually viewed from directly above. Displayed with high accuracy.

As shown in FIG. 1, the bird's-eye view video generation device 20 includes a signing imaging data acquisition unit 21, an imaging data storage unit 22, an overhead conversion unit 23, an overhead parameter estimation unit 24, and an overhead parameter storage unit 25 as functional blocks.
The sign image data acquisition unit 21 is an image acquisition unit that acquires image data in which a predetermined sign is imaged from image data captured by the camera 10. Here, the predetermined sign is a sign of an equally spaced pattern drawn on a reference plane (for example, the ground), and includes, for example, a pedestrian crossing or a zebra pattern in which white line patterns are drawn at equal intervals. .

  In addition, the sign imaging data acquisition unit 21 acquires a captured image captured by the camera 10 at a timing designated by the user using the operation unit 40 as a captured image in which a predetermined sign is captured. That is, when the vehicle travels or stops on a road on which a predetermined sign such as a pedestrian crossing is drawn, a predetermined instruction is captured by the user using the operation unit 40 to take in an instruction. A captured image can be acquired.

The imaging data storage unit 22 is an image storage unit that stores imaging data obtained by imaging a predetermined label acquired by the label imaging data acquisition unit 21.
The overhead conversion unit 23 converts the captured image into an overhead image using an overhead parameter that is a conversion parameter for converting the captured image captured by the camera 10 into an overhead image by two-dimensional projective conversion. The monitor 30 is a display unit that acquires and displays the overhead image generated by the overhead conversion unit 23.

The bird's-eye view parameter estimation unit 24 estimates a bird's-eye view parameter for performing bird's-eye conversion on an image of the captured image captured by the camera 10. The overhead parameter storage unit 25 is a parameter storage unit that stores the overhead parameter calculated by the overhead parameter estimation unit 24.
The overhead parameter estimation unit 24 includes a binarization conversion unit 24a, an interval index calculation unit 24b, and an overhead parameter calculation unit 24c as functional blocks.

The binarization conversion unit 24a compares the pixel value in the bird's-eye view image generated by the bird's-eye conversion unit 23 with a predetermined threshold related to the pixel, and generates an image with the maximum luminance (white) and the minimum luminance (black). The image is binarized by conversion. Although the amount of calculation in the calculation of the interval index value is reduced by binarization, binarization conversion is not an essential process in the present invention. That is, a predetermined sign may be extracted directly from the overhead image by image recognition such as pattern matching.
Based on a predetermined label in the binarized image generated by the binarization conversion unit 24a, the interval index calculation unit 24b displays an interval index indicating the degree to which the patterns constituting the label are aligned on the image. calculate. For example, in the case of markings of equally spaced white line patterns, the variance of the spacing between adjacent white line patterns is the spacing index.

The overhead parameter calculation unit 24c is a parameter calculation unit that calculates an overhead parameter to be converted into an appropriate overhead image based on the value of the interval index calculated by the interval index calculation unit 24b.
In the present invention, a sign (such as a pedestrian crossing) composed of equally spaced patterns drawn on a reference surface such as the ground is targeted. If an image obtained by capturing this sign is converted to a bird's-eye view, the pattern is also displayed in the bird's-eye view image. Is supposed to be expressed at regular intervals.
That is, when the bird's-eye view parameter is not appropriate, in the bird's-eye view image generated by using this, there is a possibility that the interval between the patterns that should be originally equal may be disturbed.
Therefore, the bird's-eye view parameter calculation unit 24c calculates the bird's-eye view parameter based on the value of the interval index calculated by the interval index calculation unit 24b so that the marking patterns in the bird's-eye view image are equally spaced.

  FIG. 2 is a block diagram illustrating a configuration of a navigation device including the overhead view video generation device according to the first embodiment. In FIG. 2, the navigation device 2 is a navigation device for a moving body mounted on a moving body (for example, a vehicle), and generates an overhead video that converts an image captured by the camera 10 into an overhead image and outputs it to the monitor 30. The apparatus 20 includes a navigation processing unit 3, a map database (map DB) 4, a position information acquisition unit 5, and a position calculation unit 6.

  The navigation processing unit 3 executes navigation processing such as route search, route guidance, and map display based on the vehicle position calculated by the position calculation unit 6 and the map data in the map DB 4. The map DB 4 is a database in which map data used by the navigation processing unit 3 is registered. Map data includes topographic map data, residential map data, road networks, and the like. Image information for route guidance and map display is sent from the navigation processing unit 3 to the display control unit 23 a and displayed on the monitor 30.

The position information acquisition unit 5 is a position information acquisition unit that acquires position information related to the current position, and acquires position information from satellite position and time information included in a GPS signal received from a GPS satellite, for example. This position information is output from the position information acquisition unit 5 to the position calculation unit 6.
The position calculation unit 6 calculates the current position of the host vehicle based on the position information acquired from the position information acquisition unit 5.

  The display control unit 23a is a control unit that controls display processing of the monitor 30, and includes the overhead view conversion unit 23 illustrated in FIG. Image information from the navigation processing unit 3 can be converted into an overhead image by the overhead conversion unit 23 of the display control unit 23 a and displayed on the monitor 30.

  In addition, the sign imaging data acquisition unit 21 is based on the vehicle position calculated by the position calculation unit 6 and the map data in the map DB 4. If it is determined that the vehicle is stopped, it is determined that the camera 10 is imaging the sign, and imaging data captured by the camera 10 at this time is acquired and stored in the imaging data storage unit 22. A point with a predetermined sign is a point where a crosswalk or zebra pattern such as an intersection or a curve is drawn on the road, and is set in advance in a road network, for example.

Furthermore, if the vehicle lane information of the vehicle can be identified from the vehicle position and the map data, if the information on the point is associated with the road lane information and registered in the map data, a predetermined sign is captured. Images can be obtained accurately.
Although not shown in FIG. 2, the sign imaging data acquisition unit 21 acquires the imaging data captured by the camera 10 at the timing specified by the user using the operation unit 40, as in FIG. 1. You may make it memorize | store in the data storage part 22. FIG.

Further, the sign imaging data acquisition unit 21 is configured as follows, and there is a possibility that a predetermined sign is captured from the imaging data captured by the camera 10 using the vehicle status of the host vehicle or image recognition. High imaging data may be acquired.
FIG. 3 is a block diagram illustrating a configuration example of the sign imaging data acquisition unit. The sign imaging data acquisition unit 21 illustrated in FIG. 3A determines whether or not the imaging data of the camera 10 should be acquired based on the vehicle data acquired from the vehicle control unit 7 of the host vehicle.
That is, based on the vehicle data acquired from the vehicle control unit 7 when capturing the image data captured by the camera 10 at the timing specified by the user using the operation unit 40 or the position timing based on the vehicle position and the map data. Thus, it is determined whether or not the captured image data can be captured.

  Examples of the vehicle data include vehicle speed information and steering angle information. When the captured image data acquisition unit 21 captures captured image data captured by the camera 10 and the speed of the vehicle indicated by the vehicle speed information exceeds a predetermined threshold, the vehicle 10 is high-speed and the camera 10 accurately displays the predetermined label. It is determined that imaging cannot be performed, and imaging data is not acquired from the camera 10 even at a user specified timing or position timing.

In addition, the predetermined sign may be captured obliquely in the captured image depending on the relationship between the traveling direction of the vehicle and the imaging field of view of the camera 10. In this case, based on the steering angle information of the own vehicle at the time of image capturing by the camera 10, the rotation angle at which the captured image is rotated to convert the sign into a predetermined direction may be determined.
As described above, by using the vehicle data, it is possible to improve the acquisition accuracy of the imaging data obtained by imaging the predetermined sign.

Moreover, the sign imaging data acquisition part 21 shown in FIG.3 (b) is provided with the image recognition part 21a. The image recognition unit 21a detects an image of a predetermined sign by performing an image recognition process on the image data captured by the camera 10. The sign image data acquisition unit 21 stores the image data in the image data storage unit 22 when a predetermined sign is detected by the image recognition process performed by the image recognition unit 21a.
Note that the timing at which the image recognition unit 21a performs the image recognition process may be a user-specified timing using the operation unit 40 or a position timing based on the vehicle position and map data.
By configuring in this way, it is possible to further improve the acquisition accuracy of imaging data obtained by imaging a predetermined sign.

  In addition, the sign imaging data acquisition unit 21 may detect that the weather at the time of imaging by the camera 10 is rain, snow, or fog based on weather information or time information acquired via a communication device mounted on the vehicle. When it is determined that the time is nighttime, imaging data is not acquired from the camera 10 even at a user-specified timing or position timing. By doing so, it is possible to prevent the data in which the sign is imaged from being stored in a state where the interval index value cannot be calculated.

Next, the operation will be described.
FIG. 4 is a flowchart showing the operation of the overhead view video generation apparatus according to the first embodiment.
In the normal bird's-eye view image generation device 20, the bird's-eye view conversion unit 23 acquires imaging data from the camera 10 to generate a bird's-eye view image and outputs it to the monitor 30. At this time, when the sign imaging data acquisition unit 21 performs the processing from step ST1 to step ST3 and the overhead parameter estimation unit 24 estimates the overhead parameter, the processing from step ST4 to step ST12 is performed.

  First, the sign imaging data acquisition unit 21 acquires imaging data captured by the camera 10 (step ST1). Next, the sign imaging data acquisition unit 21 determines whether or not the acquired imaging data matches the conditions for capturing the imaging data storage unit 22 (step ST2). At this time, the imaging data acquired from the camera 10 is imaged at a timing designated by the user using the operation unit 40, for example, or a position timing based on the vehicle position and map data (a point where a predetermined sign exists). Whether the captured image data is the captured image data or the captured image data from which a predetermined sign has been extracted by the image recognition process.

When the captured image data matches the capturing conditions (step ST2; YES), the sign imaged data acquisition unit 21 stores the captured image data acquired from the camera 10 in the captured data storage unit 22 (step ST3).
If the capturing condition is not met (step ST <b>2; NO), the sign imaging data acquisition unit 21 discards the imaging data acquired from the camera 10 without saving it in the imaging data storage unit 22. Thereafter, the process proceeds to step ST4.

  As described above, processing load in the vehicle surrounding display system 1 or the navigation device 2 provided with the overhead view video generation device 20 by temporarily storing the captured image used for estimation of the overhead view parameter in the imaging data storage unit 22. It is possible to perform the overhead parameter estimation processing after step ST4 by selecting a time interval with a small number of intervals. For example, a threshold is provided for the operating rate of the CPU that performs the arithmetic processing of the vehicle surrounding display system 1 or the navigation device 2, and the overhead parameter estimation processing is started when the operating rate of the CPU is lower than the threshold.

  The overhead conversion unit 23 determines whether there is data in the imaging data storage unit 22 (step ST4). If there is no data in the imaging data storage unit 22 (step ST4; NO), the process returns to step ST1. When data is stored in the imaging data storage unit 22 (step ST4; YES), the overhead conversion unit 23 reads the current overhead parameter from the overhead parameter storage unit 25, and uses the overhead parameter to use the imaging data storage unit. The image of the data read from 22 is overhead-converted (step ST5). Specifically, using the overhead parameters A to H, the coordinates are converted into the coordinates of the overhead image according to the conversion formula of the following formula (1).

The overhead parameters A to H in the above formula (1) are constants determined from the coordinates of the four vertices in the image plane. The four vertices of the output image (overhead image) corresponding to the four vertices of the input image, a (x1, y1), b (x2, y2), c (x3, y3), d (x4, y4) are represented by e (X1, When Y1), f (X2, Y2), g (X3, Y3), and h (X4, Y4), the overhead parameters A to H can be derived from the matrix of the following formula (2).

FIG. 5 is a diagram showing the overhead view conversion according to Embodiment 1, and shows a case where an image obtained by photographing a pedestrian crossing with a side camera when the vehicle travels an intersection is converted into an overhead view image by the projective transformation. Yes. The coordinates of the four vertices of the input image described in the upper part of FIG. 5 are a (0,0), b (1,0), c (1,1), d (0,1), and the four vertices of the input image When projective transformation of the above equation (2) is applied to the coordinates of the corresponding vertices, the coordinates of the four vertices of the output image (overhead image) are e (−0.5, 0), f as shown in the lower row. (1.5,0), g (1,1), h (0,1).

  The binarization conversion unit 24a acquires the overhead view image generated by the overhead view conversion unit 23, and performs binarization conversion on the overhead view image (step ST6). Thus, a predetermined sign such as a pedestrian crossing drawn on the ground is extracted from the overhead image. Thereafter, the interval index calculation unit 24b calculates an interval index indicating the degree to which the pattern interval of the label on the bird's-eye view image, that is, the label extracted by binarization is aligned (step ST7).

FIG. 6 is a diagram illustrating an overhead parameter estimation process based on the interval index according to the first embodiment. As illustrated in FIG. 6, the overhead conversion unit 23 changes the overhead parameter used in step ST5 for each predetermined amount, and executes overhead conversion using the changed overhead parameter. In the example of FIG. 6, conversion is performed so that the overhead conversion amount increases from top to bottom.
The binarization conversion unit 24a binarizes and converts each overhead view image converted by the overhead view conversion unit 23. As a result, the images shown in FIGS. 6A, 6C, 6E, and 6G are obtained. For binarization conversion, for example, simple binarization combined with a top hat filter is used.

Thereafter, the interval index calculation unit 24b calculates the number of white pixels based on the vertical direction for each of the above-described binarized images, as shown in (b), (d), (f), and (h) of FIG. As shown in FIG. Next, the interval index calculation unit 24b provides a certain threshold value for the above-described histogram, and separates into a part having the number of white pixels equal to or greater than the threshold value and the other part. Thereby, a white line portion on the image and a non-white line portion (a portion corresponding to the interval between adjacent white lines) are discriminated. The interval index calculation unit 24b calculates the width of the white line and the width of the portion that is not the white line.
Note that the width of the upper end of the image and the lower end of the image is not considered because the width cannot be obtained accurately.

  The interval index calculation unit 24b calculates the respective variances for the width of the white line and the width of the portion that is not the white line. The variance can be calculated by the following equation (3) using each width xi of the N white lines and the average value μ of the widths of the N white lines. Also, the variance is similarly calculated for each width of the portion that is not N white lines.

Next, the interval index calculation unit 24b calculates an average value of the variance of the width of the white line and the variance of the width of the portion that is not the white line to obtain an interval index value. In this case, the lower the interval index value, that is, the closer the widths are to the same, the closer the intervals are.

  Returning to the description of FIG. When the interval index value is calculated as described above, the overhead parameter calculation unit 24c increments or decrements the overhead parameter value, or adds or subtracts a predetermined value to each overhead parameter value, for example. Update the parameter value. Thereafter, the overhead view parameter calculation unit 24c determines whether or not the interval index value has been calculated based on the N overhead view parameters at the present time (step ST8).

  At present, the interval index value based on one overhead parameter is calculated. For this reason, the overhead parameter calculation unit 24c determines that the interval index value based on the N overhead parameters is not calculated (step ST8; NO). Next, the overhead parameter calculation unit 24c changes to the next overhead parameter (step ST12). In the case of FIG. 5, the overhead parameter calculation unit 24c changes, for example, the x-coordinates of the vertices e and f of the output image by 0.1 and changes them to overhead parameters for projective conversion to this value.

  Then, it returns to the process of step ST5 and repeats the above-mentioned process using the bird's-eye view parameter after a change. During this time, for example, an overhead image as shown in FIG. 6E is obtained, the number of white pixels in the binarized image is histogrammed, and a histogram as shown in FIG. 6F is obtained. In the histogram shown in (f) of FIG. 6, the widths of the white lines and the portions that are not white lines are constant. At this time, the interval index calculation unit 24b calculates an average value of these width dispersions = “0” as the interval index value.

  When the interval index value is calculated based on the N overhead parameters (step ST8; YES), the overhead parameter calculation unit 24c appropriately sets the overhead parameter when the interval index value (average value of the variance of the width) is the lowest. The parameter is estimated and stored in the overhead parameter storage unit 25 (step ST9).

  Next, the overhead conversion unit 23 deletes the imaging data for which the overhead parameter estimation by the overhead parameter estimation unit 24 has been completed from the imaging data stored in the imaging data storage unit 22 (step ST10). Thereafter, the overhead conversion unit 23 reads the optimized overhead parameter from the overhead parameter storage unit 25, generates an overhead image based on this, and displays it on the monitor 30 (step ST11). Thereby, the process of FIG. 4 is completed.

As described above, according to the first embodiment, the overhead view conversion unit 23 uses the overhead view parameters read from the overhead view parameter storage unit 25 that stores the overhead view parameters for converting the image into the overhead view image by projective transformation. 10 is converted into a bird's-eye view image, and the interval index calculation unit 24b indicates the degree to which the intervals of the marking patterns are aligned on the bird's-eye view image obtained by converting the captured image in which the markings of equally spaced patterns are captured. An interval index value is calculated, and the overhead parameter calculation unit 24c calculates an overhead parameter in which the intervals of the marking patterns are aligned within a predetermined range on the overhead image based on the calculated interval index value. 25.
In this way, the overhead view conversion can be performed with a simple calculation because projection conversion is used instead of conventional viewpoint conversion based on camera posture parameters. Furthermore, it is possible to calculate the interval index value of the marking pattern from the overhead image obtained by imaging the marking of the equally spaced pattern, and to easily adjust the overhead parameter based on this.

The equidistant pattern markings drawn on the reference surface such as the ground should be represented by the equidistant pattern in the overhead view image, but if the overhead parameters are not appropriate, the marking pattern on the overhead view image Disturbance occurs in the interval.
Therefore, in the present invention, it is possible to easily calculate an appropriate overhead parameter by calculating the interval index value of the equally spaced pattern captured in the overhead image and adjusting the overhead parameter so that the pattern intervals are aligned based on this. Can be adjusted.
Moreover, since the overhead view parameter is obtained from the overhead view image, it is not necessary to prepare a test pattern or the like in advance and it is not necessary to obtain infinity, so that the difficulty of estimating an appropriate overhead view parameter can be reduced. Further, the overhead view parameter can be estimated even with the left and right cameras of the vehicle.

Further, according to the first embodiment, the sign imaging data acquisition unit 21 that acquires a captured image in which signs having an equally spaced pattern are captured, and the imaging that stores the captured image acquired by the label imaging data acquisition unit 21. A data storage unit 22, the overhead conversion unit 23 converts the captured image read from the imaging data storage unit 22 into an overhead image, and the interval index calculation unit 24b acquires the overhead image converted by the overhead conversion unit 23. Then, an interval index value indicating the degree to which the intervals of the marking patterns are aligned on the overhead image is calculated.
With this configuration, the captured image can be temporarily stored in the captured data storage unit 22, and the overhead parameter estimation processing can be performed by selecting a gap with a small processing load.

  Furthermore, according to the first embodiment, the binarization conversion unit 24a that binarizes and converts the overhead image of the captured image in which the markings of the equally spaced pattern are captured is provided. The interval index value is calculated using the marking pattern binarized by the conversion unit 24a. At this time, the interval index calculation unit 24b creates a histogram of the number of pixels of the marking pattern binarized and converted, and the interval index value is determined using the width of the pattern determined based on this histogram and the interval between adjacent patterns. Is calculated. In this way, the interval index value can be easily calculated using an image processing technique such as binarization conversion.

  Further, according to the first embodiment, the sign image data acquisition unit 21 performs image recognition on the picked-up image picked up by the camera 10 and acquires a picked-up image obtained by picking up marks with uniform intervals. Therefore, it is possible to accurately acquire an image in which a sign drawn on a reference surface such as the ground is captured.

  Furthermore, according to the first embodiment, the operation unit 40 that receives an operation from the outside is provided, and the captured image data acquisition unit 21 is captured by the camera 10 at a timing specified by using the operation unit 40. Is acquired as a captured image in which signs having an equally spaced pattern are picked up, so that an image in which a sign drawn on a reference plane such as the ground is picked up can be acquired with high accuracy.

  Further, according to the first embodiment, the camera 10 is mounted on the vehicle and images the surroundings thereof, and the indication imaging data acquisition unit 21 indicates the equidistant pattern based on the current position and the map data. Since the captured image captured by the camera 10 at the timing when it is determined that the host vehicle has arrived at an existing point is acquired as a captured image in which a sign of an equally spaced pattern is captured, the captured image is drawn on a reference plane such as the ground. It is possible to accurately acquire an image obtained by capturing a sign.

  Furthermore, according to the first embodiment, since the navigation device 2 includes the above-described overhead view video generation device 20, the image information generated by the navigation processing is converted into an overhead view image using an appropriate overhead view parameter and presented. can do. In addition, by temporarily storing the captured image in the captured data storage unit 22, it is possible to perform the overhead parameter estimation process by selecting the interval between the navigation processes.

In the first embodiment, the interval index calculation unit 24b calculates, as the interval index value, the average value of the widths of the white lines and the portions that are not white lines based on the N overhead parameters, and the overhead parameter calculation unit 24c. However, when the overhead index when the interval index value is the lowest (the average value of the variance of the width is the lowest) is estimated to be the most accurate parameter, the overhead parameter estimation according to the present invention is It is not limited to this.
For example, in consideration of an imaging error of the camera 10, the interval between the marking patterns on the overhead image is aligned within a predetermined error range, that is, a range in which the interval index value is higher than a predetermined threshold (an average of the dispersion of widths). If the value is in a range lower than a predetermined threshold), it may be determined that the overhead parameter accuracy is high.
Furthermore, the overhead parameter storage unit 25 may not be overwritten with the overhead parameter, but may be updated in consideration of the overhead parameter calculated in advance.

Embodiment 2. FIG.
FIG. 7 is a block diagram showing a configuration of a video display system provided with the overhead view video generation apparatus according to Embodiment 2 of the present invention. Similarly to FIG. 1, a vehicle equipped with the overhead view video generation apparatus according to the present invention. 1 shows a vehicle surrounding display system. In FIG. 7, a vehicle surroundings display system 1 </ b> A according to Embodiment 2 includes a camera 10, an overhead view video generation device 20 </ b> A, and a monitor 30.
As shown in FIG. 7, the bird's-eye view image generation device 20 </ b> A includes a signing imaging data acquisition unit 21, an imaging data storage unit 22, an overhead conversion unit 23, an overhead parameter estimation unit 24 </ b> A, and an overhead parameter storage unit 25.

In the overhead parameter estimation unit 24A, an edge extraction processing unit 24d is added to the configuration shown in FIG. The edge extraction processing unit 24d is an edge extraction unit that extracts edges on the binarized image, and identifies a line on the overhead image (binarized image) by the edge extraction processing.
Although the amount of calculation in the edge extraction process and the interval index value calculation process is reduced by binarization, the binarization conversion is omitted if a predetermined sign can be identified from the original overhead image by the edge extraction process. That is, the binarization conversion unit 24a may be omitted.

  The interval index calculation unit 24b calculates the interval index value of the pattern on the overhead image indicated by the overhead image data. The pattern on the bird's-eye view image is a line specified by the edge extracted by the edge extraction processing unit 24d. That is, the interval index calculation unit 24b determines the edge extracted by the edge extraction processing unit 24d as a line on the overhead image and calculates the interval index value.

  FIG. 8 is a block diagram illustrating a configuration of a navigation device including the overhead view video generation device according to the second embodiment. In FIG. 8, a navigation device 2A is a navigation device for a mobile body mounted on a mobile body (for example, a vehicle). The overhead view video generation device 20A according to the second embodiment, the navigation processing unit 3, the map DB 4, A position information acquisition unit 5 and a position calculation unit 6 are provided. In the navigation device 2A, the configuration other than the overhead view video generation device 20A shown in FIG. 7 is the same as the configuration described with reference to FIG.

Next, the operation will be described.
FIG. 9 is a flowchart illustrating the operation of the overhead view video generation device according to the second embodiment. 9, the processing from step ST1 to step ST3 performed by the sign imaging data acquisition unit 21, the processing from step ST4 to step ST5 performed by the overhead view parameter estimation unit 24, and the processing from step ST8 to step ST12 are shown in FIG. The same as 4.
Therefore, the process of step ST6-1 and step ST7a which are different processes from Embodiment 1 will be described.

In step ST6-1, the edge extraction processing unit 24d extracts an edge on the bird's-eye view image (in which a predetermined sign is captured) binarized and converted by the binarization conversion unit 24a.
Next, the interval index calculation unit 24b calculates the interval index value of the marking by using the edge extracted by the edge extraction processing unit 24d as a line on the overhead image (step ST7a).

  FIG. 10 is a diagram illustrating an overhead parameter estimation process based on the interval index according to the second embodiment. First, the edge extraction processing unit 24d extracts the edge of the image binarized by the binarization conversion unit 24a. Thereby, the image shown in FIG. 10A is obtained. In this edge extraction process, for example, a Sobel filter that extracts edges by first-order differentiation of image values is used.

  Subsequently, the interval index calculation unit 24b extracts a straight line for the edge extracted by the edge extraction processing unit 24d. For example, a straight line is extracted using Hough transform. The Hough transform is a process of transforming the coordinates (x, y) on the image into a distance ρ from the origin and an angle θ. By using the Hough transform, the angle of each straight line can be converted into a histogram as shown in FIG. Further, the distance from the origin of each straight line can be represented by ρ.

The interval index calculation unit 24b can extract only a straight line having a constant angle θ by providing an appropriate threshold value for the result of the histogram of the angle θ. FIG. 10C shows the result of extracting only the horizontal line by the comparison process with the threshold value.
Since the distance between each straight line and the origin is represented by ρ, the width of each straight line can be calculated using the distance ρ. The interval index calculation unit 24b processes the width of each straight line in the same manner as in the first embodiment, and calculates the average value of the variances of the widths as the interval index value. The overhead parameter calculation unit 24c uses the interval index value to calculate the optimized overhead parameter as in the first embodiment.

  As described above, according to the second embodiment, the edge index processing unit 24b includes the edge extraction processing unit 24d that extracts the edge on the overhead image of the captured image in which the markings of the equally spaced pattern are captured. An interval index value is calculated using the edge of the label extracted by the edge extraction processing unit 24d. At this time, the interval index calculation unit 24b obtains the pattern width and the interval between the adjacent patterns based on the distance between the edge of the marking pattern extracted by the edge extraction processing unit 24d and the origin on the overhead image. An interval index value is calculated using the width of the pattern and the interval between adjacent patterns. In this way, the interval index value can be calculated easily. Further, even if the markings of equally spaced patterns are captured obliquely on the captured image, the pattern width and the interval between adjacent patterns can be obtained from the distance between the edge and the origin on the overhead image.

  In the present invention, within the scope of the invention, any combination of each embodiment, any component of each embodiment can be modified, or any component can be omitted in each embodiment. .

  DESCRIPTION OF SYMBOLS 1,1A Vehicle surrounding display system, 2,2A navigation apparatus, 3 navigation processing part, 4 map database (map DB), 5 position information acquisition part, 6 position calculation part, 7 vehicle control part, 10 camera, 20, 20A Video generation device, 21 Marking imaging data acquisition unit, 21a Image recognition unit, 22 Imaging data storage unit, 23 Overhead conversion unit, 23a Display control unit, 24, 24A Overhead parameter estimation unit, 24a Binary conversion unit, 24b Interval index Calculation unit, 24c Overhead parameter calculation unit, 24d Edge extraction processing unit, 25 Overhead parameter storage unit, 30 monitor, 40 operation unit.

Claims (12)

A parameter storage unit for storing conversion parameters for converting an image into an overhead image by projective conversion;
An overhead conversion unit that converts a captured image captured by the imaging unit into an overhead image using the conversion parameter read from the parameter storage unit;
An interval for obtaining a bird's-eye view image obtained by converting a picked-up image obtained by picking up an image of a pattern with equal intervals by the bird's-eye view conversion unit, and calculating an interval index value indicating the degree to which the intervals of the pattern of the marking are aligned on the bird's-eye view image An index calculation unit;
Based on the interval index value calculated by the interval index calculation unit, a conversion parameter in which the interval of the marking pattern is aligned within a predetermined range on the overhead image is calculated, and parameter calculation is stored in the parameter storage unit And a bird's-eye view image generation device. An image acquisition unit for acquiring a captured image in which the indication of the equally spaced pattern is captured;
An image storage unit that stores the captured image acquired by the image acquisition unit,
The overhead view conversion unit converts the captured image read from the image storage unit into an overhead image,
The interval index calculation unit acquires the overhead image converted by the overhead view conversion unit, and calculates an interval index value indicating the degree to which the intervals of the marking patterns are aligned on the overhead image. The overhead view video generation device according to 1. A binarization conversion unit that binarizes and converts a bird's-eye view image of a captured image obtained by imaging the equidistant pattern markings;
3. The overhead image generation according to claim 1, wherein the interval index calculation unit calculates the interval index value using the marking pattern binarized by the binarization conversion unit. apparatus.   The interval index calculation unit creates a histogram of the number of pixels of the marking pattern binarized and converts the interval index value using the width of the pattern determined based on the histogram and the interval between adjacent patterns. The overhead image generation apparatus according to claim 3, wherein the overhead image generation apparatus is calculated. An edge extraction processing unit that extracts an edge on an overhead image of a captured image in which the markings of the equally spaced patterns are captured;
The overhead image generation device according to claim 1, wherein the interval index calculation unit calculates the interval index value using an edge of the sign extracted by the edge extraction processing unit.   The interval index calculation unit obtains an interval between adjacent patterns and the width of the pattern based on the distance between the edge of the marking pattern extracted by the edge extraction processing unit and the origin on the overhead image, 6. The overhead image generation apparatus according to claim 5, wherein the interval index value is calculated using a pattern width and an interval between adjacent patterns.   The bird's-eye view according to claim 2, wherein the image acquisition unit performs image recognition on a captured image captured by the imaging unit, and acquires a captured image in which signs of equally spaced patterns are captured. Video generation device. It has an operation unit that accepts external operations,
The said image acquisition part acquires the picked-up image imaged by the said image pick-up part at the timing designated using the said operation part as a picked-up image by which the label of the pattern of equal intervals was imaged. 3. The bird's-eye view image generation device according to 2. The imaging unit is mounted on a moving body and images the surroundings,
The image acquisition unit captures a captured image captured by the imaging unit at a timing when it is determined that the moving body has arrived at a point where a sign of an equally spaced pattern exists based on the current position and map data, etc. The bird's-eye view image generation apparatus according to claim 2, wherein the image is acquired as a captured image in which an indication of an interval pattern is captured. An overhead conversion step in which the overhead conversion unit converts the captured image captured by the imaging unit into an overhead image using the conversion parameter read from the parameter storage unit that stores the conversion parameter for converting the image into an overhead image by projective conversion. When,
The interval index calculation unit inputs an overhead image obtained by converting a captured image obtained by capturing the markings of the equally spaced pattern in the overhead view conversion step, and indicates the degree to which the intervals of the marking patterns are aligned on the overhead image. An interval index calculating step for calculating a value;
Based on the interval index value calculated in the interval index calculation step, the parameter calculation unit calculates a conversion parameter in which the interval of the marking pattern is aligned within a predetermined range on the overhead image, and the parameter storage unit And a parameter calculation step stored in the overhead view video generation method. In a video display system that displays a bird's-eye view around a moving object,
An imaging unit for imaging the periphery of the moving body;
An overhead view video generation device according to any one of claims 1 to 9,
A video display system, comprising: a display unit that displays the overhead image generated by the overhead image generation device.   A navigation device comprising the overhead view video generation device according to any one of claims 1 to 9.

Images