JP3084208B2 - Image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for accurately recognizing a specific line image by enlarging a captured image information, and by enlarging the image information captured by a vehicle-mounted camera. The present invention relates to a device for accurately recognizing a lane of a running lane of a vehicle.

[0002]

2. Description of the Related Art As a device for recognizing a traveling lane based on image data captured by an on-vehicle camera, there is generally a method in which a single camera captures a situation in front. However, in such a device, if a configuration including a wide-angle lens is used in order to recognize a wide range, it is difficult to recognize a distant lane. There is a problem that the recognizable visual field becomes narrow when the mechanism is used.

As an apparatus for improving this, for example, Japanese Patent Laid-Open
One disclosed in JP-A-06-229760 is exemplified.

This device has a solid-state imaging section having a low-magnification optical system, a solid-state imaging section having a high-magnification optical system, and image data obtained by the solid-state imaging section having a low-magnification optical system. Apparatus for recognizing that a relatively distant running path ahead of a vehicle is a curved road or that the vehicle is in a state of transition to a curved road by comparing image data obtained by a solid-state imaging unit having a system. It is.

[0005]

However, according to the prior art as described above, when recognizing the shape of a relatively distant lane, a configuration is provided in which a solid-state imaging unit having a high-magnification optical system is provided. . For this reason, both low-magnification optical system and high-magnification optical system, and a solid-state imaging unit for each optical system are required, the device configuration becomes large, and it cannot be said that the device is necessarily suitable for in-vehicle use. However, there is a problem that an increase in cost is unavoidable.

In consideration of the on-vehicle environment, there has been a long-awaited demand for an apparatus configuration with as few optical systems as possible under high temperatures or in environments with large vibration amplitudes.

In view of the above-mentioned problems of the conventional apparatus, the present invention has a configuration including a plurality of optical systems for recognizing a lane for a traveling lane of a host vehicle based on image information captured by a vehicle-mounted camera. It is another object of the present invention to provide a device which can recognize a lane accurately even in a relatively distant traveling lane while securing a wide field of view.

[0008]

The following means are conceivable in order to solve the above problems.

That is, an image pickup means for picking up an image and obtaining image information, an image enlargement processing means for enlarging the size of the picked-up image information, and scanning the enlarged image information for each horizontal line. An edge pixel coordinate detecting means for examining a boundary between a specific color and another color, extracting a pixel serving as the boundary as an edge pixel, and calculating a position coordinate of the extracted edge pixel, and a position coordinate of the extracted edge pixel Is converted to coordinates before the enlargement processing, edge pixel coordinate conversion means,
A line image extracting unit that obtains a line image formed by connecting the edge pixels so as to form a straight line by referring to the transformed coordinates of the edge pixels; and an enlargement, which is an enlargement amount of image information, defined for each horizontal line. An enlargement condition storing means for storing an enlargement center coordinate indicating a ratio and a reference position for performing an enlargement process of image information.

The image enlargement processing means refers to the storage contents of the enlargement condition storage means and, for each horizontal line, uses the enlargement factor as a center and enlarges the image information at a magnification indicated by the enlargement ratio. This is a device that performs processing to increase the size.

Further, more specifically, the following embodiments can be considered.

That is, an image pickup means mounted on a vehicle for picking up an image in front of the vehicle and obtaining image information, an image enlargement processing means for enlarging the size of the imaged image information, An edge pixel coordinate detecting means for scanning line by line, examining a boundary between a specific color and another color, extracting a pixel serving as the boundary as an edge pixel, and obtaining a position coordinate of the extracted edge pixel; Edge pixel coordinate conversion means for converting the position coordinates of the extracted edge pixels into coordinates before the enlargement processing; and left and right sides formed by connecting the edge pixels so as to form a straight line with reference to the converted coordinates of the edge pixels. A lane extracting means for finding a lane; an enlarging ratio which stores an enlarging rate which is an enlarging amount of image information and an enlarging center coordinate which indicates a reference position at which enlarging processing of image information is defined for each horizontal line And a storage unit.

The image enlargement processing means refers to the contents stored in the enlargement condition storage means, and for each horizontal line, uses the enlargement ratio as a center and the magnification indicated by the enlargement ratio for each horizontal line. This is a device that performs processing to increase the size.

[0014] It is preferable that the image enlargement processing means performs processing for enlarging the size of photographed image information only in the horizontal direction.

[0015]

According to the present invention, an image in front of the vehicle is captured by the image capturing means mounted on the vehicle to obtain image information.

The enlargement condition storage means stores an enlargement ratio, which is an amount of enlargement of image information, and enlargement center coordinates indicating a reference position at which image information enlargement processing is performed, which is defined for each horizontal line.

Then, the size of the captured image information is enlarged by the image enlargement processing means.

More specifically, the image enlargement processing means refers to the contents stored in the enlargement condition storage means and, for each horizontal line, sets the image information at the magnification indicated by the enlargement ratio, centering on the enlargement center coordinates. Is performed to increase the size of.
At this time, the image enlargement processing means performs a process of enlarging the size of the captured image information only in the horizontal direction.

Further, the edge pixel coordinate detecting means scans the enlarged image information for each horizontal line, checks a boundary between a specific color and another color, and sets a pixel serving as the boundary as an edge pixel. The position coordinates of the extracted and extracted edge pixels are obtained, and the edge pixel coordinate conversion means converts the position coordinates of the extracted edge pixels into coordinates before the enlargement processing.

The lane extracting means refers to the transformed coordinates of the edge pixels, and finds the left and right lanes formed by connecting the edge pixels so as to form a straight line.

As described above, according to the present invention, the image information captured by the image capturing means is enlarged (hereinafter, simply referred to as "image zoom" as appropriate), and based on the enlarged image data, The traveling lane can be accurately recognized.

That is, by employing the image zoom,
Since the pixels (edge pixels) indicating the boundary between the road surface and the lane can be extracted while the lane width is enlarged, it is possible to accurately recognize a farther lane.

[0023]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a configuration diagram of an apparatus according to an embodiment of the present invention.

This apparatus comprises a photographing unit 1, a processing unit 5, and a memory 2.
3 and a display unit 14, and the processing unit 5 further includes an image processing unit 6 and a determination unit 7. Signals are exchanged between the vehicle state sensor 20, the vehicle motion control device 22 and the alarm device 21, and the determination unit 7.

Hereinafter, each component of the apparatus will be described.

First, the photographing section 1 will be described.

The CCD camera 2 captures a forward image,
It is an imaging unit for obtaining image information, and has a function of converting image information of a forward subject into a color analog signal. For example, when the vehicle is mounted on a vehicle, it may be installed near a grill near the front of the vehicle, in a location with little dirt, or in a location with a wide front view such as a room mirror or a sun visor in the vehicle interior.

Next, the A / D converter 3 is a CCD camera 2
Performs a process of converting the analog image signal output from the to the digital signal.

Next, the color difference conversion circuit 4 includes an A / D converter 3
, The digital signals corresponding to the three primary colors (R (red), G (green), and B (blue)) of the subject are defined by the following equation, and the luminance signal Y and the color difference signal “R−
Y, BY ”.

Y = 0.3R + 0.6G + 0.1B R−Y = 0.7R−0.6G−0.1B BY−−0.3R−0.6G + 0.9B The luminance signal and the color difference signal obtained by the above conversion. Is input to the delay circuit 16 and the buffer 25
Is stored in

Next, the processing section 5 will be described. The contents of the operation will be described later, so that the function of each component will be outlined here.

As shown in FIG. 1, the processing section 5 includes an image processing section 6 and a judgment section 7.

First, the components of the image processing section 6 will be described.

The buffer 25 is a means for storing the image information sent from the color difference conversion circuit 4, but does not necessarily need to have a capacity for one frame. For example, if the processing unit of the image processing is three lines, it may be configured to have a capacity of three lines.

The image zoom circuit 8 receives the image information from the buffer 25, and according to the zoom condition (zoom center coordinate and enlargement ratio) determined for each horizontal line by the zoom condition determining unit 9 described later. After performing a process of enlarging information only in the horizontal direction (for example, for each horizontal line), the image information subjected to the enlarging process is extracted by the extraction processing circuit 1.
0 is transferred.

Here, the zoom center coordinate is a parameter indicating the position of a reference point at which enlargement processing is performed when the image information is enlarged, and the enlargement ratio is a parameter indicating the magnification at which the image information is enlarged.

The extraction processing circuit 10 extracts a pixel corresponding to predetermined extraction color condition data from the enlarged image information, performs image processing for removing noise and the like, and then performs one horizontal line processing. Above, the position coordinate data (edge coordinate data) for the extracted pixel is transferred to the edge coordinate conversion unit 11. Note that the extracted color condition data is data of specific luminance and color difference, and usually adopts data of luminance and color difference of a road color (running lane) and stores the data in the memory 2.
3 in advance, and the extraction processing circuit 10 may be configured to retrieve the data when necessary.

When examining one horizontal line, if a change point from a pixel not satisfying the extracted color condition data to a pixel satisfying the extracted color condition data is assumed, the extracted color condition data is not satisfied. Such a pixel is referred to as a rising edge pixel, and its coordinate data is referred to as edge coordinate data of the rising edge pixel. Similarly, when examining one horizontal line, assuming a transition point from a pixel satisfying the extracted color condition data to a pixel not satisfying the extracted color condition data, a pixel satisfying the extracted color condition data is obtained. Is called a falling edge pixel, and its coordinate data is
It is called edge coordinate data of the falling edge pixel. The edge coordinate data includes information indicating whether the edge coordinate is a rising edge or a falling edge, in addition to the coordinate information.

Next, based on the zoom center coordinates and the enlargement ratio transferred from the zoom condition determination unit 9, the edge coordinate conversion unit 11 converts the edge coordinate data transferred from the extraction processing circuit 10 into the coordinates before the enlargement processing. The data is converted into data and transferred to the lane recognition unit 12. Note that the zoom center coordinates and the enlargement ratio will be described later in detail in the image zoom processing.

Next, based on the edge coordinate data transferred from the edge coordinate conversion unit 11, the lane recognition unit 12 extracts edge coordinate data for the left and right lanes of the traveling lane in which the vehicle travels, and extracts the data. The left and right lanes are recognized based on the edge coordinate data thus obtained, and the recognition result is transferred to the zoom condition determination unit 9, the danger determination unit 13, and the replacement circuit 15.

Next, the zoom condition determining unit 9 receives the lane information recognized from the lane recognizing unit 12, calculates the zoom center coordinate and the enlargement ratio, and sends them to the edge coordinate converting unit 11 and the image zoom circuit 8. Forward. The zoom condition, zoom
The system center coordinates and the enlargement ratio may be set in advance for each horizontal line, and the image enlargement processing may be performed.

Next, the components of the display unit 14 will be described.

The delay circuit 16 has a function of delaying the input image information by a time corresponding to various processing times in the processing section 5 and outputting the delayed image information. This is a circuit for performing synchronization when the processing result output from the lane recognition unit 12 is input to the replacement circuit 15.

Next, the replacement circuit 15 superimposes the processing result from the processing section 5 on the image delayed by the delay circuit 16. Specifically, a lane recognition unit 12 as shown in FIG. 15B recognizes a captured image as shown in FIG. 15A output from the delay circuit 16 without any image processing. The image information of the lane is superimposed, and FIG.
(1) Image information in which a recognition lane is drawn on a captured image as shown in FIG. In addition, when the danger determination unit 13 drives the alarm device 21, at the same time, superimpose information for displaying the alarm information on the monitor 18 is provided to the replacement circuit 15. good. FIG. 15D shows how the alarm information is displayed on the monitor 18.

Next, the encoder circuit 17 includes the replacement circuit 1
5 is a circuit for converting the image information output from 5 into an NTSC signal.

The monitor 18 receives the NTSC signal and displays the corresponding image information on the display screen.

Next, the components of the judgment section 7 will be described.

The determining unit 7 is an example in which the present invention is applied to a driver's alert device, and the determining unit itself does not form a main part of the present invention.

The vehicle state judging section 19 includes the vehicle state sensor 2
The traveling state of the own vehicle is determined based on the signal transmitted from 0, and the determination result is transferred to the danger determining unit 13.

Here, the vehicle state sensor 8 is a means for detecting the amount of exercise of the own vehicle, the intention of the driver to operate, and the like. For example, a vehicle speed sensor for measuring the speed of the own vehicle, other direction indicators, And a steer angle sensor.

The danger determining unit 13 recognizes the traveling lane of the own vehicle based on the edge coordinate data for the left and right lanes sent from the lane recognizing unit 12. The presence of a preceding vehicle, an obstacle, or the like is recognized based on edge coordinate data for an object other than a lane, and the traveling risk of the own vehicle is recognized based on data transmitted from the vehicle state determination unit 19.

For example, if the speed of the own vehicle detected by the vehicle speed sensor is equal to or higher than a predetermined value and the presence of a preceding vehicle or the like can be recognized at a specific position in the image information ahead, this is regarded as a dangerous state. Judgment is made and the vehicle motion control device 22 and the alarm device 21 are driven. The vehicle motion control device 22 includes a driving system, a braking system,
This is a device for controlling a steering system and the like. As a specific example,
Automatic braking is conceivable.

The alarm device 21 may be any device that appeals to the driver's hearing or vision and draws attention to the driver, such as a chime driving operation or LED display. And so on.

Various examples of the configuration example of the determination unit 7 described above are conceivable.

The memory 23 stores the zoom condition determining unit 9.
, And also functions as a work area when the lane recognition unit 12 and the like perform processing.

Next, the operation of the apparatus according to the present invention will be described.

In the present apparatus, roughly two processes are executed in parallel at different periods.

In the "first processing", the image information obtained by the CCD camera 2 is converted from analog to digital by the A / D converter 3, and further converted by the color difference conversion circuit 4 into luminance and color difference signals. Then, after a predetermined time delay is given by the delay circuit 16 and the image processing is performed by the function of the replacement circuit 15, the image processing result is converted into the NTS by the encoder circuit 17.
This is a series of processes for displaying a processing result on the monitor 18 as a C signal.

First, the "first processing" will be described with reference to FIGS.

When the power of the apparatus is turned on by operating the apparatus (S2), initialization is performed (S2).
4). Here, the initialization includes, for example, processing such as clearing of the work area area of the memory 23. Also,
As an operation performed by a human, initial adjustment of the photographing unit 1 and the display unit 14 and initial values of an extraction color condition and a zoom condition used in a second process described later are set by an input unit (not shown).

Next, an image was taken by the CCD camera 2,
A signal corresponding to image information ahead of the vehicle is output as an analog RGB signal (S6).

Next, the analog RGB signal is converted into a digital RGB signal by the A / D converter 3 (S8).
The digital RGB signals are converted into luminance and color difference signals (S10).

Next, the delay circuit 16 receives the image signal output from the color difference conversion circuit 4 as an input signal, and delays the input signal by a processing time required by the processing section 5 (S1).
2) Synchronization with the signal processed by the processing unit 5 is performed.

Next, the replacement circuit 15 performs a process of superimposing the processing result of the processing unit 5 on the original image information that has not been processed (S14). The signal is converted to an NTSC signal (S16), and a display image corresponding to the NTSC signal is displayed on the monitor 18 (S18).

Thereafter, the flow branches to step (S6) to obtain image information of the subject in the next cycle. The above processing is performed at the frame rate (for example, 16.7 ms)
Is performed sequentially with the period as a cycle.

Next, the “second processing” will be described. In the second processing, the image information sent from the photographing section 1 is
After the image is enlarged in the horizontal direction by the image zoom (image enlargement) process (S20), the edge pixels are extracted (S2).
2) Recognize the lane with respect to the traveling lane in which the host vehicle runs (S28), determine whether the host vehicle is in danger (S36), and drive (warning) alarm device 21 (warning) (S38).
And a series of processing up to driving (avoidance) (S40) of the vehicle motion control device 22.

For each of these processes, FIGS.
This will be described with reference to FIG.

The image zooming process (S20) will be described in detail later. The image zooming circuit 8 enlarges the image information obtained by shooting only in the horizontal direction according to the set zooming conditions, and performs the process on the road surface. This is a process for enlarging an image such as a white line or a white line. For example, a process of enlarging the original image shown in FIG. 3A is performed as shown in FIG.

In the edge extraction processing (S22), the extraction processing circuit 10 extracts edge pixels as described above. This situation is shown in FIG. Of course, edge coordinate data is also extracted.

FIG. 4 shows an edge extraction process (S2
The processing content of 2) is shown in a flowchart.

In the pixel discrimination process (S2202), the set extraction color condition is compared with the color data of all the pixels, and a pixel satisfying the extraction color condition is extracted.

Then, edge point discrimination processing (S2204)
Then, in order to remove noise, for example, a smoothing process using spatial filtering of “3 pixels × 3 pixels” is performed, and among the nine pixels, half or more of the pixels are a group of pixels satisfying the extraction color condition. For example, these nine pixels are pixels that satisfy the extraction color condition.

When the image information subjected to the enlargement processing as shown in FIG. 3 (2) is examined for each horizontal line, the pixels which do not satisfy the extraction color condition data are replaced with the pixels which satisfy the extraction color condition data. When the value changes to, a pixel satisfying the extracted color condition data is set as a rising edge pixel, and its coordinate data is set as edge coordinate data of the rising edge pixel. Similarly, when checking for each horizontal line, if a pixel that satisfies the extracted color condition data changes from a pixel that satisfies the extracted color condition data to a pixel that does not satisfy the extracted color condition data, the pixel that satisfies the extracted color condition data is A falling edge pixel is set, and its coordinate data is set as edge coordinate data of the falling edge pixel.

Then, the edge coordinate data is transferred to the edge coordinate conversion unit 11. In the example shown in FIG. 3, the image information is composed of a road surface and a lane. However, when another vehicle or the like exists on the road surface, an edge pixel that does not form a lane is extracted.

Next, in the edge coordinate conversion processing (S24), the edge coordinate data is referred to the zoom center coordinate and the enlargement ratio transferred from the zoom condition determining unit 9 as shown in FIG. The data is converted into the coordinate data before the enlargement and transferred to the lane recognition unit 12.

FIGS. 5A and 5B show image information before and after the enlargement process, respectively. In the case of the zoom center coordinate C and the enlargement ratio R of the y-th horizontal line from the upper end of the screen, the edge coordinate data P ′ in FIG. 5B can be converted into the edge coordinate data P before enlargement by the following equation. Here, W is a horizontal width of a set of pixels constituting image information.

P = (P'-W / 2) / R + C Next, in the lane recognition processing (S28), a plurality of approximate straight lines are obtained from the transferred edge coordinate data, and shown in FIG. Thus, the lane for the traveling lane in which the own vehicle travels is estimated.

That is, the lane recognition unit 12 performs the following processing.

Next, the lane recognizing unit 12 converts the edge coordinate data sent from the edge coordinate converting unit 11 into edge pixels considered to be for the left lane, edge pixels considered to be for the right lane, It is determined whether or not the pixel is an edge pixel, and classification processing is performed.

In this process, the type of the edge pixel is determined based on the edge coordinate data sent from the edge coordinate conversion unit 11. Here, the edge coordinate data includes coordinate information of each edge pixel and information of rising or falling. In general, if the road surface color is the extracted color condition data, the rising edge pixel indicates the left lane,
The falling edge pixel indicates the right lane. However,
When a preceding vehicle, an obstacle, or the like exists in the front road surface area, edge pixels other than the lane are also recognized as edge pixels of the lane.

Therefore, it is assumed that a rising edge pixel existing on the left side of the screen center line in FIG. 3 is an edge pixel on the left lane, and a falling edge existing on the right side is a right lane edge pixel. It may be determined as an edge pixel of a car or the like.

Next, based on the edge coordinate data of the edge pixels for the left and right lanes, an approximate straight line is obtained by, for example, the least square method or the like, and this is set as a recognized lane. In addition,
The edge coordinate data of the edge pixels other than the edge pixels for the left and right lanes is determined to be a pixel indicating the preceding vehicle, and the edge coordinate data closest to the position of the own vehicle is determined as the existing position of the preceding vehicle.

Next, in the danger determination process (S36), the traveling state (vehicle speed, etc.) of the own vehicle is determined from the signal from the vehicle state sensor (for example, vehicle speed sensor) 20, and the result of the determination and the lane are used. From the relationship between the determined road surface on which the vehicle can run and the location of the preceding vehicle, it is estimated whether or not the own vehicle is in a dangerous state.

For example, when the speed of the host vehicle detected by the vehicle speed sensor is equal to or higher than a predetermined value and the position of the preceding vehicle is within a predetermined range, this is determined as a dangerous state.

Next, in the warning process (S38), when the danger determination unit 13 determines that the vehicle is in a dangerous state, the warning device 21 is driven to notify the driver of the own vehicle.

Next, in the avoidance process (S40), when the danger determining unit 13 determines that the vehicle is in a dangerous state, the vehicle motion control device 2
2 is driven.

Incidentally, the vehicle motion control device 22 may be driven when it is determined that the driver's operation has been insufficient even for the danger state warning.

The apparatus operates by repeating these series of processes.

Next, the image zoom processing (S20), which is the main processing in the present invention, will be described in detail.

FIG. 6 is a flowchart for explaining the processing contents of the image zoom processing (S20).

First, referring to the coordinate data of the recognized lane obtained by processing in the previous cycle, the zoom condition determining unit 9
The zoom center coordinates and the enlargement ratio are determined for each horizontal line (S2002).

Here, the zoom center coordinate is a parameter indicating the position of a reference point at which enlargement processing is performed when the image information is enlarged, and the enlargement ratio is a parameter indicating the magnification at which the image information is enlarged. The zoom center coordinate is, for example, the center position of the own vehicle traveling lane sandwiched between the recognized left and right lanes, and the enlargement ratio is such that the enlarged lane does not protrude from the screen (for example, about 80% of the screen width W). ). Such a determination is made by the zoom condition determining unit 9.
This is performed by referring to the edge coordinate data and the like constituting the lane sent from the lane recognition unit 12 and ascertaining the center position and the width of the traveling lane between the recognized left and right lanes. . Of course, when it is not necessary to sequentially update the zoom center coordinates and the enlargement ratio, they may be determined in advance for each horizontal line and stored in the zoom condition determination unit 9.

Next, an actual scene will be described with reference to the drawings.

FIG. 7A shows an example of the result of lane recognition obtained by processing in the previous cycle. In the y-th horizontal line from the top edge of the screen, the left lane coordinate is xl and the right lane coordinate is xr
Then, the zoom center coordinates C and the enlargement ratio R are expressed by the following equations.

Zoom center coordinate: C = (xr + xl) / 2 Magnification: R = α × W / (xr−xl) where (xr−xl) / W <α <1, where α is the value after the enlargement process. Is a coefficient for preventing the lane from protruding from the screen. However, if α is smaller than the lower limit ((xr-xl) / W), R becomes smaller than “1” and the image information is reduced, and in this case, α = ( xr-xl) / W, that is, the enlargement ratio R = 1. In the following, W indicates the screen width.

FIG. 7B shows an example of image information subjected to enlargement processing when α = 0.8.

Next, FIG. 8 shows an example of a result of lane recognition when a scene on a curved road is assumed and the recognized left lane is only partially present in the screen. In this case, assuming that the left lane coordinate does not exist in the screen and the right lane coordinate is xr in the y-th horizontal line from the upper end of the screen, the zoom center coordinate C and the enlargement ratio R are expressed by the following equations.

Zoom center coordinates: C = xr / 2 Magnification ratio: R = α × W / xr This is obtained when xl = 0 in the equation for calculating the zoom center coordinates C and the magnification ratio R with respect to FIG. Equivalent to.

FIG. 9 shows an example of lane recognition results when a scene on a curved road is assumed and the recognized right lane is only partially present in the screen.
In this case, at the y-th horizontal line from the top of the screen,
Since the left lane coordinate is xl and the right lane coordinate does not exist in the screen, the zoom center coordinate C and the enlargement ratio R are expressed by the following equations.

Zoom center coordinate: C = (W + xl) / 2 Magnification ratio: R = α × W / (W−xl) This is a formula for calculating the zoom center coordinate C and the magnification ratio R with respect to FIG. This is equivalent to W.

Next, the image zoom circuit 8 performs a process of enlarging the image according to the determined zoom center coordinates and the enlargement ratio (S2004).

Here, the enlargement processing for one horizontal line will be described with reference to FIG.

As shown in FIG. 10A, the data of 8 pixels (assuming that one horizontal line is composed of 8 pixels) is obtained by using the position indicated by the arrow as the zoom center coordinate and increasing the magnification by “2 × , The case of performing the enlargement process will be described.

First, the number of pixels to be enlarged is obtained. At this time, since the number of pixels after the enlargement process must also be eight, the number of pixels to be enlarged is four in consideration of the enlargement ratio “2”.

Next, pixel data to be enlarged is selected. It is assumed that four pixels are selected around the zoom center coordinates.
Then, the size of the image information for the selected pixel is doubled and assigned to eight pixels forming one horizontal line.

By such processing, the image information shown in FIG. 10A is enlarged as shown in FIG. 10B.

By performing the above-described processing on all the horizontal lines, the entire screen information can be enlarged in the horizontal direction.

Hereinafter, a specific example of lane recognition by image enlargement processing will be described with reference to FIGS.

FIG. 12A shows a detailed view of the part A (part of the right lane) shown in FIG. 11A. FIG. 12B is a diagram showing the result of performing image enlargement processing (enlargement ratio: twice) on FIG. 12A. As mentioned above,
In the edge point discrimination processing (FIG. 4: S2204), spatial filtering of “3 pixels × 3 pixels” is performed to remove noise.

That is, if more than half of the nine pixels satisfy the extraction color condition, it is determined that the nine pixels satisfy the extraction color condition. FIG. 11C shows the state of the change of the enlargement ratio set for each horizontal line. Specifically, the horizontal line located below the Nth horizontal line continuously decreases. In order to do so, the magnification is set.

Now, it is assumed that FIGS. 12A and 12B are separated by a cluster of “3 pixels × 3 pixels” as shown by a thick line. In the case of FIG. 12A, “3 pixels × 3” having a white line color
The number of “pixels” is one, and in the edge point determination processing, it is considered that pixels are configured as shown in FIG. On the other hand, in the case of FIG. 12B, the number of “3 pixels × 3 pixels” having a white line color is four, and in the edge point discrimination processing, pixels are configured as shown in FIG. 12B ′. Is considered to be

Therefore, in the enlarged image, the ratio of the white line in the area A (see FIG. 11) is "2 /
3 ", and the ratio of the white line is larger than" 1/3 "when the enlargement processing is not performed. For this reason, white lines that cannot be recognized unless the enlargement processing is performed are enlarged and the percentage of the white lines in the image information is increased, whereby the white lines can be recognized. Therefore, by the enlargement process, it is possible to recognize the lane with respect to the traveling lane in which the own vehicle travels to a farther distance.

Hereinafter, other processing methods of the image enlargement processing will be enumerated.

As described above, as an example of the image enlargement processing, the method of calculating and obtaining the enlargement ratio for each horizontal line has been described. However, in the image enlargement processing, as shown in FIG. 13, the enlargement ratio may be changed discontinuously around a certain horizontal line.

More specifically, as shown in FIG.
A predetermined enlargement factor is set for horizontal lines (including the M-th horizontal line) located above the M-th horizontal line, and for a horizontal line located below the M-th horizontal line. An enlargement ratio different from the predetermined enlargement ratio (in the case of FIG. 13, the enlargement ratio = 1, the enlargement processing is not performed) is set. FIG. 13B shows a processing example when the enlargement ratio is discontinuous.
FIG. 13A shows an original image.

As described above, a horizontal line located above a specific horizontal line is enlarged at a predetermined enlargement ratio, and an enlargement process is not performed on a horizontal line located below. It should be noted that any specific enlargement ratio may be set for a specific horizontal line.

According to this method, the amount of calculation can be greatly reduced as compared with the method of calculating the enlargement ratio for each horizontal line.

The following is conceivable as a method for setting the horizontal line for changing the enlargement ratio and the enlargement ratio.
As the simplest method, the horizontal line for changing the enlargement ratio and the enlargement ratio are set in advance, and the image zoom circuit 8 performs the image enlargement process with reference to the horizontal line. A method is also conceivable in which only the enlargement ratio is fixed and the horizontal line for which the enlargement ratio is changed is variable so that the lane width on the horizontal line at which the enlargement ratio is switched is constant.

As shown in FIG. 14, the center position of the running lane between the left and right lanes of the horizontal line for changing the magnification may be used as the zoom center coordinate.

As described above, according to the present invention,
By the process of enlarging an image, it is possible to realize a device that can accurately recognize a distant lane with a simple configuration.

[0122]

As described above, according to the present invention, the image information captured by the image capturing apparatus is expanded in the horizontal direction, thereby achieving a distant view with a simple configuration while securing a wide field of view. It is possible to realize a device capable of accurately recognizing the lane even for the traveling road.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing an example of a system configuration according to the present invention.

FIG. 2 is a flowchart showing the entire processing content.

FIG. 3 is an explanatory diagram showing an image generated by the processing.

FIG. 4 is a flowchart illustrating an edge extraction process.

FIG. 5 is an explanatory diagram of the principle of edge coordinate conversion processing.

FIG. 6 is a flowchart illustrating processing contents of an electronic zoom processing.

FIG. 7 is an explanatory diagram for obtaining zoom center coordinates in a certain scene.

FIG. 8 is an explanatory diagram for obtaining zoom center coordinates in a certain scene.

FIG. 9 is an explanatory diagram for obtaining zoom center coordinates in a certain scene.

FIG. 10 is an explanatory diagram of the principle of the enlargement process.

FIG. 11 is an explanatory diagram of a screen and an enlargement ratio before and after an enlargement process.

FIG. 12 is an explanatory diagram of the principle of edge determination processing.

FIG. 13 is an explanatory diagram of a screen and an enlargement ratio before and after an enlargement process when the enlargement ratio changes discontinuously.

FIG. 14 is an explanatory diagram illustrating zoom center coordinates when the enlargement ratio changes discontinuously.

FIG. 15 is an explanatory diagram of a processing result by a replacement circuit.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Photographing part, 5 ... Processing part, 6 ... Image processing part, 7 ... Judgment part, 8 ... Image zoom circuit, 9 ... Zoom condition determination part, 10
... Extraction processing circuit, 11 ... Edge coordinate conversion unit, 12 ... Lane recognition unit, 14 ... Display unit

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Tatsuhiko Moji 2520 Takada, Hitachinaka-shi, Ibaraki Automobile Equipment Division, Hitachi, Ltd. No. 20 No. 1 Semiconductor Division, Hitachi, Ltd. (56) References JP-A-3-194670 (JP, A) JP-A-6-124340 (JP, A) JP-A-6-229760 (JP, A) ( 58) Surveyed field (Int.Cl. ⁷ , DB name) G06T 1/00, 3/40 G06T 7/00, 9/20 G08G 1/16

Claims (8)

(57) [Claims] An image pickup means for picking up an image and obtaining image information, an image enlargement processing means for enlarging the size of the picked up image information, and scanning the enlarged image information for each horizontal line. An edge pixel coordinate detecting means for examining a boundary between a specific color and another color, extracting a pixel serving as the boundary as an edge pixel, and calculating a position coordinate of the extracted edge pixel, and a position coordinate of the extracted edge pixel And an edge pixel coordinate conversion means for converting the coordinates to the coordinates before the enlargement processing, and a line image extraction means for obtaining a line image formed by connecting the edge pixels so as to form a straight line with reference to the converted coordinates of the edge pixels. ,
An enlarging condition storing unit that stores an enlarging ratio, which is an amount of enlarging image information, defined for each horizontal line, and an enlarging center coordinate indicating a reference position at which the enlarging process of the image information is performed; Performs a process of enlarging the size of image information at the magnification indicated by the enlargement ratio, centering on the enlargement center coordinates, for each horizontal line, with reference to the storage contents of the enlargement condition storage means. An image processing apparatus characterized by the above-mentioned. 2. An image pickup device mounted on a vehicle for picking up an image in front of the vehicle and obtaining image information; an image enlargement processing unit for enlarging the size of the image information taken; An edge pixel coordinate detecting means for scanning line by line, examining a boundary between a specific color and another color, extracting a pixel serving as the boundary as an edge pixel, and obtaining a position coordinate of the extracted edge pixel; Edge pixel coordinate conversion means for converting the position coordinates of the extracted edge pixels into coordinates before the enlargement processing; and left and right sides formed by connecting the edge pixels so as to form a straight line with reference to the converted coordinates of the edge pixels. Lane extracting means for finding lanes, and enlarging condition storing means for storing an enlarging rate, which is an amount of expansion of image information, defined for each horizontal line, and expansion center coordinates indicating a reference position at which the image information is to be expanded. The image enlargement processing unit refers to the storage content of the enlargement condition storage unit, and for each horizontal line, the magnification of the image information at a magnification indicated by the enlargement ratio with the enlargement center coordinate as a center. An image processing apparatus characterized by performing a process of enlarging the image. 3. The image processing apparatus according to claim 1, wherein said image enlargement processing means determines a size of the photographed image information.
An image processing apparatus for performing processing for enlarging only in a horizontal direction. 4. The enlargement condition storage means according to claim 1, wherein the enlargement ratio is previously stored in the enlargement condition storage means as a value which changes continuously for each horizontal line. An image processing apparatus characterized by the above-mentioned. 5. The method according to claim 2, wherein said lane extracting means includes an enlargement factor for each horizontal line based on coordinate data of the extracted lane obtained by processing in the previous lane extraction cycle. An image processing apparatus having a function of obtaining enlargement center coordinates and storing the coordinates in the enlargement condition storage means. 6. The lane extracting means according to claim 2, wherein said lane extracting means divides a predetermined screen width by a value obtained by dividing a width of a vehicle traveling lane sandwiched between left and right lanes in each horizontal line. An image processing apparatus having a function of obtaining an enlargement ratio for each horizontal line based on the enlargement ratio and storing the enlargement ratio in the enlargement condition storage means. 7. The lane extracting means according to claim 2, wherein the lane extracting means determines the center position of the own vehicle traveling lane sandwiched between the left and right lanes in each horizontal line by the enlarged center coordinates for each horizontal line. An image processing apparatus characterized by having a function of obtaining the above as a condition and storing it in the enlargement condition storing means. 8. The enlargement ratio according to claim 1, wherein the enlargement ratio is a value for a horizontal line existing vertically above a specific horizontal line. And a value for a horizontal line that includes the specific horizontal line and exists below the horizontal line in the vertical direction.