JP2009294719A - Road marking recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a road marking recognition device for learning road markings in each area. <P>SOLUTION: A similarity degree between an image of the front of a vehicle imaged by an imaging unit for imaging surroundings of the vehicle and a road marking pattern "STOP" is calculated. When the maximum value of the calculation result of the similarity degree is x<SB>1</SB>, the image of a display unit is switched to an image of the vehicle surroundings since the similarity degree x<SB>1</SB>is within a similar allowable range [A<SB>0</SB>, B<SB>0</SB>] of the similarity degree. When the maximum value of the calculation result of the similarity degree is x<SB>2</SB>, and a brake is on and the vehicle speed is not higher than a predetermined speed, the image on the display unit is switched to the image of the vehicle surroundings by changing the similarity allowable range [A<SB>0</SB>, B<SB>0</SB>] to [A<SB>1</SB>, B<SB>1</SB>] since the similarity degree x<SB>2</SB>is out of the similarity allowable range but within the maximum allowable range [α, β]. Here, B<SB>1</SB>is equal to x<SB>2</SB>, and B<SB>0</SB>-A<SB>0</SB>is equal to B<SB>1</SB>-A<SB>1</SB>. The similarity allowable range [A<SB>1</SB>, B<SB>1</SB>] after the change is within the similarity allowable range corresponding to the road marking "STOP" in a district where the vehicle is located. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a road marking recognition device that recognizes a road marking using an image captured by an imaging unit.

The vehicle is equipped with a display unit that displays various images and an imaging unit that captures the periphery of the vehicle. In many cases, the imaging unit images a place that is a blind spot of the driver.
The display unit normally displays a car navigation image (hereinafter referred to as a car navigation image), a television image, or the like. However, when the vehicle speed is equal to or lower than the predetermined speed, an image captured by the image capturing unit is displayed on the display unit automatically or when the driver manually operates the switch.

By the way, before entering the priority road from a non-priority road, for example, on a crossroad, a T-shaped road, etc., the driver needs to stop the vehicle and check the surrounding safety.
Conventionally, there has been proposed a safe driving support device that recognizes road markings such as “stop” and “stop” written on the road surface to indicate the stop of the vehicle using an image captured by an imaging unit (Patent Literature). 1).
When a road marking such as “stop” or “stop” is present on the road surface in front of the vehicle, the safe driving support device disclosed in Patent Document 1 displays an image displayed on the display unit as a car navigation image or a television image. Etc. to switch to an image of a place where the driver is blind spot.
As a result, the driver can easily confirm the surrounding safety with reference to the image displayed on the display unit.

In order to recognize the road marking, the safe driving support apparatus stores in advance a reference character / symbol pattern including a character string such as “stop” or “stop”. The safe driving support device extracts the feature of the image captured by the imaging unit to obtain a character symbol pattern of the road sign, and collates the obtained character symbol pattern with the reference character symbol pattern.
JP 2002-219998 A

  However, the shape and size of character strings such as “stop” and “stop” may differ depending on the region. Therefore, in order to accurately recognize the road marking, it is necessary to store reference character / symbol patterns corresponding to character strings such as “stop” and “stop” in various places. In other words, the manufacturer of the safe driving support device must prepare various types of reference character / symbol patterns, and in order to store these, a storage device with a large storage capacity is installed in the safe driving support device. Must.

  The present invention has been made in view of such circumstances, and its main purpose is when the similarity between the image captured by the imaging unit and the road marking pattern is outside the similar allowable range and within the maximum allowable range. In addition, an object of the present invention is to provide a road marking recognition apparatus that can automatically learn road markings in various places by changing the upper and lower limits of the similar allowable range.

  Another object of the present invention is to change the upper and lower limit values of the similarity allowable range when the similarity between the image captured by the imaging unit and the road marking pattern is out of the similarity allowable range and the operation unit is operated. The object of the present invention is to provide a road marking recognition apparatus that can manually learn road markings in various places.

  Another object of the present invention is to change the upper and lower limit values of the similar allowable range so that the difference value between the upper limit value and the lower limit value is within a predetermined range. It is an object of the present invention to provide a road marking recognition apparatus capable of preventing the inconvenience of adversely affecting the learning results of other local road markings.

  Still another object of the present invention is to display the display unit when the similarity between the image captured by the imaging unit and the road marking pattern is within the similar allowable range, or when changing the upper and lower limit values of the similar allowable range. By switching the image displayed on the vehicle to an image around the vehicle imaged by the imaging unit, the road marking recognition device can switch the image on the display unit when a road sign is present on the road surface ahead of the vehicle. Is to provide.

  A road marking recognition device according to a first aspect of the present invention includes an imaging unit that captures the periphery of a vehicle, and a storage unit that stores a road marking pattern related to the stop of the vehicle, and uses the image captured by the imaging unit, In a road sign recognition device for recognizing a road sign, similarity calculation means for calculating a similarity between at least a part of an image ahead of the vehicle imaged by the imaging unit and a road marking pattern stored in the storage unit; A coincidence determination unit that determines whether or not the calculation result of the similarity calculation unit is within the similarity allowable range of the similarity, and when the match determination unit determines NO, the calculation result is the similarity allowable range A result determination means for determining whether or not the vehicle is within a wider maximum allowable range; and whether or not a stop condition for stopping the vehicle is satisfied when the result determination unit determines that the vehicle is within the maximum allowable range. judge Vehicle determination means, and a change means for changing an upper limit value and / or a lower limit value of the similar permissible range based on the calculation result when it is determined that the stop determination means is established. To do.

  A road marking recognition apparatus according to a second aspect of the present invention includes an imaging unit that captures an image of the periphery of a vehicle, and a storage unit that stores a road marking pattern related to the stop of the vehicle, and uses the image captured by the imaging unit, In a road marking recognition apparatus for recognizing a road marking, a similarity between an operation unit, at least a part of an image in front of the vehicle captured by the imaging unit, and a road marking pattern stored in the storage unit is calculated. Similarity calculating means, a coincidence determining means for determining whether the calculation result of the similarity calculating means is within the similarity allowable range of the similarity, and when the coincidence determining means determines NO, the operation unit operates An operation determination unit that determines whether or not the operation determination unit has been operated, and a change unit that changes an upper limit value and / or a lower limit value of the similar allowable range based on the calculation result when it is determined that the operation determination unit has been operated. To prepare And butterflies.

  In the road marking recognition apparatus according to a third aspect of the present invention, the changing means has an upper limit value and / or a lower limit value of the similar allowable range so that a difference value between the upper limit value and the lower limit value of the similar allowable range is within a predetermined range. The value is changed based on the calculation result.

  The road marking recognition device according to a fourth aspect of the present invention is the case where the display unit for displaying an image and the coincidence determining means determine that the similarity is within the similar allowable range, or the changing means is an upper limit value of the similar allowable range and When the lower limit value is changed, image switching means for switching an image displayed on the display unit to an image around the vehicle imaged by the imaging unit is provided.

The first invention includes an imaging unit and a storage unit, and further includes a similarity calculation unit, a coincidence determination unit, a result determination unit, a stop determination unit, and a change unit.
The imaging unit images the vicinity of the vehicle. The road marking recognition apparatus recognizes a road marking related to the stop of the vehicle using an image captured by the imaging unit. Road markings using character strings such as “stop” and “to rare” may have different shapes, dimensions, etc. depending on the region. However, the shape and dimensions of character strings such as “Stop” and “Torrare” are naturally restricted by the regulations of the Road Traffic Law and Road Law, so the shape and dimensions of road markings will not be significantly different. .
Hereinafter, the road marking related to the stop of the vehicle is referred to as a stopped road marking, and the road marking other than the stopped road marking is referred to as another type road marking.

The storage unit stores a stop road marking pattern.
For example, the manufacturer of the road sign recognition device prepares a stop road sign pattern having a standard shape, dimensions, etc., and stores it in the storage unit in advance when the road sign recognition device is manufactured. . There may be one type of stop road marking pattern, and one type may be used for each of different character strings such as a character string marked “stop” and a character string marked “stop”. However, for example, it is not necessary to store a stop road marking pattern in advance for each character string having a different shape, dimension, or the like marked as “stop”.

The similarity calculation means calculates a similarity between at least a part of an image in front of the vehicle captured by the imaging unit (for example, an image of a road surface positioned in front of the vehicle) and a stop road marking pattern stored in the storage unit. .
The coincidence determining unit determines whether or not the calculation result of the similarity calculating unit is within the similarity allowable range of the similarity. The similarity allowable range is stored in, for example, a readable / writable nonvolatile storage device, and is changed under predetermined conditions as will be described later. However, when the road sign recognition device is shipped from the factory, it is desirable that the manufacturer calculates a similar allowable range based on a stop road sign having, for example, a standard shape, size, etc., and stores it in the road sign recognition device in advance. .

When the stop road sign exists in front of the vehicle, the stop road sign is shown in the image ahead of the vehicle captured by the imaging unit. At this time, the calculation result of the similarity calculation means falls within the similarity allowable range of the similarity.
On the other hand, when there is another road sign in front of the vehicle or there is no road sign, the other kind road sign is reflected in the image in front of the vehicle captured by the imaging unit, or the road sign is not captured. At this time, the calculation result of the similarity calculation unit is out of the similarity allowable range of the similarity.

However, even if the calculation result of the similarity calculation means is out of the similarity allowable range, a stop road sign may exist in front of the vehicle. For example, a similar permissible range corresponding to the stop road sign in the X region is stored in the road sign recognition device, and the stop road sign in the X region is shaped in the image in front of the vehicle captured by the imaging unit. This is a case where a stop road sign in Y region with different dimensions is shown. If the similar permissible range corresponding to the stop road sign in Y region is stored in the road sign recognition device, the calculation result of the similarity calculating means should be within the similar permissible range.
Therefore, when the coincidence determination unit determines NO, that is, when the calculation result of the similarity calculation unit is out of the similarity allowable range of the similarity, the result determination unit determines that the calculation result of the similarity calculation unit is within the maximum allowable range. It is determined whether or not. The maximum allowable range is wider than the similar allowable range.

The upper limit value and the lower limit value of the maximum allowable range are calculated by the manufacturer based on, for example, stopped road markings having various shapes and dimensions in various places throughout the country, and are read-only non-volatile memory when the road marking recognition device is manufactured. This is pre-stored in the apparatus.
By the way, in the place where the stop road sign is written, the vehicle must stop temporarily. For this reason, for example, the driver depresses the brake pedal and sets the vehicle speed to “0”.
When it is determined that the result determination unit is within the maximum allowable range, the stop determination unit determines whether a stop condition for stopping the vehicle is satisfied. The case where the stop condition is satisfied is, for example, a case where the brake pedal is depressed and the vehicle speed is equal to or lower than a predetermined speed.

When the stop determination means determines that the stop condition is satisfied, that is, when the vehicle is temporarily stopped at the place where the stop road sign exists, the changing means sets the upper limit value and / or the lower limit value of the similar allowable range, It changes based on the calculation result of the similarity calculation means. For example, when the calculation result of the similarity calculation unit is larger than the upper limit value of the similarity allowance range, the upper limit value of the similarity allowance range is set to a value equal to or greater than the calculation result of the similarity calculation unit.
That is, the similar allowable range is changed based on the stop road sign existing at the place where the vehicle has stopped. Since the similar allowable range changed in this way is a similar allowable range corresponding to the stop road marking in the region where the vehicle is currently located, the road sign recognition device will be used in the future in which the vehicle is currently located. It is difficult to make a mistake in determining whether there is a stop road sign.

  By the way, if the stop determination means determines NO, that is, if the vehicle does not stop once, it is clear that there is no stop road sign, so even if the calculation result of the similarity calculation means is within the maximum allowable range, No change in tolerance is made. In addition, when the result determination means determines NO, it is clear that there is no stop road marking, so the similarity allowable range is not changed.

In the second invention, an imaging unit, a storage unit, and an operation unit are provided, and further, similarity calculation means, coincidence determination means, operation determination means, and change means are provided.
The imaging unit images the vicinity of the vehicle. The road marking recognition apparatus recognizes a road marking related to the stop of the vehicle using an image captured by the imaging unit. Road markings using character strings such as “stop” and “to rare” may have different shapes, dimensions, etc. depending on the region. However, the shape and dimensions of character strings such as “Stop” and “Torrare” are naturally restricted by the regulations of the Road Traffic Law and Road Law, so the shape and dimensions of road markings will not be significantly different. .
Hereinafter, the road marking related to the stop of the vehicle is referred to as a stopped road marking, and the road marking other than the stopped road marking is referred to as another type road marking.

The storage unit stores a stop road marking pattern.
For example, the manufacturer of the road sign recognition device prepares a stop road sign pattern having a standard shape, dimensions, etc., and stores it in the storage unit in advance when the road sign recognition device is manufactured. . There may be one type of stop road marking pattern, and one type may be used for each of different character strings such as a character string marked “stop” and a character string marked “stop”. However, for example, it is not necessary to store a stop road marking pattern in advance for each character string having a different shape, dimension, or the like marked as “stop”.

By the way, in the place where the stop road sign is written, the vehicle must stop temporarily.
The driver operates the operation unit when the road sign recognition device learns the stop road sign. The case where the road sign recognition device learns the stop road sign is a case where the road sign recognition device does not recognize the stop road sign even though the stop road sign exists on the road surface ahead of the vehicle. The driver can easily determine whether or not the road sign recognition device recognizes the stop road sign depending on, for example, whether or not the image on the display unit is switched to an image around the vehicle.

The similarity calculation means calculates a similarity between at least a part of an image in front of the vehicle captured by the imaging unit (for example, an image of a road surface positioned in front of the vehicle) and a stop road marking pattern stored in the storage unit. .
The coincidence determining unit determines whether or not the calculation result of the similarity calculating unit is within the similarity allowable range of the similarity. The similarity allowable range is stored in, for example, a readable / writable nonvolatile storage device, and is changed under predetermined conditions as will be described later. However, when the road sign recognition device is shipped from the factory, it is desirable that the manufacturer calculates a similar allowable range based on a stop road sign having, for example, a standard shape, size, etc., and stores it in the road sign recognition device in advance. .

When the stop road sign exists in front of the vehicle, the stop road sign is shown in the image ahead of the vehicle captured by the imaging unit. At this time, the calculation result of the similarity calculation means falls within the similarity allowable range of the similarity.
On the other hand, when there is another road sign in front of the vehicle or there is no road sign, the other kind road sign is reflected in the image in front of the vehicle captured by the imaging unit, or the road sign is not captured. At this time, the calculation result of the similarity calculation unit is out of the similarity allowable range of the similarity.

However, even if the calculation result of the similarity calculation means is out of the similarity allowable range, a stop road sign may exist in front of the vehicle. For example, a similar permissible range corresponding to the stop road sign in the X region is stored in the road sign recognition device, and the stop road sign in the X region is shaped in the image in front of the vehicle captured by the imaging unit. This is a case where a stop road sign in Y region with different dimensions is shown. If the similar permissible range corresponding to the stop road sign in Y region is stored in the road sign recognition device, the calculation result of the similarity calculating means should be within the similar permissible range.
Therefore, when the coincidence determination unit determines NO, that is, when the calculation result of the similarity calculation unit is out of the similarity allowable range of the similarity, the operation determination unit determines whether or not the operation unit has been operated. .

When the operation determining unit determines that the operation unit has been operated, that is, when there is a stop road sign in front of the vehicle, the changing unit calculates the upper limit value and / or lower limit value of the similar allowable range as the calculation result of the similarity calculating unit. Change based on. For example, when the calculation result of the similarity calculation unit is smaller than the lower limit value of the similarity allowance range, the lower limit value of the similarity allowance range is set to a value less than or equal to the calculation result of the similarity calculation unit.
That is, the similar allowable range is changed based on the stop road sign existing in front of the vehicle. Since the similar allowable range changed in this way is a similar allowable range corresponding to the stop road marking in the region where the vehicle is currently located, the road sign recognition device will be used in the future in which the vehicle is currently located. It is difficult to make a mistake in determining whether there is a stop road sign.

  By the way, when the operation determination unit determines that the operation unit is not operated, that is, when the driver does not feel the necessity of learning, the similarity allowable range is not changed.

In the third invention, the changing means changes the upper limit value and / or lower limit value of the similar allowable range based on the calculation result of the similarity calculating means. At this time, the upper limit value and lower limit value of the similar allowable range are changed. So that the difference value is within a predetermined range.
This is because the similar permissible range corresponding to the stop road markings in each place has a difference between the upper limit value and the lower limit value, although the upper and lower limit values are different, and falls within a predetermined range.

  If the similar allowable range is changed to unlimited, inconvenience occurs in the recognition of the stop road sign. For example, the road sign recognition device for a vehicle traveling in the X region stores a similar allowable range corresponding to the stop road sign in the X region, and when the vehicle moves to the Y region, the upper limit value of the similar allowable range is set. It is assumed that the lower limit value of the similar allowable range is decreased in correspondence with the stop road sign in the Z region when the vehicle moves to the Z region and is increased in correspondence with the stop road sign in the Y region. Thereafter, when the vehicle moves to the X region, a similar allowable range that is uselessly wider than the similar allowable range corresponding to the stop road marking in the X region is stored in the road sign recognition device. There is a possibility of misrecognizing a seed road sign as a stop road sign. That is, there is a disadvantage that the learning result of the stop road marking in the Y and Z regions has an adverse effect on the learning result of the stop road marking in the X region.

  As a result, it is possible to prevent inconvenience that the learning result of one local road marking adversely affects the learning result of another local road marking. For this reason, even when the vehicle travels over a region where the shape, dimensions, etc. of the road markings are relatively different, the road markings at various locations can be accurately recognized.

In the fourth aspect of the invention, the image processing apparatus includes a display unit that displays an image, and the image displayed on the display unit is a car navigation image, a television image, or the like (hereinafter referred to as a normal image) displayed in a normal case. The image is switched to an image around the vehicle (hereinafter referred to as a surrounding image) captured by the imaging unit. It is desirable that the surrounding image shows a place that becomes a blind spot of the driver.
Switching from the normal image to the peripheral image is performed when the coincidence determining unit determines that the value is within the similar allowable range, or when the changing unit changes the upper limit value and / or the lower limit value of the similar allowable range.

Even when the coincidence determining unit determines that the value is within the similar allowable range, or when the changing unit changes the upper limit value and / or the lower limit value of the similar allowable range, the stop road sign is displayed on the image in front of the vehicle captured by the imaging unit. When the image is captured, that is, when the vehicle is temporarily stopped, a peripheral image is displayed on the display unit in order to support safe start of the vehicle that has stopped.
On the other hand, when the coincidence determination unit determines NO, the change unit does not change the upper limit value and / or the lower limit value of the similar allowable range, and the vehicle does not stop temporarily. Is displayed.

As a result, when a road sign such as “stop” or “stop” exists on the road surface in front of the vehicle, the image on the display unit can be switched to a peripheral image around the vehicle. For this reason, safe driving of the vehicle can be supported.
On the other hand, when there is no road sign such as “stop” or “stop” on the road surface in front of the vehicle, the image on the display unit can be left as the image displayed in the normal case. For this reason, when the display of a surrounding image is unnecessary, the problem that the image of a display part switches wastefully can be prevented.

According to the road marking recognition apparatus of the present invention, the road marking recognition apparatus can automatically learn road markings in various places. For this reason, the road marking recognition apparatus can accurately recognize road markings in various places. In addition, it is not necessary for the driver to manually learn the road markings at various locations in the road marking recognition device, and the driver can concentrate on driving.
Furthermore, it is not necessary for the manufacturer of the safe driving support device to prepare various types of road marking patterns corresponding to character strings such as “stop” and “stop” in various places. Moreover, in order to memorize | store these, it is not necessary to mount a memory | storage device with a large memory capacity in a safe driving assistance apparatus.

In the case of the road marking recognition apparatus according to the present invention, the driver can cause the road marking recognition apparatus to manually learn road markings in various places. For this reason, the road marking recognition apparatus can accurately recognize road markings in various places. Moreover, there is no need for the road sign recognition device to automatically learn road signs in various locations, and it prevents the inconvenience of incorrect learning results based on misrecognition, misjudgment, etc. of the road sign recognition device being accumulated in the road sign recognition device. can do.
Furthermore, it is not necessary for the manufacturer of the safe driving support device to prepare various types of road marking patterns corresponding to character strings such as “stop” and “stop” in various places. Moreover, in order to memorize | store these, it is not necessary to mount a memory | storage device with a large memory capacity in a safe driving assistance apparatus.

  Hereinafter, the present invention will be described in detail with reference to the drawings illustrating embodiments thereof.

Embodiment 1.
FIG. 1 is a block diagram showing a main configuration of the road marking recognition apparatus according to Embodiment 1 of the present invention.
In the figure, reference numeral 1 denotes a road sign recognition device, and the road sign recognition device 1 is mounted on a vehicle 6. As shown in FIG. 1, the road marking recognition apparatus 1 includes a recognition processing unit 2, an imaging unit 3, a display unit 41, and an image switching unit 42. The vehicle 6 includes a car navigation device 61, a vehicle speed sensor 62, and a brake 63 in addition to the road sign recognition device 1.

  The car navigation device 61 displays the current position, travel route, and the like of the vehicle 6 on the display unit 41 so as to notify the driver of the current position and guide the shortest route to the destination. For this purpose, the car navigation device 61 outputs image data indicating a car navigation image in which the current position, the travel route, and the like are described, to the image switching unit 42.

The vehicle speed sensor 62 detects the vehicle speed of the vehicle 6. The detection result of the vehicle speed sensor 62 is output to the recognition processing unit 2. The vehicle speed of the stopped vehicle 6 is “0”, and the vehicle speed from the departure to the complete stop exceeds “0”. Further, the vehicle speed of the vehicle 6 that is stopping is equal to or lower than a predetermined first speed (for example, 20 km / h), and the vehicle speed of the vehicle 6 immediately before the stop is equal to or lower than a second speed (for example, 5 km / h) that is lower than the first speed. is there.
The brake 63 is used when the vehicle 6 is decelerated and stopped. While the brake pedal is depressed by the driver of the vehicle 6, the brake 63 continuously outputs a brake signal indicating that the brake 63 is turned on to the recognition processing unit 2, and the brake pedal is depressed. While it is not, the brake signal output is stopped.

  By the way, in the place where the stop road sign is written, the driver must stop the vehicle 6 once. For this reason, the driver depresses the brake pedal and sets the vehicle speed to “0”. Thus, in the present embodiment, the vehicle 6 stops when the brake signal is output to the recognition processing unit 2 and the vehicle speed indicated by the detection result of the vehicle speed sensor 62 is equal to or lower than the second speed. Assume that the condition is met.

The imaging unit 3 is a camera unit having a lens unit 30, an imaging element 31, and a DSP (Digital Signal Processor) 32 for image processing, and images the periphery of the vehicle 6. The imaging unit 3 according to the present embodiment is configured to capture an image in front of the vehicle 6 over about 180 ° in the horizontal direction (see FIG. 2 described later), and a predetermined control signal (from the recognition processing unit 2). Hereinafter, the front of the vehicle 6 is continuously imaged while the imaging signal is being input, and imaging is not performed while the imaging signal is not being input.
Hereinafter, an image in front of the vehicle 6 is referred to as a front image.
The lens unit 30 shown in FIG. 1 includes one or a plurality of lenses having a wide angle of view such as a wide-angle lens or a fish-eye lens. The lens unit 30 is disposed in the center in the horizontal direction at the front end of the vehicle 6 or in the rear mirror, and condenses light incident from the front of the vehicle 6 onto the image sensor 31.

The image pickup device 31 uses a CCD, CMOS, or the like, detects the light collected by the lens unit 30, and outputs the detection result to the DSP 32 continuously or intermittently.
The DSP 32 generates, in a time series, image data indicating still images captured at a predetermined time interval (for example, 0.1 second interval) based on the output of the image sensor 31. Image data generated by the DSP 32 (that is, image data indicating a forward image captured by the imaging unit 3) is output to the recognition processing unit 2 and the image switching unit 42, respectively.
In the present embodiment, the imaging unit 3 operates when the vehicle speed is equal to or lower than the first speed, and the imaging unit 3 stops operating when the vehicle speed exceeds the first speed. However, the present invention is not limited to this. Is not to be done. For example, the imaging unit 3 may be always operating.

The display unit 41 is an in-vehicle display using, for example, a liquid crystal display panel, and displays an image based on the image data input from the image switching unit 42. Such a display part 41 is arrange | positioned in the position which a driver | operator can visually recognize easily.
Normally, a car navigation image is displayed on the display unit 41, but the image displayed on the display unit 41 may be switched from the car navigation image to the front image, as will be described later.

  In addition, the image displayed on the display part 41 is not limited to a car navigation image and a front image. For example, a television image may be displayed instead of the car navigation image. Further, instead of the front image, side images on the left and right sides of the vehicle 6 may be displayed. For this purpose, a plurality of imaging units 3 for imaging the front and side of the vehicle 6 may be provided, and an image captured by a single imaging unit 3 that captures images from the front to the side of the vehicle 6. The side image obtained by cutting out a part of the image may be displayed on the display unit 41.

  In the present embodiment, the image switching unit 42 is an ECU (Electronic Control Unit) having an image switching function for switching an image displayed on the display unit 41. The image switching unit 42 includes a plurality of types (this embodiment). 2 types) of image data and a switching signal for instructing switching of the image data. More specifically, the image switching unit 42 receives image data output from the DSP 32 of the imaging unit 3, image data output from the car navigation device 61, and a switching signal output from the recognition processing unit 2. Is done. The image switching unit 42 includes an image processing unit that performs image processing such as distortion correction, viewpoint conversion, enlargement / reduction, partial clipping, and superimposition of other images on the input image data. It is desirable that

When one type of image data is input, the image switching unit 42 outputs the input one type of image data to the display unit 41. When two types of image data are input, the image switching unit 42 outputs one of the two types of input image data to the display unit 41, but when a switching signal is input The image data to be output to the display unit 41 is switched from one to the other. For example, when image data input from the car navigation device 61 is output to the display unit 41, when a switching signal is input, the image data is switched to image data input from the DSP 32 of the imaging unit 3. The image data is output to the display unit 41.
Image data indicating a car navigation image is normally output from the image switching unit 42 to the display unit 41.

  Note that the physical configuration of the image switching unit 42 is not limited to an independent ECU as in the present embodiment. For example, instead of including the image switching unit 42, an image switching circuit having an image switching function may be incorporated in the display unit 41, the recognition processing unit 2, the car navigation device 61, or the like. In this case, the image switching circuit is controlled by a control unit such as each device or unit to switch the image displayed on the display unit 41. As a result, the device configuration can be simplified.

In the following, as road markings for instructing the stop of the vehicle 6 (that is, stopped road markings), examples using the character strings of “stop” and “to rare” are not limited to this, but “stop male”, A character string such as “stop” or “STOP” may be used. Further, the stop road marking is not limited to a character string instructing to stop the vehicle 6, and may be a symbol (for example, a stop line) indicating a stop position of the vehicle 6.
FIG. 2 is a plan view schematically showing a vehicle equipped with the road marking recognition apparatus according to Embodiment 1 of the present invention and a stop road marking written on the road surface in front of the vehicle. It is a schematic diagram which shows the image ahead of the vehicle which the imaging part with which this road marking recognition apparatus is provided imaged.

FIG. 2 illustrates a vehicle 6 that is about to enter a priority road in the horizontal direction in the figure from a non-priority road in the vertical direction in the figure on a T-shaped road. At the end of the non-priority road, a stop line and a stop road sign 71 using a character string “stop” are written. A two-dot chain line in FIG. 2 indicates a viewing angle of the imaging unit 3.
FIG. 3 illustrates a front image 50 captured by the imaging unit 3 in the T-junction shown in FIG. The front image 50 includes a stop road sign 71.
On the other hand, the stop road sign 71 is not marked on the priority road. For this reason, the stop road sign 71 is not shown in the front image which the imaging part 3 with which the vehicle 6 located in a priority road is equipped imaged. Even if the vehicle is located on the non-priority road, the stop road sign 71 is not shown in the front image captured by the imaging unit 3 until the vehicle 6 reaches the end of the non-priority road.

  The recognition processing unit 2 shown in FIG. 1 is an ECU for performing processing for recognizing a stop road sign 71 as shown in FIG. 2 based on the forward image 50 as shown in FIG. RAM 22, EEPROM 23, image processing unit 24, data memory 25, and image memory 26.

A ROM 21 shown in FIG. 1 is a read-only nonvolatile memory, and stores a control program and data for the road marking recognition apparatus 1. The data stored in the ROM 21 includes an upper limit value and a lower limit value of a similar allowable range, which will be described later, and default values of the upper limit value and the lower limit value of the maximum allowable range.
The RAM 22 is a readable / writable volatile memory.
The control unit 20 is a control center of the road sign recognition apparatus 1, and uses the RAM 22 as a work area and executes various processes according to the control program and data stored in the ROM 21. For example, the control unit 20 controls the imaging unit 3 to start / end imaging, and outputs a switching signal to the image switching unit 42 to switch the image displayed on the display unit 41. In addition, a detection result of the vehicle speed sensor 62 is input from the vehicle speed sensor 62 to the control unit 20, and a brake signal is input from the brake 63.

The EEPROM 23 is a readable / writable nonvolatile memory, and the control unit 20 writes the default values of the upper and lower limit values of the similar allowable range stored in the ROM 21 at a predetermined timing. This predetermined timing is, for example, when the road marking recognition apparatus 1 is shipped from the factory or when a reset switch (not shown) is operated by the driver.
The upper and lower limits of the similar allowable range written in the EEPROM 23 may be changed as will be described later.

The data memory 25 is a read-only or readable / writable nonvolatile memory using a mask ROM, EEPROM, or the like. In the data memory 25, various tables, filters, and the like used when the image processing unit 24 performs image processing such as distortion correction and edge extraction on the image data, and “stop” characters as shown in FIG. A road marking pattern 51 of a stop road marking 71 using a column is stored. That is, the data memory 25 functions as a storage unit.
The “stop” road marking pattern 51 uses binary image data, and is stored in the data memory 25 when the road marking recognition apparatus 1 is manufactured. Further, although not shown, the data memory 25 stores a road marking pattern of a stop road sign (hereinafter referred to as a road sign pattern 52 of the stop road sign 72) using a character string “Tare”. The road marking pattern 52 also uses binary image data.

The character strings of “stop” and “tore” of the stop road markings 71 and 72 may differ in shape, size, etc. depending on the region, but the data memory 25 corresponds to the stop road markings 71 and 72 in various parts of the country. The road marking pattern to be stored is not stored, and only the road marking patterns 51 and 52 corresponding to the “stop” and “stop” stop road markings 71 and 72 having standard shapes and dimensions are stored. .
The image memory 26 is a readable / writable volatile memory using SRAM, DRAM, or the like, and temporarily stores image data.

The image processing unit 24 uses an ASIC (Application Specified Integrated Circuit), and the image data output from the imaging unit 3 to the recognition processing unit 2 is input to the image processing unit 24. The image processing unit 24 performs predetermined image processing and similarity calculation processing on the image data input from the imaging unit 3 using the image memory 26 as a work area.
Here, the image processing performed by the image processing unit 24 is processing such as distortion correction, edge extraction, enlargement / reduction, and partial cutout, for example. The similarity calculation process performed by the image processing unit 24 is a process of calculating the similarity between the image data subjected to the image processing by the image processing unit 24 and each of the road marking patterns 51 and 52.

  The front image captured through the lens unit 30 is an image having distortion that is gradually curved from the center toward the end. In order to remove this distortion, the image processing unit 24 controlled by the control unit 20 uses the correction table stored in advance in the data memory 25 to perform image processing on the image data input from the imaging unit 3. A distortion correction process for correcting the distortion is performed. The distortion correction process is a process of moving the pixel value of each pixel of the input image data to a pixel at a predetermined position and interpolating the pixel value between the moved pixels, for example, by linear interpolation. In the correction table, each pixel and a pixel value destination are stored in association with each other.

  Further, the image processing unit 24 controlled by the control unit 20 performs edge extraction processing on the image data indicating the front image. In the edge extraction process, for example, when the difference between pixel values of one pixel and other pixels adjacent to the pixel exceeds a threshold value, the one pixel is extracted as an edge component. For this reason, in the edge extraction process, the image data is binarized using an edge extraction filter such as a Sobel filter or a pre-wit filter stored in the data memory 25. As a result, the front image subjected to the edge extraction process is a binary image.

  The similarity calculation process performed by the image processing unit 24 is a pre-process for the control unit 20 to determine whether or not the stop road sign 71 or the stop road sign 72 is shown in the forward image.

  As shown in FIG. 3, in the image processing unit 24, first, the road marking pattern 51 stored in the data memory 25 is used as a template, and the road marking pattern 51 and the road marking pattern 51 are converted into a template matching method. The degree of similarity with a part of the corresponding forward image (hereinafter referred to as the corresponding image) is calculated. The similarity is calculated by sequentially changing the corresponding image to another part of the front image (for example, shifting the corresponding position by one column / one row in the direction of the white arrow in the figure) with respect to the entire front image. All the calculated similarities are input to the control unit 20. Similarly, the image processing unit 24 calculates the similarity between the road marking pattern 52 stored in the data memory 25 and the corresponding image corresponding to the road marking pattern 52.

  In addition, it is not limited to the structure which performs template matching with respect to the whole image which the imaging part 3 imaged, A part (for example, horizontal direction center) with a high probability that the stop road sign is reflected among the images which the imaging part 3 imaged The template matching may be performed on the subordinate side.

In the control unit 20, one of the stop road markings 71 and 72 exists in front of the vehicle 6 based on the maximum similarity value x calculated by the image processing unit 24 (hereinafter referred to as the similarity x). It is determined whether or not to do so. For this determination, the similarity allowable range and the maximum allowable range of the similarity are used.
Here, the stop road sign 71 in the X region has a substantially standard shape, dimension, etc., and the stop road sign 71 in the Y region has a shape, dimension, etc. slightly different from the standard one, Z A case will be described in which the local stop road sign 71 has a shape, size, etc. that are relatively different from the standard one.

When template matching is performed by applying the road marking pattern 51 to the forward image 50 captured in the X region, the similarity x falls within a range of 95% to 100%, for example. Accordingly, a range of 95% to 100% is used as the similar allowable range corresponding to the stop road sign 71 in the X region.
However, when template matching is performed by applying the road marking pattern 51 to the forward image 50 captured in the Y region, the similarity x falls within a range of 90% to 95%, for example, and the forward image captured in the Z region. When the template matching is performed by applying the road marking pattern 51 to the image 50, the similarity x falls within a range of 80% to 85%, for example. Therefore, in the Y region, the range of 90% to 95% should be used as the similar allowable range, and in the Z region, the range of 80% to 85% should be used as the similar allowable range.

For this purpose, the road sign recognition apparatus 1 learns the stop road sign 71 in the region where the vehicle 6 is currently located, and appropriately changes the upper and lower limit values of the similar allowable range.
If the maximum value of the upper limit value of the similar allowable range is, for example, 100% and the minimum value of the lower limit value of the similar allowable range is 80% in all regions such as the X region, the Y region, and the Z region. A range of 80% to 100% is used as the maximum allowable range. This is because, when the road marking pattern 51 is applied to the captured forward image and a similarity x of 80% or more is obtained, a stop road marking 71 in any local area is shown in the forward image. Because there is a possibility. On the other hand, when a similarity x of less than 80% is obtained, it is obvious that the forward road image 71 is not shown in the stop road sign 71.

The lower limit default value A ₀ (hereinafter referred to as the lower limit value A ₀ ) and the upper limit default value B ₀ (hereinafter referred to as the upper limit value B ₀ ) of the similar permissible range are determined when the road marking recognition device 1 is shipped from the factory. It is calculated based on a stop road sign having a standard shape, dimensions, etc. (hereinafter referred to as a standard stop road sign) and stored in the ROM 21. The similar permissible range corresponding to the standard stop road marking is a range between the lower limit value A ₀ and the upper limit value B ₀ .

The maximum permissible range is a range between the lower limit value α and the upper limit value β. The lower limit value α and the upper limit value β of the maximum permissible range are various values throughout the country when the road sign recognition device 1 is shipped from the factory. It is calculated based on the stop road marking having the shape, dimensions, etc., and is stored in the ROM 21. Therefore, the lower limit value α ≦ the lower limit value A ₀ and the upper limit value β ≧ the upper limit value B ₀ . However, the similar permissible range and the maximum permissible range do not match (that is, the condition that the lower limit value α = the lower limit value A ₀ and the upper limit value β = the upper limit value B ₀ are compatible).

FIG. 4 is an explanatory diagram showing upper and lower limit values of the similar allowable range and the maximum allowable range used in the road marking recognition apparatus according to Embodiment 1 of the present invention.
FIG. 4A shows a number line representing the magnitude of the similarity x and the upper and lower limits A ₀ and B _{0 of} the similar allowable range and the upper and lower limits α and β of the maximum allowable range. FIG. 4B shows the similarity x, the upper and lower limits A ₁ and B _{1 of} the similar allowable range changed from the upper and lower limits A ₀ and B _{0 of} the similar allowable range, and the upper and lower limits of the maximum allowable range. A number line representing the magnitude of the values α and β is shown.

When the similarity x between the forward image and the road marking pattern is the similarity x ₁ shown in FIG. 4A, the lower limit value A ₀ ≦ similarity x ₁ ≦ the upper limit value B ₀ . That is, the similarity x ₁ is within the similar allowable range [A ₀ , B ₀ ]. Since the similar permissible range [A ₀ , B ₀ ] corresponds to the standard stop road sign, it can be seen that the standard stop road sign is shown in the front image. It can be seen that there is a stop road sign. Further, it can be seen that the vehicle 6 is located in the region X where the standard stop road marking is marked on the road surface.

On the other hand, when the similarity x between the forward image and the road marking pattern is the similarity x ₀ or the similarity x ₂ , the similarity x ₀ <the lower limit value A ₀ or the similarity x ₂ > the upper limit value B ₀ . That is, the similarities x ₀ and x ₂ are out of the similarity allowable range [A ₀ , B ₀ ]. Accordingly, it can be seen that at least the standard stop road sign is not shown in the front image, and that at least the standard stop road sign does not exist in front of the vehicle 6.

However, the similarity x ₂ is lower limit value α ≦ similarity x ₂ ≦ upper limit value β. That is, the similarity x ₂ is within the maximum allowable range [α, β]. At this time, if the stop condition is satisfied, it can be seen that a stop road sign other than the standard stop road sign is shown in the front image, and the standard stop road is ahead of the vehicle 6. It can be seen that there is a stop road sign other than the sign. Further, it can be seen that the vehicle 6 is located in a region Y where a stop road sign other than the standard stop road sign is marked on the road surface.

Since the similar allowable range [A ₀ , B ₀ ] corresponds to the standard stop road marking, the similar allowable range [A ₀ , B ₀ ] stored in the EEPROM 23 is the road surface of the region Y where the vehicle 6 is located. Is changed to the similar allowable range [A ₁ , B ₁ ] corresponding to the stop road marking described in ( ₁ ).
On the other hand, even if the similarity x ₂ is within the maximum allowable range [α, β], if the stop condition is not satisfied, it can be seen that the stop road sign is not shown in the front image, It can be seen that there is no stop road sign ahead of the vehicle 6. Therefore, the similar allowable range [A ₀ , B ₀ ] is not changed.
By the way, for the similarity x ₀ , the similarity x ₀ <the lower limit value α. That is, the similarity x ₀ is out of the maximum allowable range. Therefore, it can be seen that the stop road sign is not shown in the front image, and that the stop road sign is not present in front of the vehicle 6.

Next, when the newly calculated similarity x is similarity x ₂ , lower limit value A ₁ ≦ similarity x ₂ ≦ upper limit value B _{1 as shown} in FIG. That is, the similarity x ₂ is within the similar allowable range [A ₁ , B ₁ ]. Since the similar allowable range [A ₁ , B ₁ ] corresponds to the stop road sign of the region Y, it can be seen that the stop road sign of the region Y exists in front of the vehicle 6.

On the other hand, when the similarity x between the forward image and the road marking pattern is the similarity x ₁ , the similarity x ₁ <the lower limit value A ₁ . That is, the similarity x ₁ is out of the similarity allowable range [A ₁ , B ₁ ]. Therefore, it can be seen that there is no stop road sign for at least the region Y in front of the vehicle 6.
However, lower limit value α ≦ similarity x ₁ ≦ upper limit value β. That is, the similarity x ₁ is within the maximum allowable range [α, β]. At this time, if the stop condition is satisfied, it can be seen that there is a stop road sign for the region Z other than the region Y in front of the vehicle 6.

Similar tolerance [A _1, B _1] is to correspond to the stop road sign of local Y, similar tolerances are stored in the EEPROM23 [A _1, B _1] corresponds to the stop road sign in rural Z shown The similar allowable range [A ₂ , B ₂ ] is not changed.
On the other hand, even if the similarity x ₁ is within the maximum allowable range [α, β], if the stop condition is not satisfied, it can be seen that the stop road sign is not shown in the front image, It can be seen that there is no stop road sign ahead of the vehicle 6.
By the way, for the similarity x ₀ , the similarity x ₀ <the lower limit value α. That is, the similarity x ₀ is out of the maximum allowable range. Therefore, it can be seen that the stop road sign is not shown in the front image, and that the stop road sign is not present in front of the vehicle 6.

Hereinafter, how to determine the upper and lower limit values of the similar allowable range will be described. Here, using the integer i (i ≧ 0), the upper and lower limit values before change are upper and lower limit values A _i and B _i , and the upper and lower limit values after change are A _{i + 1} and B _{i + 1} .
In the present embodiment, when similarity x> upper limit value _Bi , the upper limit value B _{i + 1} after change is equal to the similarity degree x, and the difference value between the upper and lower limit values A _i and B _i before the change is changed. The changed upper and lower limit values A _{i + 1} and B _{i + 1} are obtained so that the difference between the subsequent upper and lower limit values A _{i + 1} and B _{i + 1} becomes equal. That is, the changed upper and lower limit values A _{i + 1} and B _{i + 1} are obtained using the equations (2) and (3) under the condition of the following equation (1).

B _i <x (1)
B _{i + 1} = x (2)
B _i −A _i = B _{i + 1} −A _{i + 1} (3)
FIG. 4B shows the similar allowable range [A ₁ , B ₁ ] obtained based on the equations (1) to (3) when i = 0.

On the other hand, when similarity x <lower limit value A _i , lower limit value A _{i + 1} after change is equal to similarity degree x, and the difference between upper and lower limit values A _i and B _i before change and upper and lower limit values after change The upper and lower limit values A _{i + 1} and B _{i + 1} after the change are obtained so that the difference value between the values A _{i + 1} and B _{i + 1} becomes equal. That is, the changed upper and lower limit values A _{i + 1} and B _{i + 1} are obtained using the equations (5) and (6) under the condition of the following equation (4).
A _i > x (4)
A _{i + 1} = x (5)
B _i −A _i = B _{i + 1} −A _{i + 1} (6)

In addition, the calculation formulas of the upper and lower limit values A _{i + 1} and B _{i + 1} after the change are not limited to the formulas (1) to (6). For example, an appropriate positive number c is used, and the following formulas (7) and (8) may be used instead of the formulas (2) and (5).
B _{i + 1} = x + c (7)
However, B _{i + 1} ≦ β
A _{i + 1} = x−c (8)
However, A _{i + 1} ≧ α

Further, positive numbers d and e satisfying d ≦ B _i −A _i ≦ e and d ≠ e may be used, and the following formula (9) may be used instead of formulas (3) and (6).
d ≦ B _{i + 1} −A _{i + 1} ≦ e (9)
In this case, as long as the formula (9) is satisfied, the upper limit value B _i is changed to the upper limit value B _{i + 1} based on the formula (2) or the formula (7), and the lower limit value A _i is changed to the lower limit value. A _i may be set as the lower limit A _{i + 1} as it is. Or, the lower limit value A _i, Equation (5) or changed to the lower limit value A _{i + 1} based on equation (8), the upper limit value B _i, as the upper limit the upper limit B _i B _{i + 1} And it is sufficient.

The reason why the changed upper and lower limit values A _{i + 1} and B _{i + 1} are calculated using the equations (3), (6) or (9) is similar to the stop road markings 71 and 72 in various parts of the country. Although the allowable range is different between the upper and lower limit values A and B, the difference value B−A between the upper limit value B and the lower limit value A is substantially constant or falls within the predetermined range [d, e]. . If the formula (3), (6) or to calculate the upper and lower limit values _{A i + 1, B i +} 1 without using the equation (9), finally bounds similar tolerance values A _{i + 1} and B _{i + 1} may coincide with the upper and lower limit values α and β of the maximum allowable range, and the meaning of learning the stop road markings 71 and 72 is lost.
In this embodiment, the upper and lower limit values A and B of the similar allowable range and the upper and lower limit values α and β of the maximum allowable range are common to both the stop road signs 71 and 72. Each upper and lower limit value may be set for each.

5 and 6 are flowcharts showing the procedure of the road marking recognition process executed by the road marking recognition apparatus according to Embodiment 1 of the present invention.
The control unit 20 determines whether or not the vehicle speed is equal to or lower than the first speed based on the detection result of the vehicle speed sensor 62 input from the vehicle speed sensor 62 (S11). When the vehicle speed exceeds the first speed (NO in S11), since the vehicle 6 is traveling at a high speed, the control unit 20 repeatedly executes the process of S11 without performing the processes after S12 described later. To do.

  If the vehicle speed is equal to or lower than the first speed (YES in S11), the vehicle 6 may be stopped, so the control unit 20 starts outputting the imaging signal to the imaging unit 3 (S12). The imaging unit 3 to which the imaging signal is input starts imaging around the vehicle 6 and outputs image data of the captured image to the image processing unit 24. The image processing unit 24 to which the image data has been input performs image processing on the input image data, and is based on the image data on which the image processing has been performed and road marking patterns 51 and 52 stored in the data memory 25. The similarity is calculated and the calculation result is input to the control unit 20. That is, the image processing unit 24 functions as a similarity calculation unit.

Based on the similarity calculation result input from the image processing unit 24, the control unit 20 obtains the similarity x that is the maximum value of the calculation result (S13), and the obtained similarity x is stored in the EEPROM 23. It is determined whether or not it is within a certain permissible range [A _i , B _i ] (S14). That is, the control unit 20 in S14 functions as a coincidence determination unit.

If a x <A _i Matawax> B _i (NO at S14), since the similarity x obtained in S13, similar tolerances are stored in the EEPROM23 [A _i, B _i] is out of the control The unit 20 determines whether or not the similarity x obtained in S13 is within the maximum allowable range [α, β] (S15). That is, the control unit 20 in S15 functions as a result determination unit.
If α ≦ x ≦ β (YES in S15), there may be a stop road sign 71 or a stop road sign 72 that does not correspond to the similar allowable range [A _i , B _i ] in front of the vehicle 6. The control unit 20 determines whether a brake signal is input from the brake 63 (S16). Further, the control unit 20 determines whether or not the vehicle speed is equal to or lower than the second speed based on the detection result of the vehicle speed sensor 62 input from the vehicle speed sensor 62 (S17). That is, the control unit 20 in S16 and S17 functions as a stop determination unit.

If the brake signal is input (YES in S16) and the vehicle speed is equal to or lower than the second speed (YES in S17), the stopping condition is satisfied, and therefore the control unit 20 performs the expressions (1) to ( 3) or upper and lower limit values A _{i + 1} and B _{i + 1} of the similar allowable range are calculated using equations (4) to (6) (S18), and the calculated upper and lower limit values A _{i + 1} and B _i are calculated. ₊₁ is stored in the EEPROM 23 (S19). As a result, the upper and lower limit values of the similar allowable range are changed from the upper and lower limit values A _i and B _i to the upper and lower limit values A _{i + 1} and B _{i + 1} . That is, the control unit 20 in S18 and S19 functions as a changing unit.

When the process of S19 is completed, since there is a stop road sign 71 or a stop road sign 72 corresponding to the similar allowable range [A _{i + 1} , B _{i + 1} ] in front of the vehicle 6, the control unit 20 The process proceeds to S20.
When A _i ≦ x ≦ B _i (YES in S14), a stop road sign 71 or a stop road sign 72 corresponding to the similar allowable range [A _i , B _i ] stored in the EEPROM 23 is ahead of the vehicle 6. Therefore, the control unit 20 moves the process to S20.

The control unit 20 outputs a switching signal to the image switching unit 42 (S20). By executing S20, the image switching unit 42 switches the image data output to the display unit 41 from the image data of the car navigation image to the image data of the front image. As a result, the image displayed on the display unit 41 is switched from the car navigation image to the front image. That is, the control unit 20 and the image switching unit 42 in S20 function as an image switching unit.
After the process of S20 is completed, the control unit 20 determines whether or not the vehicle speed exceeds the first speed based on the detection result of the vehicle speed sensor 62 input from the vehicle speed sensor 62 (S21). When the vehicle speed is equal to or lower than the first speed (NO in S21), the control unit 20 repeatedly executes the process of S21.

When the vehicle speed exceeds the first speed (YES in S21), the control unit 20 outputs a switching signal to the image switching unit 42 again (S22). By executing S22, the image switching unit 42 switches the image data output to the display unit 41 from the image data of the front image to the image data of the car navigation image. As a result, the image displayed on the display unit 41 is switched from the front image to the car navigation image.
After completing the process of S22, the control unit 20 returns the process to S11.

  If x <α or x> β (NO in S15), the similarity x determined in S13 is out of the maximum allowable range, and therefore there is no stop road sign 71 and stop road sign 72 ahead of the vehicle 6. I understand that. If no brake signal is input (NO in S16) or if the vehicle speed exceeds the second speed (NO in S17), the stop condition is not satisfied, so the vehicle stops in front of the vehicle 6. It can be seen that the road sign 71 and the stop road sign 72 do not exist. When the stop road sign 71 and the stop road sign 72 do not exist in front of the vehicle 6, the control unit 20 stops outputting the imaging signal to the imaging unit 3 (S23), and returns the process to S11. The imaging unit 3 that stops receiving the imaging signal stops imaging.

  Since the road marking recognition apparatus 1 as described above can automatically learn stop road markings in various parts of the country, the “not” and “to rare” road sign patterns in various parts of the country are stored in the data memory 25. Even without this, stop road markings throughout the country can be accurately recognized.

Embodiment 2. FIG.
FIG. 7 is a block diagram showing a main configuration of the road marking recognition apparatus according to Embodiment 2 of the present invention.
In the figure, reference numeral 1 denotes a road marking recognition apparatus, and the road marking recognition apparatus 1 of the present embodiment has a configuration substantially similar to that of the road marking recognition apparatus 1 of the first embodiment (see FIG. 1).
However, as shown in FIG. 7, the road marking recognition apparatus 1 of the present embodiment includes an operation unit 43, and the operation unit 43 is disposed at a position where the driver can easily operate. The operation unit 43 operated by the driver outputs an operation signal to the control unit 20, but the operation unit 43 that is not operated does not output the operation signal.
Other parts corresponding to those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  The driver operates the operation unit 43 when the road sign recognition device 1 learns the stop road sign. The case where the road sign recognition device 1 learns the stop road sign is a case where the image on the display unit 41 does not switch from the car navigation image to the front image even though the stop road sign exists in front of the vehicle 6.

When the similarity x between the forward image and the road marking pattern 51 or the road marking pattern 52 is the similarity x ₂ shown in FIG. 4A, the upper limit value B ₀ <similarity x ₂ . That is, the similarity x ₂ is out of the similarity allowable range [A ₀ , B ₀ ]. Therefore, it can be seen that at least a standard stop road sign does not exist in front of the vehicle 6.
However, lower limit value α ≦ similarity x ₂ ≦ upper limit value β. That is, the similarity x ₂ is within the maximum allowable range [α, β]. At this time, if the driver operates the operation unit 43, it can be seen that there is a stop road sign other than the standard stop road sign in front of the vehicle 6.

Since the similar allowable range [A ₀ , B ₀ ] corresponds to the standard stop road marking, the similar allowable range [A ₀ , B ₀ ] stored in the EEPROM 23 is changed to the similar allowable range [A ₁ , B ₁ ]. Be changed.
On the other hand, even if the similarity x ₂ is within the maximum allowable range [α, β], if the driver does not operate the operation unit 43, there may be no stop road sign ahead of the vehicle 6. Recognize. Therefore, the similar allowable range [A ₀ , B ₀ ] is not changed.

  FIG. 8 is a flowchart showing a procedure of road marking recognition processing executed by the road marking recognition apparatus according to Embodiment 2 of the present invention. The road sign recognition process of the present embodiment is substantially the same as the road sign recognition process of the first embodiment (see FIGS. 5 and 6), but instead of the processes of S15 to S17 shown in FIG. The process of S30 shown in FIG. 8 is executed. Below, the description of the process of S11-S14 and S18-S23 is abbreviate | omitted.

If a x <A _i Matawax> B _i (NO at S14), since the similarity x obtained in S13, similar tolerances are stored in the EEPROM23 [A _i, B _i] is out of the control The unit 20 determines whether or not an operation signal is input from the operation unit 43 (S30). That is, the control unit 20 determines whether or not the operation unit 43 has been operated. That is, the control unit 20 in S30 functions as an operation determination unit.
When an operation signal is input from the operation unit 43 (YES in S30), the control unit 20 executes the process of S18. That is, the control unit 20 in S18 and S19 functions as a changing unit.
When the operation signal is not input from the operation unit 43 (NO in S30), the control unit 20 executes the processes after S23.

  Since the road marking recognition apparatus 1 as described above can manually learn the stopped road markings in various parts of the country, the “stop” and “to rare” road sign patterns in various parts of the country are stored in the data memory 25. Even if it is not memorized, stop road markings throughout the country can be accurately recognized.

It is a block diagram which shows the principal part structure of the road marking recognition apparatus which concerns on Embodiment 1 of this invention. It is a top view which shows typically the vehicle carrying the road marking recognition apparatus which concerns on Embodiment 1 of this invention, and the stop road marking written on the road surface ahead of a vehicle. It is a schematic diagram which shows the image ahead of the vehicle imaged by the imaging part with which the road marking recognition apparatus which concerns on Embodiment 1 of this invention is provided. It is explanatory drawing which shows the upper and lower limit value of each of the similar permissible range and the maximum permissible range used in the road marking recognition apparatus according to Embodiment 1 of the present invention. It is a flowchart which shows the procedure of the road marking recognition process performed with the road marking recognition apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the procedure of the road marking recognition process performed with the road marking recognition apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the principal part structure of the road marking recognition apparatus which concerns on Embodiment 2 of this invention. It is a flowchart which shows the procedure of the road marking recognition process performed with the road marking recognition apparatus which concerns on Embodiment 2 of this invention.

Explanation of symbols

1 road marking recognition device 2 recognition processing unit 25 data memory (storage unit)
3 Imaging unit 41 Display unit 42 Image switching unit (image switching unit)
43 Operation unit 50 Front image (image in front of the vehicle)
51 Road marking pattern (Road marking pattern)
6 Vehicles 71 Stop road marking (road marking)

Claims (4)

An imaging unit for imaging the periphery of the vehicle;
A storage unit for storing road marking patterns related to vehicle stops,
In the road marking recognition device that recognizes the road marking using the image captured by the imaging unit,
Similarity calculation means for calculating a similarity between at least a part of an image in front of the vehicle imaged by the imaging unit and a road marking pattern stored in the storage unit;
A coincidence determination means for determining whether the calculation result of the similarity calculation means is within the similarity allowable range of the similarity;
A result determination means for determining whether or not the calculation result is within a maximum allowable range wider than the similar allowable range when the coincidence determination means determines NO;
When the result determining means determines that the vehicle is within the maximum allowable range, a stop determining means for determining whether a stop condition for stopping the vehicle is satisfied;
A road marking recognition device, comprising: a changing unit that changes an upper limit value and / or a lower limit value of the similar allowable range based on the calculation result when it is determined that the stop determination unit is established. . An imaging unit for imaging the periphery of the vehicle;
A storage unit for storing road marking patterns related to vehicle stops,
In the road marking recognition device that recognizes the road marking using the image captured by the imaging unit,
An operation unit;
Similarity calculation means for calculating a similarity between at least a part of an image in front of the vehicle imaged by the imaging unit and a road marking pattern stored in the storage unit;
A coincidence determination means for determining whether the calculation result of the similarity calculation means is within the similarity allowable range of the similarity;
An operation determining unit that determines whether or not the operation unit has been operated when the coincidence determining unit determines NO;
A road sign recognition apparatus, comprising: a changing unit that changes an upper limit value and / or a lower limit value of the similar allowable range based on the calculation result when it is determined that the operation determination unit is operated.   The changing means changes the upper limit value and / or the lower limit value of the similar allowable range based on the calculation result so that a difference value between the upper limit value and the lower limit value of the similar allowable range is within a predetermined range. The road marking recognition apparatus according to claim 1, wherein the road marking recognition apparatus is configured as described above. A display for displaying an image;
When it is determined that the coincidence determining unit is within the similar allowable range, or when the changing unit changes the upper limit value and / or the lower limit value of the similar allowable range, an image displayed on the display unit is displayed. The road marking recognition apparatus according to claim 1, further comprising: an image switching unit that switches to an image around the vehicle captured by the imaging unit.

Images