JP4613738B2 - Intersection recognition system and intersection recognition method - Google Patents

Description

  The present invention relates to an intersection recognition system and an intersection recognition method.

  Conventionally, when a vehicle is traveling along a road, a first intersection that recognizes a stop line in front of the vehicle and recognizes an approach to the intersection of the vehicle according to the recognition result of the stop line A recognition method is provided (see, for example, Patent Document 1).

In addition, the side of the vehicle is photographed by a camera, and a travel road line along the traveling direction of the vehicle is detected based on an image acquired along with the photographing, and the traveling road road line is in front of the traveling direction of the vehicle. Provided is a second intersection recognition method for recognizing entry of a vehicle into an intersection when a road intersection line extending in a direction different from the road road line is detected from the interrupted position. (For example, see Patent Document 2).
JP 2003-85562 A JP 2004-102827 A

  However, in the first and second intersection recognition methods of the related art, in the case of the first intersection recognition method, for an intersection with a stop line, it is possible to recognize the approach of the vehicle to the intersection. For an intersection without a mark, it is impossible to recognize the approach of the vehicle to the intersection.

  Further, in the second intersection recognition method, the roadway road line and the roadway intersection line may not be detected by the edge, and the approach of the vehicle to the intersection cannot be accurately recognized.

  The present invention solves the problems of the conventional first and second intersection recognition methods, and provides an intersection recognition system and an intersection recognition method capable of accurately recognizing the approach of a vehicle to an intersection. Objective.

  Therefore, in the intersection recognition system of the present invention, an imaging device that is attached to a predetermined portion of the vehicle and images the front of the vehicle, and a radar device that irradiates the irradiation medium and measures the distance to the irradiated object Image recognition processing means for performing image recognition processing on the image data of a plurality of frames sent from the imaging device, detecting an intersection vehicle moving at the intersection as a vehicle candidate, and the detection between each frame A moving variable calculation processing means for calculating an actual moving amount of the vehicle candidate and calculating a moving speed of the vehicle candidate based on the actual moving amount and a switching period of each frame; Is provided as a parameter to the radar apparatus, and distance acquisition processing means for acquiring the distance to the vehicle candidate measured by the radar apparatus.

  According to the present invention, image recognition processing is performed on image data, an intersecting vehicle moving at an intersection is detected as a vehicle candidate, a moving speed of the detected vehicle candidate is calculated, and the moving speed is used as a parameter. Since the distance to the vehicle candidate measured by the radar apparatus is given to the radar apparatus, the approach of the vehicle to the intersection can be recognized accurately.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing a vehicle control system according to an embodiment of the present invention.

  In the figure, reference numeral 10 denotes a control unit, and the control unit 10 is a power train that performs shifting at a predetermined gear ratio, for example, a continuously variable transmission (CVT) as an automatic transmission, a stepped transmission (automatic transmission). Control of an electric drive device etc. is performed, and other various controls are performed. Therefore, the control unit 10 is a control device that performs overall control of the control unit 10 and a CPU 21 as an arithmetic device, a RAM 22 that is used as a working memory when the CPU 21 performs various arithmetic processes, In addition to the program for the above, a ROM 23 in which various programs for searching for a route to the destination, route guidance, etc. are recorded, a flash memory (not shown) used for recording various data, programs, etc. Prepare.

  Reference numeral 14 is an information terminal, for example, a navigation device as an in-vehicle device mounted on a vehicle, 63 is a network, 51 is an information center as an information provider, and the control unit 10, navigation device 14, network 63, information A vehicle control system is configured by the center 51 and the like.

  The navigation device 14 includes a GPS (Global Positioning System) sensor 15 as a current position detection unit for detecting a current position, a data recording unit 16 as an information recording unit in which various information is recorded in addition to map data, and input information. The navigation processing unit 17 that performs various arithmetic processing such as navigation processing, the direction sensor 18 as a direction detection unit that detects the vehicle direction, and a driver who is an operator performs predetermined input. The operation unit 34 as a first input unit for display, various displays by an image displayed on a screen (not shown), and a display unit 35 as a first output unit for notifying the driver Voice input unit 36 as a second input unit for performing input, various displays are performed by voice, and the driver is notified A voice output unit 37 as a second output unit, and a communication unit 38 as a transmission / reception unit functioning as a communication terminal. The navigation processing unit 17 includes a GPS sensor 15, a data recording unit 16, an orientation sensor 18, an operation The unit 34, the display unit 35, the audio input unit 36, the audio output unit 37, and the communication unit 38 are connected.

  The navigation processing unit 17 includes the control unit 10, an accelerator sensor 42 as an engine load detection unit that detects an accelerator pedal operation by the driver based on an accelerator opening, and a brake pedal operation amount by the driver. A brake sensor 43 as a braking detection unit for detecting the vehicle speed, a vehicle speed sensor 44 as a vehicle speed detection unit for detecting the vehicle speed, a light distribution device 46 for performing light distribution of a headlight (not shown) as an irradiating device, a predetermined front end of the vehicle A front monitoring device 48 for monitoring the front of the vehicle, an imaging device 49 for photographing the periphery of the vehicle, a steering wheel, and a steering angle of the vehicle. A steering sensor 50 or the like serving as a rudder angle detector for detecting the steering angle is connected. The imaging device 49 is disposed at the front end of the vehicle and is an imaging device 49F such as a front camera for photographing the front of the vehicle, and the imaging device 49B such as a back camera is disposed at the rear end of the vehicle for photographing the rear of the vehicle. , And imaging devices 49L and 49R such as side cameras that are disposed on the left and right side mirrors of the vehicle and photograph the side of the vehicle. The accelerator sensor 42, the brake sensor 43, the steering sensor 50, and the like constitute an operation information detection unit for detecting vehicle operation information by the driver. Further, the vehicle speed sensor 44 and the steering sensor 50 constitute a vehicle information detection unit for detecting vehicle information.

  The GPS sensor 15 detects the current location on the earth by receiving radio waves generated by an artificial satellite, and also detects the time. In the present embodiment, the GPS sensor 15 is used as the current position detection unit. Instead of the GPS sensor 15, the steering sensor 50, a distance sensor (not shown), an altimeter, etc. are used alone or It can also be used in combination. Further, a gyro sensor, a geomagnetic sensor, or the like can be used as the direction sensor 18.

  The data recording unit 16 includes a map database including map data files, and map data is recorded in the map database. The map data includes intersection data related to intersections, node data related to nodes, road data related to road links, search data processed for searching, facility data related to facilities, and feature data related to features. .

  The feature is a display object that is installed or formed on a road in order to provide various driving information to the driver or to perform various driving guidance. , Crosswalks, manholes, traffic lights, etc. The display line includes a stop line for stopping the vehicle, a vehicle lane boundary line that divides each lane, a partition line that indicates a parking space, and the road sign indicates an advancing direction in each lane. , A guidance sign for notifying a stop point such as “stop”, and the like. The feature data includes position information that represents the position of each feature by coordinates, image information that represents each feature by image, and the like. The temporary stop points include an entry point from a non-priority road to a priority road, a railroad crossing, an intersection where a red light flashes, and the like.

  The road data includes lane data including lane numbers assigned to each lane on the road, lane position information, and the like. Data for outputting predetermined information by the audio output unit 37 is also recorded in the data recording unit 16.

  Further, the data recording unit 16 includes a statistical database composed of statistical data files, a travel history database composed of travel history data files, and the like. Statistical data is stored in the statistical data file, and travel history data is stored in the travel history data file. However, both are recorded as performance data.

  The statistical data is a history of traffic information provided in the past, that is, history information representing a history, and a road traffic information center (not shown) such as a VICS (registered trademark: Vehicle Information and Communication System) center as an information provider. Traffic information provided in the past, etc., road traffic census information that is data representing traffic volume by road traffic census provided by the Ministry of Land, Infrastructure, Transport and Tourism, road timetable information provided by the Ministry of Land, Infrastructure, Transport and Tourism alone, or It is created by using in combination, processing as needed, and applying statistical processing. It is possible to add traffic jam prediction information for predicting traffic jams to the statistical data. In that case, in creating the statistical data, detailed conditions such as date and time, day of the week, weather, various events, seasons, facility information (presence / absence of large facilities such as department stores and supermarkets) are added to the history information.

  The data items of the statistical data include a link number for each road link, a direction flag indicating a traveling direction, an information type indicating a type of information, a congestion level at each predetermined timing, and when traveling on each road link. It consists of a link required time representing a required time for each predetermined timing, average data for each day of the link required time (for example, day average data), and the like.

  Further, the travel history data is collected from a plurality of vehicles, that is, the own vehicle or other vehicles by the information center 51, and is a record of the travel of the vehicle on the road on which each vehicle travels, that is, the track record information representing the track record. It is calculated and accumulated as probe data based on the running data.

  The data items of the travel history data include a link required time at each predetermined timing when traveling on each road link, a degree of congestion at each predetermined timing when traveling on each road link, and the like. Note that travel history data can be added to the statistical data. Further, in the present embodiment, the congestion level is used as a congestion index that represents the degree of congestion, and is represented separately for congestion, congestion, and non-congestion.

  The data recording unit 16 includes a disk (not shown) such as a hard disk, a CD, a DVD, and an optical disk in order to record the various data, and also reads / writes for reading and writing various data. A head (not shown) such as a head is provided. A memory card or the like can be used for the data recording unit 16.

  In the present embodiment, the data recording unit 16 is provided with the map database, statistical database, travel history database, and the like. However, in the information center 51, the map database, statistical database, travel history, etc. A history database or the like can also be provided.

  The navigation processing unit 17 is a control device that performs overall control of the navigation device 14 and a CPU 31 as an arithmetic device, a RAM 32 that is used as a working memory when the CPU 31 performs various arithmetic processes, In addition to the above program, a ROM 33 in which various programs for searching for a route to the destination, route guidance and the like are recorded, and a flash memory (not shown) used for recording various data, programs and the like are provided. .

  In the present embodiment, various programs can be recorded in the ROM 33 and various data can be recorded in the data recording unit 16, but the programs, data, and the like can also be recorded on a disk or the like. In this case, the program, data, etc. can be read from a disk or the like and written to the flash memory. Therefore, the program, data, etc. can be updated by exchanging the disk or the like. Further, the control program, data, and the like of the control unit 10 can be recorded on a disk or the like. Further, the program, data, and the like can be received via the communication unit 38 and written into the flash memory of the navigation processing unit 17.

  The operation unit 34 is operated by the driver to correct the current location at the start of traveling, to input a departure point and a destination, to input a passing point, and to activate the communication unit 38. As the operation unit 34, a keyboard, a mouse, or the like disposed independently of the display unit 35 can be used. Further, as the operation unit 34, a predetermined input operation is performed by touching or clicking an image operation unit such as various keys, switches, and buttons displayed as images on the screen formed in the display unit 35. It is possible to use a touch panel that can be used.

  A display is used as the display unit 35. Then, on various screens formed on the display unit 35, the current position of the vehicle is displayed as the vehicle position, the vehicle direction is displayed as the vehicle direction, a map, a search route, guidance information along the search route, traffic In addition to displaying information and the like, displaying the distance to the next intersection on the searched route, and the traveling direction at the next intersection, the operation of the image operation unit, operation unit 34, voice input unit 36, etc. It is possible to display guidance, operation menus, key guidance, FM multiplex broadcast programs, and the like.

  The voice input unit 36 includes a microphone (not shown) and the like, and can input necessary information by voice. Further, the voice output unit 37 includes a voice synthesizer and a speaker (not shown), and the search route, guidance information, traffic information, and the like are output from the voice output unit 37, for example, as voice synthesized by the voice synthesizer. .

  The communication unit 38 transmits various information such as current traffic information and general information transmitted from the road traffic information center to radio wave beacons via radio wave beacon devices, optical beacon devices and the like arranged along the road. A beacon receiver for receiving as an optical beacon, an FM receiver for receiving as an FM multiplex broadcast via an FM broadcast station, and the like. The traffic information includes traffic jam information, regulation information, parking lot information, traffic accident information, service area congestion status information, and the like, and general information includes news, weather forecasts, and the like. Further, the beacon receiver and the FM receiver are unitized and arranged as a VICS receiver, but can be arranged separately.

  The traffic information includes an information type that indicates the type of information, a mesh number for specifying a mesh, a road link that connects two points (for example, intersections), and a link that indicates whether it is up / down. Number, including link information representing the content of information provided corresponding to the link number. For example, when the traffic information is traffic jam information, the link information is a distance from the start point of the road link to the head of the traffic jam Congestion head data representing traffic congestion degree, congestion section, traffic jam length representing the distance from the beginning of the traffic jam to the end of the traffic jam, link required time representing the time required to travel on the road link, and the like.

  And the communication part 38 can receive various information, such as traffic information and general information other than data, such as the said map data, statistical data, and driving history data, from the said information center 51 via the network 63. .

  For this purpose, the information center 51 includes a server 53, a communication unit 57 connected to the server 53, a database (DB) 58 as an information recording unit, and the like. The server 53 controls the entire server 53. CPU 54 as a control device and an arithmetic device, RAM 55 used as a working memory when the CPU 54 performs various arithmetic processes, control programs, route search to the destination, route guidance, etc. A ROM 56 and the like in which various programs for performing are recorded. The database 58 records data similar to the various data recorded in the data recording unit 16, for example, the map data, statistical data, travel history data, and the like. Further, the information center 51 provides various information such as current traffic information and general information transmitted from the road traffic information center, and travel history data collected from a plurality of vehicles (own vehicle or other vehicles) in real time. can do.

  Further, the light distribution device 46 changes the inclination of the optical axis of the headlight and irradiates light of a low beam and an up beam.

  Further, the forward monitoring device 48 is composed of a radar device such as a millimeter wave radar or a laser radar, and irradiates the front irradiation object with a millimeter wave, a laser or the like as an irradiation medium, and a distance to the irradiation object. Is measured, the vehicle traveling ahead, that is, the preceding vehicle is monitored, the temporary stop point, the obstacle, and the like are monitored. In addition, the front monitoring device 48 detects the relative speed indicating the relative vehicle speed with respect to the preceding vehicle, the approach speed with respect to the temporary stop point, the approach speed with respect to the obstacle, etc. as the vehicle surrounding information, and the inter-vehicle distance, the inter-vehicle time, etc. Or calculate. The measurement, monitoring, detection, calculation, and the like are performed every predetermined measurement period (for example, 100 [ms]) with respect to a preceding vehicle, an obstacle, and the like included in an irradiation range such as millimeter wave radar and laser radar.

  In the present embodiment, the imaging device 49 is composed of a color CCD element, and in order to monitor the periphery of the own vehicle, the optical axis is attached obliquely downward, and in addition to the road that is running, A building or a structure on the side of the road is photographed as an object to be photographed, image data of the photographed object, that is, image data is generated and sent to the CPU 21. The CPU 21 reads the image data and performs image recognition processing on the image data, thereby recognizing each object to be photographed in the image as a recognition object. In the present embodiment, a CCD element is used as the imaging device 49, but a C-MOS element or the like can be used instead of the CCD element.

  The vehicle control system, the control unit 10, the navigation processing unit 17, the CPUs 21, 31, 54, the server 53, and the like function as a computer singly or in combination of two or more, and based on various programs, data, and the like. Perform arithmetic processing. The data recording unit 16, RAMs 22, 32, 55, ROMs 23, 33, 56, database 58, flash memories, etc. constitute a recording medium. An MPU or the like can be used instead of the CPUs 21, 31, and 54 as the arithmetic unit.

  Next, a basic operation when the vehicle control system having the above configuration is used as an intersection recognition system will be described.

  First, when the operation unit 34 is operated by the driver and the navigation device 14 is activated, a navigation initialization processing unit (not shown) of the CPU 31 performs a navigation initialization process and detects the current location of the vehicle detected by the GPS sensor 15. The vehicle direction detected by the direction sensor 18 is read, and various data are initialized. Next, the matching processing means (not shown) of the CPU 31 performs matching processing, and based on the trajectory of the read current location and the shape, arrangement, etc. of each road link constituting the road around the current location, The current location is identified by determining whether it is located on the road link.

  Subsequently, a basic information acquisition processing unit (not shown) of the CPU 31 performs a basic information acquisition process and acquires the map data by reading it from the data recording unit 16 or receiving it from the information center 51 or the like via the communication unit 38. And get. In addition, when acquiring map data from the information center 51 etc., the said basic information acquisition process means records the received map data in flash memory.

  A display processing unit (not shown) of the CPU 31 performs display processing and forms various screens on the display unit 35. For example, the map display processing means of the display processing means performs a map display process, forms a map screen on the display unit 35, displays a surrounding map on the map screen, and sets the current position as the vehicle position and the direction of the vehicle. Is displayed as the vehicle direction.

  Accordingly, the driver can drive the vehicle according to the map, the vehicle position, and the vehicle direction.

  Further, when the driver operates the operation unit 34 to input a destination, a destination setting processing unit (not shown) of the CPU 31 performs a destination setting process to set the destination. Note that the departure place can be input and set as necessary. Moreover, a predetermined point can be registered in advance, and the registered point can be set as a destination. Subsequently, when the driver operates the operation unit 34 to input a search condition, a search condition setting processing unit (not shown) of the CPU 31 performs a search condition setting process to set the search condition.

  When the destination and the search condition are set in this way, the route search processing unit (not shown) of the CPU 31 performs the route search process, reads the current location, the destination, the search condition, and the like from the data recording unit 16. Based on the current location, the destination, and the search data, a route from the departure point to the destination represented by the current location is searched based on the search condition, and route data representing the searched route is output. At this time, the route with the smallest total link cost assigned to each road link is set as the searched route.

  When a plurality of lanes are formed on the road and a traveling lane is detected, the route search processing means searches for a search route for each lane. In this case, the route data includes a lane number and the like.

  Subsequently, a guidance processing means (not shown) of the CPU 31 performs guidance processing and provides route guidance. For this purpose, the route display processing means of the guidance processing means performs route display processing, reads the route data, and displays the searched route on the map screen according to the route data. When a search route for each lane is searched, route guidance for each lane is performed at a predetermined point, for example, a guidance intersection, and a travel lane is displayed in the enlarged intersection view. If necessary, the voice output processing means of the guidance processing means performs voice output processing, and outputs a search route by voice from the voice output unit 37 to provide route guidance.

  The information center 51 can perform a route search process. In that case, the CPU 31 transmits the current location, the destination, search conditions, and the like to the information center 51. When the information center 51 receives the current location, destination, search conditions, etc., the route search processing means (not shown) of the CPU 54 performs route search processing similar to that of the CPU 31 and reads the search data from the database 58 to obtain the current location, destination. Based on the location and the search data, a route from the departure point to the destination is searched under the search conditions, and route data representing the searched route is output. Subsequently, a transmission processing unit (not shown) of the CPU 54 performs transmission processing and transmits the route data to the navigation device 14. Therefore, in the navigation device 14, when the basic information acquisition processing means receives the route data from the information center 51, the guidance processing means performs route guidance as described above.

  And when a guidance intersection exists on the searched route, when the vehicle reaches a route guidance point in front of the guidance intersection by a predetermined distance (for example, X [m]), the intersection enlarged map display processing means of the guidance processing means Performs an enlarged intersection map display process, forms an enlarged intersection map as described above in a predetermined area of the map screen, provides route guidance by the enlarged intersection map, and displays the map around the guided intersection in the enlarged intersection map. When the route guidance for the lane unit is being performed, the travel lane or the like is displayed. Further, if necessary, the voice output processing means outputs a voice such as “This is X [m] to the left” from the voice output unit 37 and performs route guidance.

  Incidentally, in the present embodiment, an image recognition system is configured by the control unit 10 and the imaging device 49F, and a forward monitoring system is configured by the control unit 10 and the forward monitoring device 48. The intersection can be recognized by the recognition system and the forward monitoring system.

  Therefore, in the control unit 10, an intersection recognition processing unit (not shown) of the CPU 21 performs an intersection recognition process to detect another vehicle (hereinafter referred to as an “intersection vehicle”) that moves in the lateral direction at the intersection in front of the host vehicle. In addition to recognizing the intersection, the distance from the own vehicle to the other vehicle is calculated.

  2 is a main flowchart showing the operation of the intersection recognition processing means in the embodiment of the present invention, FIG. 3 is a diagram showing a subroutine of image recognition processing in the present embodiment, and FIG. 4 is an image pickup apparatus in the embodiment of the present invention. FIG. 5 is a diagram showing an imaging region in the embodiment of the present invention, FIG. 6 is a diagram showing details of the mounting state of the imaging device in the embodiment of the present invention, and FIG. FIG. 8 is a first diagram illustrating detection result correction processing according to the embodiment, and FIG. 8 is a second diagram illustrating detection result correction processing according to the embodiment of the present invention.

  By the way, as the forward monitoring device 48 (FIG. 1), a radar device such as a millimeter wave radar or a laser radar can be used. In the present embodiment, a millimeter wave radar is used. In the millimeter wave radar, when a moving object moving in the front-rear direction with respect to the traveling direction of the host vehicle is detected in front, the speed of the moving object is detected by one detection due to the characteristics of the millimeter wave radar (Doppler effect). In addition to being able to measure, it is possible to predict the moving distance for each measurement period of the moving object based on the measured speed of the moving object, and to detect them as the same moving object. However, the millimeter wave radar cannot measure the speed of the moving object due to the characteristics of the millimeter wave radar with respect to the moving object moving in the lateral direction with respect to the traveling direction of the vehicle ahead. Therefore, the moving distance for each measurement cycle cannot be predicted and cannot be detected as the same moving object.

  On the other hand, by performing image recognition according to the image data captured by the imaging device 49F, it is possible to easily detect a moving object moving in the lateral direction in the front, but the imaging device 49F has a monocular configuration. Therefore, it is difficult to detect the distance from the imaging device 49F to the moving object by image recognition, and the detection accuracy is lowered.

  Therefore, in the present embodiment, by performing image recognition processing on the image data sent from the imaging device 49F for each frame at a predetermined cycle, an intersecting vehicle is detected as a vehicle candidate, and the vehicle from the own vehicle to the vehicle is detected. An approximate distance to the candidate is calculated, and then the distance from the vehicle to the vehicle candidate is measured based on the approximate distance by a millimeter wave radar.

  For this purpose, the activation determination processing means of the intersection recognition processing means performs activation determination processing to determine whether the image recognition system is activated. When the image recognition system is activated, the forward monitoring system is activated. Determine if you are. When the forward monitoring system is activated, the image recognition processing means of the intersection recognition processing means performs image recognition processing.

  That is, the day / night determination processing means of the image recognition processing means performs day / night determination processing, determines whether it is daytime or nighttime by turning on / off the illumination signal sent to the light distribution device 46, and at daytime. In some cases, the daytime image recognition processing means as the first recognition processing means of the image recognition processing means performs daytime image recognition processing as the first recognition processing, and in the case of night, the image recognition processing means. The night image recognition processing means as the second recognition processing means performs night image recognition processing as the second recognition processing.

  The daytime image recognition processing means detects a vehicle candidate in the following procedure.

  That is, the daytime image recognition processing means first receives the image data of the first frame sent from the imaging device 49F, and performs feature amount matching using the edge feature on the image area composed of the image data. Then, an object of the same color is extracted from the image area. Subsequently, the daytime image recognition processing means further performs a labeling process to extract an object having a feature similar in shape to the vehicle.

  Then, the daytime image recognition processing means calculates a similarity between the shape of the extracted object and the shape of the vehicle by giving a predetermined shape parameter to the extracted object. Subsequently, the daytime image recognition processing means extracts and detects an object having a similarity higher than a threshold value as a vehicle candidate.

  In the daytime image recognition processing algorithm, it is difficult to perform feature amount matching at night, to extract an object of the same color, or to extract an object having a feature similar in shape to a vehicle. However, since it is possible to detect white illuminants such as headlights and tail lamps, red illuminants, etc. at night, the night image recognition processing means uses the same algorithm for the night image recognition processing. A vehicle candidate is extracted and detected without extracting a colored object or extracting an object having a feature similar in shape to the vehicle.

  When the vehicle candidate is detected in this way, the distance calculation processing means of the intersection recognition processing means performs a distance calculation process to calculate the distance from the own vehicle to the vehicle candidate in the image area.

  4 to 6, 61 is a CCD element as an imaging device, f is a focal point of the imaging device 49F, Ra is a road surface, h is a height of the imaging device 49F expressed by a distance from the road surface Ra to the focal point f ( Strictly speaking, it is the height of the focal point f, but substantially represents the height of the imaging device 49F. Further, E1 is a line extending directly from the focal point f, E2 is a line indicating the lower edge of the photographing range, E3 is an optical axis of the imaging device 49F, E4 is a line extending from the focal point f toward the vehicle candidate, and E5 is A line E6 extending from the focal point f toward the vanishing point is a line indicating the upper edge of the photographing range.

  P1 is a point immediately below where the line E1 and the road surface Ra intersect, P2 is a lower edge point where the line E2 and the road surface Ra intersect, P3 is an optical axis point where the optical axis E3 and the road surface Ra intersect, and P4 is a line A vehicle candidate point at which E4 and the road surface Ra intersect, L1 is a distance from the direct point P1 to the lower edge point P2, L2 is a distance from the direct point P1 to the optical axis point P3, and La is a vehicle candidate from the direct point P1. The distance to the point P4, Lb is the distance from the lower edge point P2 to the vehicle candidate point P4, Pc is the number of pixels from the center on the image area to the vehicle candidate, and Pd is the upper edge from the center on the image area or This is the number of pixels up to the bottom edge. Since the focal length Lf from the CCD element 61 to the focal point f can be ignored with respect to the distance La, the distance La substantially represents the horizontal distance from the imaging device 49F to the vehicle candidate. .

In addition, θ1 is a vertical field angle that represents the photographing range in the vertical direction of the imaging device 49F as an angle, θ2 is a half field angle that represents a range from the optical axis E3 to the line E2, and the half field angle θ2 is
θ2 = θ1 / 2
It is. Θ3 is the depression angle of the imaging device 49F, and θ4 is the angle formed by the optical axis E3 and the line E4.

By the way, in FIG.
tan (θ3-θ2) = L1 / h
So
tan θ2 = (h · tan θ3−L1) / (h + L1 · tan θ3) (1)
become. Also,
tan (θ3 + θ4) = La / h
Therefore, the distance La is
La = h · (tan θ3 + tan θ4)
/ (1-tan θ3 · tan θ4) (2)
become.

Here, since each pixel has a unit size ρ,
tan θ2 = Pd · ρ / Lf (3)
tan θ4 = Pc · ρ / Lf
And
tan θ4 = Pc · tan θ2 / Pd (4)
become. Therefore, substituting tan θ2 of equation (1) into equation (4),
tan θ4 = Pc · {(h · tan θ3−L1)
/ (H + L1 · tan θ3)} / Pd (5)
become.

  Therefore, when tan θ4 in equation (5) is substituted into equation (2), distance La can be expressed by distance L1, number Pc, Pd, height h, and depression angle θ3. The distance L1, the height h, and the depression angle θ3 are determined by the mounting state of the imaging device 49F, and the numbers Pc and Pd are defined by the specification of the imaging device 49F, so that the distance La can be calculated.

  In FIG. 5, each direct point P1, lower edge point P2, optical axis point P3, vehicle candidate point P4, distances L1, L2, La, and Lb in the image area of the image data captured by the imaging device 49F are shown. Indicated. The straight lines passing through the optical axis point P3 and the vehicle candidate point P4 are lines indicating the distances L2-L1 and Lb from the lower end of the image area, respectively.

  In the present embodiment, tan θ2 of equation (1) is substituted into equation (4), but tan θ2 of equation (3) can be substituted. However, since the numbers Pc and Pd and the unit size ρ are defined by the specification of the imaging device 49F, the value of the expression (2) tan θ2 varies depending on the tolerance of the imaging device 49F, the positive contact with the imaging device 49F, and the like. However, the calculation accuracy of the distance La is lowered. On the other hand, tan θ2 in Expression (1) is expressed by the distance L1, the height h, and the depression angle θ3, so that the value of tan θ2 does not fluctuate and the calculation accuracy of the distance La can be increased.

  Next, the image recognition processing means receives the image data of the second frame sent from the imaging device 49F, performs the same image recognition processing on the image area composed of the image data, and detects a vehicle candidate. When a vehicle candidate is detected, the distance calculation processing means calculates a distance La from the own vehicle (imaging device 49F) shown in FIG. 5 to the vehicle candidate.

  The similarity determination processing means of the intersection recognition processing means performs similarity determination processing, and each vehicle candidate, in the present embodiment, two vehicle candidates detected by two image recognition processes. To determine if they are similar. For this purpose, the similarity determination processing means calculates the similarity Q between two vehicle candidates, and determines whether each vehicle candidate is similar according to whether the similarity Q is equal to or greater than a threshold value Qth. To do.

  When the similarity Q is equal to or greater than the threshold value Qth and the vehicle candidates are similar, the vehicle information acquisition processing means of the intersection recognition processing means performs vehicle information acquisition processing, and the vehicle of the host vehicle is used as a correction variable. The vehicle speed and rudder angle in the information are read. And the intersection vehicle information calculation processing means of the intersection recognition processing means performs intersection vehicle information calculation processing, and from the own vehicle to the vehicle candidate based on the vehicle speed, the steering angle and the switching period τ of each frame. The distance La is corrected.

  7 and 8, R1 and R2 are roads, C1 is an intersection where the roads R1 and R2 intersect, wh1 is the own vehicle, and wh2 is an intersection vehicle. Then, at the first timing when the first shooting is performed and the image data of the first frame is sent from the imaging device 49F, the own vehicle wh1 is traveling at the point p1, and the crossing vehicle wh2 is traveling at the point p2. Then, at the second timing when the next shooting is performed and the image data of the second frame is sent from the imaging device 49F, the own vehicle wh1 has moved to the point p3 and the crossing vehicle wh2 has moved to the point p4. Suppose.

In this case, the distance that the own vehicle wh1 moves between the first and second timings, that is, the own vehicle moving distance ΔL1 is the switching period of each frame represented by the time between the first and second timings. Is represented by the period τ and the vehicle speed V,
ΔL1 = V · τ
And expressed by a function f (V, τ) of the vehicle speed V and the period τ.

Further, when the distance La from the own vehicle wh1 to the crossing vehicle wh2 at the first timing is Laf, and the distance La from the own vehicle wh1 to the crossing vehicle wh2 at the second timing is Las,
Las = Laf−ΔL1
It becomes.

  By the way, when the own vehicle wh1 moves and approaches the intersection C1 between the first timing and the second timing, as shown in FIG. 8, the image in the image area becomes closer to the intersection C1. The distance in the image area between any points is similarly increased.

Therefore, when the movement amount on the image area from the position of the crossing vehicle wh2 at the first timing to the position of the crossing vehicle wh2 at the second timing is LE, and the actual movement amount is LA, the movement amount LE is It becomes longer than the movement amount LA by the ratio of the distances Laf and Las,
LE = LA ・ Las / Laf
become.

Therefore, the movement variable calculation processing means of the intersection recognition processing means performs movement variable calculation processing, calculates the movement amount LE in the image area, corrects the movement amount LE based on the distances Laf, Las, Movement amount LA as the first movement variable of wh2
LA = LE · Laf / Las
Is calculated. Note that the movement amount LA can also be calculated using the steering angle θ. Subsequently, the movement variable calculation processing means divides the movement amount LA by the period τ to thereby move the movement speed Vwh2 as the second movement variable of the crossing vehicle wh2.
Vwh2 = LA / τ
Is calculated.

  By the way, as described above, the millimeter wave radar usually detects the distance from the own vehicle wh1 to the moving object moving in the front-rear direction by a single scan using the Doppler effect, and detects the moving speed. However, the distance to the moving object that moves in the lateral direction cannot be detected. Therefore, in the present embodiment, when the moving speed calculated based on the image recognition process is given to the millimeter wave radar, the distance to the moving object moving in the lateral direction can be predicted, and the same for each period τ. It can be detected as an object. Therefore, the distance to the moving object that moves in the lateral direction can be detected by the millimeter wave radar. Further, since the millimeter wave radar is irradiated in the traveling direction of the vehicle, the distance from the own vehicle wh1 measured by the millimeter wave radar to the moving object can be set as the distance to the intersection.

  In the present embodiment, the distance acquisition processing means of the intersection recognition processing means performs distance acquisition processing, calculates the moving speed Vwh2 as a parameter of the intersection vehicle wh2 in the millimeter wave radar, and uses the parameter as the millimeter wave radar. To give. Then, the millimeter wave radar reads the parameter, and based on the parameter, measures the distance Lr from the own vehicle wh1 to the crossing vehicle wh2. Therefore, the distance acquisition processing means acquires the measured distance Lr and sets it as the distance R from the own vehicle wh1 to the intersection C1. As the parameter, in addition to the moving speed Vwh2, a spread angle from the millimeter wave irradiation axis to the front end and the rear end of the crossing vehicle wh2 can be given to the millimeter wave radar.

  Thus, the distance Lr from the own vehicle wh1 to the crossing vehicle wh2 can be measured by calculating the moving speed Vwh2 of the crossing vehicle wh2 by the image recognition processing and giving the moving speed Vwh2 to the millimeter wave radar as a parameter. it can. As a result, an accurate distance R from the own vehicle wh1 to the intersection C1 can be acquired.

  Therefore, even if there is no stop line or the like at the intersection C1, it is possible to accurately recognize the approach of the host vehicle wh1 to the intersection C1. In addition, it is necessary to detect a traveling road line along the traveling direction of the own vehicle wh1, or to detect a traveling road intersection line extending in a direction different from the traveling road line from a position where the traveling road road line is interrupted. Therefore, the intersection C1 can be detected with high accuracy, and the approach of the host vehicle wh1 to the intersection C1 can be recognized more accurately.

  Furthermore, since the distance R from the own vehicle wh1 to the intersection C1 can be accurately obtained, in the matching process in the navigation device 14, the current location is any road based on the distance R from the own vehicle wh1 to the intersection C1. It can be determined whether it is located on the link. Therefore, the current location can be specified with high accuracy.

Next, the flowchart of FIG. 2 will be described.
Step S1: It is determined whether the image recognition system is activated. If the image recognition system is activated, the process proceeds to step S2, and if not activated, the process ends.
Step S2: It is determined whether the forward monitoring system is activated. If the forward monitoring system is activated, the process proceeds to step S3, and if not activated, the process is terminated.
Step S3 Image recognition processing is performed.
Step S4: Determine whether a vehicle candidate is detected. If a vehicle candidate is detected, the process proceeds to step S5, and if not detected, the process ends.
Step S5: Calculate the distance to the vehicle candidate.
Step S6: Image recognition processing is performed.
Step S7: Determine whether a vehicle candidate has been detected. If a vehicle candidate is detected, the process proceeds to step S8, and if not detected, the process ends.
Step S8: A distance La to the vehicle candidate is calculated.
Step S9: The similarity Q is calculated.
Step S10: It is determined whether the similarity Q is equal to or greater than a threshold value Qth. If the similarity Q is equal to or greater than the threshold value Qth, the process proceeds to step S11, and if not, the process ends.
Step S11: Vehicle information is acquired.
Step S12 The moving speed Vwh2 of the intersecting vehicle wh2 is calculated.
Step S13: The moving speed Vwh2 of the crossing vehicle wh2 is used as a parameter.
Step S14: The distance Lr is measured by the millimeter wave radar.
Step S15: The measured distance Lr is set as the distance R to the intersection C1, and the process is terminated.

Next, the flowchart of FIG. 3 will be described.
Step S3-1: It is determined whether it is noon. If it is noon, the process proceeds to step S3-2. If not, the process proceeds to step S3-3.
Step S3-2: Perform daytime image recognition processing and return.
Step S3-3 Perform night image recognition processing and return.

  In the present embodiment, image recognition processing is performed on the image data of the first and second frames in order to calculate the moving speed Vwh2, but in other embodiments, three or more Image recognition processing can be performed on the frame image data.

  In the present embodiment, the moving speed Vwh2 is calculated by the image recognition process, and the moving speed Vwh2 is used as a parameter of the millimeter wave radar, thereby calculating the distance Lr from the own vehicle wh1 to the crossing vehicle wh2. However, it is recognized by the image recognition process that the crossing vehicle wh2 has entered the irradiation range of the millimeter wave radar of the own vehicle wh1, and from the own vehicle wh1 of the millimeter wave radar to the crossing vehicle wh2 at that time It is also possible to calculate the distance Lr and acquire the distance R from the own vehicle wh1 to the intersection C1.

  In addition, this invention is not limited to the said embodiment, It can change variously based on the meaning of this invention, and does not exclude them from the scope of the present invention.

It is a figure which shows the vehicle control system in embodiment of this invention. It is a main flowchart which shows operation | movement of the intersection recognition process means in embodiment of this invention. It is a figure which shows the subroutine of the image recognition process in this Embodiment. It is a figure which shows the attachment state of the imaging device in embodiment of this invention. It is a figure which shows the imaging area in embodiment of this invention. It is a figure which shows the detail of the attachment state of the imaging device in embodiment of this invention. It is a 1st figure explaining the detection result correction | amendment process in embodiment of this invention. It is a 2nd figure explaining the detection result correction | amendment process in embodiment of this invention.

Explanation of symbols

10 Control Unit 14 Navigation Device 21 CPU
48 Forward monitoring device 49F Imaging device 51 Information center 63 Network C1 Intersection wh2 Crossing vehicle

Claims (3)

An imaging device attached to a predetermined portion of the vehicle and photographing the front of the vehicle;
A radar device that irradiates the irradiation medium and measures the distance to the irradiated object;
Image recognition processing means for performing image recognition processing on the image data of a plurality of frames sent from the imaging device, and detecting an intersection vehicle moving at the intersection as a vehicle candidate;
A movement variable calculation that calculates the actual movement amount of the detected vehicle candidate between the frames and calculates the movement speed of the vehicle candidate based on the actual movement amount and the switching period of each frame. Processing means;
An intersection recognition system comprising: a distance acquisition processing unit that applies the moving speed as a parameter to the radar apparatus and acquires a distance to a vehicle candidate measured by the radar apparatus. Based on the image data of each frame, and having a distance calculation processing means for calculating the distance from the vehicle to the vehicle candidate when shooting is performed on the image data of each frame,
The movement variable calculation processing means calculates the movement amount of the vehicle candidate on the image area between the frames based on the image data of each frame, and also calculates the vehicle candidate on the image area between the frames. The actual amount of movement of the vehicle candidate between frames is calculated based on the amount of movement and the distance from the vehicle to the vehicle candidate when the image data of each frame is captured. Intersection recognition system. The front of the vehicles taken by an imaging device attached to a predetermined portion of the vehicle,
Irradiate the irradiation medium with a radar device and measure the distance to the irradiated object.
Image recognition processing is performed on the image data of a plurality of frames sent from the imaging device, and an intersection vehicle moving at the intersection is detected as a vehicle candidate.
Calculate the actual movement amount of the detected vehicle candidate between the frames, and calculate the moving speed of the vehicle candidate based on the actual movement amount and the switching period of each frame ,
An intersection recognition method comprising: providing the moving speed as a parameter to the radar apparatus, and acquiring a distance to a vehicle candidate measured by the radar apparatus.

Images