WO2022249504A1 - Vehicle information processing device - Google Patents

Vehicle information processing device Download PDF

Info

Publication number
WO2022249504A1
WO2022249504A1 PCT/JP2021/033632 JP2021033632W WO2022249504A1 WO 2022249504 A1 WO2022249504 A1 WO 2022249504A1 JP 2021033632 W JP2021033632 W JP 2021033632W WO 2022249504 A1 WO2022249504 A1 WO 2022249504A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
mirror
image
vehicle
curvature
Prior art date
Application number
PCT/JP2021/033632
Other languages
French (fr)
Japanese (ja)
Inventor
政彦 渡辺
伸 坂本
薇▲ふぇん▼ 謝
Original Assignee
株式会社Ｎｔｔデータオートモビリジェンス研究所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｎｔｔデータオートモビリジェンス研究所 filed Critical 株式会社Ｎｔｔデータオートモビリジェンス研究所
Publication of WO2022249504A1 publication Critical patent/WO2022249504A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems

Definitions

  • the present invention relates to a vehicle information processing apparatus that generates driving assistance information that assists the driving of a vehicle traveling on a road with curved mirrors.
  • Patent Document 1 an information processing device described in Patent Document 1 is known.
  • a curved mirror reflected in the image is detected based on image information representing an image obtained by a camera device (imaging device) mounted on the vehicle, and an object (bicycle, walking, etc.) reflected in the curved mirror is detected. person, etc.) is detected. Then, the position of the detected object in the curved mirror (above the reference, below the reference, behind the reflected road, near the reflected road, etc.) is detected, and the position of the detected object is determined. A degree of danger to the vehicle is estimated. Driving assistance information is generated and output based on the estimated degree of danger.
  • the size of the object reflected in the curved mirror is calculated based on the image information, and the degree of danger to the vehicle is estimated based on the size of the object reflected in the curved mirror.
  • Driving assistance information is generated and output based on the estimated degree of danger.
  • the degree of danger corresponding to the state of an object reflected in a curved mirror at an intersection or the like can be determined. Therefore, the degree of danger can be accurately determined from the image of the deformed object reflected on the curved mirror surface of the curved mirror, and safe driving of the vehicle can be supported in locations with poor visibility such as intersections.
  • the degree of danger is estimated based on the position of the image of the object reflected on the curved mirror on the curved mirror, and driving support information is generated based on the estimated degree of danger.
  • the driving support information based on the estimated degree of danger is effective in calling attention to the driver driving the vehicle, it is not based on the actual position of the object reflected in the curved mirror. do not have.
  • the driving support information is information that should be used for vehicle driving control based on the actual positional relationship (actual situation reflected in the curved mirror) with objects (vehicles, bicycles, pedestrians, etc.) reflected in the curved mirror. It's hard to say enough.
  • the present invention has been made in view of such circumstances, and provides a vehicle information processing apparatus capable of providing driving support information based on the actual situation reflected in the curved mirror.
  • a vehicle information processing apparatus is for assisting driving of a vehicle based on mirror image information representing an image including an image of a curved mirror installed on a road, which is obtained by a camera device mounted on the vehicle.
  • a vehicle information processing device for generating driving support information, wherein the curvature of the mirror represents the degree of curvature of the mirror surface of the curved mirror based on the information of the image of the curved mirror included in the image represented by the mirror image information.
  • a first information acquisition unit that acquires information
  • a second information acquisition unit that acquires mirror position information representing the position of the curved mirror
  • a third information acquisition unit that acquires mirror orientation information indicating the direction in which the mirror surface of the curved mirror faces
  • a position of the object of interest based on information on the image of the object of interest detected in the image represented by the mirror image information, the information on the curvature of the mirror, the information on the position of the mirror, and the information on the orientation of the mirror.
  • an object position detection unit that generates object position information representing: and generates the driving support information based on the object position information obtained by the object position detection unit.
  • the mirror curvature information representing the degree of curvature of the curved mirror is acquired based on the information of the image of the curved mirror included in the image represented by the mirror image information obtained by the camera device mounted on the vehicle.
  • mirror position information representing the position of the curved mirror is acquired.
  • mirror orientation information representing the direction in which the mirror surface of the curved mirror faces the vehicle is acquired.
  • the image of the object of interest reflected on the curved mirror included in the image represented by the mirror image information is detected, and along with the information of the detected image of the object of interest, the mirror curvature obtained as described above is detected.
  • Object position information representing the position of the target object is generated based on the information, the mirror position information, and the mirror orientation information, and driving support information is generated based on the object position information.
  • the first information acquisition unit obtains table information in which external characteristic information representing the characteristic of the external shape of each of a plurality of types of curved mirrors and the degree of curvature corresponding thereto are determined.
  • a storage unit for storing, a mirror external shape feature acquiring unit for acquiring external shape feature information representing the characteristics of the external shape of the curved mirror from information on the image of the curved mirror included in the image represented by the mirror image information; a curvature determining unit configured to determine the degree of curvature corresponding to the external feature information acquired by the mirror external feature acquiring unit in the table information as the degree of curvature to be represented by the mirror curvature information.
  • the external feature information representing the characteristic of the external shape of the curved mirror is acquired from the information of the image of the curved mirror included in the image represented by the mirror image information, and the acquired information is stored in the table information.
  • the degree of curvature corresponding to the external feature information is determined as the degree of curvature to be represented by the mirror curvature information, and the mirror curvature information representing the degree of curvature is acquired.
  • the second information acquisition section is represented by a distance acquisition section that acquires mirror distance information representing a distance from the vehicle to the curved mirror, and the mirror image information.
  • a mirror position detection unit that generates the mirror position information based on information on the image of the curved mirror included in the image and the mirror distance information acquired by the distance acquisition unit.
  • the mirror distance information representing the distance from the vehicle to the curved mirror is acquired, and based on the image information of the curved mirror included in the image represented by the mirror image information and the mirror distance information. mirror position information representing the position of the curved mirror is generated.
  • the third information acquiring section acquires the above information based on the image information of the curved mirror and the mirror curvature information included in the image represented by the mirror image information. and a mirror normal calculation unit that generates the normal direction of the center point of the mirror surface of the curved mirror as the mirror orientation information.
  • the normal line of the center point of the mirror surface of the curved mirror is calculated based on the information of the curved mirror image included in the image represented by the mirror image information and the curved mirror curvature information (degree of curvature).
  • Direction is generated as mirror orientation information.
  • the object position detection unit detects the image based on the information of the image of the object of interest reflected on the curved mirror in the image represented by the mirror image information and the mirror curvature information.
  • a distance calculation unit that calculates a distance between the curved mirror and the target object as a real object; information on an image of the target object reflected on the curved mirror in the image represented by the mirror image information; the mirror position information; and a reflection direction calculation unit for calculating, based on the mirror orientation information, a mirror reflection direction representing a direction toward the object of interest as a real object from the image of the object of interest reflected on the curved mirror in the line of sight from the vehicle. and generating the object position information based on the distance between the curved mirror and the object of interest as a real object obtained by the distance calculation unit and the mirror reflection direction.
  • the curved mirror and the target object as a real object are determined based on the information of the image of the target object reflected on the curved mirror in the image represented by the mirror image information and the mirror curvature information (degree of curvature) of the curved mirror. is calculated, and based on the information of the image of the object of interest reflected on the curved mirror in the image represented by the mirror image information, the mirror position information, and the mirror tilt information, the line of sight from the vehicle is reflected on the curved mirror A mirror reflection direction representing a direction toward the object of interest as a real object is calculated from the image of the object of interest. Then, object position information is generated based on the calculated distance and the mirror reflection direction.
  • the a blind spot area detection unit that generates blind area information representing a blind area for the vehicle that is driven by the vehicle, and generates the driving support information based on the blind area information obtained by the blind area detection unit.
  • the blind area information representing the blind area for the vehicle on the target road is generated based on the map information together with the mirror curvature information, the mirror position information, and the mirror direction information acquired as described above. . Then, driving support information is generated based on this blind spot information.
  • a vehicle information processing apparatus is for assisting driving of a vehicle based on mirror image information representing an image including an image of a curved mirror installed on a road, which is obtained by a camera device mounted on the vehicle.
  • a vehicle information processing device for generating driving support information, wherein the curvature of the mirror represents the degree of curvature of the mirror surface of the curved mirror based on the information of the image of the curved mirror included in the image represented by the mirror image information.
  • a first information acquisition unit that acquires information
  • a second information acquisition unit that acquires mirror position information representing the position of the curved mirror
  • a third information acquisition unit that acquires mirror orientation information representing the direction in which the mirror surface of the mirror faces; the mirror curvature information, the mirror position information, the mirror orientation information, the vehicle position information, and road information representing a road on which the vehicle travels
  • a blind spot area detection unit that generates blind spot area information representing a blind spot area for the vehicle formed on the road of interest based on map information including The driving support information is generated based on.
  • the mirror curvature information representing the degree of curvature of the curved mirror is acquired based on the information of the image of the curved mirror included in the image represented by the mirror image information obtained by the camera device mounted on the vehicle.
  • mirror position information representing the position of the curved mirror is acquired.
  • mirror orientation information representing the direction in which the mirror surface of the curved mirror faces is acquired.
  • blind area information representing a blind area for the vehicle on the road of interest is generated, and this blind area information is generated.
  • Driving support information is generated based on.
  • the blind spot area detection unit generates direct visibility limit line information representing a limit line of an area of the road of interest that is directly visible from the vehicle based on the map information.
  • a mirror visibility limit detection unit that generates mirror visibility limit information, and generates the blind area information based on the direct visibility limit information and the mirror visibility limit information.
  • the direct visibility limit line information representing the limit line of the area of the road of interest that is directly visible from the vehicle is generated based on the map information, and the map information, the mirror curvature information (curvature), and the mirror position information are generated.
  • map information, the mirror curvature information (curvature), and the mirror position information are generated.
  • mirror orientation information mirror visibility limit line information representing the limit line of the area visible from the vehicle through the curved mirror on the road of interest is generated.
  • Blind spot area information representing a blind spot area for the vehicle is generated based on the direct visibility limit information and the mirror visibility limit information.
  • the degree of curvature of the mirror surface of the curved mirror object position information representing the position of the object of interest is generated based on mirror curvature information representing the position of the curved mirror, mirror position information representing the position of the curved mirror, and mirror orientation information representing the direction in which the mirror surface of the curved mirror faces. Therefore, based on the object position information, it is possible to provide driving support information based on the actual position (actual situation) of the target object (object) reflected on the curved mirror.
  • mirror curvature information representing the degree of curvature of the mirror surface of the curve mirror
  • mirror position information representing the position of the curve mirror
  • mirror orientation representing the direction of the mirror surface of the curve mirror
  • FIG. 1A is a diagram showing an example of the relationship between a vehicle and a moving object (bicycle) at an intersection.
  • FIG. 1B is a diagram showing an example of a curved mirror (mirror body) in which a moving object (bicycle) is reflected.
  • FIG. 2 is a block diagram showing the configuration of the vehicle information processing apparatus according to the first embodiment of the present invention.
  • 3A is a flow chart showing the flow (1) of processing in an information processing unit in the vehicle information processing apparatus shown in FIG. 2.
  • FIG. 3B is a flow chart showing a flow (part 2) of processing in an information processing unit in the vehicle information processing apparatus shown in FIG. 2.
  • FIG. 4 is a diagram showing the relationship between an image Ic obtained by photographing by a camera device and the photographing range SA of the camera device.
  • FIG. 5 is a diagram showing the relationship between the features (diameter, rectangular shape) of the outer shape of the curved mirror and the radius of curvature of the curved mirror.
  • FIG. 6 is a diagram showing a mirror surface of a curved mirror and a normal line passing through the center point of the mirror surface (mirror plane).
  • FIG. 7 is a diagram showing the relationship between how the mirror surface of a curved mirror appears as a mirror plane and the normal line passing through the center of the mirror plane.
  • FIG. 8 is a diagram showing the relationship between an image Ic (including a curved mirror on which a moving object is captured) captured by a camera device and a shooting range SA of the camera device (including a curved mirror on which a moving object is captured).
  • FIG. 9 is a diagram showing a positional relationship between a camera device, a curved mirror reflecting a moving object, and a moving object (actual object).
  • FIG. 10 is a diagram showing an example of a blind spot area of a vehicle heading for an intersection.
  • FIG. 11A is a flowchart showing the flow (part 1) of processing in an information processing unit in the vehicle information processing apparatus according to the second embodiment of the present invention.
  • FIG. 11B is a flow chart showing the flow (part 2) of processing in the information processing unit in the vehicle information processing apparatus according to the second embodiment of the present invention.
  • FIG. 12 is a diagram showing an example of the relationship between the position of the vehicle with respect to the intersection and the blind spot area.
  • FIG. 1A assume the situation shown in FIG. 1A.
  • a road R1 and a road R2 intersect at an intersection X.
  • FIG. A curved mirror 50 is installed at the corner of the intersection X where the vehicle 100 on the road R1 can see the road R2 side in order to ensure the visibility of the vehicle 100 traveling on the road R1 toward the intersection X on the road R2 side.
  • the curved mirror 50 has a structure in which a mirror body 51 having a curved mirror surface is fixed to the tip of a post 52 erected at the corner of the intersection X.
  • FIG. 1A a road R1 and a road R2 intersect at an intersection X.
  • FIG. A curved mirror 50 is installed at the corner of the intersection X where the vehicle 100 on the road R1 can see the road R2 side in order to ensure the visibility of the vehicle 100 traveling on the road R1 toward the intersection X on the road R2 side.
  • the curved mirror 50 has a structure in which a mirror body 51 having a curved
  • the curved mirror 50 (mirror The image Io(t1) (see FIG. 1B) of the bicycle 200(t1) reflected on the mirror surface of the main body 51) can be visually recognized. Further, from the vehicle 100 (t2) at time t2 when the vehicle 100 (t2) approaches the intersection X, the image Io (t2) (see FIG. 1B) of the bicycle 200 (t2) reflected on the mirror surface of the curved mirror 50 (mirror body 51) can be visually recognized. can be done.
  • FIG. 2 An in-vehicle system that is mounted in a vehicle 100 and includes the vehicle information processing device 10 according to the first embodiment of the present invention is configured as shown in FIG. 2, this in-vehicle system includes a vehicle information processing device 10, a camera device 20 for capturing an image of the front of the vehicle 100, and similarly to the camera device 20, captures an image of the front of the vehicle 100 and outputs image information including depth information. a stereo camera 21, a vehicle speed sensor 22 that outputs a wheel speed pulse according to the rotation of the wheels of the vehicle 100, a GPS unit 23 that receives signals from a plurality of positioning satellites and outputs a detection signal according to the position of the vehicle 100. , and a navigation device 30 containing map information.
  • a vehicle information processing device 10 for capturing an image of the front of the vehicle 100, and similarly to the camera device 20, captures an image of the front of the vehicle 100 and outputs image information including depth information.
  • a stereo camera 21 a vehicle speed sensor 22 that outputs a wheel speed pulse according
  • the vehicle information processing device 10 has an information processing unit 11 including a CPU, an image processing section 12 , a storage section 13 and an output processing section 14 .
  • the image processing unit 12 processes image information from the camera device 20 and the stereo camera 21 and provides image information in a predetermined format to the information processing unit 11 .
  • the information processing unit 11 executes processing for generating driving support information based on image information from the image processing section 12 according to a program.
  • the storage unit 13 stores various programs to be executed by the information processing unit 11, various information used in the information processing unit 11 such as table information described later, information generated by processing in the information processing unit 11, and the like.
  • the output processing unit 14 outputs the generated driving support information to an information output unit (display device, voice output device, etc.), steering in the vehicle 100 according to the type of the driving support information. system, accelerator system, and braking system.
  • the vehicle information processing device 10 In a situation where a vehicle 100 traveling on a road R1 approaches an intersection X and a bicycle traveling on a road R2 (hereinafter referred to as a moving object) 200 (object of interest) approaches the intersection X, the vehicle information processing device 10 The information processing unit 11 in generates object position information representing the actual position of the moving object 200 (object of interest) reflected on the mirror surface of the curved mirror 50 (mirror body 51) (object position detection section). This processing is executed according to the procedure shown in FIGS. 3A and 3B. Then, based on the obtained object position information, the information processing unit 11 moves the moving object 200 (target object) along the road R2 that intersects the road R1 toward the intersection X approached by the vehicle 100 traveling on the road R1. Generate driving assistance information in moving situations.
  • the information processing unit 11 acquires image information from the camera device 20 through the image processing unit 12, and based on the signal from the GPS unit 23, the position of the vehicle 100 (position on the map). Position information (for example, latitude and longitude) is acquired (S11). The acquired image information and vehicle position information are stored in the storage unit 13 . The information processing unit 11 determines whether the image represented by the acquired image information includes the image of the curved mirror 50 (S12). For example, whether the image represented by the acquired image information includes the image of the curved mirror 50 according to the method described in Patent Document 1 or the method according to the image recognition technology and machine learning algorithm.
  • the information processing unit 11 acquires image information and vehicle position information from the camera device 20 (S11) and stores them until image information representing an image including the image of the curved mirror 50 is acquired (YES in S12). Storing (updating) in the unit 13 is repeatedly executed.
  • the information processing unit 11 calculates the mirror position representing the position of the curved mirror 50.
  • Information is acquired (S13: second information acquisition unit). Specifically, the information processing unit 11 uses the three-dimensional image information (including depth information) obtained from the stereo camera 21 through the image processing unit 12 to determine the distance from the vehicle 100 to the curved mirror 50 . Acquire distance information (distance acquisition unit). Further, the information processing unit 11 uses the optical parameters (focal length, magnification, angle of view, etc.) of the camera device 20 to obtain a curve represented by mirror image information obtained by photographing, as shown in FIG.
  • the information processing unit 11 stores the two-dimensional positional relationship of objects in the photographing area SA obtained from the mirror image information, and mirror distance information (depth information) representing the distance from the vehicle 100 to the curved mirror 50.
  • the mirror position information representing the position of the curved mirror 50 with reference to the position of the vehicle 100 (hereinafter referred to as the host vehicle coordinate system) is calculated (mirror position detector).
  • the mirror position information acquired in this manner is stored in the storage unit 13 .
  • the location of the curved mirror 50 installed at the intersection X to which the vehicle 50 is heading is determined based on the map information. You may make it acquire the mirror positional information showing a position.
  • information on the installation position of the curved mirror 50 is transmitted by a beacon, and the vehicle 100 approaching the intersection X receives the beacon, thereby determining the position of the curved mirror 50. It is also possible to acquire mirror position information representing
  • the information processing unit 11 calculates the mirror surface of the mirror main body 51 (curved mirror 50) based on the information of the image Ic50 of the curved mirror 50 included in the image Ic (see FIG. 4) represented by the acquired mirror image information.
  • the information processing unit 11 also acquires mirror orientation information representing the direction in which the mirror surface of the mirror body 51 (the curved mirror 50) faces the vehicle 100 based on the acquired mirror curvature information and the information on the image Ic50 of the curved mirror 50.
  • S14 third information acquisition unit.
  • the acquired mirror curvature information and mirror orientation information are stored in the storage unit 13 .
  • mirror curvature information is obtained as follows.
  • the external shape characteristic information (diameter ⁇ of round shape, vertical and horizontal size of square shape) representing the characteristics of the external shape of each of the five types of curved mirrors shown in FIG.
  • Predetermined table information (see FIG. 5) is stored in the storage unit 13 .
  • Any curve mirror actually installed on the road is represented by the mirror image information on the premise that it has one of the five types of external shapes shown in the table information shown in FIG.
  • the degree of curvature (mirror curvature information) of the actual curved mirror 50 (mirror body 51) is determined based on the information of the image Ic50 of the curved mirror 50 included in the image Ic. For example, from the information (see FIG.
  • the optical parameters (focal length, magnification, angle of view, etc.) of the camera device 20 Using the distance from the vehicle 100 to the curved mirror 50, the diameter (outer shape) of the mirror main body 51 of the actual curved mirror 50 is estimated and calculated (mirror outer shape feature acquisition unit).
  • the diameter corresponding to the diameter (outer shape) obtained by the estimation calculation specifically, the diameter closest to the diameter obtained by the estimation calculation is determined as the mirror curvature information (curvature determination unit).
  • the curve installed at the intersection X to which the vehicle 50 is headed is determined based on the map information.
  • Mirror curvature information about the mirror 50 may be obtained.
  • mirror curvature information representing the degree of curvature of the mirror surface of the curved mirror 50 is transmitted by a beacon together with information on the installation position of the curved mirror 50, and a vehicle approaching the intersection X is transmitted.
  • mirror curvature information (degree of curvature) of the curved mirror 50 can also be obtained.
  • the mirror orientation information representing the orientation of the mirror surface of the mirror body 51 (curved mirror 50) with respect to the vehicle 100 (optical axis direction of the camera device 20) (combined direction of the rotation direction on the vertical plane and the rotation direction on the horizontal plane) is It is obtained as follows.
  • the mirror surface 51a is the same as that of the mirror body 51, as shown in FIG. It can be a partial portion (convex portion) of the spherical surface 500 with a specular curvature radius R (see FIG. 5) corresponding to the diameter ⁇ .
  • the shape of the mirror surface portion of the image Ic50 of the curved mirror 50 (mirror body 51) included in the two-dimensional image Ic (see FIG. 4) represented by the mirror image information is a planar circular or elliptical shape. is.
  • this circular or elliptical plane is defined as a mirror plane (mirror plane 51b in FIG. 6).
  • the mirror plane 51b is defined by the positional relationship between the vehicle 100 (camera device 20) and the curved mirror 50 and the inclination of the mirror body 51 itself (rotation angle on the vertical plane: depression angle: for example, 5° to 10°).
  • the mirror plane 51b(0) obtained from the mirror image information (information on the image Ic of the curved mirror 50) is circular. If the vehicle 100 (camera device 20) and the curved mirror 50 face each other obliquely (see, for example, FIG. 1), and if the mirror body 51 itself is tilted, the mirror image information (see Ic in FIG. 4) indicates that , elliptical mirror planes 51b(1), 51b(2), 51b( 3), 51b(4) are obtained.
  • the center point (the point P in FIG. 6, the 7) are detected. Then, the normal vector n (n0 to n4) of the mirror plane 51b (51b(0) to 51b(4)) at the center point P (P0 to P4) is determined (mirror normal calculating section).
  • the normal vector n determined on the image Ic (image Ic50) in this way is obtained using the optical parameters (focal length, magnification, angle of view, etc.) of the camera device 20 and the distance from the vehicle 100 to the curved mirror 50. is expressed in the own vehicle coordinate system.
  • the normal vector n passing through the center point of the mirror plane 51b expressed in the host vehicle coordinate system corresponds to the normal line passing through the center point of the mirror surface 51a, which is a part of the spherical surface, and the normal vector n is defined as mirror orientation information representing the direction in which the mirror surface 51a of the curved mirror 50 (mirror body 51) faces.
  • the information processing unit 11 calculates the image Ic ( 4) includes an image of a moving object (object of interest) such as a bicycle, a car (another vehicle), or a person reflected in the curved mirror 50. (S15). For example, from the image Ic50 of the curved mirror 50 included in the image Ic represented by the mirror image information, the image of the moving object reflected on the curved mirror 50 is extracted by a known process for extracting moving object candidates and a technique according to a machine learning algorithm. parts can be detected.
  • the information processing unit 11 performs the above-described processes (S11, S12, S13, S14) until the mirror image information representing the image Ic including the image Ic50 of the curved mirror 50 reflecting the moving object (object of interest) is obtained. Execute repeatedly. During this time, the vehicle position information, mirror image information, mirror position information (vehicle coordinate system), and mirror direction information (vehicle coordinate system) stored in the storage unit 13 are sequentially updated as the vehicle 100 moves.
  • the information processing unit 11 determines that the image Ic50 of the curved mirror 50 included in the image Ic represented by the mirror image information includes the image of the moving object reflected on the curved mirror 50 (YES in S15 : object image detection unit), hereinafter, information on the image of the moving object detected in the image Ic represented by the mirror image information, mirror curvature information and mirror position information stored in the storage unit 13 at this time , and a process of generating object position information representing the position of the moving object based on the mirror direction information (object position detection unit). This process is performed as follows (S16-S24).
  • the information processing unit 11 calculates the size (e.g., height m) of the image portion Ic200 of the moving object 200 extracted from the image Ic represented by the mirror image information obtained as shown in FIG.
  • the position of the portion Ic200 within the mirror surface (in-camera coordinate system) in the image Ic50 of the curved mirror 50 is calculated (S16). Then, the information processing unit 11 determines the size (height m) of the image portion Ic200 of the moving object 200 in the image Ic and the position of the image portion Ic200 within the mirror surface in the image Ic50 of the curved mirror 50.
  • the curved mirror 50 (mirror surface of the mirror body 51) within the imaging area SA of the camera device 20
  • the size e.g., height h
  • the position of the image Io of the moving object 200 within the curved mirror 50 (mirror surface of the mirror body 51) (own vehicle coordinate system) is estimated and calculated (S17).
  • the information processing unit 11 proceeds to the procedure shown in FIG. Based on the position in , the size (height H) of the moving object 200 as a real object is obtained, and the enlargement ratio (h/H) by the curved mirror 50 is calculated (S18).
  • images Io and the size (height h) are machine-learned in advance, and the relationship is stored in the storage unit 13 as table information.
  • the table information it corresponds to the type (curvature) of the curved mirror 50, and the position in the mirror surface and the size (height h) of the image Io of the moving object 200 obtained by the estimation calculation (S17).
  • the actual size (height H) of the moving object 200 can be acquired.
  • the information processing unit 11 based on the position in the mirror surface of the curved mirror 50 (mirror body 51) of the image Io of the moving object 200 and the mirror position information, the camera device 20 (vehicle 100) and the curved mirror shown in FIG. An incident ray IR connecting the image Io of the moving object 200 on 50 (mirror body 51) is calculated (S19). Further, the information processing unit 11 calculates the normal line np of the mirror surface of the curved mirror 50 (mirror body 51) at the position of the image Io of the moving object 200 (S20). This normal np can be calculated as follows.
  • the center O of the spherical surface 5001 including the mirror surface 51a is obtained from the normal vector n passing through the center point of the mirror surface 51a (mirror plane 51b: see FIGS. 6 and 7) of the curved mirror 50. Then, a straight line extending from the center O and passing through the position within the mirror surface of the image Io of the moving object 200 is calculated as the normal line np.
  • the information processing unit 11 uses the magnification (h/H: magnification) of the mirror surface of the curved mirror 50 (mirror body 51) and the mirror curvature information (curvature radius R: corresponding to the focal length f), According to the law of geometrical optics, the distance D between the position of the image Io of the moving object 200 on the mirror surface of the curved mirror 50 (mirror body 51) and the actual moving object 200 is estimated and calculated (S21: distance calculation unit). . Further, as shown in FIG.
  • the information processing unit 11 determines whether or not a predetermined termination condition (for example, stop driving, driving support mode OFF, etc.) is satisfied (S25), and in a situation where the termination condition is not satisfied (NO in S25),
  • a predetermined termination condition for example, stop driving, driving support mode OFF, etc.
  • the above-described processes (S11 to S24: FIGS. 3A and 3B) are repeatedly executed. During that time, every time the vehicle 100 passes through a place where a curved mirror is installed, such as an intersection X or a curved road, object position information representing the position of the moving object (object of interest) reflected in the curved mirror obtained by the above-described processing is generated. be done.
  • driving support information for the vehicle 100 is generated.
  • the generated driving support information is output from the output processing unit 14 according to the type of driving support information under the control of the information processing unit 11 that recognizes the traveling speed of the vehicle 100 based on the signal from the vehicle speed sensor 22.
  • unit display, audio output device, etc.
  • ECU for controlling various actuators of the steering system, the accelerator system, and the braking system in the vehicle 100, and the like. For example, the following driving assistance information can be generated.
  • the object position information representing the position in the own vehicle coordinate system can be converted into position information representing the position in the absolute coordinate system (latitude and longitude) using the own vehicle position information (see S11).
  • Display information for displaying the position of the moving object 200 (object of interest) on the display together with the road map can be generated as the driving support information using the position information.
  • the steering system, accelerator system, and A control signal to be supplied to an ECU that controls various actuators of the braking system can be generated as driving support information.
  • the image Io of the moving object 200 reflected on the curved mirror 50 (mirror body 51) detected in the mirror image information representing the image Ic including the image Ic50 of the curved mirror 50 is detected.
  • the information of the actual object based on mirror curvature information representing the degree of curvature of the mirror surface of the curve mirror 50, mirror position information representing the position of the curve mirror 50, and mirror orientation information representing the direction in which the mirror surface of the curve mirror 50 faces.
  • Object position information representing the position of the moving object 200 is generated. Therefore, based on the object position information, driving support information based on the actual position (actual situation) of the moving object 200 reflected in the curved mirror 50 can be provided.
  • the method for acquiring the mirror curvature information, the mirror position information, and the mirror orientation information is not limited to the processing described above, and other methods may be used.
  • a vehicle information processing apparatus is configured as the vehicle information processing apparatus 10 in the in-vehicle system shown in FIG. 2, as in the case of the first embodiment.
  • This vehicle information processing apparatus 10 is characterized by providing driving support information based on a blind spot area on a road (such as an intersection X) on which a curved mirror 50 is installed.
  • a curved mirror 50 (mirror body 51) is installed at the corner of the intersection X where the vehicle 100 can see the road R2 side of the vehicle 100 in order to secure the visibility of the vehicle 100 on the road R2 side.
  • the information processing unit 11 of the vehicle information processing device 10 in the in-vehicle system mounted on the vehicle 100 generates blind area information representing the blind area with respect to the vehicle 100 (blind area detection unit). This processing is executed according to the procedure shown in FIGS. 11A and 11B.
  • the information processing unit 11 acquires image information from the camera device 20 via the image processing unit 12, as in the case of the first embodiment (see FIG. 3A), and also acquires the signal from the GPS unit 23. (S31).
  • the acquired image information and vehicle position information are stored in the storage unit 13 .
  • the information processing unit 11 repeatedly acquires the image information and the vehicle position information until the image information representing the image including the image of the curved mirror 50 is acquired (S32). 13 to update the image information and the own vehicle position information.
  • information processing unit 11 obtains mirror position information representing the position of curved mirror 50 according to the same method as in the first embodiment.
  • S33 second information acquisition unit.
  • the information processing unit 11 acquires (third information acquisition unit) mirror orientation information representing the direction in which the mirror surface of the curved mirror 50 (mirror body 51) faces, and also acquires the degree of curvature of the mirror surface of the curved mirror 50 (mirror body 51).
  • first information acquisition unit S34
  • the mirror position information, mirror orientation information, and mirror curvature information are stored in the storage unit 13 .
  • the information processing unit 11 proceeds to the procedure shown in FIG. 11B.
  • the information processing unit 11 calculates the position of the obstacle point A at the front left corner of the intersection X (S35). Specifically, similarly to the method of calculating the position of the curved mirror 50 in the first embodiment, the optical parameters (focus distance, magnification, angle of view, etc.), the two-dimensional positional relationship of objects (including the obstacle point A) within the imaging area SA corresponding to the image Ic is estimated and calculated. Obstacle point A from vehicle 100 obtained from the two-dimensional positional relationship of objects (obstacle point A) within photographing area SA and three-dimensional image information (including depth information) from stereo camera 21 . The position (position information) of the obstacle point A with respect to the position of the vehicle 100 is calculated based on the distance information (depth information).
  • the position of each corner at the intersection X (position (latitude and longitude) on the map) is embedded in the map information, the position of the left corner before the intersection X to which the vehicle 50 is headed can be determined from the map information. It may be acquired.
  • the position of each corner of the intersection X is transmitted by a beacon, and the beacon is received by the vehicle 100 approaching the intersection X, thereby obtaining the position of the front left corner of the intersection X. You can also
  • the information processing unit 11 calculates a limit line (direct visibility limit line Ldv) that is directly visible from the vehicle 100 that is the limit at the obstacle point A.
  • Ldv direct visibility limit line
  • S36 direct visibility limit detector
  • the information processing unit 11 uses the image portion Ic50 of the curved mirror 50 (mirror main body 51) included in the image Ic represented by the mirror position information (own vehicle coordinate system) and the mirror image information. Using optical parameters (focal length, magnification, angle of view, etc.), the line of sight Lm from the vehicle 100 to the outermost edge point of the mirror surface of the curved mirror 50 (mirror body 51) in the visual range of the vehicle 100 is calculated. (S37). Then, the information processing unit 11 calculates the mirror field of view, which is the line of sight line Lm reflected by the curved mirror 50 (mirror body 51) and which is the limit line of the area visible by the curved mirror 50 (mirror body 51).
  • a limit line Lmv (mirror view limit line information) is calculated (S38: mirror view limit detector). Specifically, as described above, the center O of the spherical surface 500 (see FIG. 6) including the mirror surface of the curved mirror 50 (mirror body 51) is obtained, and the curved mirror 50 (mirror body 51) extends from the center O. A normal vector ne passing through the outermost edge point of the mirror surface of 51) is obtained. Then, using the visual field line Lm and the normal vector ne, the mirror visual field limit line Lmv (mirror visual field limit line information) is calculated according to the principle of reflection (law of geometric optics).
  • the information processing unit 11 based on the mirror visibility limit line Lmv and map information (expressing road information including road R2) , the position (position information) of the obstacle point C, which is the intersection of the mirror visibility limit line Lmv and the edge line of the road R2, is calculated (S39).
  • the information processing unit 11 also calculates the position (positional information) of the visibility limit point B, which is the intersection of the mirror visibility limit line Lmv and the direct visibility limit line Ldv (S40). Then, the information processing unit 11 generates, from the obstacle point A, the visibility limit point B, and the obstacle point C, blind area information representing a triangular area surrounded by these three points as a blind area Zd (S41 ).
  • the information processing unit 11 determines whether or not a predetermined termination condition (for example, stop driving, driving support mode OFF, etc.) is satisfied (S42), and in a situation where the termination condition is not satisfied (NO in S42), The above-described processes (S31 to S41: FIGS. 11A and 11B) are repeatedly executed. During this time, blind spot area information representing a blind spot area Zd that gradually narrows as the vehicle 100 approaches the intersection X is generated.
  • a predetermined termination condition for example, stop driving, driving support mode OFF, etc.
  • the information processing unit 11 generates driving support information for the vehicle 100 based on the blind area information representing the blind area Zd that narrows in sequence each time the blind area information is obtained.
  • the generated driving support information is output from the output processing unit 14 according to the type of driving support information under the control of the information processing unit 11 that recognizes the traveling speed of the vehicle 100 based on the signal from the vehicle speed sensor 22.
  • unit display, audio output device, etc.
  • ECU for controlling various actuators of the steering system, the accelerator system, and the braking system in the vehicle 100, and the like.
  • the following driving assistance information can be generated.
  • Display information including the vehicle 100, the intersection X between the roads R1 and R2, and the blind area Zd can be generated as driving support information based on the blind area information and the road map.
  • an image as shown in FIG. 10 can be displayed on the display.
  • the steering system of the vehicle 100 based on the blind spot area information representing the blind spot area Zd that gradually narrows as it approaches the intersection X, the steering system of the vehicle 100 to decelerate and stop particularly at the intersection X without a stop line as shown in FIG.
  • a control signal to be supplied to an ECU that controls various actuators of an accelerator system and a braking system can be generated as driving support information.
  • warning information mark, sound
  • mirror curvature information representing the degree of curvature of the mirror surface of the curve mirror 50 (mirror body 51), mirror position information representing the position of the curve mirror 50, Blind spot area information representing a blind spot area Zd for the vehicle formed at the intersection X between the road R1 and the road R2 is generated based on the map information together with the mirror direction information representing the direction. Therefore, based on the blind spot information, it is possible to provide driving support information based on the blind spot actually formed with respect to the vehicle (obtained from the actual situation reflected in the curved mirror 50).
  • each position, direction, and various straight line information can be expressed in any of the own vehicle coordinate system, the absolute coordinate system, and any other arbitrary coordinate system. be able to.
  • object position information is generated in the vehicle information processing apparatus 10 according to the first embodiment and blind area information is generated in the vehicle information processing apparatus 10 according to the second embodiment
  • the object Both position information and blind spot information may be generated, and driving assistance information based on these two types of information may be provided.
  • the present invention is not limited to this. Even in other situations, such as a situation where a curved mirror is provided on a curved road, the above-described vehicle condition processing device 10 uses the same method to obtain an object position representing the actual position of a moving object reflected on the curved mirror. Information, or blind area information representing the blind area for the vehicle, can be generated.
  • a vehicle information processing apparatus has the effect of being able to provide driving assistance information based on the actual situation reflected in a curved mirror, and is installed in a vehicle traveling on a road provided with a curved mirror. It is useful as an information processing device for vehicles.

Abstract

[Problem] To provide a vehicle information processing device that can provide driving assistance information based on the actual position of an object reflected in a roadside mirror. [Solution] The vehicle information processing device comprises: a first information acquisition unit (S14) that acquires mirror curvature information on the basis of information of an image in a roadside mirror 50 included in mirror image information; a second information acquisition unit (S13) that acquires mirror position information; a third information acquisition unit (S14) that acquires mirror orientation information on the basis of the mirror curvature information and the image in the roadside mirror 50; and an object position detection unit that generates object position information representing the position of an object of interest reflected in the roadside mirror 50 on the basis of the information of an image of the object of interest detected in the image represented by the mirror image information, the mirror curvature information, the mirror position information, and the mirror orientation information, and generates the driving assistance information on the basis of the object position information obtained by the object position detection unit.

Description

車両用情報処理装置Vehicle information processing device
　本発明は、カーブミラーの設けられた道路を走行する車両の運転を支援する運転支援情報を生成する車両用情報処理装置に関する。 The present invention relates to a vehicle information processing apparatus that generates driving assistance information that assists the driving of a vehicle traveling on a road with curved mirrors.
　従来、特許文献１に記載される情報処理装置が知られている。この情報処理装置では、車両に搭載されたカメラ装置（撮像装置）により得られる画像を表す画像情報に基づいてその画像に映るカーブミラーが検出され、更に、そのカーブミラーに映る物体（自転車、歩行者等）が検出される。そして、その検出された物体のカーブミラー内における位置（基準より上側、基準より下側、映る道路の奥側、映る道路の手前側等）が検出され、その検出された物体の位置に応じて当該車両に対する危険度合いが推定される。その推定された危険度合いに基づいた運転支援情報が生成され、出力される。また、前記画像情報に基づいて前記カーブミラーに映る物体のサイズが演算され、そのカーブミラーに映る物体のサイズに基づいて当該車両に対する危険度合いが推定される。その推定された危険度合いに基づいた運転支援情報が生成され、出力される。 Conventionally, an information processing device described in Patent Document 1 is known. In this information processing device, a curved mirror reflected in the image is detected based on image information representing an image obtained by a camera device (imaging device) mounted on the vehicle, and an object (bicycle, walking, etc.) reflected in the curved mirror is detected. person, etc.) is detected. Then, the position of the detected object in the curved mirror (above the reference, below the reference, behind the reflected road, near the reflected road, etc.) is detected, and the position of the detected object is determined. A degree of danger to the vehicle is estimated. Driving assistance information is generated and output based on the estimated degree of danger. Further, the size of the object reflected in the curved mirror is calculated based on the image information, and the degree of danger to the vehicle is estimated based on the size of the object reflected in the curved mirror. Driving assistance information is generated and output based on the estimated degree of danger.
　このような情報処理装置によれば、交差点等においてカーブミラーに映る物体の状態に応じた危険度合いが判る。このため、カーブミラーの湾曲した鏡面に変形して映る物体の像から危険度合いを的確に判断することができ、交差点等の見通しの悪い場所において、車両の安全運転を支援することができる。 According to such an information processing device, the degree of danger corresponding to the state of an object reflected in a curved mirror at an intersection or the like can be determined. Therefore, the degree of danger can be accurately determined from the image of the deformed object reflected on the curved mirror surface of the curved mirror, and safe driving of the vehicle can be supported in locations with poor visibility such as intersections.
特開２０１８－１０６６６６号公報JP 2018-106666 A
　上述した情報処理装置では、カーブミラーに映る物体の像の当該カーブミラーにおける位置に基づいて危険度合いが推定され、その推定される危険度合いに基づいた運転支援情報が生成される。しかし、上記推定される危険度合いに基づいた運転支援情報は、車両を運転する運転者に対して注意喚起するものとしては有効ではあるが、カーブミラーに映る物体の現実の位置に基づいたものではない。このため、上記運転支援情報は、カーブミラーに映る物体（車両、自転車、歩行者等）との実際の位置関係（カーブミラーに映る実際の状況）に基づく車両の運転制御に供すべき情報としては充分なものとは言い難い。 In the information processing device described above, the degree of danger is estimated based on the position of the image of the object reflected on the curved mirror on the curved mirror, and driving support information is generated based on the estimated degree of danger. However, although the driving support information based on the estimated degree of danger is effective in calling attention to the driver driving the vehicle, it is not based on the actual position of the object reflected in the curved mirror. do not have. For this reason, the driving support information is information that should be used for vehicle driving control based on the actual positional relationship (actual situation reflected in the curved mirror) with objects (vehicles, bicycles, pedestrians, etc.) reflected in the curved mirror. It's hard to say enough.
　本発明は、このような事情に鑑みてなされたもので、カーブミラーに映る実際の状況に基づいた運転支援情報を提供することのできる車両用情報処理装置を提供するものである。 The present invention has been made in view of such circumstances, and provides a vehicle information processing apparatus capable of providing driving support information based on the actual situation reflected in the curved mirror.
　本発明に係る車両用情報処理装置は、車両に搭載されるカメラ装置により得られる道路に設置されたカーブミラーの像を含む画像を表すミラー画像情報に基づいて前記車両の運転を支援するための運転支援情報を生成する車両用情報処理装置であって、前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報に基づいて当該カーブミラーの鏡面の湾曲度を表すミラー湾曲情報を取得する第１情報取得部と、前記カーブミラーの位置を表すミラー位置情報を取得する第２情報取得部と、前記ミラー湾曲情報及び前記カーブミラーの像の情報に基づいて前記車両に対する前記カーブミラーの鏡面の向く方向を表すミラー向き情報を取得する第３情報取得部と、前記ミラー画像情報にて表される画像に含まれる前記カーブミラーに映った注目物体の像を検出する物体像検出部と、前記ミラー画像情報にて表される前記画像において検出された前記注目物体の像の情報、前記ミラー湾曲情報、前記ミラー位置情報、及び前記ミラー向き情報に基づいて前記注目物体の位置を表す物体位置情報を生成する物体位置検出部と、を有し、前記物体位置検出部にて得られる前記物体位置情報に基づいて前記運転支援情報を生成する、構成となる。 A vehicle information processing apparatus according to the present invention is for assisting driving of a vehicle based on mirror image information representing an image including an image of a curved mirror installed on a road, which is obtained by a camera device mounted on the vehicle. A vehicle information processing device for generating driving support information, wherein the curvature of the mirror represents the degree of curvature of the mirror surface of the curved mirror based on the information of the image of the curved mirror included in the image represented by the mirror image information. a first information acquisition unit that acquires information; a second information acquisition unit that acquires mirror position information representing the position of the curved mirror; a third information acquisition unit that acquires mirror orientation information indicating the direction in which the mirror surface of the curved mirror faces; and a position of the object of interest based on information on the image of the object of interest detected in the image represented by the mirror image information, the information on the curvature of the mirror, the information on the position of the mirror, and the information on the orientation of the mirror. and an object position detection unit that generates object position information representing: and generates the driving support information based on the object position information obtained by the object position detection unit.
　このような構成により、車両に搭載されるカメラ装置により得られるミラー画像情報からそれが表す画像に含まれるカーブミラーの像の情報に基づいて当該カーブミラーの湾曲度を表すミラー湾曲情報が取得され、そのカーブミラーの位置を表すミラー位置情報が取得される。また、取得されたミラー湾曲情報及び前記ミラー画像情報が表す画像に含まれる前記カーブミラーの像に基づいて前記車両に対する前記カーブミラーの鏡面の向く方向を表すミラー向き情報が取得される。そして、前記ミラー画像情報にて表される画像に含まれる前記カーブミラーに映った注目物体の像が検出され、その検出された注目物体の像の情報とともに、前述したように取得されたミラー湾曲情報、ミラー位置情報、及びミラー向き情報に基づいて、前記注目物体の位置を表す物体位置情報が生成され、その物体位置情報に基づいて運転支援情報が生成される。 With such a configuration, the mirror curvature information representing the degree of curvature of the curved mirror is acquired based on the information of the image of the curved mirror included in the image represented by the mirror image information obtained by the camera device mounted on the vehicle. , mirror position information representing the position of the curved mirror is acquired. Further, based on the image of the curved mirror included in the image represented by the acquired mirror curvature information and the mirror image information, mirror orientation information representing the direction in which the mirror surface of the curved mirror faces the vehicle is acquired. Then, the image of the object of interest reflected on the curved mirror included in the image represented by the mirror image information is detected, and along with the information of the detected image of the object of interest, the mirror curvature obtained as described above is detected. Object position information representing the position of the target object is generated based on the information, the mirror position information, and the mirror orientation information, and driving support information is generated based on the object position information.
　本発明に係る車両用情報処理装置において、前記第１情報取得部は、複数種のカーブミラーのそれぞれの外形形状の特徴を表す外形特徴情報とそれに対応する湾曲度とが定められたテーブル情報を記憶する記憶部と、前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報から前記カーブミラーの外形形状の特徴を表す外形特徴情報を取得するミラー外形特徴取得部と、前記テーブル情報において前記ミラー外形特徴取得部により取得された前記外形特徴情報に対応する湾曲度を、前記ミラー湾曲情報が表すべき湾曲度として決定する湾曲度決定部と、を有する構成とすることができる。 In the information processing apparatus for a vehicle according to the present invention, the first information acquisition unit obtains table information in which external characteristic information representing the characteristic of the external shape of each of a plurality of types of curved mirrors and the degree of curvature corresponding thereto are determined. a storage unit for storing, a mirror external shape feature acquiring unit for acquiring external shape feature information representing the characteristics of the external shape of the curved mirror from information on the image of the curved mirror included in the image represented by the mirror image information; a curvature determining unit configured to determine the degree of curvature corresponding to the external feature information acquired by the mirror external feature acquiring unit in the table information as the degree of curvature to be represented by the mirror curvature information. can.
　このような構成により、ミラー画像情報にて表される画像に含まれるカーブミラーの像の情報から、前記カーブミラーの外形形状の特徴を表す外形特徴情報が取得され、テーブル情報においてその取得された外形特徴情報に対応する湾曲度がミラー湾曲情報が表すべき湾曲度として決定され、その湾曲度を表す当該ミラー湾曲情報が取得される。 With such a configuration, the external feature information representing the characteristic of the external shape of the curved mirror is acquired from the information of the image of the curved mirror included in the image represented by the mirror image information, and the acquired information is stored in the table information. The degree of curvature corresponding to the external feature information is determined as the degree of curvature to be represented by the mirror curvature information, and the mirror curvature information representing the degree of curvature is acquired.
　本発明に係る車両用情報処理装置において、前記第２情報取得部は、前記車両から前記カーブミラーまでの距離を表すミラー距離情報を取得する距離取得部と、前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報と前記距離取得部により取得された前記ミラー距離情報とに基づいて前記ミラー位置情報を生成するミラー位置検出部と、を有する構成とすることができる。 In the vehicle information processing apparatus according to the present invention, the second information acquisition section is represented by a distance acquisition section that acquires mirror distance information representing a distance from the vehicle to the curved mirror, and the mirror image information. A mirror position detection unit that generates the mirror position information based on information on the image of the curved mirror included in the image and the mirror distance information acquired by the distance acquisition unit.
　このような構成により、車両からカーブミラーまでの距離を表すミラー距離情報が取得され、ミラ―画像情報にて表される画像に含まれる前記カーブミラーの像の情報と、前記ミラー距離情報に基づいてカーブミラーの位置を表すミラー位置情報が生成される。 With such a configuration, the mirror distance information representing the distance from the vehicle to the curved mirror is acquired, and based on the image information of the curved mirror included in the image represented by the mirror image information and the mirror distance information. mirror position information representing the position of the curved mirror is generated.
　本発明に係る車両用情報処理装置において、前記第３情報取得部は、前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報と前記ミラー湾曲情報とに基づいて、前記カーブミラーの鏡面の中心点の法線方向を前記ミラー向き情報として生成するミラー法線演算部とを有する構成とすることができる。 In the information processing apparatus for a vehicle according to the present invention, the third information acquiring section acquires the above information based on the image information of the curved mirror and the mirror curvature information included in the image represented by the mirror image information. and a mirror normal calculation unit that generates the normal direction of the center point of the mirror surface of the curved mirror as the mirror orientation information.
　このような構成により、ミラー画像情報にて表される画像に含まれるカーブミラーの像の情報と前記カーブミラー湾曲情報（湾曲度）とに基づいて、前記カーブミラーの鏡面の中心点の法線方向がミラー向き情報として生成される。 With such a configuration, the normal line of the center point of the mirror surface of the curved mirror is calculated based on the information of the curved mirror image included in the image represented by the mirror image information and the curved mirror curvature information (degree of curvature). Direction is generated as mirror orientation information.
　本発明に係る車両用情報処理装置において、前記物体位置検出部は、前記ミラー画像情報にて表される画像における前記カーブミラーに映る前記注目物体の像の情報と前記ミラー湾曲情報とに基づいて前記カーブミラーと実物としての前記注目物体との距離を演算する距離演算部と、前記ミラー画像情報にて表される画像における前記カーブミラーに映る前記注目物体の像の情報、前記ミラー位置情報、及び前記ミラー向き情報に基づいて、前記車両からの視線の前記カーブミラーに映る前記注目物体の像から実物としての前記注目物体に向う方向を表すミラー反射方向を演算する反射方向演算部と、を有し、前記距離演算部にて得られる前記カーブミラーと実物としての前記注目物体との距離と前記ミラー反射方向とに基づいて前記物体位置情報を生成する、構成とすることができる。 In the vehicle information processing apparatus according to the present invention, the object position detection unit detects the image based on the information of the image of the object of interest reflected on the curved mirror in the image represented by the mirror image information and the mirror curvature information. a distance calculation unit that calculates a distance between the curved mirror and the target object as a real object; information on an image of the target object reflected on the curved mirror in the image represented by the mirror image information; the mirror position information; and a reflection direction calculation unit for calculating, based on the mirror orientation information, a mirror reflection direction representing a direction toward the object of interest as a real object from the image of the object of interest reflected on the curved mirror in the line of sight from the vehicle. and generating the object position information based on the distance between the curved mirror and the object of interest as a real object obtained by the distance calculation unit and the mirror reflection direction.
　このような構成により、ミラー画像情報にて表される画像におけるカーブミラーに映る注目物体の像の情報とカーブミラーのミラー湾曲情報（湾曲度）とに基づいて当該カーブミラーと実物としての注目物体との距離が演算され、前記ミラー画像情報にて表される画像におけるカーブミラーに映る注目物体の像の情報、ミラー位置情報、及びミラー傾き情報に基づいて車両からの視線の前記カーブミラーに映る前記注目物体の像から実物としての前記注目物体に向う方向を表すミラー反射方向が演算される。そして、演算された前記距離と前記ミラー反射方向とに基づいて物体位置情報が生成される。 With such a configuration, the curved mirror and the target object as a real object are determined based on the information of the image of the target object reflected on the curved mirror in the image represented by the mirror image information and the mirror curvature information (degree of curvature) of the curved mirror. is calculated, and based on the information of the image of the object of interest reflected on the curved mirror in the image represented by the mirror image information, the mirror position information, and the mirror tilt information, the line of sight from the vehicle is reflected on the curved mirror A mirror reflection direction representing a direction toward the object of interest as a real object is calculated from the image of the object of interest. Then, object position information is generated based on the calculated distance and the mirror reflection direction.
　本発明に係る車両用情報処理装置において、更に、前記ミラー湾曲情報、前記ミラー位置情報、前記ミラー向き情報及び車両走行用の道路を表す道路情報を含む地図情報に基づいて、注目する道路において形成される前記車両に対する死角領域を表す死角領域情報を生成する死角領域検出部を有し、前記死角領域検出部により得られる前記死角領域情報に基づいて前記運転支援情報を生成する、構成とすることができる。 In the information processing apparatus for a vehicle according to the present invention, furthermore, based on map information including the mirror curvature information, the mirror position information, the mirror direction information, and road information representing a road for vehicle travel, the a blind spot area detection unit that generates blind area information representing a blind area for the vehicle that is driven by the vehicle, and generates the driving support information based on the blind area information obtained by the blind area detection unit. can be done.
　このような構成により、前述したように取得されたミラー湾曲情報、ミラー位置情報、及びミラー向き情報とともに地図情報に基づいて、注目する道路において当該車両に対する死角領域を表す死角領域情報が生成される。そして、この死角領域情報に基づいて運転支援情報が生成される。 With such a configuration, the blind area information representing the blind area for the vehicle on the target road is generated based on the map information together with the mirror curvature information, the mirror position information, and the mirror direction information acquired as described above. . Then, driving support information is generated based on this blind spot information.
　本発明に係る車両用情報処理装置は、車両に搭載されるカメラ装置により得られる道路に設置されたカーブミラーの像を含む画像を表すミラー画像情報に基づいて前記車両の運転を支援するための運転支援情報を生成する車両用情報処理装置であって、前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報に基づいて当該カーブミラーの鏡面の湾曲度を表すミラー湾曲情報を取得する第１情報取得部と、前記カーブミラーの位置を表すミラー位置情報を取得する第２情報取得部と、前記ミラー湾曲情報及び前記カーブミラーの像の情報に基づいて車両に対する前記カーブミラーの鏡面の向く方向を表すミラー向き情報を取得する第３情報取得部と、前記ミラー湾曲情報、前記ミラー位置情報、前記ミラー向き情報、前記車両位置情報及び車両走行用の道路を表す道路情報を含む地図情報に基づいて、注目する道路において形成される前記車両に対する死角領域を表す死角領域情報を生成する死角領域検出部と、を有し、前記死角領域検出部により得られる前記死角領域情報に基づいて前記運転支援情報を生成する、構成となる。 A vehicle information processing apparatus according to the present invention is for assisting driving of a vehicle based on mirror image information representing an image including an image of a curved mirror installed on a road, which is obtained by a camera device mounted on the vehicle. A vehicle information processing device for generating driving support information, wherein the curvature of the mirror represents the degree of curvature of the mirror surface of the curved mirror based on the information of the image of the curved mirror included in the image represented by the mirror image information. a first information acquisition unit that acquires information; a second information acquisition unit that acquires mirror position information representing the position of the curved mirror; a third information acquisition unit that acquires mirror orientation information representing the direction in which the mirror surface of the mirror faces; the mirror curvature information, the mirror position information, the mirror orientation information, the vehicle position information, and road information representing a road on which the vehicle travels; a blind spot area detection unit that generates blind spot area information representing a blind spot area for the vehicle formed on the road of interest based on map information including The driving support information is generated based on.
　このような構成により、車両に搭載されるカメラ装置により得られるミラー画像情報からそれが表す画像に含まれるカーブミラーの像の情報に基づいて当該カーブミラーの湾曲度を表すミラー湾曲情報が取得され、そのカーブミラーの位置を表すミラー位置情報が取得される。また、取得されたミラー湾曲情報及び前記ミラー画像情報が表す画像に含まれる前記カーブミラーの像に基づいて前記カーブミラーの鏡面の向く方向を表すミラー向き情報が取得される。そして、前述したように取得されたミラー湾曲情報、ミラー位置情報、及びミラー向き情報とともに地図情報に基づいて、注目する道路において当該車両に対する死角領域を表す死角領域情報が生成され、この死角領域情報に基づいて運転支援情報が生成される。 With such a configuration, the mirror curvature information representing the degree of curvature of the curved mirror is acquired based on the information of the image of the curved mirror included in the image represented by the mirror image information obtained by the camera device mounted on the vehicle. , mirror position information representing the position of the curved mirror is acquired. Further, based on the image of the curved mirror included in the image represented by the acquired mirror curvature information and the mirror image information, mirror orientation information representing the direction in which the mirror surface of the curved mirror faces is acquired. Then, based on map information together with the acquired mirror curvature information, mirror position information, and mirror direction information as described above, blind area information representing a blind area for the vehicle on the road of interest is generated, and this blind area information is generated. Driving support information is generated based on.
　本発明に係る車両用情報処理装置において、前記死角領域検出部は、前記地図情報に基づいて前記注目する道路における前記車両から直接視認可能な領域の限界線を表す直接視界限界線情報を生成する直接視界限界検出部と、前記地図情報、前記ミラー湾曲情報、前記ミラー位置情報、及び前記ミラー向き情報に基づいて前記注目する道路における前記車両から前記カーブミラーにより視認可能な領域の限界線を表すミラー視界限界線情報を生成するミラー視界限界検出部と、を有し、前記直接視界限界線情報及び前記ミラー視界限界線情報に基づいて前記死角領域情報を生成する、構成とすることができる。 In the vehicle information processing apparatus according to the present invention, the blind spot area detection unit generates direct visibility limit line information representing a limit line of an area of the road of interest that is directly visible from the vehicle based on the map information. A direct visibility limit detection unit and a limit line of an area visible from the vehicle through the curved mirror on the road of interest based on the map information, the mirror curvature information, the mirror position information, and the mirror direction information. and a mirror visibility limit detection unit that generates mirror visibility limit information, and generates the blind area information based on the direct visibility limit information and the mirror visibility limit information.
　このような構成により、地図情報に基づいて注目する道路における車両から直接視認可能な領域の限界線を表す直接視界限界線情報が生成され、地図情報、ミラー湾曲情報（湾曲度）、ミラー位置情報、及びミラー向き情報に基づいて注目する道路における前記車両からカーブミラーにより視認可能な領域の限界線を表すミラー視界限界線情報が生成される。そして、前記直接視界限界線情報及び前記ミラー視界限界線情報に基づいて前記車両に対する死角領域を表す死角領域情報が生成される。 With such a configuration, the direct visibility limit line information representing the limit line of the area of the road of interest that is directly visible from the vehicle is generated based on the map information, and the map information, the mirror curvature information (curvature), and the mirror position information are generated. , and mirror orientation information, mirror visibility limit line information representing the limit line of the area visible from the vehicle through the curved mirror on the road of interest is generated. Blind spot area information representing a blind spot area for the vehicle is generated based on the direct visibility limit information and the mirror visibility limit information.
　本発明に係る車両用情報処理装置によれば、カーブミラーの像を含む画像を表すミラー画像情報において検出された前記カーブミラーに映る注目物体の像の情報とともに、前記カーブミラーの鏡面の湾曲度を表すミラー湾曲情報、前記カーブミラーの位置を表すミラー位置情報、及び前記カーブミラーの鏡面の向く方向を表すミラー向き情報に基づいて、前記注目物体の位置を表す物体位置情報が生成される。このため、その物体位置情報に基づいて、前記カーブミラーに映る注目物体（物体）の実際の位置（実際の状況）に基づいた運転支援情報を提供することができる。 According to the information processing apparatus for a vehicle according to the present invention, along with the information of the image of the target object reflected in the curved mirror detected in the mirror image information representing the image including the image of the curved mirror, the degree of curvature of the mirror surface of the curved mirror object position information representing the position of the object of interest is generated based on mirror curvature information representing the position of the curved mirror, mirror position information representing the position of the curved mirror, and mirror orientation information representing the direction in which the mirror surface of the curved mirror faces. Therefore, based on the object position information, it is possible to provide driving support information based on the actual position (actual situation) of the target object (object) reflected on the curved mirror.
　また、本発明に係る車両用情報処理装置によれば、カーブミラーの鏡面の湾曲度を表すミラー湾曲情報、前記カーブミラーの位置を表すミラー位置情報、前記カーブミラーの鏡面の向きを表すミラー向き情報とともに地図情報に基づいて、注目する道路において形成される車両に対する死角領域を表す死角領域情報が生成される。このため、その死角領域情報に基づいて、車両に対して実際に形成される死角領域（カーブミラーに映る実際の状況から得られる）に基づいた運転支援情報を提供することができる。 Further, according to the vehicle information processing apparatus of the present invention, mirror curvature information representing the degree of curvature of the mirror surface of the curve mirror, mirror position information representing the position of the curve mirror, and mirror orientation representing the direction of the mirror surface of the curve mirror are provided. Based on the map information together with the information, blind area information representing the blind area for the vehicle formed on the road of interest is generated. Therefore, based on the blind spot information, it is possible to provide driving support information based on the blind spot actually formed with respect to the vehicle (obtained from the actual situation reflected in the curved mirror).
図１Ａは、交差点における車両と移動物体（自転車）との関係の一例を示す図である。FIG. 1A is a diagram showing an example of the relationship between a vehicle and a moving object (bicycle) at an intersection. 図１Ｂは、移動物体（自転車）が映るカーブミラー（ミラー本体）の一例を示す図である。FIG. 1B is a diagram showing an example of a curved mirror (mirror body) in which a moving object (bicycle) is reflected. 図２は、本発明の第１の実施の形態に係る車両用情報処理装置の構成を示すブロック図である。FIG. 2 is a block diagram showing the configuration of the vehicle information processing apparatus according to the first embodiment of the present invention. 図３Ａは、図２に示す車両用情報処理装置における情報処理ユニットでの処理の流れ（その１）を示すフローチャートである。3A is a flow chart showing the flow (1) of processing in an information processing unit in the vehicle information processing apparatus shown in FIG. 2. FIG. 図３Ｂは、図２に示す車両用情報処理装置における情報処理ユニットでの処理の流れ（その２）を示すフローチャートである。3B is a flow chart showing a flow (part 2) of processing in an information processing unit in the vehicle information processing apparatus shown in FIG. 2. FIG. 図４は、カメラ装置の撮影により得られた画像Ｉｃと、当該カメラ装置の撮影範囲ＳＡとの関係を示す図である。FIG. 4 is a diagram showing the relationship between an image Ic obtained by photographing by a camera device and the photographing range SA of the camera device. 図５は、カーブミラーの外形形状の特徴（直径、矩形形状）とカーブミラーの鏡面曲率半径との関係を示す図である。FIG. 5 is a diagram showing the relationship between the features (diameter, rectangular shape) of the outer shape of the curved mirror and the radius of curvature of the curved mirror. 図６は、カーブミラーの鏡面と、その鏡面（ミラー平面）の中心点を通る法線を示す図である。FIG. 6 is a diagram showing a mirror surface of a curved mirror and a normal line passing through the center point of the mirror surface (mirror plane). 図７は、カーブミラーの鏡面のミラー平面としての見え方とそのミラー平面の中心を通る法線との関係とを示す図である。FIG. 7 is a diagram showing the relationship between how the mirror surface of a curved mirror appears as a mirror plane and the normal line passing through the center of the mirror plane. 図８は、カメラ装置の撮影により得られた画像Ｉｃ（移動物体が映るカーブミラーを含む）と、当該カメラ装置の撮影範囲ＳＡ（移動物体が映るカーブミラーを含む）との関係を示す図である。FIG. 8 is a diagram showing the relationship between an image Ic (including a curved mirror on which a moving object is captured) captured by a camera device and a shooting range SA of the camera device (including a curved mirror on which a moving object is captured). be. 図９は、カメラ装置、移動物体が映るカーブミラー及び移動物体（実物）との位置的関係を示す図である。FIG. 9 is a diagram showing a positional relationship between a camera device, a curved mirror reflecting a moving object, and a moving object (actual object). 図１０は、交差点に向う車両の死角領域の一例を示す図である。FIG. 10 is a diagram showing an example of a blind spot area of a vehicle heading for an intersection. 図１１Ａは、本発明の第２の実施の形態に係る車両用情報処理装置における情報処理ユニットでの処理の流れ（その１）を示すフローチャートである。FIG. 11A is a flowchart showing the flow (part 1) of processing in an information processing unit in the vehicle information processing apparatus according to the second embodiment of the present invention. 図１１Ｂは、本発明の第２の実施の形態に係る車両用情報処理装置における情報処理ユニットでの処理の流れ（その２）を示すフローチャートである。FIG. 11B is a flow chart showing the flow (part 2) of processing in the information processing unit in the vehicle information processing apparatus according to the second embodiment of the present invention. 図１２は、交差点に対する車両の位置と死角領域との関係の一例を示す図である。FIG. 12 is a diagram showing an example of the relationship between the position of the vehicle with respect to the intersection and the blind spot area.
　以下、本発明の実施の形態について図面を用いて説明する。 Embodiments of the present invention will be described below with reference to the drawings.
　例えば、図１Ａに示すような状況を想定する。図１Ａにおいて、道路Ｒ１と道路Ｒ２とが交差点Ｘにおいて交差する。道路Ｒ１を走行して交差点Ｘに向う車両１００の道路Ｒ２側の視界を確保するために道路Ｒ１の車両１００から見通しのきく交差点Ｘのコーナ部にカーブミラー５０が設置されている。カーブミラー５０は、図１Ｂに示すように、交差点Ｘのコーナ部に立設する支柱５２の先端部に湾曲した鏡面を有するミラー本体５１が固定された構造となっている。 For example, assume the situation shown in FIG. 1A. In FIG. 1A, a road R1 and a road R2 intersect at an intersection X. In FIG. A curved mirror 50 is installed at the corner of the intersection X where the vehicle 100 on the road R1 can see the road R2 side in order to ensure the visibility of the vehicle 100 traveling on the road R1 toward the intersection X on the road R2 side. As shown in FIG. 1B, the curved mirror 50 has a structure in which a mirror body 51 having a curved mirror surface is fixed to the tip of a post 52 erected at the corner of the intersection X. As shown in FIG.
　車両１００が道路Ｒ１を交差点Ｘに向って走行するとともに、自転車２００（注目物体）が道路Ｒ２を交差点Ｘに向って走行する状況において、時刻ｔ１の車両１００（ｔ１）から、カーブミラー５０（ミラー本体５１）の鏡面に映る自転車２００（ｔ１）の像Ｉｏ（ｔ１）（図１Ｂ参照）を視認することができる。また、更に交差点Ｘに近づいた時刻ｔ２の車両１００（ｔ２）から、カーブミラー５０（ミラー本体５１）の鏡面に映る自転車２００（ｔ２）の像Ｉｏ（ｔ２）（図１Ｂ参照）を視認することができる。 In a situation where the vehicle 100 travels on the road R1 toward the intersection X and the bicycle 200 (object of interest) travels on the road R2 toward the intersection X, the curved mirror 50 (mirror The image Io(t1) (see FIG. 1B) of the bicycle 200(t1) reflected on the mirror surface of the main body 51) can be visually recognized. Further, from the vehicle 100 (t2) at time t2 when the vehicle 100 (t2) approaches the intersection X, the image Io (t2) (see FIG. 1B) of the bicycle 200 (t2) reflected on the mirror surface of the curved mirror 50 (mirror body 51) can be visually recognized. can be done.
　車両１００に搭載され、本発明の第１の実施の形態に係る車両用情報処理装置１０を含む車載システムは、図２に示すように構成される。図２において、この車載システムは、車両用情報処理装置１０、車両１００の前方を撮影するカメラ装置２０、カメラ装置２０と同様に車両１００の前方を撮影して奥行の情報を含む画像情報を出力するステレオカメラ２１、車両１００の車輪の回転に応じた車輪速パルスを出力する車速センサ２２、複数の測位衛星からの信号を受信して車両１００の位置に応じた検出信号を出力するＧＰＳユニット２３、及び地図情報を含むナビゲーション装置３０を有している。車両用情報処理装置１０は、ＣＰＵを含む情報処理ユニット１１、画像処理部１２、記憶部１３及び出力処理部１４を有している。画像処理部１２は、情報処理ユニット１１の制御のもと、カメラ装置２０及びステレオカメラ２１のそれぞれからの画像情報を処理して所定形式の画像情報を情報処理ユニット１１に提供する。情報処理ユニット１１は、プログラムに従って画像処理部１２からの画像情報に基づいて運転支援情報を生成する処理を実行する。記憶部１３は、情報処理ユニット１１にて実行すべき種々のプログラム、後述するテーブル情報等の情報処理ユニット１１で利用する種々の情報、及び情報処理ユニット１１での処理により生成された情報等を記憶する。出力処理部１４は、情報処理ユニット１１の制御のもと、生成された運転支援情報を、その運転支援情報の種類に応じて情報出力部（表示器、音声出力器等）、車両１００における操舵系、アクセル系、及び制動系の各種アクチュエータを制御するＥＣＵに供給する。 An in-vehicle system that is mounted in a vehicle 100 and includes the vehicle information processing device 10 according to the first embodiment of the present invention is configured as shown in FIG. 2, this in-vehicle system includes a vehicle information processing device 10, a camera device 20 for capturing an image of the front of the vehicle 100, and similarly to the camera device 20, captures an image of the front of the vehicle 100 and outputs image information including depth information. a stereo camera 21, a vehicle speed sensor 22 that outputs a wheel speed pulse according to the rotation of the wheels of the vehicle 100, a GPS unit 23 that receives signals from a plurality of positioning satellites and outputs a detection signal according to the position of the vehicle 100. , and a navigation device 30 containing map information. The vehicle information processing device 10 has an information processing unit 11 including a CPU, an image processing section 12 , a storage section 13 and an output processing section 14 . Under the control of the information processing unit 11 , the image processing unit 12 processes image information from the camera device 20 and the stereo camera 21 and provides image information in a predetermined format to the information processing unit 11 . The information processing unit 11 executes processing for generating driving support information based on image information from the image processing section 12 according to a program. The storage unit 13 stores various programs to be executed by the information processing unit 11, various information used in the information processing unit 11 such as table information described later, information generated by processing in the information processing unit 11, and the like. Remember. Under the control of the information processing unit 11, the output processing unit 14 outputs the generated driving support information to an information output unit (display device, voice output device, etc.), steering in the vehicle 100 according to the type of the driving support information. system, accelerator system, and braking system.
　道路Ｒ１を走行する車両１００が交差点Ｘに近づくとともに道路Ｒ２を走行する自転車（以下、自転車を一般化して移動物体という）２００（注目物体）が交差点Ｘに近づく状況において、車両用情報処理装置１０における情報処理ユニット１１は、カーブミラー５０（ミラー本体５１）の鏡面に映る移動物体２００（注目物体）の実際の位置を表す物体位置情報を生成する（物体位置検出部）。この処理は、図３Ａ及び図３Ｂに示す手順に従って実行される。そして、情報処理ユニット１１は、得られた物体位置情報に基づいて、道路Ｒ１を走行する車両１００が接近する交差点Ｘに向ってその道路Ｒ１に交差する道路Ｒ２を移動物体２００（注目物体）が移動している状況における運転支援情報を生成する。 In a situation where a vehicle 100 traveling on a road R1 approaches an intersection X and a bicycle traveling on a road R2 (hereinafter referred to as a moving object) 200 (object of interest) approaches the intersection X, the vehicle information processing device 10 The information processing unit 11 in generates object position information representing the actual position of the moving object 200 (object of interest) reflected on the mirror surface of the curved mirror 50 (mirror body 51) (object position detection section). This processing is executed according to the procedure shown in FIGS. 3A and 3B. Then, based on the obtained object position information, the information processing unit 11 moves the moving object 200 (target object) along the road R2 that intersects the road R1 toward the intersection X approached by the vehicle 100 traveling on the road R1. Generate driving assistance information in moving situations.
　図３Ａにおいて、情報処理ユニット１１は、カメラ装置２０から画像処理部１２を経た画像情報を取得するとともに、ＧＰＳユニット２３からの信号に基づいて車両１００の位置（地図上の位置）を表す自車位置情報（例えば、緯度・経度）を取得する（Ｓ１１）。取得された画像情報及び自車位置情報は記憶部１３に格納される。情報処理ユニット１１は、取得した画像情報が表す画像にカーブミラー５０の像が含まれているか否かを判定する（Ｓ１２）。例えば、特許文献１に記載された手法や、画像認識の技術及び機械学習のアルゴリズムに従った手法等に従って、取得される画像情報にて表される画像にカーブミラー５０の像が含まれているか否かを判定することができる。情報処理ユニット１１は、カーブミラー５０の像を含む画像を表す画像情報が取得される（Ｓ１２でＹＥＳ）まで、カメラ装置２０からの画像情報及び自車位置情報の取得（Ｓ１１）及びそれらの記憶部１３への格納（更新）を繰り返し実行する。 In FIG. 3A, the information processing unit 11 acquires image information from the camera device 20 through the image processing unit 12, and based on the signal from the GPS unit 23, the position of the vehicle 100 (position on the map). Position information (for example, latitude and longitude) is acquired (S11). The acquired image information and vehicle position information are stored in the storage unit 13 . The information processing unit 11 determines whether the image represented by the acquired image information includes the image of the curved mirror 50 (S12). For example, whether the image represented by the acquired image information includes the image of the curved mirror 50 according to the method described in Patent Document 1 or the method according to the image recognition technology and machine learning algorithm. It is possible to determine whether or not The information processing unit 11 acquires image information and vehicle position information from the camera device 20 (S11) and stores them until image information representing an image including the image of the curved mirror 50 is acquired (YES in S12). Storing (updating) in the unit 13 is repeatedly executed.
　その過程で、カーブミラー５０の像を含む画像を表す画像情報（以下、ミラー画像情報という）が取得されると（Ｓ１２でＹＥＳ）、情報処理ユニット１１は、カーブミラー５０の位置を表すミラー位置情報を取得する（Ｓ１３：第２情報取得部）。具体的には、情報処理ユニット１１は、ステレオカメラ２１から画像処理部１２を経て得らえる３次元画像情報（奥行の情報を含む）に基づいて車両１００からカーブミラー５０までの距離を表すミラー距離情報を取得する（距離取得部）。また、情報処理ユニット１１は、カメラ装置２０の光学的パラメータ（焦点距離、倍率、画角等）を用いて、図４に示すように、撮影により得られたミラー画像情報にて表されるカーブミラー５０の像Ｉｃ５０を含む画像Ｉｃから、その撮影領域ＳＡ内の物（カーブミラー５０を含む）の２次元的な位置関係を推定演算する。そして、情報処理ユニット１１は、前記ミラー画像情報から得られる撮影領域ＳＡ内の物の２次元的な位置関係と、車両１００からカーブミラー５０までの距離を表すミラー距離情報（奥行の情報）とに基づいて、車両１００の位置を基準とした（以下、自車座標系という）カーブミラー５０の位置を表すミラー位置情報を演算する（ミラー位置検出部）。このようにして取得されたミラー位置情報は記憶部１３に格納される。 In the process, when image information representing an image including the image of the curved mirror 50 (hereinafter referred to as mirror image information) is acquired (YES in S12), the information processing unit 11 calculates the mirror position representing the position of the curved mirror 50. Information is acquired (S13: second information acquisition unit). Specifically, the information processing unit 11 uses the three-dimensional image information (including depth information) obtained from the stereo camera 21 through the image processing unit 12 to determine the distance from the vehicle 100 to the curved mirror 50 . Acquire distance information (distance acquisition unit). Further, the information processing unit 11 uses the optical parameters (focal length, magnification, angle of view, etc.) of the camera device 20 to obtain a curve represented by mirror image information obtained by photographing, as shown in FIG. From the image Ic including the image Ic50 of the mirror 50, the two-dimensional positional relationship of the objects (including the curved mirror 50) within the imaging area SA is estimated and calculated. Then, the information processing unit 11 stores the two-dimensional positional relationship of objects in the photographing area SA obtained from the mirror image information, and mirror distance information (depth information) representing the distance from the vehicle 100 to the curved mirror 50. , the mirror position information representing the position of the curved mirror 50 with reference to the position of the vehicle 100 (hereinafter referred to as the host vehicle coordinate system) is calculated (mirror position detector). The mirror position information acquired in this manner is stored in the storage unit 13 .
　なお、カーブミラー５０の設置位置の情報（地図上での位置（緯度・経度））が地図情報に埋め込まれている場合、その地図情報から車両５０が向かう交差点Ｘに設置されたカーブミラー５０の位置を表すミラー位置情報を取得するようにしてもよい。また、カーブミラー５０の設置された交差点Ｘにおいて、カーブミラー５０の設置位置の情報をビーコンにより発信させておき、交差点Ｘに近づいた車両１００においてそのビーコンを受信することにより、カーブミラー５０の位置を表すミラー位置情報を取得することもできる。 Note that when the information on the installation position of the curved mirror 50 (the position (latitude and longitude) on the map) is embedded in the map information, the location of the curved mirror 50 installed at the intersection X to which the vehicle 50 is heading is determined based on the map information. You may make it acquire the mirror positional information showing a position. At the intersection X where the curved mirror 50 is installed, information on the installation position of the curved mirror 50 is transmitted by a beacon, and the vehicle 100 approaching the intersection X receives the beacon, thereby determining the position of the curved mirror 50. It is also possible to acquire mirror position information representing
　次に、情報処理ユニット１１は、取得したミラー画像情報にて表される画像Ｉｃ（図４参照）に含まれるカーブミラー５０の像Ｉｃ５０の情報に基づいてミラー本体５１（カーブミラー５０）の鏡面の湾曲度を表すミラー湾曲情報を取得する（Ｓ１４：第１情報取得部）。また、情報処理ユニット１１は、その取得したミラー湾曲情報及び前記カーブミラー５０の像Ｉｃ５０の情報に基づいて車両１００に対するミラー本体５１（カーブミラー５０）の鏡面の向く方向を表すミラー向き情報を取得する（Ｓ１４：第３情報取得部）。これら取得されたミラー湾曲情報及びミラー向き情報は記憶部１３に格納される。 Next, the information processing unit 11 calculates the mirror surface of the mirror main body 51 (curved mirror 50) based on the information of the image Ic50 of the curved mirror 50 included in the image Ic (see FIG. 4) represented by the acquired mirror image information. (S14: first information acquisition unit). The information processing unit 11 also acquires mirror orientation information representing the direction in which the mirror surface of the mirror body 51 (the curved mirror 50) faces the vehicle 100 based on the acquired mirror curvature information and the information on the image Ic50 of the curved mirror 50. (S14: third information acquisition unit). The acquired mirror curvature information and mirror orientation information are stored in the storage unit 13 .
　ミラー湾曲情報は例えば次のようにして取得される。 For example, mirror curvature information is obtained as follows.
　日本の道路反射鏡設置指針（日本道路協会）では、外形形状の特徴（丸型の直径φ、角型の縦横サイズ）で分類されるカーブミラーの種類と鏡面の曲率半径（ミラー湾曲情報）との関係が、図５に示すように規定されている。具体的には、直径φ＝６００ｍｍの丸型のカーブミラーの鏡面曲率半径ＲがＲ＝２２００ｍｍであり、直径φ＝８００ｍｍの丸型のカーブミラーの鏡面曲率半径ＲがＲ＝３０００ｍｍであり、直径φ＝１０００ｍｍの丸型のカーブミラーの鏡面曲率半径ＲがＲ＝３６００ｍｍである。また、縦横サイズが４５０ｍｍ×６００ｍｍの角型のカーブミラーの鏡面曲率半径ＲがＲ＝２２００ｍｍであり、縦横サイズが６００ｍｍ×８００ｍｍの角型のカーブミラーの鏡面曲率半径ＲがＲ＝３０００ｍｍである。図５に示すような５種類のカーブミラーのそれぞれの外形形状の特徴を表す外形特徴情報（丸型の直径φ、角型の縦横サイズ）とそれに対応する湾曲度（鏡面曲率半径Ｒ）とが定められたテーブル情報（図５参照）が記憶部１３に記憶されている。 In Japan's road reflector installation guidelines (Japan Road Association), the types of curved mirrors are classified by the characteristics of the external shape (round diameter φ, square vertical and horizontal sizes), the curvature radius of the mirror surface (mirror curvature information), and is defined as shown in FIG. Specifically, the radius of curvature R of a round curved mirror with a diameter of φ=600 mm is R=2200 mm, the radius of curvature R of a round curved mirror with a diameter of φ=800 mm is R=3000 mm, and the radius of curvature R is R=3000 mm. The radius of curvature R of a round curved mirror with φ=1000 mm is 3600 mm. Further, the radius of curvature R of a square curved mirror with a size of 450 mm×600 mm is R=2200 mm, and the radius of curvature R of a square curved mirror with a size of 600 mm×800 mm is R=3000 mm. The external shape characteristic information (diameter φ of round shape, vertical and horizontal size of square shape) representing the characteristics of the external shape of each of the five types of curved mirrors shown in FIG. Predetermined table information (see FIG. 5) is stored in the storage unit 13 .
　道路に実際に設置されるいずれのカーブミラーも、図５に示すテーブル情報で示される５種類の外形形状のうちのいずれかを有するものであることを前提に、前記ミラー画像情報にて表される画像Ｉｃに含まれるカーブミラー５０の像Ｉｃ５０の情報に基づいて実際のカーブミラー５０（ミラー本体５１）の湾曲度（ミラー湾曲情報）が決められる。例えば、取得されたミラー画像情報にて表される画像Ｉｃに含まれるカーブミラー５０の像Ｉｃ５０の情報（図４参照）から、カメラ装置２０の光学的パラメータ（焦点距離、倍率、画角等）及び車両１００からカーブミラー５０までの距離を用いて、実際のカーブミラー５０のミラー本体５１の直径（外形形状）が推定演算される（ミラー外形特徴取得部）。記憶部１３に記憶された前記テーブル情報（図５参照）において、前記推定演算により得られた直径（外形形状）に対応する、具体的には、その推定演算により得られた直径に最も近い直径に対応する鏡面曲率半径Ｒがミラー湾曲情報として決定される（湾曲度決定部）。 Any curve mirror actually installed on the road is represented by the mirror image information on the premise that it has one of the five types of external shapes shown in the table information shown in FIG. The degree of curvature (mirror curvature information) of the actual curved mirror 50 (mirror body 51) is determined based on the information of the image Ic50 of the curved mirror 50 included in the image Ic. For example, from the information (see FIG. 4) of the image Ic50 of the curved mirror 50 included in the image Ic represented by the acquired mirror image information, the optical parameters (focal length, magnification, angle of view, etc.) of the camera device 20 Using the distance from the vehicle 100 to the curved mirror 50, the diameter (outer shape) of the mirror main body 51 of the actual curved mirror 50 is estimated and calculated (mirror outer shape feature acquisition unit). In the table information (see FIG. 5) stored in the storage unit 13, the diameter corresponding to the diameter (outer shape) obtained by the estimation calculation, specifically, the diameter closest to the diameter obtained by the estimation calculation is determined as the mirror curvature information (curvature determination unit).
　なお、カーブミラー５０の鏡面の湾曲度を表すミラー湾曲情報がそのカーブミラー５０の設置位置の情報とともに地図情報に埋め込まれている場合、その地図情報から車両５０が向かう交差点Ｘに設置されたカーブミラー５０についてのミラー湾曲情報を取得するようにしてもよい。また、カーブミラー５０の設置された交差点Ｘにおいて、カーブミラー５０の鏡面の湾曲度を表すミラー湾曲情報をそのカーブミラー５０の設置位置の情報とともにビーコンにより発信させておき、交差点Ｘに近づいた車両１００においてそのビーコンを受信することにより、カーブミラー５０のミラー湾曲情報（湾曲度）を取得することもできる。 If the mirror curvature information representing the degree of curvature of the mirror surface of the curved mirror 50 is embedded in the map information together with the information on the installation position of the curved mirror 50, the curve installed at the intersection X to which the vehicle 50 is headed is determined based on the map information. Mirror curvature information about the mirror 50 may be obtained. At an intersection X where a curved mirror 50 is installed, mirror curvature information representing the degree of curvature of the mirror surface of the curved mirror 50 is transmitted by a beacon together with information on the installation position of the curved mirror 50, and a vehicle approaching the intersection X is transmitted. By receiving the beacon at 100, mirror curvature information (degree of curvature) of the curved mirror 50 can also be obtained.
　車両１００（カメラ装置２０の光軸方向）に対するミラー本体５１（カーブミラー５０）の鏡面の向く方向（垂直面での回転方向と水平面での回転方向との合成方向）を表すミラー向き情報は、次のようにして取得される。 The mirror orientation information representing the orientation of the mirror surface of the mirror body 51 (curved mirror 50) with respect to the vehicle 100 (optical axis direction of the camera device 20) (combined direction of the rotation direction on the vertical plane and the rotation direction on the horizontal plane) is It is obtained as follows.
　丸型のミラー本体５１を有するカーブミラー５０の場合（角型のミラ―本体の場合も基本的に同じ）、その鏡面５１ａは、図６（立体図）に示すように、そのミラー本体５１の直径φに対応する鏡面曲率半径Ｒ（図５参照）の球面５００の一部部分（凸面部分）とすることができる。そして、前記ミラー画像情報にて表される２次元的な画像Ｉｃ（図４参照）に含まれるカーブミラー５０（ミラー本体５１）の像Ｉｃ５０の鏡面部分の形状は、平面状の円形または楕円形である。ここで、この円形または楕円形の平面をミラー平面と定義する（図６において、ミラー平面５１ｂ）。 In the case of a curved mirror 50 having a round mirror body 51 (basically the same applies to a rectangular mirror body), the mirror surface 51a is the same as that of the mirror body 51, as shown in FIG. It can be a partial portion (convex portion) of the spherical surface 500 with a specular curvature radius R (see FIG. 5) corresponding to the diameter φ. The shape of the mirror surface portion of the image Ic50 of the curved mirror 50 (mirror body 51) included in the two-dimensional image Ic (see FIG. 4) represented by the mirror image information is a planar circular or elliptical shape. is. Here, this circular or elliptical plane is defined as a mirror plane (mirror plane 51b in FIG. 6).
　ミラー平面５１ｂは、図７（平面図）に示すように、車両１００（カメラ装置２０）とカーブミラー５０との位置関係及びミラー本体５１自体の傾き（垂直面での回転角度：俯角：例えば、５°～１０°）に応じて変化する。図７において、車両１００（カメラ装置２０）とミラー本体５１とが正対した状態では、ミラー画像情報（カーブミラー５０の像Ｉｃの情報）から得られるミラー平面５１ｂ（０）は円形である。車両１００（カメラ装置２０）とカーブミラー５０とが斜めに対向し（例えば、図１参照）、また、ミラー本体５１自体が傾いていると、前記ミラー画像情報（図４におけるＩｃ参照）からは、車両１００（カメラ装置２０）とカーブミラー５０との対向におけるその斜めの程度やミラー本体５１自体の傾きの程度に応じて異なる楕円形のミラー平面５１ｂ（１）、５１ｂ（２）、５１ｂ（３）、５１ｂ（４）が得られる。 As shown in FIG. 7 (plan view), the mirror plane 51b is defined by the positional relationship between the vehicle 100 (camera device 20) and the curved mirror 50 and the inclination of the mirror body 51 itself (rotation angle on the vertical plane: depression angle: for example, 5° to 10°). In FIG. 7, when the vehicle 100 (camera device 20) faces the mirror body 51, the mirror plane 51b(0) obtained from the mirror image information (information on the image Ic of the curved mirror 50) is circular. If the vehicle 100 (camera device 20) and the curved mirror 50 face each other obliquely (see, for example, FIG. 1), and if the mirror body 51 itself is tilted, the mirror image information (see Ic in FIG. 4) indicates that , elliptical mirror planes 51b(1), 51b(2), 51b( 3), 51b(4) are obtained.
　前記ミラー画像情報にて表される画像Ｉｃに含まれるカーブミラー５０の画像Ｉｃ５０において、幾何学的な演算手法に従って、ミラー平面５１ｂ（例えば、楕円形状）の中心点（図６における点Ｐ、図７における点Ｐ０、Ｐ１、Ｐ２、Ｐ３、Ｐ４）が検出される。そして、その中心点Ｐ（Ｐ０～Ｐ４）におけるミラー平面５１ｂ（５１ｂ（０）～５１ｂ（４））の法線ベクトルｎ（ｎ０～ｎ４）が決められる（ミラー法線演算部）。このように画像Ｉｃ（像Ｉｃ５０）上で決められたその法線ベクトルｎは、カメラ装置２０の光学的パラメータ（焦点距離、倍率、画角等）及び車両１００からカーブミラー５０までの距離を用いて自車座標系にて表される。このように自車座標系にて表される前記ミラー平面５１ｂの中心点を通る法線ベクトルｎは球面の一部である鏡面５１ａの中心点を通る法線に対応し、その法線ベクトルｎがカーブミラー５０（ミラー本体５１）の鏡面５１ａが向く方向を表すミラー向き情報として定義される。 In the image Ic50 of the curved mirror 50 included in the image Ic represented by the mirror image information, the center point (the point P in FIG. 6, the 7) are detected. Then, the normal vector n (n0 to n4) of the mirror plane 51b (51b(0) to 51b(4)) at the center point P (P0 to P4) is determined (mirror normal calculating section). The normal vector n determined on the image Ic (image Ic50) in this way is obtained using the optical parameters (focal length, magnification, angle of view, etc.) of the camera device 20 and the distance from the vehicle 100 to the curved mirror 50. is expressed in the own vehicle coordinate system. Thus, the normal vector n passing through the center point of the mirror plane 51b expressed in the host vehicle coordinate system corresponds to the normal line passing through the center point of the mirror surface 51a, which is a part of the spherical surface, and the normal vector n is defined as mirror orientation information representing the direction in which the mirror surface 51a of the curved mirror 50 (mirror body 51) faces.
　図３Ａに戻って、上述したようにミラー位置情報（Ｓ１３）、ミラー湾曲情報及びミラー向き情報（Ｓ１４）が取得されると、情報処理ユニット１１は、ミラー画像情報にて表される画像Ｉｃ（図４参照）に含まれるカーブミラー５０の像Ｉｃ５０に、そのカーブミラー５０に映った自転車、自動車（他車）、人等の移動物体（注目物体）の像が含まれているか否かを判定する（Ｓ１５）。例えば、ミラー画像情報が表す画像Ｉｃに含まれるカーブミラー５０の像Ｉｃ５０から、移動物体の候補を抽出する公知の処理及び機械学習のアルゴリズムに従った手法によって、カーブミラー５０に映る移動物体の像部分を検出することができる。そして、情報処理ユニット１１は、移動物体（注目物体）が映ったカーブミラー５０の像Ｉｃ５０を含む画像Ｉｃを表すミラー画像情報が得られるまで、上述した処理（Ｓ１１、Ｓ１２、Ｓ１３、Ｓ１４）を繰り返し実行する。その間、車両１００の移動とともに記憶部１３に記憶される自車位置情報、ミラー画像情報、ミラー位置情報（自車座標系）、及びミラー向き情報（自車座標系）が順次更新される。 Returning to FIG. 3A, when the mirror position information (S13), the mirror curvature information, and the mirror orientation information (S14) are obtained as described above, the information processing unit 11 calculates the image Ic ( 4) includes an image of a moving object (object of interest) such as a bicycle, a car (another vehicle), or a person reflected in the curved mirror 50. (S15). For example, from the image Ic50 of the curved mirror 50 included in the image Ic represented by the mirror image information, the image of the moving object reflected on the curved mirror 50 is extracted by a known process for extracting moving object candidates and a technique according to a machine learning algorithm. parts can be detected. Then, the information processing unit 11 performs the above-described processes (S11, S12, S13, S14) until the mirror image information representing the image Ic including the image Ic50 of the curved mirror 50 reflecting the moving object (object of interest) is obtained. Execute repeatedly. During this time, the vehicle position information, mirror image information, mirror position information (vehicle coordinate system), and mirror direction information (vehicle coordinate system) stored in the storage unit 13 are sequentially updated as the vehicle 100 moves.
　情報処理ユニット１１は、ミラー画像情報にて表される画像Ｉｃに含まれるカーブミラー５０の像Ｉｃ５０に、そのカーブミラー５０に映った移動物体の像が含まれていると判定すると（Ｓ１５でＹＥＳ：物体像検出部）、以下、前記ミラー画像情報にて表される画像Ｉｃにおいて検出された前記移動物体の像の情報、この時点で記憶部１３に格納されているミラー湾曲情報、ミラー位置情報、及び前記ミラー向き情報に基づいて前記移動物体の位置を表す物体位置情報を生成する処理を実行する（物体位置検出部）。この処理は、次のように行われる（Ｓ１６～Ｓ２４）。 When the information processing unit 11 determines that the image Ic50 of the curved mirror 50 included in the image Ic represented by the mirror image information includes the image of the moving object reflected on the curved mirror 50 (YES in S15 : object image detection unit), hereinafter, information on the image of the moving object detected in the image Ic represented by the mirror image information, mirror curvature information and mirror position information stored in the storage unit 13 at this time , and a process of generating object position information representing the position of the moving object based on the mirror direction information (object position detection unit). This process is performed as follows (S16-S24).
　情報処理ユニット１１は、図８に示すように取得された前記ミラー画像情報にて表される画像Ｉｃから抽出された移動物体２００の像部分Ｉｃ２００のサイズ（例えば、高さｍ）と、その像部分Ｉｃ２００のカーブミラー５０の像Ｉｃ５０における鏡面内での位置（カメラ内座標系）とを算出する（Ｓ１６）。そして、情報処理ユニット１１は、画像Ｉｃ内での移動物体２００の像部分Ｉｃ２００のサイズ（高さｍ）と、その画像部分Ｉｃ２００のカーブミラー５０の像Ｉｃ５０における鏡面内での位置とから、カメラ装置２０の光学的パラメータ（焦点距離、倍率、画角等）及び車両１００からカーブミラー５０までの距離を用いて、カメラ装置２０の撮影領域ＳＡ内のカーブミラー５０（ミラー本体５１の鏡面）に実際に映る移動物体２００の像Ｉｏのサイズ（例えば、高さｈ）と、カーブミラー５０（ミラー本体５１の鏡面）内でのその移動物体２００の像Ｉｏの位置（自車座標系）と、を推定演算する（Ｓ１７）。 The information processing unit 11 calculates the size (e.g., height m) of the image portion Ic200 of the moving object 200 extracted from the image Ic represented by the mirror image information obtained as shown in FIG. The position of the portion Ic200 within the mirror surface (in-camera coordinate system) in the image Ic50 of the curved mirror 50 is calculated (S16). Then, the information processing unit 11 determines the size (height m) of the image portion Ic200 of the moving object 200 in the image Ic and the position of the image portion Ic200 within the mirror surface in the image Ic50 of the curved mirror 50. Using the optical parameters (focal length, magnification, angle of view, etc.) of the device 20 and the distance from the vehicle 100 to the curved mirror 50, the curved mirror 50 (mirror surface of the mirror body 51) within the imaging area SA of the camera device 20 The size (e.g., height h) of the image Io of the moving object 200 that is actually reflected, the position of the image Io of the moving object 200 within the curved mirror 50 (mirror surface of the mirror body 51) (own vehicle coordinate system), is estimated and calculated (S17).
　その後、情報処理ユニット１１は、図３Ｂに示す手順に進み、カーブミラー５０に映る移動物体２００の像Ｉｏのサイズ（高さｈ）及びその像Ｉｏのカーブミラー５０（ミラー本体５１の鏡面）内での位置に基づいて、実物としての移動物体２００のサイズ（高さＨ）を求め、カーブミラー５０による拡大率（ｈ／Ｈ）を算出する（Ｓ１８）。ここで、例えば、種々の移動物体（自動車、自転車、人等）の実サイズ（高さＨ）とカーブミラー５０の種類（湾曲度：図５参照）毎の鏡面上の各位置での像Ｉｏのサイズ（高さｈ）との関係を予め機械学習させて、それらの関係をテーブル情報として記憶部１３に記憶させておく。そして、そのテーブル情報を参照して、カーブミラー５０の種類（湾曲度）、及び前記推定演算（Ｓ１７）により得られた移動物体２００の像Ｉｏの鏡面内位置及びサイズ（高さｈ）に対応した移動物体２００の実サイズ（高さＨ）を取得することができる。 After that, the information processing unit 11 proceeds to the procedure shown in FIG. Based on the position in , the size (height H) of the moving object 200 as a real object is obtained, and the enlargement ratio (h/H) by the curved mirror 50 is calculated (S18). Here, for example, images Io and the size (height h) are machine-learned in advance, and the relationship is stored in the storage unit 13 as table information. Then, referring to the table information, it corresponds to the type (curvature) of the curved mirror 50, and the position in the mirror surface and the size (height h) of the image Io of the moving object 200 obtained by the estimation calculation (S17). The actual size (height H) of the moving object 200 can be acquired.
　また、情報処理ユニット１１は、移動物体２００の像Ｉｏのカーブミラー５０（ミラー本体５１）の鏡面内位置及びミラー位置情報に基づいて、図９に示す、カメラ装置２０（車両１００）とカーブミラー５０（ミラー本体５１）における移動物体２００の像Ｉｏとを結ぶ入射線ＩＲを算出する（Ｓ１９）。更に、情報処理ユニット１１は、移動物体２００の像Ｉｏの位置でのカーブミラー５０（ミラー本体５１）の鏡面の法線ｎｐを算出する（Ｓ２０）。この法線ｎｐは、次のようにして算出することができる。 Further, the information processing unit 11, based on the position in the mirror surface of the curved mirror 50 (mirror body 51) of the image Io of the moving object 200 and the mirror position information, the camera device 20 (vehicle 100) and the curved mirror shown in FIG. An incident ray IR connecting the image Io of the moving object 200 on 50 (mirror body 51) is calculated (S19). Further, the information processing unit 11 calculates the normal line np of the mirror surface of the curved mirror 50 (mirror body 51) at the position of the image Io of the moving object 200 (S20). This normal np can be calculated as follows.
　前述したように、カーブミラー５０の鏡面５１ａ（ミラー平面５１ｂ：図６及び図７参照）の中心点を通る法線ベクトルｎから鏡面５１ａを含む球面５００ｌの中心Ｏが求められる。そして、その中心Ｏから延びて前述した移動物体２００の像Ｉｏの鏡面内位置を通る直線が法線ｎｐとして算出される。 As described above, the center O of the spherical surface 5001 including the mirror surface 51a is obtained from the normal vector n passing through the center point of the mirror surface 51a (mirror plane 51b: see FIGS. 6 and 7) of the curved mirror 50. Then, a straight line extending from the center O and passing through the position within the mirror surface of the image Io of the moving object 200 is calculated as the normal line np.
　このようにして、カーブミラー５０の拡大率（ｈ／Ｈ：倍率：Ｓ１８）、入射線ＩＲ（図９参照；Ｓ１９）、及び移動物体２００の像Ｉｏの鏡面内位置での法線ｎｐ（Ｓ２０）が得られると、情報処理ユニット１１は、カーブミラー５０（ミラー本体５１）の鏡面の拡大率（ｈ／Ｈ：倍率）及びミラー湾曲情報（曲率半径Ｒ：焦点距離ｆに対応）を用い、幾何光学の法則に従って、カーブミラー５０（ミラー本体５１）の鏡面における移動物体２００の像Ｉｏの位置と、実物としての移動物体２００との間の距離Ｄを推定演算する（Ｓ２１：距離演算部）。また、情報処理ユニット１１は、図９に示すように、入射線ＩＲ及び法線ｎｐを用い、鏡面での反射の法則（入射角θ＝反射角θ）に従って、反射線ＲＲ（ミラー反射方向）を算出する（Ｓ２２：反射方向演算部）。そして、情報処理ユニット１１は、反射線ＲＲ（方向）と、カーブミラー５０（ミラー本体５１）の鏡面における移動物体２００の像Ｉｏと実物としての移動物体２００との間の距離Ｄとに基づいて、移動物体２００の位置（自車座標系）を算出する（Ｓ２３）。そして、情報処理ユニット１１は、算出された位置に基づいて移動物体の位置情報（物体位置情報）を生成する（Ｓ２４）。 In this way, the magnification ratio (h/H: magnification: S18) of the curved mirror 50, the incident line IR (see FIG. 9; S19), and the normal np (S20 ) is obtained, the information processing unit 11 uses the magnification (h/H: magnification) of the mirror surface of the curved mirror 50 (mirror body 51) and the mirror curvature information (curvature radius R: corresponding to the focal length f), According to the law of geometrical optics, the distance D between the position of the image Io of the moving object 200 on the mirror surface of the curved mirror 50 (mirror body 51) and the actual moving object 200 is estimated and calculated (S21: distance calculation unit). . Further, as shown in FIG. 9, the information processing unit 11 uses the incident line IR and the normal line np to calculate the reflection line RR (mirror reflection direction) according to the law of reflection on a mirror surface (incident angle θ=reflection angle θ). is calculated (S22: reflection direction calculator). Based on the reflection line RR (direction) and the distance D between the image Io of the moving object 200 on the mirror surface of the curved mirror 50 (mirror body 51) and the real moving object 200, the information processing unit 11 , the position of the moving object 200 (own vehicle coordinate system) is calculated (S23). Then, the information processing unit 11 generates position information (object position information) of the moving object based on the calculated position (S24).
　情報処理ユニット１１は、所定の終了条件（例えば、走行停止、運転支援モードＯＦＦ等）が満たされるか否かを判定し（Ｓ２５）、終了条件が満たされていない（Ｓ２５でＮＯ）状況において、前述した処理（Ｓ１１～Ｓ２４：図３Ａ、図３Ｂ）を繰り返し実行する。その間、交差点Ｘやカーブ道路等、車両１００がカーブミラーの設置された場所を通過する毎に、上述した処理により得られるカーブミラーに映る移動物体（注目物体）の位置を表す物体位置情報が生成される。 The information processing unit 11 determines whether or not a predetermined termination condition (for example, stop driving, driving support mode OFF, etc.) is satisfied (S25), and in a situation where the termination condition is not satisfied (NO in S25), The above-described processes (S11 to S24: FIGS. 3A and 3B) are repeatedly executed. During that time, every time the vehicle 100 passes through a place where a curved mirror is installed, such as an intersection X or a curved road, object position information representing the position of the moving object (object of interest) reflected in the curved mirror obtained by the above-described processing is generated. be done.
　そして、情報処理ユニット１１は、そのカーブミラー５０に映る移動物体２００（注目物体）の位置を表す物体位置情報（自車座標系での位置を表す）が得られる毎に、その物体位置情報に基づいて車両１００の運転支援情報を生成する。生成された運転支援情報は、車速センサ２２からの信号に基づいて車両１００の走行速度を認識する情報処理ユニット１１の制御のもと、出力処理部１４から運転支援情報の種類に応じて情報出力部（表示器、音声出力器等）、車両１００における操舵系、アクセル系、及び制動系の各種アクチュエータを制御するＥＣＵ等に供給される。例えば、次のような運転支援情報を生成することができる。 Every time the information processing unit 11 obtains object position information (representing the position in the own vehicle coordinate system) representing the position of the moving object 200 (object of interest) reflected in the curved mirror 50, Based on this, driving support information for the vehicle 100 is generated. The generated driving support information is output from the output processing unit 14 according to the type of driving support information under the control of the information processing unit 11 that recognizes the traveling speed of the vehicle 100 based on the signal from the vehicle speed sensor 22. unit (display, audio output device, etc.), ECU for controlling various actuators of the steering system, the accelerator system, and the braking system in the vehicle 100, and the like. For example, the following driving assistance information can be generated.
　自車座標系での位置を表す前記物体位置情報を、自車位置情報（Ｓ１１参照）を用いて絶対座標系（緯度経度）での位置を表す位置情報に変換することができる。その位置情報を用いて移動物体２００（注目物体）の位置を表示器に道路地図とともに表示させる表示情報を運転支援情報として生成することができる。また、車両１００と移動物体２００（注目物体）との相対的な位置関係を表す物体位置情報に基づいて、交差点Ｘ（図１参照）を通過する際の車両１００における操舵系、アクセル系、及び制動系の各種アクチュエータを制御するＥＣＵに供給すべき制御信号を運転支援情報として生成することができる。また、車両１００と移動物体２００（注目物体）との相対的な位置関係を表す物体位置情報に基づいて、その相対的な位置関係に応じた警報情報を運転支援情報として生成することができる。 The object position information representing the position in the own vehicle coordinate system can be converted into position information representing the position in the absolute coordinate system (latitude and longitude) using the own vehicle position information (see S11). Display information for displaying the position of the moving object 200 (object of interest) on the display together with the road map can be generated as the driving support information using the position information. Further, based on the object position information representing the relative positional relationship between the vehicle 100 and the moving object 200 (object of interest), the steering system, accelerator system, and A control signal to be supplied to an ECU that controls various actuators of the braking system can be generated as driving support information. Also, based on the object position information representing the relative positional relationship between the vehicle 100 and the moving object 200 (object of interest), it is possible to generate warning information corresponding to the relative positional relationship as driving support information.
　上述したような車両用情報処理装置１０によれば、カーブミラー５０の像Ｉｃ５０を含む画像Ｉｃを表すミラー画像情報において検出された前記カーブミラー５０（ミラー本体５１）に映る移動物体２００の像Ｉｏの情報とともに、カーブミラー５０の鏡面の湾曲度を表すミラー湾曲情報、カーブミラー５０の位置を表すミラー位置情報、及びカーブミラー５０の鏡面の向く方向を表すミラー向き情報に基づいて、実物としての移動物体２００の位置を表す物体位置情報が生成される。このため、その物体位置情報に基づいて、カーブミラー５０に映る移動物体２００の実際の位置（実際の状況）に基づいた運転支援情報を提供することができる。 According to the vehicle information processing apparatus 10 as described above, the image Io of the moving object 200 reflected on the curved mirror 50 (mirror body 51) detected in the mirror image information representing the image Ic including the image Ic50 of the curved mirror 50 is detected. along with the information of the actual object based on mirror curvature information representing the degree of curvature of the mirror surface of the curve mirror 50, mirror position information representing the position of the curve mirror 50, and mirror orientation information representing the direction in which the mirror surface of the curve mirror 50 faces. Object position information representing the position of the moving object 200 is generated. Therefore, based on the object position information, driving support information based on the actual position (actual situation) of the moving object 200 reflected in the curved mirror 50 can be provided.
　前述した各位置や方向の情報は、自車座標系や絶対座標系において表されたが、これに限定されず、他の任意の座標系において表すことができる。 Although the information on each position and direction described above is expressed in the own vehicle coordinate system and the absolute coordinate system, it is not limited to this, and can be expressed in any other coordinate system.
　また、ミラー湾曲情報、ミラー位置情報、ミラー向き情報を取得するための手法は、前述した処理に限定されず、他の手法であってもよい。 Also, the method for acquiring the mirror curvature information, the mirror position information, and the mirror orientation information is not limited to the processing described above, and other methods may be used.
　次に、本発明の第２の実施の形態に係る車両用情報処理装置について説明する。 Next, a vehicle information processing apparatus according to a second embodiment of the present invention will be described.
　本発明の第２の実施の形態に係る車両用情報処理装置は、第１の実施の形態の場合と同様に、図２に示す車載システムにおける車両用情報処理装置１０として構成される。この車両用情報処理装置１０は、カーブミラー５０の設置された道路（交差点Ｘ等）での死角領域に基づいた運転支援情報を提供することを特徴としている。 A vehicle information processing apparatus according to the second embodiment of the present invention is configured as the vehicle information processing apparatus 10 in the in-vehicle system shown in FIG. 2, as in the case of the first embodiment. This vehicle information processing apparatus 10 is characterized by providing driving support information based on a blind spot area on a road (such as an intersection X) on which a curved mirror 50 is installed.
　例えば、図１０に示すように、道路Ｒ２に交差する道路Ｒ１をその交差点Ｘに向かって車両１００が走行する場合を想定する。そして、車両１００の道路Ｒ２側の視界を確保するために車両１００から見通しのきく交差点Ｘのコーナ部にカーブミラー５０（ミラー本体５１）が設置されている。 For example, as shown in FIG. 10, it is assumed that the vehicle 100 travels toward the intersection X on the road R1 that intersects the road R2. A curved mirror 50 (mirror body 51) is installed at the corner of the intersection X where the vehicle 100 can see the road R2 side of the vehicle 100 in order to secure the visibility of the vehicle 100 on the road R2 side.
　車両１００に搭載された車載システムにおける車両情報処理装置１０の情報処理ユニット１１は、車両１００に対する死角領域を表す死角領域情報を生成する（死角領域検出部）。この処理は、図１１Ａ及び図１１Ｂに示す手順に従って実行される。 The information processing unit 11 of the vehicle information processing device 10 in the in-vehicle system mounted on the vehicle 100 generates blind area information representing the blind area with respect to the vehicle 100 (blind area detection unit). This processing is executed according to the procedure shown in FIGS. 11A and 11B.
　図１１Ａにおいて、情報処理ユニット１１は、第１の実施の形態の場合（図３Ａ参照）と同様に、カメラ装置２０から画像処理部１２を経た画像情報を取得するとともに、ＧＰＳユニット２３からの信号に基づいて車両１００の位置（地図上の位置）を表す自車位置情報を取得する（Ｓ３１）。取得された画像情報及び自車位置情報は記憶部１３に記憶される。情報処理ユニット１１は、カーブミラー５０の像を含む画像を表す画像情報が取得される（Ｓ３２）まで、画像情報及び自車位置情報を繰り返し取得し、取得されたそれらの情報に基づいて記憶部１３に格納される画像情報及び自車位置情報を更新する。 In FIG. 11A, the information processing unit 11 acquires image information from the camera device 20 via the image processing unit 12, as in the case of the first embodiment (see FIG. 3A), and also acquires the signal from the GPS unit 23. (S31). The acquired image information and vehicle position information are stored in the storage unit 13 . The information processing unit 11 repeatedly acquires the image information and the vehicle position information until the image information representing the image including the image of the curved mirror 50 is acquired (S32). 13 to update the image information and the own vehicle position information.
　情報処理ユニット１１は、カーブミラー５０の像を含む画像を表す画像情報が得られると（Ｓ３２でＹＥＳ）、第１の実施の形態と同様の手法に従って、カーブミラー５０の位置を表すミラー位置情報を取得する（Ｓ３３：第２情報取得部）。また、情報処理ユニット１１は、カーブミラー５０（ミラー本体５１）の鏡面の向く方向を表すミラー向き情報を取得する（第３情報取得部）とともにカーブミラー５０（ミラー本体５１）の鏡面の湾曲度を表すミラー湾曲情報を取得する（第１情報取得部：Ｓ３４）。それら、ミラー位置情報、ミラー向き情報及びミラー湾曲情報は記憶部１３に記憶される。 When image information representing an image including the image of curved mirror 50 is obtained (YES in S32), information processing unit 11 obtains mirror position information representing the position of curved mirror 50 according to the same method as in the first embodiment. (S33: second information acquisition unit). In addition, the information processing unit 11 acquires (third information acquisition unit) mirror orientation information representing the direction in which the mirror surface of the curved mirror 50 (mirror body 51) faces, and also acquires the degree of curvature of the mirror surface of the curved mirror 50 (mirror body 51). (first information acquisition unit: S34). The mirror position information, mirror orientation information, and mirror curvature information are stored in the storage unit 13 .
　その後、情報処理ユニット１１は、図１１Ｂに示す手順に移行する。 After that, the information processing unit 11 proceeds to the procedure shown in FIG. 11B.
　図１０に示す状況において、情報処理ユニット１１は、交差点Ｘの手前左側角部の障害物点Ａの位置を算出する（Ｓ３５）。具体的には、第１の実施の形態におけるカーブミラー５０の位置を算出した手法と同様に、カーブミラー５０の像Ｉｃ５０を含む画像Ｉｃを表すミラー画像情報からカメラ装置２０の光学的パラメータ（焦点距離、倍率、画角等）を用いて、画像Ｉｃに対応する撮影領域ＳＡ内の物（障害物点Ａを含む）の２次元的な位置関係が推定演算される。そして、その撮影領域ＳＡ内の物（障害物点Ａ）の二次元的な位置関係と、ステレオカメラ２１からの３次元画像情報（奥行の情報を含む）から得られる車両１００から障害物点Ａまでの距離情報（奥行の情報）とに基づいて、車両１００の位置を基準とした障害物点Ａの位置（位置情報）が算出される。 In the situation shown in FIG. 10, the information processing unit 11 calculates the position of the obstacle point A at the front left corner of the intersection X (S35). Specifically, similarly to the method of calculating the position of the curved mirror 50 in the first embodiment, the optical parameters (focus distance, magnification, angle of view, etc.), the two-dimensional positional relationship of objects (including the obstacle point A) within the imaging area SA corresponding to the image Ic is estimated and calculated. Obstacle point A from vehicle 100 obtained from the two-dimensional positional relationship of objects (obstacle point A) within photographing area SA and three-dimensional image information (including depth information) from stereo camera 21 . The position (position information) of the obstacle point A with respect to the position of the vehicle 100 is calculated based on the distance information (depth information).
　なお、交差点Ｘにおける各角部の位置（地図上での位置（緯度・経度））が地図情報に埋め込まれている場合、その地図情報から車両５０が向かう交差点Ｘの手前左側角部の位置を取得するようにしてもよい。また、交差点Ｘにおいて、その交差点Ｘの各角部の位置をビーコンにより発信させておき、交差点Ｘに近づいた車両１００においてそのビーコンを受信することにより、交差点Ｘの手前左側角部の位置を取得することもできる。 If the position of each corner at the intersection X (position (latitude and longitude) on the map) is embedded in the map information, the position of the left corner before the intersection X to which the vehicle 50 is headed can be determined from the map information. It may be acquired. At the intersection X, the position of each corner of the intersection X is transmitted by a beacon, and the beacon is received by the vehicle 100 approaching the intersection X, thereby obtaining the position of the front left corner of the intersection X. You can also
　次に、情報処理ユニット１１は、障害物点Ａの位置（自車座標系）に基づいて、その障害物点Ａで限界となる車両１００から直接視認可能な限界線（直接視界限界線Ｌｄｖ）を表す直接視界限界線情報を生成する（Ｓ３６：直接視界限界検出部）。なお、自車位置情報に基づいて、前記直接視界限界線Ｌｄｖ（直接視界限界線情報）は、絶対座標系において表すこともできる。 Next, based on the position of the obstacle point A (in the own vehicle coordinate system), the information processing unit 11 calculates a limit line (direct visibility limit line Ldv) that is directly visible from the vehicle 100 that is the limit at the obstacle point A. (S36: direct visibility limit detector). Note that the direct line of sight Ldv (direct line of sight line information) can also be expressed in an absolute coordinate system based on the vehicle position information.
　更に、情報処理ユニット１１は、ミラー位置情報（自車座標系）、ミラー画像情報にて表される画像Ｉｃに含まれるカーブミラー５０（ミラー本体５１）の像部分Ｉｃ５０に基づき、カメラ装置２０の光学的パラメータ（焦点距離、倍率、画角等）を用いて、車両１００の視野範囲における当該車両１００からカーブミラー５０（ミラー本体５１）の鏡面の最外方エッジ点に至る視野線Ｌｍを算出する（Ｓ３７）。そして、情報処理ユニット１１は、前記視野線Ｌｍの前記カーブミラー５０（ミラー本体５１）での反射線であって、カーブミラー５０（ミラー本体５１）により視認可能な領域の限界線であるミラー視界限界線Ｌｍｖ（ミラー視界限界線情報）を算出する（Ｓ３８：ミラー視界限界検出部）。具体的には、前述したように、カーブミラー５０（ミラー本体５１）の鏡面を含む球面５００（図６参照）の中心Ｏが求められ、その中心Ｏから延びて前述したカーブミラー５０（ミラー本体５１）の鏡面の最外方エッジ点を通る法線ベクトルｎｅが求められる。そして、前記視野線Ｌｍ及び法線ベクトルｎｅを用い、反射の原理（幾何光学の法則）に従って、前記ミラー視界限界線Ｌｍｖ（ミラー視界限界線情報）が算出される。 Further, the information processing unit 11 uses the image portion Ic50 of the curved mirror 50 (mirror main body 51) included in the image Ic represented by the mirror position information (own vehicle coordinate system) and the mirror image information. Using optical parameters (focal length, magnification, angle of view, etc.), the line of sight Lm from the vehicle 100 to the outermost edge point of the mirror surface of the curved mirror 50 (mirror body 51) in the visual range of the vehicle 100 is calculated. (S37). Then, the information processing unit 11 calculates the mirror field of view, which is the line of sight line Lm reflected by the curved mirror 50 (mirror body 51) and which is the limit line of the area visible by the curved mirror 50 (mirror body 51). A limit line Lmv (mirror view limit line information) is calculated (S38: mirror view limit detector). Specifically, as described above, the center O of the spherical surface 500 (see FIG. 6) including the mirror surface of the curved mirror 50 (mirror body 51) is obtained, and the curved mirror 50 (mirror body 51) extends from the center O. A normal vector ne passing through the outermost edge point of the mirror surface of 51) is obtained. Then, using the visual field line Lm and the normal vector ne, the mirror visual field limit line Lmv (mirror visual field limit line information) is calculated according to the principle of reflection (law of geometric optics).
　上述したようにミラー視界限界線Ｌｍｖ（ミラー視界限界線情報）が得られると、情報処理ユニット１１は、前記ミラー視界限界線Ｌｍｖと地図情報（道路Ｒ２を含む道路情報を表す）とに基づいて、前記ミラー視界限界線Ｌｍｖと道路Ｒ２の縁線との交点である障害物点Ｃの位置（位置情報）を算出する（Ｓ３９）。また、情報処理ユニット１１は、前記ミラー視界限界線Ｌｍｖと前記直接視界限界線Ｌｄｖとの交点である視界限界点Ｂの位置（位置情報）を算出する（Ｓ４０）。そして、情報処理ユニット１１は、前記障害物点Ａ、前記視界限界点Ｂ、及び前記障害物点Ｃから、それら３点で囲まれる三角形領域を死角領域Ｚｄとして表す死角領域情報を生成する（Ｓ４１）。 When the mirror visibility limit line Lmv (mirror visibility limit line information) is obtained as described above, the information processing unit 11, based on the mirror visibility limit line Lmv and map information (expressing road information including road R2) , the position (position information) of the obstacle point C, which is the intersection of the mirror visibility limit line Lmv and the edge line of the road R2, is calculated (S39). The information processing unit 11 also calculates the position (positional information) of the visibility limit point B, which is the intersection of the mirror visibility limit line Lmv and the direct visibility limit line Ldv (S40). Then, the information processing unit 11 generates, from the obstacle point A, the visibility limit point B, and the obstacle point C, blind area information representing a triangular area surrounded by these three points as a blind area Zd (S41 ).
　情報処理ユニット１１は、所定の終了条件（例えば、走行停止、運転支援モードＯＦＦ等）が満たれるか否かを判定し（Ｓ４２）、終了条件が満たされていない（Ｓ４２でＮＯ）状況において、前述した処理（Ｓ３１～Ｓ４１：図１１Ａ、図１１Ｂ）を繰り返し実行する。その間、車両１００が交差点Ｘに近づくに従って順次狭くなる死角領域Ｚｄを表す死角領域情報が生成される。 The information processing unit 11 determines whether or not a predetermined termination condition (for example, stop driving, driving support mode OFF, etc.) is satisfied (S42), and in a situation where the termination condition is not satisfied (NO in S42), The above-described processes (S31 to S41: FIGS. 11A and 11B) are repeatedly executed. During this time, blind spot area information representing a blind spot area Zd that gradually narrows as the vehicle 100 approaches the intersection X is generated.
　そして、情報処理ユニット１１は、順次狭くなる死角領域Ｚｄを表す死角領域情報が得られる毎に、その死角領域情報に基づいて車両１００の運転支援情報を生成する。生成された運転支援情報は、車速センサ２２からの信号に基づいて車両１００の走行速度を認識する情報処理ユニット１１の制御のもと、出力処理部１４から運転支援情報の種類に応じて情報出力部（表示器、音声出力器等）、車両１００における操舵系、アクセル系、及び制動系の各種アクチュエータを制御するＥＣＵ等に供給される。例えば、次のような運転支援情報を生成することができる。 Then, the information processing unit 11 generates driving support information for the vehicle 100 based on the blind area information representing the blind area Zd that narrows in sequence each time the blind area information is obtained. The generated driving support information is output from the output processing unit 14 according to the type of driving support information under the control of the information processing unit 11 that recognizes the traveling speed of the vehicle 100 based on the signal from the vehicle speed sensor 22. unit (display, audio output device, etc.), ECU for controlling various actuators of the steering system, the accelerator system, and the braking system in the vehicle 100, and the like. For example, the following driving assistance information can be generated.
　死角領域情報とともに道路地図に基づいて、車両１００、道路Ｒ１と道路Ｒ２との交差点Ｘ、死角領域Ｚｄを含む表示情報を運転支援情報として生成することができる。これにより、例えば、図１０に示すような画像を表示器に表示させることができる。また、交差点Ｘに近づくに従って徐々に狭くなる死角領域Ｚｄを表す死角領域情報に基づいて、特に、図１２に示すような停止線の無い交差点Ｘにおいて、減速・停止すべき車両１００における操舵系、アクセル系、及び制動系の各種アクチュエータを制御するＥＣＵに供給すべき制御信号を運転支援情報として生成することができる。また、前記死角領域情報に基づいて、その死角領域Ｚｄの存在を知らせる警報情報（マーク、音声）を運転支援情報として生成することができる。 Display information including the vehicle 100, the intersection X between the roads R1 and R2, and the blind area Zd can be generated as driving support information based on the blind area information and the road map. As a result, for example, an image as shown in FIG. 10 can be displayed on the display. Further, based on the blind spot area information representing the blind spot area Zd that gradually narrows as it approaches the intersection X, the steering system of the vehicle 100 to decelerate and stop particularly at the intersection X without a stop line as shown in FIG. A control signal to be supplied to an ECU that controls various actuators of an accelerator system and a braking system can be generated as driving support information. Further, based on the blind spot information, it is possible to generate warning information (mark, sound) for notifying the presence of the blind spot Zd as driving support information.
　上述したような車両用情報処理装置１０によれば、カーブミラー５０（ミラー本体５１）の鏡面の湾曲度を表すミラー湾曲情報、カーブミラー５０の位置を表すミラー位置情報、カーブミラー５０の鏡面の向きを表すミラー向き情報とともに地図情報に基づいて、道路Ｒ１と道路Ｒ２との交差点Ｘにおいて形成される車両に対する死角領域Ｚｄを表す死角領域情報が生成される。このため、その死角領域情報に基づいて、車両に対して実際に形成される死角領域（カーブミラー５０に映る実際の状況から得られる）に基づいた運転支援情報を提供することができる。 According to the vehicle information processing apparatus 10 as described above, mirror curvature information representing the degree of curvature of the mirror surface of the curve mirror 50 (mirror body 51), mirror position information representing the position of the curve mirror 50, Blind spot area information representing a blind spot area Zd for the vehicle formed at the intersection X between the road R1 and the road R2 is generated based on the map information together with the mirror direction information representing the direction. Therefore, based on the blind spot information, it is possible to provide driving support information based on the blind spot actually formed with respect to the vehicle (obtained from the actual situation reflected in the curved mirror 50).
　第２の実施の形態に係る車両用情報処理装置１０においても、各位置や方向、また各種直線の情報は、自車座標系、絶対座標系、及び他の任意の座標系のいずれにおいても表すことができる。 In the vehicle information processing apparatus 10 according to the second embodiment, each position, direction, and various straight line information can be expressed in any of the own vehicle coordinate system, the absolute coordinate system, and any other arbitrary coordinate system. be able to.
　なお、第１の実施の形態に係る車両用情報処理装置１０では物体位置情報が生成され、第２の実施の形態に係る車両用情報処理装置１０では死角領域情報が生成されたが、前記物体位置情報及び死角領域情報の双方が生成され、それらの２種の情報に基づいた運転支援情報を提供するものであってもよい。 Although object position information is generated in the vehicle information processing apparatus 10 according to the first embodiment and blind area information is generated in the vehicle information processing apparatus 10 according to the second embodiment, the object Both position information and blind spot information may be generated, and driving assistance information based on these two types of information may be provided.
　また、前述した各実施の形態では、交差点Ｘにカーブミラー５０が設置される状況について説明したが、これに限定されない。他の状況、例えば、カーブ道路にカーブミラーが設けられた状況であっても、前述した車両用状処理装置１０は、同様の手法により、カーブミラーに映る移動物体の現実の位置を表す物体位置情報、あるいは、車両に対する死角領域を表す死角領域情報を生成することができる。 Also, in each of the above-described embodiments, the situation in which the curved mirror 50 is installed at the intersection X has been described, but the present invention is not limited to this. Even in other situations, such as a situation where a curved mirror is provided on a curved road, the above-described vehicle condition processing device 10 uses the same method to obtain an object position representing the actual position of a moving object reflected on the curved mirror. Information, or blind area information representing the blind area for the vehicle, can be generated.
　本発明に係る車両用情報処理装置は、カーブミラーに映る実際の状況に基づいた運転支援情報を提供することができるという効果を有し、カーブミラーの設けられた道路を走行する車両に搭載される車両用情報処理装置として有用である。 INDUSTRIAL APPLICABILITY A vehicle information processing apparatus according to the present invention has the effect of being able to provide driving assistance information based on the actual situation reflected in a curved mirror, and is installed in a vehicle traveling on a road provided with a curved mirror. It is useful as an information processing device for vehicles.
　１０　車両用情報処理装置
　１１　情報処理ユニット
　１２　画像処理部
　１３　記憶部
　１４　出力処理部
　２０　カメラ装置
　２１　ステレオカメラ
　２２　車速センサ
　２３　ＧＰＳユニット
　３０　ナビゲーション装置
　１００　車両
　２００　移動物体（注目物体） REFERENCE SIGNS LIST 10 vehicle information processing device 11 information processing unit 12 image processing unit 13 storage unit 14 output processing unit 20 camera device 21 stereo camera 22 vehicle speed sensor 23 GPS unit 30 navigation device 100 vehicle 200 moving object (object of interest)

Claims (8)

  1. 　車両に搭載されるカメラ装置により得られる道路に設置されたカーブミラーの像を含む画像を表すミラー画像情報に基づいて前記車両の運転を支援するための運転支援情報を生成する車両用情報処理装置であって、
    　前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報に基づいて当該カーブミラーの鏡面の湾曲度を表すミラー湾曲情報を取得する第１情報取得部と、
    　前記カーブミラーの位置を表すミラー位置情報を取得する第２情報取得部と、
    　前記ミラー湾曲情報及び前記カーブミラーの像の情報に基づいて車両に対する前記カーブミラーの鏡面の向く方向を表すミラー向き情報を取得する第３情報取得部と、
    　前記ミラー画像情報にて表される画像に含まれる前記カーブミラーに映った注目物体の像を検出する物体像検出部と、
    　前記ミラー画像情報にて表される前記画像において検出された前記注目物体の像の情報、前記ミラー湾曲情報、前記ミラー位置情報、及び前記ミラー向き情報に基づいて前記注目物体の位置を表す物体位置情報を生成する物体位置検出部と、を有し、
    　前記物体位置検出部にて得られる前記物体位置情報に基づいて前記運転支援情報を生成する、車両用情報処理装置。     A vehicle information processing device for generating driving assistance information for assisting the driving of the vehicle based on mirror image information representing an image including an image of a curved mirror installed on a road obtained by a camera device mounted on the vehicle. and
    a first information acquisition unit that acquires mirror curvature information representing the degree of curvature of the mirror surface of the curved mirror based on information of the image of the curved mirror included in the image represented by the mirror image information;
    a second information acquisition unit that acquires mirror position information representing the position of the curved mirror;
    a third information acquiring unit that acquires mirror orientation information representing the direction in which the mirror surface of the curved mirror faces the vehicle based on the information on the curvature of the mirror and the information on the image of the curved mirror;
    an object image detection unit that detects an image of an object of interest reflected on the curved mirror included in the image represented by the mirror image information;
    An object position representing the position of the object of interest based on information on the image of the object of interest detected in the image represented by the mirror image information, the information on the curvature of the mirror, the information on the position of the mirror, and the information on the direction of the mirror. an object position detection unit that generates information,
    An information processing device for a vehicle that generates the driving support information based on the object position information obtained by the object position detection unit.
  2. 　前記第１情報取得部は、
    　複数種のカーブミラーのそれぞれの外形形状の特徴を表す外形特徴情報とそれに対応する湾曲度とが定められたテーブル情報を記憶する記憶部と、
    　前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報から前記カーブミラーの外形形状の特徴を表す外形特徴情報を取得するミラー外形特徴取得部と、
    　前記テーブル情報において前記ミラー外形特徴取得部により取得された前記外形特徴情報に対応する湾曲度を、前記ミラー湾曲情報が表すべき湾曲度として決定する湾曲度決定部と、を有する請求項１記載の車両用情報処理装置。     The first information acquisition unit
    a storage unit for storing table information in which external feature information representing features of the external shape of each of a plurality of types of curved mirrors and degrees of curvature corresponding thereto are defined;
    a mirror outer shape feature acquisition unit that acquires outer shape feature information representing features of the outer shape of the curved mirror from information of the image of the curved mirror included in the image represented by the mirror image information;
    2. The curvature determination unit according to claim 1, further comprising a curvature determination unit configured to determine, in the table information, the curvature corresponding to the external feature information acquired by the mirror external feature acquisition unit as the curvature to be represented by the mirror curvature information. Vehicle information processing device.
  3. 　前記第２情報取得部は、
    　前記車両から前記カーブミラーまでの距離を表すミラー距離情報を取得する距離取得部と、
    　前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報と前記距離取得部により取得された前記ミラー距離情報とに基づいて前記ミラー位置情報を生成するミラー位置検出部と、を有する請求項１または２記載の車両用情報処理装置。     The second information acquisition unit
    a distance acquisition unit that acquires mirror distance information representing the distance from the vehicle to the curved mirror;
    a mirror position detection unit configured to generate the mirror position information based on information of the curved mirror image included in the image represented by the mirror image information and the mirror distance information acquired by the distance acquisition unit; 3. The vehicle information processing device according to claim 1 or 2, comprising:
  4. 　前記第３情報取得部は、
    　前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報と前記ミラー湾曲情報とに基づいて、前記カーブミラーの鏡面の中心点の法線方向を前記ミラー向き情報として生成するミラー法線演算部とを有する請求項１乃至３のいずれかに記載の車両用情報処理装置。     The third information acquisition unit
    A normal direction of a center point of the mirror surface of the curved mirror is generated as the mirror orientation information based on information on the image of the curved mirror and the information on the curvature of the mirror included in the image represented by the mirror image information. 4. The vehicle information processing apparatus according to any one of claims 1 to 3, further comprising a mirror normal calculation section.
  5. 　前記物体位置検出部は、
    　前記ミラー画像情報にて表される画像における前記カーブミラーに映る前記注目物体の像の情報と前記ミラー湾曲情報とに基づいて前記カーブミラーと実物としての前記注目物体との距離を演算する距離演算部と、
    　前記ミラー画像情報にて表される画像における前記カーブミラーに映る前記注目物体の像の情報、前記ミラー位置情報、及び前記ミラー向き情報に基づいて、前記車両からの視線の前記カーブミラーに映る前記注目物体の像から実物としての前記注目物体に向う方向を表すミラー反射方向を演算する反射方向演算部と、を有し、
    　前記距離演算部にて得られる前記カーブミラーと実物としての前記注目物体との距離と前記ミラー反射方向とに基づいて前記物体位置情報を生成する、請求項１乃至４のいずれかに記載の車両用情報処理装置。     The object position detection unit is
    Distance calculation for calculating a distance between the curved mirror and the object of interest as a real object based on the information of the image of the object of interest reflected on the curved mirror in the image represented by the mirror image information and the information on the curvature of the mirror. Department and
    Based on the information of the image of the object of interest reflected on the curved mirror in the image represented by the mirror image information, the mirror position information, and the mirror direction information, a reflection direction calculation unit that calculates a mirror reflection direction representing a direction toward the target object as a real object from the image of the target object,
    5. The vehicle according to any one of claims 1 to 4, wherein the object position information is generated based on the distance between the curved mirror and the object of interest as a real object obtained by the distance calculation unit and the reflection direction of the mirror. Information processing equipment for
  6. 　更に、前記ミラー湾曲情報、前記ミラー位置情報、前記ミラー向き情報及び車両走行用の道路を表す道路情報を含む地図情報に基づいて、注目する道路において形成される前記車両に対する死角領域を表す死角領域情報を生成する死角領域検出部を有し、
    　前記死角領域検出部により得られる前記死角領域情報に基づいて前記運転支援情報を生成する、請求項１乃至５のいずれかに記載の車両用情報処理装置。     Further, a blind spot area representing a blind spot area for the vehicle formed on a road of interest based on map information including the mirror curvature information, the mirror position information, the mirror orientation information, and road information representing a road on which the vehicle travels. Having a blind area detection unit that generates information,
    6. The vehicle information processing apparatus according to any one of claims 1 to 5, wherein said driving support information is generated based on said blind spot area information obtained by said blind spot area detection unit.
  7. 　車両に搭載されるカメラ装置により得られる道路に設置されたカーブミラーの像を含む画像を表すミラー画像情報に基づいて前記車両の運転を支援するための運転支援情報を生成する車両用情報処理装置であって、
    　前記ミラー画像情報にて表される画像に含まれる前記カーブミラーの像の情報に基づいて当該カーブミラーの鏡面の湾曲度を表すミラー湾曲情報を取得する第１情報取得部と、
    　前記カーブミラーの位置を表すミラー位置情報を取得する第２情報取得部と、
    　前記ミラー湾曲情報及び前記カーブミラーの像の情報に基づいて車両に対する前記カーブミラーの鏡面の向く方向を表すミラー向き情報を取得する第３情報取得部と、
    　前記ミラー湾曲情報、前記ミラー位置情報、前記ミラー向き情報、前記車両位置情報及び車両走行用の道路を表す道路情報を含む地図情報に基づいて、注目する道路において形成される前記車両に対する死角領域を表す死角領域情報を生成する死角領域検出部と、を有し、
    　前記死角領域検出部により得られる前記死角領域情報に基づいて前記運転支援情報を生成する、車両用情報処理装置。     A vehicle information processing device for generating driving assistance information for assisting the driving of the vehicle based on mirror image information representing an image including an image of a curved mirror installed on a road obtained by a camera device mounted on the vehicle. and
    a first information acquisition unit that acquires mirror curvature information representing the degree of curvature of the mirror surface of the curved mirror based on information of the image of the curved mirror included in the image represented by the mirror image information;
    a second information acquisition unit that acquires mirror position information representing the position of the curved mirror;
    a third information acquiring unit that acquires mirror orientation information representing the direction in which the mirror surface of the curved mirror faces the vehicle based on the information on the curvature of the mirror and the information on the image of the curved mirror;
    Based on the map information including the mirror curvature information, the mirror position information, the mirror direction information, the vehicle position information, and the road information representing the road on which the vehicle travels, a blind spot area for the vehicle formed on the road of interest is determined. and a blind area detection unit that generates blind area information representing
    A vehicle information processing device that generates the driving support information based on the blind area information obtained by the blind area detection unit.
  8. 　前記死角領域検出部は、
    　前記地図情報に基づいて前記注目する道路における前記車両から直接視認可能な領域の限界線を表す直接視界限界線情報を生成する直接視界限界検出部と、
    　前記地図情報、前記ミラー湾曲情報、前記ミラー位置情報、前記ミラー向き情報に基づいて前記注目する道路における前記車両から前記カーブミラーにより視認可能な領域の限界線を表すミラー視界限界線情報を生成するミラー視界限界検出部と、を有し、
    　前記直接視界限界線情報及び前記ミラー視界限界線情報に基づいて前記死角領域情報を生成する、請求項６または７記載の車両用情報処理装置。



    　     The blind area detection unit is
    a direct visibility limit detection unit that generates, based on the map information, direct visibility limit line information representing a limit line of an area of the road of interest that is directly visible from the vehicle;
    Mirror visibility limit line information representing a limit line of an area on the road of interest that can be visually recognized from the vehicle through the curved mirror is generated based on the map information, the mirror curvature information, the mirror position information, and the mirror direction information. and a mirror visibility limit detection unit,
    8. The vehicle information processing apparatus according to claim 6, wherein said blind spot area information is generated based on said direct visibility limit line information and said mirror visibility limit line information.
PCT/JP2021/033632 2021-05-25 2021-09-14 Vehicle information processing device WO2022249504A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021087846A JP7422704B2 (en) 2021-05-25 2021-05-25 Vehicle information processing device
JP2021-087846 2021-05-25

Publications (1)

Publication Number Publication Date
WO2022249504A1 true WO2022249504A1 (en) 2022-12-01

Family

ID=84229651

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/033632 WO2022249504A1 (en) 2021-05-25 2021-09-14 Vehicle information processing device

Country Status (2)

Country Link
JP (1) JP7422704B2 (en)
WO (1) WO2022249504A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009211624A (en) * 2008-03-06 2009-09-17 Aisin Aw Co Ltd Driving support device, driving support method, and computer program
JP2010122821A (en) * 2008-11-18 2010-06-03 Fujitsu Ten Ltd Vehicle driving support device
JP2017162178A (en) * 2016-03-09 2017-09-14 パナソニックＩｐマネジメント株式会社 Determination device, determination method and determination program
JP2019204139A (en) * 2018-05-21 2019-11-28 株式会社ゼンリン Control system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009211624A (en) * 2008-03-06 2009-09-17 Aisin Aw Co Ltd Driving support device, driving support method, and computer program
JP2010122821A (en) * 2008-11-18 2010-06-03 Fujitsu Ten Ltd Vehicle driving support device
JP2017162178A (en) * 2016-03-09 2017-09-14 パナソニックＩｐマネジメント株式会社 Determination device, determination method and determination program
JP2019204139A (en) * 2018-05-21 2019-11-28 株式会社ゼンリン Control system

Also Published As

Publication number Publication date
JP2022181025A (en) 2022-12-07
JP7422704B2 (en) 2024-01-26

Similar Documents

Publication Publication Date Title
US10878256B2 (en) Travel assistance device and computer program
EP3091338B1 (en) Misrecognition determination device
US11535155B2 (en) Superimposed-image display device and computer program
US8094192B2 (en) Driving support method and driving support apparatus
JP6189815B2 (en) Traveling line recognition system
EP1961613B1 (en) Driving support method and driving support device
JP6819681B2 (en) Imaging control devices and methods, and vehicles
JP5454695B2 (en) Risk calculation device
JP6981377B2 (en) Vehicle display control device, vehicle display control method, and control program
JP2004056763A (en) Monitoring apparatus, monitoring method, and program for monitor
JP2009211624A (en) Driving support device, driving support method, and computer program
JP2009184554A (en) Safe travel assisting system
KR20200131832A (en) Information processing devices, mobile devices and methods, and programs
JP2007102691A (en) View-field support apparatus for vehicle
JP3857698B2 (en) Driving environment recognition device
JP2007257299A (en) Vehicle external recognition system
JP2009105741A (en) In-vehicle perimeter monitoring system
JP2013054496A (en) Display device for vehicle
WO2022249504A1 (en) Vehicle information processing device
JP2006164197A (en) Running support system
JP6798133B2 (en) Driving support method and driving support device
JP2008305283A (en) Drive support apparatus and drive support method
JP2006010328A (en) Vehicle position identify device
JP2022086686A (en) Target detection device and vehicle equipped with the same
JP4234071B2 (en) Vehicle road surface sign detection device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21943138

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE