JP2008060988A - Apparatus for acquiring travel environment information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the accuracy of object identification even for an object located far, to shorten the processing time, and to reduce cost by reducing the required performance of a processor. <P>SOLUTION: An obstacle detection device 100 uses and images an object in the moving direction of a mobile to acquire the object as travel environment information. An object detection unit 111 detects the position and the direction of the object, located in the moving direction of the mobile. A camera setting calculation unit 112 calculates, from the position and the direction of the object, the camera setting comprising the lens direction, the focal point of lens and zoom setting which are preset for each type of the object and with which imaging can be performed, in an object-recognizable image size; and a camera control unit 114 controls a camera 104 for imaging the object with the camera setting calculated for each object, and the image is acquired by an image acquisition unit 115. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a traveling environment information acquisition device that is mounted on a moving body such as an automobile and detects an obstacle by recognizing the environment around the moving body.

  2. Description of the Related Art Conventionally, in order to assist a driver, an apparatus that is mounted on a vehicle such as an automobile and recognizes an object that obstructs traveling that exists in the traveling direction of the host vehicle by image processing is known.

  As an apparatus for detecting an obstacle in this way, for example, a traveling environment recognition apparatus disclosed in Patent Document 1 is well known. In this traveling environment recognition device, the range of the object identification target is limited (specified) according to the distance and direction to the target object from the distance and direction to the target object detected by the radar and the image captured by the camera, Object identification is performed with images within the range.

  A process for identifying an object using an image captured by a conventional travel environment recognition apparatus will be described with reference to FIGS.

  FIG. 10 is a diagram showing the positional relationship between the traveling vehicle and the object, and FIG. 11 is a method for specifying the object range in the image from the positional relationship between the vehicle and the object according to the conventional method using the positional relationship of FIG. FIG. 12 is a conceptual diagram showing a method for extracting an object image in an image from the positional relationship between the host vehicle and the object in the conventional method.

  In the traveling vehicle shown in FIG. 10, when performing object identification of the objects A to C located in the moving direction, the conventional traveling environment apparatus photographs the objects A to C existing in the traveling direction using a camera ( (See FIG. 11A). As shown in FIG. 11 (a), depending on one image 1 on which the objects A to C are displayed, the distance to the objects A to C acquired by the radar, and the directions of the objects A to C, FIG. As shown, the object recognition ranges 2 to 4 are set for the objects A to C, respectively.

Next, as shown in FIG. 12, the object recognition ranges 2 to 4 are extracted from the screen 1, and each of them is subjected to conversion processing (shown as enlarged images 6 to 8 in FIG. 12) that can identify the objects. Execute the process.
JP 2004-265432 A

  In the object recognition method used in the traveling environment recognition apparatus disclosed in Patent Document 1, when the object recognition target is performed on the objects A to C as shown in FIGS. This is performed using images of the objects A to C.

  Therefore, in the conventional driving environment recognition device, a distant object appears small in the image, so the number of pixels to be configured is small and the outline of the object becomes unclear. There is a problem that separation is difficult and accuracy of object identification is lowered.

  In addition, when a high-resolution image is used, the number of pixels constituting the detection part of a distant object increases, but the number of pixels constituting a nearby object also increases, so that the processing time required for object identification increases. There is a problem.

  Furthermore, in the conventional driving environment recognition device, the processing time required for object identification is reduced by making the size of the image of the target portion the same (see FIG. 12). There is a problem that it takes a long processing time, resulting in an increase in the processing time.

  The present invention has been made in view of the above points, and provides a travel environment information acquisition device capable of improving the accuracy of object identification, shortening the processing time, and reducing the cost due to the reduction in the required performance of the processing processor. Objective.

  The traveling environment information acquisition device of the present invention is a traveling environment information acquisition device that is mounted on a moving body and acquires the object as traveling environment information using an image of the object in the moving direction of the moving body, Object detection means for detecting the position and direction of the target object in the moving direction of the moving body, and imaging with an image size that can be recognized by a target object set in advance for each type of target object from the position and direction of the target object A camera setting calculating means for calculating a camera setting composed of a lens orientation, a lens focus, and a zoom setting, and a camera that controls the camera and images the object with the camera setting calculated for each object. A configuration including a control unit and an image acquisition unit that acquires an image of an object captured by the camera based on the calculated camera setting is adopted.

  As described above, according to the present invention, it is possible to improve the accuracy of object identification, shorten the processing time, and reduce the cost by reducing the required performance of the processing processor.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, the obstacle detection is carried out by detecting the object using an image of the object that is mounted on the moving body and can be an obstacle in the moving direction of the moving body. The description will be applied to the apparatus.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of an obstacle detection apparatus 100 to which the traveling environment information acquisition apparatus according to Embodiment 1 of the present invention is applied.

  As illustrated in FIG. 1, the obstacle detection apparatus 100 includes an object detection sensor 102, a camera 104, a speaker 105, a monitor 106, and an obstacle detection processing unit 110.

  The obstacle detection processing unit 110 includes an object detection unit 111, a camera setting calculation unit 112, a shooting order calculation unit 113, a camera control unit 114, an image acquisition unit 115, an object recognition unit 116, and a risk determination unit. 117 and a notification unit 118.

  The object detection sensor 102 acquires the distance and direction from the current position to a plurality of objects to be identified using a camera, radar, or the like, and outputs the acquired information to the object detection unit 111. . When a radar or sonar is used as the object detection sensor 102, the direction and distance of the target object are determined from the reflection direction, the intensity of the reflected wave, and the arrival time of the reflected wave using the reflected wave of the radar or sonar. Will be calculated. With this configuration, it is possible to improve the accuracy of object identification of a distant object and reduce the processing time required for object identification.

  Based on the distance and direction information to a plurality of objects input from the object detection sensor 102, the object detection unit 111 is a set of objects that are within a specified distance range and that are within a specified direction range. Judge as one object. The determined information is output to the camera setting calculation unit 112 together with information from the object detection sensor 102.

  The distance range and the direction range to be specified are set from the allowable value of the length of the object to be identified by the object recognizing unit 116, for example, the allowable value of the total length in the case of an automobile. An allowable value of the width that the object can take from the distance to the object, for example, in the case of an automobile, a range within a viewing angle corresponding to the width and distance of the automobile is set as the allowable value.

  The camera setting calculation unit 112 is suitable for recognizing an object by the object recognition unit 116 when shooting using the camera 104 for each of a plurality of objects detected by the object detection unit 111 using the input information. Calculate the camera settings for the selected image size. This image size is an object recognizable in advance set for each type of object (pedestrian, car, etc.), and has the same resolution and the same size.

  Specifically, the camera setting calculation unit 112 determines, for each of a plurality of targets detected by the object detection unit 111, an image suitable for object recognition by the object recognition unit 116 based on the distance from the current position to the target. Calculation of the zoom setting value of the camera as the size and the setting value of the lens focus are calculated. In addition, the camera setting calculation unit 112 has an image size suitable for object recognition by the object recognition unit 116 from the direction from the current position with respect to the target for each of the plurality of objects detected by the object detection unit 111. Set values in the vertical and horizontal directions of the camera for shooting are calculated.

  The shooting order calculation unit 113 determines the shooting order of the objects based on the distance and direction based on the camera setting value for each target object calculated by the camera setting calculation unit 112 and the distance and direction to the target object.

  This photographing order calculation unit 113 reduces the load on the camera 104 and shortens the photographing time required for photographing all objects, shortens the processing time required for object identification, and shortens the time required for camera control. The camera life can be extended by reducing the load on the camera.

  The camera control unit 114 controls the camera 104 based on the information input from the shooting order calculation unit 113 to capture an object.

  Specifically, the camera control unit 114 sets the zoom of the camera 104, the focus of the lens, and the setting of the lens direction in accordance with the shooting order of the shooting order calculation unit 113 and the setting value of the camera for each object.

  In addition, the camera control unit 114 instructs the camera 104 to perform shooting with the set contents, controls the camera 104, and repeatedly executes shooting of an image of the target object for the number of objects.

  Further, the camera control unit 114 outputs the camera setting for each target object input from the imaging order calculation unit 113 to the image acquisition unit 115.

  The image acquisition unit 115 acquires an image captured by the camera 104 and also acquires the distance and direction to the target corresponding to the captured image from the camera control unit 114, and creates image information for the number of targets. . The image acquisition unit 115 saves the created image information for the number of objects and outputs it to the object recognition unit 116.

  The object recognition unit 116 identifies the target object from the target image acquired by the image acquisition unit 115, and generates object information from the distance and direction to the target object and the target identification result.

  The degree-of-risk determination unit 117 determines the degree of risk of the object based on the object information including the distance and direction to the object generated by the object recognition unit 116 and the identification result of the object.

  Specifically, using the type of the object recognized by the object recognition unit 116 and the distance and direction from the host vehicle to the object, the degree of risk is determined based on the type and distance of the object, the inside and outside of the same lane, and the like.

  For example, when the type of the object is a vehicle, the risk determination unit 117 is dangerous when the object is in the range of the own lane and the distance is within 30 m, and when the object is within the range of 30 m to 60 m. Attention, if it exceeds 60m, it will be judged as safe, if the object is outside the range of its own lane and the distance is within 30m, it will be judged as caution, and if it exceeds 30m, it will be judged as safe.

  Further, in the risk determination unit 117, when the type of the object is a person, the determination is made with half of the case of the vehicle. Note that the distance division for judging danger is different from the above example, and the same effect can be obtained with a small number of divisions or a larger number of divisions.

  The risk determination unit 117 having the above-described configuration may dynamically change the distance range based on the traveling speed of the host vehicle.

  Further, the determination whether the object to be determined is the same traveling lane as the traveling lane of the host vehicle may be performed by detecting a white line or a yellow line displayed on the road surface that distinguishes the lane from the image. Good. By using such a traveling lane detection method, it can be easily realized.

  The notification unit 118 creates a risk notification content based on the risk of collision between the host vehicle and the object determined by the risk determination unit 117, and notifies the driver from the speaker 105 or the monitor 106.

  Next, the operation of the obstacle detection apparatus 100 having the above configuration will be described with reference to FIGS.

  2 and 3 are flowcharts for explaining the operation of the obstacle detection apparatus 100 according to Embodiment 1 of the present invention. 2 shows processing up to the end of photographing, and FIG. 3 shows processing for performing object recognition using the acquired image of the target object.

  In step S201, in the obstacle detection apparatus 100, the object detection sensor 102 obtains the distance and direction from the current position of the host vehicle to the object existing in the moving direction of the host vehicle, and proceeds to step S202. In step S201, when the object detection sensor 102 detects a plurality of objects, the detection information on the distance and direction from the host vehicle to the object is output to the object detection unit 111 for each of the plurality of objects.

  In step S202, the object detection unit 111 is within the specified distance range from the current position and the direction from the current position for each target object input from the object detection sensor 102 to the target object, and the specified direction. The objects within the range are set as the same object set as one piece of object information, and the distance and direction to the detected object are calculated as the center point of the set of objects, and the process proceeds to step S203. Note that the object detection unit 111 calculates the position of the object by calculating the distance and the direction.

  In step S <b> 203, the camera setting calculation unit 112 matches the image size used for object recognition in the object recognition unit 116 for each target object based on the distance and direction to the target object input from the object detection unit 111. Then, a zoom setting value is calculated based on the distance to the object, and further, a lens focus setting value for adjusting the focal position of the lens is calculated from the zoom setting value and the distance to the object. Further, based on the direction of the object, a setting value for the camera orientation is calculated, and the process proceeds to step S204.

  In step S204, the imaging order calculation unit 113 calculates the camera zoom setting value, the lens focus setting value, the camera orientation setting value, or the target object calculated for each object input from the camera setting calculation unit 112. The order in which the camera setting change is minimized is calculated from the distance and the direction, and the shooting order of the object is calculated, and the process proceeds to step S205.

  Note that the calculation of the order that minimizes the setting change of the camera 104 in step S204 can be easily calculated by applying the shortest path search method. For example, it is possible to calculate in the order in which the amount of zoom setting change is minimized, and the time required for camera setting change is minimized from the time required for zoom setting, lens focus setting, and lens direction setting. It is possible to calculate the shooting order.

  In step S205, the camera control unit 114 performs zoom setting, lens focus setting, and lens direction setting on the camera 104 based on the camera setting of the object in accordance with the calculated shooting order, and the process proceeds to step S206. .

  In step S206, the camera control unit 114 instructs the camera 104 to shoot with the set content after completing the setting of the camera 104, and shoots an image of the object with the camera 104, and the process proceeds to step S207.

  In step S207, the image acquisition unit 115 acquires an image of the object photographed by the camera 104, acquires the distance and direction from the camera control unit 114 to the object corresponding to the object, and acquires the acquired object. The distance between the image and the object corresponding to the object is associated with the direction, and the process proceeds to step S208. In step S207, the image information of the target that is associated with the distance to the target by the image acquisition unit 115 in the direction is stored until the object recognition is completed.

  In step S <b> 208, the image acquisition unit 115 uses the image input from the camera 104 and the distance and direction to the target input from the camera control unit 114 to determine whether or not shooting of all the targets is complete. .

  In step S208, if the shooting of all objects has not been completed, the process returns to step S205, and the processes in and after step S205 are repeatedly executed to shoot the next object, and the shooting of all objects has been completed. If so, the shooting of the object ends.

  In this way, the obstacle detection apparatus 100 acquires all images of the object corresponding to the object detected by the object detection sensor 102.

  Next, the obstacle detection apparatus 100 performs object recognition processing as shown in FIG.

  In step S301 illustrated in FIG. 3, the object recognition unit 116 controls the image of the target object stored in the image acquisition unit 115 and the distance and direction to the target object corresponding to the image via the image acquisition unit 115. Acquired from the unit 114, and the process proceeds to step S302.

  In step S302, the object recognition unit 116 identifies the acquired image of the target object, and proceeds to step S303.

  In step S303, the risk level determination unit 117 determines the risk level of the target object from the type of target object identified by the object recognition unit 116, the distance to the target object, and the direction, and the process proceeds to step S304.

  In step S304, the risk determination unit 117 determines whether or not the danger notification can be performed based on the determination result of the risk of the object. If notification is to be made, the process proceeds to step S305 and the notification is not performed. In this case, the process proceeds to step S306.

  In step S305, the notification unit 118 receives the information from the risk determination unit 117, creates danger notification information from the type of object, the distance to the object, and the direction, and proceeds to step S306.

  In step S306, whether or not to end identification of all objects is determined. If not completed, the process returns to step S301, and the subsequent processing is repeatedly executed. If completed, in step S307, the notification unit 118 The danger notification is executed using the speaker 105, the monitor 106, or the like.

  FIG. 4 is a conceptual diagram illustrating an outline of the object detection process performed by the obstacle detection apparatus 100 according to the first embodiment. The positional relationship between the host vehicle on which the camera 104 shown in FIG. 4 is mounted and the objects A to C is the same as the positional relationship shown in FIG. 10 in order to clarify the difference from the conventional method. And

  That is, the obstacle detection device 100 according to the first embodiment is mounted on the traveling vehicle as the host vehicle in the positional relationship of FIG. 10, and the obstacle detection device 100 exists in the moving direction. It is assumed that C images are acquired and object recognition processing is performed on each of them.

  As shown in FIG. 4, the obstacle detection apparatus 100 first recognizes an object and calculates camera settings for each object. In this camera setting, the direction setting, the focus setting, and the zoom setting are performed so that the image acquired by photographing the object according to the setting has an image size suitable for object recognition by the object recognition unit 116. .

  Then, according to the calculated camera settings (direction, focus, zoom setting), camera settings are made for the camera 104, and an object corresponding to the camera settings is photographed using the camera 104 set for the camera.

  Here, the objects A, B, and C are photographed in this order, and FIG. 4A shows the direction of the camera 104 when photographing the object A in the positional relationship shown in FIG. The object A image 11 photographed in FIG. FIG. 4B shows the object B image 12 photographed in this direction in the direction of the camera 104 when photographing the object B in the same positional relationship. 4 (3) shows the object C image 13 photographed in this direction in the direction of the camera 104 when photographing the object C in the same positional relationship.

  Here, photographing of the objects A to C in the positional relationship of FIG. 10 includes the amount of change in the lens direction, the amount of lens movement for focusing the lens, and the amount of lens movement for zoom setting in the camera when photographing each of them. Set to the minimum order. Here, it is assumed that the objects A, B, and C are imaged in this order.

  As shown in FIG. 4, the object images acquired by photographing each of the objects A to C with the camera 104, here, the object A image 11, the object B image 12, and the object C image 13 are each as they are. The image size is such that the object recognition process can be performed.

  The target object is recognized by executing the target object recognition process using these target object images 11, 12, and 13 without being processed.

  According to this configuration, the distance and direction to the target object are detected for all the target objects in the moving direction, and the lens direction of the camera 104 is adjusted according to the detected distance and direction for each target object. Change the focus and zoom settings and shoot.

  As a result, even an object located far away can be photographed with an image having a sufficient number of pixels, so that the object identification rate can be improved.

  In addition, since a sufficient image size can be secured with an image of a low-resolution camera, an object can be identified using a low-cost camera as the camera 104.

  Furthermore, the object recognizable image size set in advance for each type of the object is the same size with the same resolution, and the object is photographed with this size.

  For this reason, it is not necessary to convert the size of the image after imaging, so that the processing necessary for image conversion is not required, and the cost can be reduced due to the reduction in the required performance of the processing processor, or the processing time can be shortened. An obstacle detection device having an effect can be provided.

  As described above, according to the first embodiment, an image suitable for object identification can be acquired regardless of the distance and direction to the target object, and conversion of the image size suitable for object identification is not required. Therefore, it is possible to simplify image processing, improve the accuracy of object identification of a distant object, shorten the processing time, and reduce the cost by reducing the required performance of the processing processor. .

(Embodiment 2)
FIG. 5 is a block diagram showing a configuration of an obstacle detection apparatus 400 to which the traveling environment information acquisition apparatus according to Embodiment 2 of the present invention is applied. The obstacle detection device 400 has the same basic configuration as the obstacle detection device 100 corresponding to the first embodiment shown in FIG. 1, and the same components are denoted by the same reference numerals. The description is omitted.

  An obstacle detection apparatus 400 shown in FIG. 5 includes a plurality of shooting cameras (cameras 104 in FIG. 1) of the object in the configuration of the obstacle detection apparatus 100 of FIG. Here, the obstacle detection apparatus 400 includes a first camera 402 for shooting an object and a second camera 404 in addition to one camera for shooting the object in the configuration of the obstacle detection apparatus 100. It has a configuration. The obstacle detection device 400 in FIG. 5 is configured to include two cameras for shooting an object such as the first camera 402 and the second camera 404 for convenience, but is not limited thereto, and is configured to include three or more. Also good.

  In the obstacle detection apparatus 400, in the obstacle detection processing unit 410 having the same configuration as the obstacle detection processing unit 110, the camera control unit 414 controls the first camera 402 and the second camera 404 based on the input information. Control and image the object.

  Specifically, the camera control unit 414 distributes the objects to be photographed for each camera from the photographing order of the objects calculated by the photographing order calculation unit 113 and performs control for each camera. The first camera 402 and the second camera 404) perform shooting.

  Here, as a method in which the camera control unit 414 distributes each camera, when there are two cameras as shown in FIG. 5, for example, the odd-numbered objects are the first camera 402 and the even-numbered objects are the first. There is a method of photographing with two cameras 404. Further, the first camera 402 may be used for the first half of the shooting order and the second camera 404 may be used for the second half of the shooting order. In addition, a threshold value is set according to the distance to the object, and the first camera 402 is used for shooting when the distance is within the threshold range, and the second camera 404 is used for shooting when the distance is outside the threshold range. You may make it perform distribution for every camera.

  In this case, for example, it is more preferable if a plurality of cameras (first camera 402 and second camera 404) provided in the obstacle detection device 400 have different imaging distances.

  That is, in the case where a threshold is provided according to the distance to the object, the first camera 402 to be imaged when the distance is within the threshold range is a short-distance camera including a wide-angle lens, and when the distance is outside the threshold range. The second camera 404 to be photographed is composed of a medium-distance / long-distance camera equipped with a telephoto lens.

  Thus, the camera control unit 414 performs sorting for each camera including the shooting order so that the object corresponding to the camera-specific imaging distance for each camera can be shot.

  In other words, the obstacle detection apparatus 400 includes a plurality of cameras for long-distance shooting, medium-distance shooting, and short-distance shooting, which are each equipped with a telephoto lens, a wide-angle lens, and the like and have different imaging distances. The imaging order is assigned based on the specific imaging distance.

  Therefore, an image acquired using each of the plurality of cameras has high accuracy, and an image that makes recognition easier in object recognition can be acquired.

  Further, from the direction of the detected object, the first camera 402 is applied to the object located in the right half, and the second camera 404 is used to assign the object to the object located in the left half. You may make it perform.

  In addition, as the control for each camera, the camera control unit 414 is configured for each camera (the first camera 402 and the second camera 404) that captures the distributed object according to the camera setting corresponding to each distributed object. Set the zoom, set the lens focus, and set the lens direction.

  In addition, the camera control unit 414 instructs the first camera 402 and the second camera 404 to perform shooting with the set contents, controls the first camera 402 and the second camera 404, and respectively follows the shooting order. The imaging of each object is repeatedly executed for the number of objects.

  Similar to the camera control unit 114, the camera control unit 414 outputs the camera settings for each target object input from the imaging order calculation unit 113 to the image acquisition unit 115.

  Next, the operation of the obstacle detection apparatus 400 having the above configuration will be described with reference to FIG.

  FIG. 6 is a flowchart for explaining the operation of the obstacle detection apparatus 400 according to Embodiment 2 of the present invention.

  First, the obstacle detection apparatus 400 performs the processing of steps S501 to S504 similar to steps S201 to S204 of the obstacle detection apparatus 100 of the first embodiment.

  That is, the obstacle detection apparatus 400 acquires object information (direction and distance) by the processing of step S501 to step S504, and images the position of the object and each object, similarly to the obstacle detection apparatus 100. The camera settings and shooting order are calculated.

  After calculating the shooting order in step S504 (the same processing as step S204), the process proceeds to step S505. In step S505, the camera control unit 414 performs shooting for each camera based on the shooting order calculated by the shooting order calculation unit 113. Sort the target object.

  Note that, in this step S505, the camera control unit 414 is realized by the above-described method as the method of distributing for each camera.

  The processing after step S506 after step S505 indicates processing for shooting for each camera to which the shooting order and the camera settings set for each target object of the shooting order are distributed. Here, steps S506 to S509 of the photographing process in the first camera 402 are the same as the processes from step S205 to step S208 of FIG.

  Similarly, steps S510 to S513 of the photographing process in the second camera 404 are the same as steps S205 to S208 in FIG.

  That is, in step S506, the camera control unit 414 performs zoom setting, lens focus setting, and lens direction setting on the first camera 402 based on the camera setting of the object in accordance with the calculated shooting order. The process proceeds to S507.

  In step S507, after the setting of the first camera 402 is completed, the camera control unit 414 instructs the first camera 402 to shoot with the set content, and the first camera 402 shoots an image of the object. The process proceeds to S508.

  In step S508, the image acquisition unit 115 acquires the image of the target object captured by the first camera 402 and acquires the distance and direction from the camera control unit 414 to the target object corresponding to the target object. The distance between the object image and the object corresponding to the object is associated with the direction, and the process proceeds to step S509. In step S508, the image information of the target associated with the distance to the target and the direction by the image acquisition unit 115 is stored until the object recognition is completed.

  In step S <b> 509, the image acquisition unit 115 uses the image input from the first camera 402 and the distance and direction to the target object input from the camera control unit 414 to determine whether or not shooting of all the targets is complete. I do.

  In step S509, if the shooting of all the objects has not been completed, the process returns to step S506, the processes from step S506 to step S509 are repeatedly executed, the next object is shot, and the shooting of all the objects is completed. If so, the shooting of the object is terminated.

  As the imaging process in the second camera 404, in step S510, the camera control unit 414 performs the zoom setting and the lens focus setting on the second camera 404 based on the camera setting of the object according to the calculated shooting order. The direction of the lens is set and the process proceeds to step S511.

  In step S511, after the setting of the second camera 404 is completed, the camera control unit 414 instructs the second camera 404 to shoot with the set content, and the second camera 404 shoots an image of the object. The process proceeds to S512.

  In step S512, the image acquisition unit 115 acquires the image of the object photographed by the second camera 404, acquires the distance and direction from the camera control unit 414 to the object corresponding to the object, and acquires the acquired object. The distance between the object image and the object corresponding to the object is associated with the direction, and the process proceeds to step S513. Note that, in step S512, the image information of the target associated with the direction to the distance to the target by the image acquisition unit 115 is stored until the object recognition is completed.

  In step S513, the image acquisition unit 115 uses the image input from the second camera 404 and the distance and direction to the target object input from the camera control unit to determine whether or not shooting of all the target objects is complete. Do.

  In step S513, if the shooting of all the objects has not been completed, the process returns to step S510, the processes from step S510 to step S513 are repeatedly executed, the next object is shot, and the shooting of all the objects is completed. If so, the shooting of the object is terminated.

  In this way, the obstacle detection device 400 converts all images of the object corresponding to the object detected by the object detection sensor 102 into a plurality of cameras (first camera 402, second camera) according to the calculated imaging order. Obtained by the camera 404). All of these acquired images are acquired with an image size that can be recognized by the object recognition unit 116 without being processed.

  Even if the number of cameras is two or more, the same processing can be realized.

  In the obstacle detection device 400, the processing after the end of photographing is the same as the processing of the obstacle detection device 100 shown in FIG.

  As described above, according to the obstacle detection device 400 according to the second embodiment of the present invention, a plurality of cameras for photographing an object are provided, the objects to be photographed are distributed to the plurality of cameras, and the images are photographed for each camera. Depending on the distance and direction to the target object, you can shoot all target objects with an image size suitable for object recognition.Set the camera zoom, lens focus and lens direction. Camera setting calculation unit to perform and camera control amount necessary for shooting multiple objects, or shooting order calculation unit to calculate the shooting order that minimizes the time required for control and camera settings and targets for each target By providing a camera control unit that controls the camera according to the shooting order of objects, the accuracy of object identification and the processing time are shortened, the cost is reduced due to a decrease in the required performance of the processing processor, and the camera It may enable shortening of the reduction of the failure rate and the imaging time by reducing the photographing load to.

  Here, the distance to the object using the image of the camera is the position of the object in the two images taken by shifting the installation positions of the two cameras in the horizontal direction or the vertical direction. It is known that the direction of the object can be realized by a method of calculating from the position in the captured image.

  In addition, the shooting order calculation unit 113 calculates the shooting order of the object, and the camera control unit 414 divides the calculated shooting order and performs setting for each of the plurality of cameras corresponding to the order. Based on this, the object is photographed. This reduces the load on the camera and the shooting time required to shoot all objects, shortens the processing time required for object identification, reduces the time required to control the camera, and loads the camera. The life of the camera can be extended by the reduction.

(Embodiment 3)
FIG. 7 is a block diagram showing a configuration of an obstacle detection apparatus 600 to which the traveling environment information acquisition apparatus according to Embodiment 3 of the present invention is applied. The obstacle detection device 600 has the same basic configuration as the obstacle detection device 100 corresponding to the first embodiment shown in FIG. 1, and the same components are denoted by the same reference numerals. The description is omitted.

  The obstacle detection apparatus 600 according to the third embodiment includes a plurality of A cameras 602 and B cameras 604 instead of the object detection sensor 102 and the camera 104 in the obstacle detection apparatus 100, and the plurality of A cameras 602, The B camera 604 is used to calculate the direction and distance of the object to detect the position and direction of the object and to capture an image of the object for identifying the object.

  The obstacle detection apparatus 600 includes an A camera 602, a B camera 604, a speaker 105, a monitor 106, and an obstacle detection processing unit 610.

  The obstacle detection processing unit 610 includes an object detection unit 611, a camera setting calculation unit 112, a shooting order calculation unit 113, a camera control unit 614, an image acquisition unit 615, an object recognition unit 116, and a risk determination unit. 117, a notification unit 118, and a shooting switching unit 618.

  The plurality of A cameras 602 and B cameras 604 are arranged at different positions in the vehicle, and image one object or separate objects.

  The plurality of A cameras 602 and B cameras 604 are controlled by the object detection unit 611 or the camera control unit 614 via the shooting switching unit 618, pick up an image of the object, and from the current position to the object to be identified. The distance and direction of the object are acquired, or an image of the object for identifying the object is acquired. Information acquired by the plurality of A cameras 602 and B cameras 604 is output to the object detection unit 611 and the image acquisition unit 615.

  The imaging switching unit 618 switches between imaging for calculating the distance and direction to the target by the plurality of A cameras 602 and B cameras 604 and imaging for identifying the target.

  The imaging switching unit 618 outputs the captured image and the input source of the camera control for the A camera 602 and the B camera 604 according to the control switching instruction for the A camera 602 and the B camera 604 from the object detection unit 611 and the camera control unit 614. Switch the destination. Specifically, when the plurality of A cameras 602 and B cameras 604 perform imaging for calculating the distance and direction to the target, the imaging switching unit 618 determines the output destination of the acquired information as the object detection unit 611. If the image is captured for identification of the object, the information output destination is set in the image acquisition unit 615.

  Also, the imaging switching unit 618 is instructed by the obstacle detection processing unit 610 to switch control between the A camera 602 and the B camera 604, and the A camera 602 and the B camera 604 are connected to the object via the imaging switching unit 618. It is good also as a structure which sets the camera for detecting a distance and a direction, acquires an image, and calculates the distance and direction from the present position to the several target object used as identification object.

  The object detection unit 611 is calculated by imaging a target object with the plurality of A cameras 602 and B cameras 604 via the shooting switching unit 618 and using the parallax of the plurality of A cameras 602 and B cameras 604. Get the direction and distance of the object.

  The object detection unit 611 is within the specified distance range using the information on the distance to the target and the direction calculated from information input from the plurality of A cameras 602 and B cameras 604, and the specified direction range. A set of objects inside is determined as one object. The determined information is output to the camera setting calculation unit 112.

  The distance range and the direction range to be specified are set from the allowable value of the length of the object to be identified by the object recognizing unit 116, for example, the allowable value of the total length in the case of an automobile. An allowable value of the width that the object can take from the distance to the object, for example, in the case of an automobile, a range within a viewing angle corresponding to the width and distance of the automobile is set as the allowable value.

  The camera control unit 614 controls the A camera 602 and the B camera 604 via the shooting switching unit 618 based on the information input from the shooting order calculation unit 113 and images the target object.

  Specifically, the camera control unit 614 instructs the shooting switching unit 618 to switch control between the A camera 602 and the B camera 604.

  In addition, the camera control unit 614 sends the A camera 602 and B to the A camera 602 and B camera 604 via the shooting switching unit 618 according to the shooting order of the shooting order calculation unit 113 and the setting value of the camera for each object. The zoom setting of the camera 604, the focus of the lens, and the lens direction are set.

  In addition, the camera control unit 614 instructs the plurality of A cameras 602 and B cameras 604 to perform shooting with the set contents via the shooting switching unit 618, and controls the plurality of A cameras 602 and B cameras 604. Under control, the imaging of the image of the object is repeatedly executed for the number of objects.

  In addition, the camera control unit 614 outputs the camera setting for each object input from the imaging order calculation unit 113 to the image acquisition unit 615.

  The image acquisition unit 615 acquires the images captured by the plurality of A cameras 602 and B cameras 604 via the capture switching unit 618, and the distance and direction to the object corresponding to the captured images from the camera control unit 614. And image information for the number of objects is created.

  The created image information for the number of objects is output to the object recognition unit 116.

  Next, the operation of the obstacle detection apparatus 600 having the above configuration will be described with reference to FIGS.

  8 and 9 are flowcharts for explaining the operation of the obstacle detection apparatus 600 according to Embodiment 3 of the present invention. FIG. 8 shows an example of an object acquisition method for detecting the distance and direction to the object using a camera.

  As shown in FIG. 8, first, in step S701, the object detection unit 611 sets a camera control input source and a captured image output destination in the shooting switching unit 618, and the process proceeds to step S702.

  In step S702, the object detection unit 611 performs zoom setting, lens focus setting, and lens direction setting for the A camera 602 via the shooting switching unit 618, and the process proceeds to step S703.

  In step S703, the object detection unit 611 performs zoom setting, lens focus setting, and lens direction setting for the B camera 604 via the shooting switching unit 618, and the process proceeds to step S704.

  In step S704, the object detection unit 611 issues a shooting instruction to the A camera 602 via the shooting switching unit 618, takes an image using the A camera 602, and proceeds to step S705.

  In step S <b> 705, the object detection unit 611 issues a shooting instruction to the B camera 604 via the shooting switching unit 618, and takes an image of the same object as that captured by the A camera 602 using the B camera 604. Then, the process proceeds to step S706.

  In step S706, the image picked up by the A camera 602 and the B camera 604 is output from the photographing switching unit 618 to the object detecting unit 611, whereby the object detecting unit 611 acquires the image, and the process proceeds to step S707.

  In step S707, the object detection unit 611 calculates the distance and direction to the target object based on the images of the A camera 602 and the B camera 604 from the image acquired from the shooting switching unit 618, and the current position to the target object. From the distance and direction, the objects within the specified distance range and within the specified direction range are set as one object information as the same object set, and the center point of the set of objects is detected. Calculated as the distance and direction to the object.

  Note that the object detection unit 611 calculates the position of the target object by calculating the distance and direction.

  Next, as illustrated in FIG. 9, the obstacle detection apparatus 600 performs image acquisition for object recognition using the camera used to acquire the target object information.

  First, the object detection unit 611 calculates the position (direction, distance) of the object using the object information of the distance to the object and the direction acquired by the calculation shown in FIG. (Step S800) Next, in step S801, the camera setting calculation unit 112 performs object recognition in the object recognition unit 116 for each target based on the distance and direction to the target input from the object detection unit 611. Calculates the zoom setting value based on the distance to the object according to the image size to be used, and further calculates the lens focus setting value for adjusting the focal position of the lens from the zoom setting value and the distance to the object. To do. Further, based on the direction of the object, a setting value for the camera orientation is calculated, and the process proceeds to step S802.

  In step S <b> 802, the shooting order calculation unit 113 calculates the camera zoom setting value, the lens focus setting value, the camera orientation setting value, or the target object calculated for each object input from the camera setting calculation unit 112. The order in which the camera setting change is minimized is calculated from the distance and the direction, and the shooting order of the object is calculated, and the process proceeds to step S803.

  The calculation of the order that minimizes the camera setting change in step S802 can be easily performed by applying the shortest path search method. For example, it is possible to calculate in the order in which the amount of zoom setting change is minimized, and the time required for camera setting change is minimized from the time required for zoom setting, lens focus setting, and lens direction setting. It is possible to calculate the shooting order.

  In step S803, the camera control unit 614 sorts objects to be photographed for each camera based on the photographing order calculated by the photographing order calculation unit 113, determines the camera to be photographed, and proceeds to step S804.

  In step S804, the camera control unit 614 sets the camera control input source to the camera control unit 614 with respect to the shooting switching unit 618, and sets the output destination of the images shot by the A camera 602 and the B camera 604 to the image acquisition unit 615. Set to.

  The processing of step S805 to step S808 and the processing of step S809 to step S812 explain the imaging processing of the object based on the shooting order and the set camera settings for each of the plurality of cameras (A camera 602, B camera 604). To do.

  That is, in step S805, the camera control unit 614 performs zoom setting and lens focus setting on the A camera 602 via the shooting switching unit 618 based on the camera setting of the object according to the calculated shooting order. The direction of the lens is set and the process proceeds to step S806.

  In step S806, the camera control unit 614 instructs the A camera 602 to shoot with the set contents after the setting of the A camera 602 via the shooting switching unit 618, and the A camera 602 displays an image of the object. The photograph is taken and the process proceeds to step S807.

  In step S807, the image acquisition unit 615 acquires the image of the object captured by the A camera 602, acquires the distance and direction from the camera control unit 614 to the object corresponding to the object, and acquires the acquired object. The distance and direction to the target object corresponding to the target object are associated with each other, and the process proceeds to step S808. In step S807, the image information of the target object that is associated with the distance to the target object by the image acquisition unit 615 is stored until the object recognition is completed.

  In step S808, the image acquisition unit 615 uses the image input from the A camera 602 and the distance and direction to the target input from the camera control unit via the shooting switching unit 618, and uses all the objects. It is determined whether or not shooting is complete.

  In step S808, if the shooting of all objects has not been completed, the process returns to step S805, the processes from step S805 to step S808 are repeatedly executed, the next object is shot, and the shooting of all objects is completed. If so, the shooting of the object is terminated.

  On the other hand, as an imaging process in the B camera 604, in step S 809, the camera control unit 614 sends the B camera 604 to the B camera 604 via the shooting switching unit 618 based on the camera setting of the target object. The zoom setting, lens focus setting, and lens direction setting are performed, and the process proceeds to step S810.

  In step S <b> 810, the camera control unit 614 instructs the B camera 604 to shoot with the set content after the setting of the B camera 604 via the shooting switching unit 618 is completed, and performs shooting. An image of the object is captured by the B camera 604 via the switching unit 618, and the process proceeds to step S811.

  In step S811, the image acquisition unit 615 acquires an image of the object photographed by the B camera 604 via the photographing switching unit 618 and determines the distance and direction from the camera control unit 614 to the object corresponding to the object. The acquired image of the object is associated with the distance to the object corresponding to the object and the direction, and the process proceeds to step S812. Note that in step S811, the image acquisition unit 615 stores the image information of the target associated with the distance to the target and the direction until the object recognition is completed.

  In step S812, the image acquisition unit 615 uses the image input from the B camera 604 and the distance and direction to the object input from the camera control unit via the shooting switching unit 618, and uses all the objects. It is determined whether or not shooting is complete.

  In step S812, if the shooting of all objects has not been completed, the process returns to step S809, the processes from step S809 to step S812 are repeatedly executed, the next object is shot, and the shooting of all objects is completed. If so, the shooting of the object is terminated.

  In this way, the obstacle detection apparatus 600 converts all the images of the objects corresponding to the objects acquired by the plurality of cameras (A camera 602, B camera 604) into a plurality of cameras ( A camera 602 and B camera 604). All of these acquired images are acquired with an image size that can be recognized by the object recognition unit 116 without being processed.

  The obstacle detection apparatus 600 includes a plurality of cameras 602 and 604 that perform imaging of the object for calculating the distance and direction to the object and imaging of the object for identification of the object. An imaging switching unit 618 that switches between imaging for calculating the distance and direction to the object by the plurality of cameras 602 and 604 and imaging for identifying the object is provided.

  By switching the photographing switching unit 618, the object detection unit 611 detects the position and direction of the target object by calculating the direction and distance of the target object using the parallax of the A camera 602 and the B camera 604, and the camera control unit 614 captures an image of the object for identifying the object using the A camera 602 and the B camera 604.

  That is, the obstacle detection apparatus 600 includes a plurality of cameras, and the cameras 602 and 604 for capturing an image to detect the distance and direction to the object from the image and the image for identifying the object are captured. Use a common camera.

  As a result, the cost is reduced by sharing the camera for shooting, the accuracy of object identification is improved and the processing time is reduced, the cost is reduced due to the reduction in the required performance of the processing processor, and the shooting load on the camera is reduced. It is possible to reduce the failure rate and the shooting time.

  In addition, although the several camera with which the obstacle detection apparatus 600 of this Embodiment 3 is provided is A camera 602 and B camera 604, it replaces with these A camera 602 and B camera 604, and several imaging part of the same function. You may use a compound eye camera provided with.

  The obstacle detection apparatus according to the present invention is not limited to the above embodiments, and can be implemented with various modifications.

  The obstacle detection device according to the present invention can be mounted on an obstacle detection device that is mounted on a vehicle and identifies whether an object in the traveling direction is an obstacle, and thus has the same effect as described above. An obstacle detection device having an effect can be provided.

  Here, the case where the present invention is configured by hardware has been described as an example, but the present invention can also be realized by software. For example, a function similar to that of the apparatus according to the present invention is realized by describing the algorithm of the travel information acquisition method according to the present invention in a program language, storing the program in a memory, and causing the information processing means to execute the algorithm. be able to.

  The travel environment information acquisition device according to the present invention has the effect of improving the accuracy of object identification, shortening the processing time, and reducing the cost due to the reduction in the required performance of the processing processor, and is mounted on a moving body such as an automobile. Therefore, it is useful as an obstacle detection device that detects an obstacle in the moving direction using a radar or an image.

The block diagram which shows the structure of the obstruction detection apparatus to which the driving environment information acquisition apparatus which concerns on Embodiment 1 of this invention is applied. The flowchart for demonstrating operation | movement of the obstruction detection apparatus in Embodiment 1 of this invention. The flowchart for demonstrating operation | movement of the obstruction detection apparatus in Embodiment 1 of this invention. The conceptual diagram explaining the outline | summary of the detection process of the target object which the obstacle detection apparatus of this Embodiment 1 performs. The block diagram which shows the structure of the obstruction detection apparatus to which the driving environment information acquisition apparatus which concerns on Embodiment 2 of this invention is applied. The flowchart for demonstrating operation | movement of the obstruction detection apparatus in Embodiment 2 of this invention. The block diagram which shows the structure of the obstruction detection apparatus to which the driving environment information acquisition apparatus which concerns on Embodiment 3 of this invention is applied. The flowchart for demonstrating operation | movement of the obstruction detection apparatus in Embodiment 3 of this invention. The flowchart for demonstrating operation | movement of the obstruction detection apparatus in Embodiment 3 of this invention. The figure which shows the positional relationship of a traveling vehicle and a target object FIG. 10 is a conceptual diagram illustrating a method for specifying an object range in an image based on the positional relationship between the vehicle and the object according to the conventional method, using the positional relationship of FIG. The conceptual diagram which shows the extraction method of the target object image in the image from the positional relationship of the own vehicle and a target object in the conventional system

Explanation of symbols

100, 400, 600 Obstacle detection device 102 Object detection sensor 104 Camera 111, 611 Object detection unit (object detection means)
112 camera setting calculation unit 113 imaging order calculation unit (imaging order calculation unit)
114, 414, 614 Camera control unit (camera control means)
115, 615 Image acquisition unit (image acquisition means)
116 Object recognition unit 117 Risk determination unit 118 Notification unit 402 First camera 404 Second camera 602 A camera 604 B camera 618 Shooting switching unit (imaging switching unit)

Claims (7)

A traveling environment information acquisition device that is mounted on a moving object and acquires the object as traveling environment information using an image of the object in the moving direction of the moving object,
Object detection means for detecting the position and direction of the object in the moving direction of the moving body;
Camera setting calculation that calculates camera settings including lens orientation, lens focus, and zoom settings that can be captured with an object-recognizable image size set in advance for each object type from the position and direction of the object Means,
Camera control means for controlling the camera to image the object with the camera settings calculated for each object;
Image acquisition means for acquiring an image of an object imaged by the camera based on the calculated camera setting;
A travel environment information acquisition device comprising: An object recognition unit that determines the type of the target using the captured target image and the set camera setting, and determines the risk of the target from the type, distance, and direction of the target When,
A danger notification means for performing a danger notification corresponding to the danger level of the object determined by the object recognition means;
The travel environment information acquisition device according to claim 1, further comprising: When the plurality of objects are detected by the object detection means and all detected objects are imaged,
Based on the camera settings for each of the plurality of objects calculated by the camera setting calculation means, the amount of change in lens orientation, the amount of lens movement for focusing the lens, and the amount of lens movement for zoom setting are minimized. The travel environment information acquisition apparatus according to claim 1, further comprising an imaging order calculation unit that calculates an imaging order of the target object. Comprising a plurality of cameras for imaging the object;
The camera control means assigns the imaging order calculated by the imaging order calculation means to each of the plurality of cameras, and sets the camera for an object to be imaged in the imaging order using each of the plurality of cameras. The traveling environment information acquisition apparatus according to claim 1, wherein the object is imaged. The plurality of cameras have different imaging distances,
The travel environment information acquisition apparatus according to claim 4, wherein the camera control unit assigns the imaging order to the plurality of cameras based on imaging distances of the plurality of cameras. A plurality of cameras that perform imaging of the object for calculating the distance and direction to the object and imaging of the object for identification of the object;
An imaging switching unit that switches between imaging for calculating the distance and direction to the object by the plurality of cameras and imaging for identifying the object;
By switching the imaging switching unit,
The object detection means detects the position and direction of the object by calculating the direction and distance of the object using the parallax of the plurality of cameras,
The travel environment information acquisition apparatus according to claim 1, wherein the camera control unit captures an image of an object for identifying the object using the plurality of cameras. A traveling environment information acquisition method that is mounted on a moving object and acquires the object as traveling environment information using an image of the object in the moving direction of the moving object,
An object detection step of detecting a position and a direction of the object in a moving direction of the moving body;
Camera setting calculation that calculates camera settings including lens orientation, lens focus, and zoom settings that can be captured with an object-recognizable image size set in advance for each object type from the position and direction of the object Steps,
A camera control step of controlling the camera and imaging the object with the camera settings;
An image acquisition step of acquiring an image of an object imaged by the camera based on the calculated camera setting;
A driving environment information acquisition method comprising:

Images