JP2012198207A - Image processing device and method, and mobile-body collision prevention apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide: an image processing device and method for accurately recognizing an object within a short range; and a mobile-body collision prevention apparatus.SOLUTION: An image processing device includes an imaging part and an image processing part for processing an image captured by the imaging part. The imaging part includes an imaging lens, a focusing mechanism for driving the imaging lens to change a focus distance, and an imaging element. The focusing mechanism performs focus sweep to drive the imaging lens and sequentially change the focusing distance so as to clearly image the objects located in different positions in an optical axis direction of the imaging lens on the imaging device. The imaging part captures respective images in accordance with the different focusing distances in the focus sweep. The image processing part recognizes an object clearly imaged in each image from the captured plural images to generate an object distribution image that indicates the position of the recognized object in accordance with the focus distances when the respective images have been captured.

Description

  The present invention relates to an image processing apparatus, for example, a moving object collision preventing apparatus using image processing. The present invention also relates to such an image processing method.

  Japanese Patent Laid-Open No. 2000-304856 (Patent Document 1) and Japanese Patent Laid-Open No. 11-120499 (Patent Document 2) disclose a global positioning system (GPS) mounted on an airplane or ship or a collision prevention device using a radar. Is described. Japanese Patent No. 3260645 (Patent Document 3) describes a car navigation device using GPS mounted on a ground transportation device such as a vehicle. Japanese Unexamined Patent Publication No. 2001-337724 (Patent Document 4) describes a technique for preventing an unmanned vehicle from entering a specific area using GPS. Furthermore, Japanese Patent Laid-Open No. 5-143158 (Patent Document 5) describes a technique for preventing a collision between unmanned vehicles using a wireless monitoring device.

  In recent years, in a mobile collision prevention apparatus having a GPS receiver, a speed detection unit, and a direction detection unit, by transmitting GPS position information, speed information, and direction information acquired by the above-described components to other mobile units The mobile body that has received these pieces of information can determine whether or not a collision occurs.

  However, in the conventional technology using such GPS, the error in the calculation of the distance between the moving bodies is usually several meters (m) to several tens of meters due to the measurement accuracy of the GPS itself and the accuracy of the GPS receiving equipment. There may be. Therefore, there is a possibility that the position of the moving body is erroneously detected within a short range where a collision accident may occur.

JP 2000-304856 A Japanese Patent Laid-Open No. 11-120499 Japanese Patent No. 3260645 JP 2001-337724 Japanese Unexamined Patent Publication No. 5-143158

  An object of the present invention is to provide an image processing apparatus and method capable of accurately recognizing an object in a short distance range, and a moving object collision preventing apparatus in view of such problems in the conventional technology.

  An image processing apparatus according to the present invention includes an imaging unit and an image processing unit that processes an image captured by the imaging unit. The imaging unit drives the imaging lens and the focusing lens to change a focusing distance. A focusing mechanism and an imaging device, and the focusing mechanism sequentially focuses so as to drive the imaging lens to clearly image an object at a different position in the optical axis direction of the imaging lens on the imaging device. A focus sweep for changing a distance is performed, and the imaging unit captures images according to a plurality of different focusing distances in the focus sweep, and the image processing unit is configured to detect the plurality of images from the plurality of captured images. Recognize the clearly imaged object in each of the images, and correspond to the focusing distance at the time of capturing each of the images And generating an object distribution picture to see the position of the serial recognized objects.

  According to the image processing apparatus of the present invention, an image showing the distribution of objects within the imaging range is generated. This object distribution image can be applied to equipment location identification and monitoring control at a production site, viewing angle switching in live sports games, an interactive entertainment system, and a mobile collision prevention apparatus according to the present invention described later. .

  In the image processing apparatus according to the present invention, it is desirable that the focusing distance is changed with a step width of 1 m or less in the focus sweep. Thereby, the precision of an object distribution image can be improved.

  In the image processing apparatus according to the present invention, it is preferable that the imaging unit captures images at a frequency of 500 images / second or more. Thereby, an object distribution image can be generated in more real time.

  In the image processing apparatus according to the present invention, the imaging unit repeatedly performs the focus sweep a plurality of times, and the image processing unit generates a plurality of the object distribution images corresponding to the plurality of focus sweeps. Is desirable. By repeatedly performing the focus sweep, the object distribution image can be updated as needed, and the real-time property of the object distribution image can be further improved.

  In the image processing apparatus according to the present invention, it is preferable that the imaging unit performs the focus sweep five times or more per second. Thereby, the real time property which updates an object distribution image can further be improved.

  In the image processing apparatus according to the present invention, the image pickup means may use a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS), or may use an infrared image pickup device. As the imaging device, a type suitable for an environment to be used can be selected.

  In the image processing apparatus according to the present invention, the imaging lens includes a zoom lens, and the imaging unit includes a zooming mechanism that drives the zoom lens and changes a focal distance thereof, and the zooming mechanism has the focusing distance. When the zoom lens is longer than a predetermined value, it is desirable to drive the zoom lens to increase its focal length, and when the focusing distance is shorter than a predetermined value, it is preferable to drive the zoom lens to shorten the focal length.

  An image processing method according to the present invention includes an imaging lens, a focusing mechanism that drives the imaging lens to change a focusing distance, an imaging unit having an imaging element, and an image processing unit that processes an image captured by the imaging unit. In the image processing apparatus, in the imaging unit, the focusing mechanism drives the imaging lens so that objects at different positions in the optical axis direction of the imaging lens are clearly imaged on the imaging element. A focus sweep that sequentially changes the focusing distance is performed, and each image is captured according to a plurality of different focusing distances in the focus sweep, and the plurality of captured images are picked up from the captured plurality of images by the image processing means. Recognizes a clearly imaged object in each of the images, and captures each of the plurality of images at the time of imaging. And generating an object distribution image displaying the position of the recognized object in accordance with the Kashingu distance.

  In the image processing method according to the aspect of the invention, it is preferable that the imaging unit repeatedly performs the focus sweep a plurality of times, and the image processing unit generates a plurality of the object distribution images according to the plurality of focus sweeps. .

  A moving object collision preventing apparatus according to the present invention is provided on a moving object, and includes an imaging unit that images the periphery of the moving object, an image processing unit that processes an image captured by the imaging unit, and a collision determination unit, The imaging means includes an imaging lens, a focusing mechanism that drives the imaging lens to change a focusing distance, and an imaging element. The focusing mechanism drives the imaging lens to move from the moving body. A focus sweep is performed to sequentially change a focusing distance so that a plurality of distant objects are clearly imaged on the image sensor, and the imaging unit displays each image according to a plurality of different focusing distances in the focus sweep. The image processing means recognizes an object that is clearly imaged in each of the plurality of images from the plurality of captured images, and the plurality of images An object distribution image that displays a position of the recognized object corresponding to a focusing distance at the time of capturing each of the images is generated, and the collision determination unit recognizes the moving object based on the object distribution image. It is characterized by determining whether or not the object collides.

  According to the mobile object collision preventing apparatus of the present invention, the collision determination unit can effectively avoid the collision accident of the mobile object by determining the collision from the object distribution image generated by the image processing unit.

  The moving body collision preventing apparatus of the present invention has display means for displaying an object distribution image, and when the collision determination means determines that the moving body and the recognized object collide, the display means , Display the object in different colors. In addition, a notification means for notifying a pilot of a mobile object is provided, and when the collision determination means determines that the mobile object and the recognized object collide, the notification means generates a warning message to the pilot. . Furthermore, it is preferable that display means for displaying the object distribution image is provided, and the image processing means adds display of grid lines representing distance intervals to the object distribution image.

  As described above, when the collision determination unit determines that a collision occurs, the display unit or the notification unit promptly warns the driver of the moving body or the driver of the vehicle and prompts the collision to be dealt with. Thereby, a collision accident can be avoided.

  The moving object collision preventing apparatus of the present invention further includes moving object control means for controlling the movement of the moving object, and the movement determining means determines that the moving object and the recognized object collide with each other. It is desirable that the body control means controls the brake, accelerator or steering of the moving body. In this way, it is possible to steer the moving body by the moving body control means, avoid a collision accident, and realize automatic traveling.

  The moving body collision preventing apparatus of the present invention further includes speed detecting means for detecting the speed of the moving body, and the collision determining means is configured to detect the moving body from the speed of the moving body detected by the speed detecting means. It is desirable to determine whether or not the recognized object collides. By providing the speed detection means, the collision determination means can determine whether or not to collide more accurately.

  In the moving object collision preventing apparatus of the present invention, the imaging lens is a zoom lens, and the imaging means includes a zooming mechanism that drives the zoom lens and changes a focal distance thereof, and the zooming mechanism includes a focusing distance. When the zoom lens is longer than a predetermined value, it is desirable to drive the zoom lens to increase its focal length, and when the focusing distance is shorter than a predetermined value, it is preferable to drive the zoom lens to shorten the focal length. Thereby, the accuracy at the time of recognizing an object by the image processing means is further improved, and an object located far away can be recognized more accurately.

  A moving object collision prevention apparatus according to the present invention is provided on a moving object, and includes an imaging unit that images the periphery of the moving unit, an image processing unit that processes an image captured by the imaging unit, and a display unit. The means includes an imaging lens, a focusing mechanism that drives the imaging lens to change a focusing distance, and an imaging element. The focusing mechanism drives the imaging lens and moves away from the moving body. A focus sweep that sequentially changes a focusing distance so that a plurality of objects are clearly imaged on the image sensor, and the imaging unit captures a single image according to a plurality of different focusing distances in the focus sweep. The image processing means recognizes an object that is clearly imaged in each of the plurality of images from the plurality of captured images, and the plurality of images. Each corresponds to focusing distance at the time of imaging to produce an object distribution image displaying the position of the recognized object, the display means, and displaying the said object distribution image. The object distribution image may be a three-dimensional image.

  By displaying the object distribution image generated by the image processing means on the display means, the driver of the moving body or the driver of the vehicle can confirm the position of the object around the moving body, further improving safety. To do.

FIG. 1 is a schematic block diagram showing a configuration of a moving object collision preventing apparatus according to an embodiment of the present invention. FIG. 2 is a side sectional view schematically showing the configuration of the imaging unit in the embodiment shown in FIG. FIG. 3 is a schematic diagram for explaining the imaging range of the imaging unit. Or 4A to 4E are diagrams illustrating examples of a plurality of images captured by the imaging unit in one focus sweep. FIG. 5 is a flow chat of image processing in the embodiment shown in FIG. 6 is a schematic diagram similar to FIG. 3, showing an example of an object distribution image generated by the image processing unit shown in FIG. FIG. 7 is a diagram for explaining a method in which the image processing unit generates a three-dimensional object distribution map.

  Hereinafter, embodiments of a moving object collision prevention apparatus using an image processing apparatus according to the present invention will be described with reference to the drawings. The moving object collision preventing apparatus according to the present embodiment uses the object distribution image generated by the image processing apparatus according to the present embodiment to determine whether a moving object such as a vehicle collides with another object or an obstacle such as a human body. Can be determined accurately. Throughout this patent application, the term “object” may be interpreted broadly including the human body.

  FIG. 1 is a block diagram schematically showing a configuration of a moving object collision preventing apparatus 10 according to an embodiment of the present invention. The moving body collision preventing apparatus 10 according to the present embodiment is disposed in a moving body, for example, the vehicle 20, and can avoid a collision accident of the mounted vehicle 20.

  As shown in FIG. 1, the collision prevention apparatus 10 includes an imaging unit 120, an image processing unit 140, a speed detection unit 150, a display unit 200, an audio unit 210, a vibration unit 220, and a read-only memory (ROM). 230, a collision determination unit 310, a vehicle control unit 400, and a control unit 100 that controls each component of the collision prevention apparatus 10, which are connected as shown in the figure.

  The imaging unit 120 is an imaging device that is installed so as to secure the field of view in front of the normal traveling direction of the vehicle 20 and captures an object scene in front of the vehicle 20. This generates a plurality of image frames in which the object scene is imaged while performing a focus sweep as will be described later, and transfers them to the image processing unit 140 via the control unit 100.

  The image processing unit 140 receives the image frame sent from the imaging unit 120, determines the presence or absence of an image clearly formed in each image frame, and recognizes or detects an object or a human body corresponding to the clear image Has a processing function. Further, the image processing unit 140 generates an object distribution image indicating the distribution of relative positions between the two based on the distance between the clearly imaged object and the human body vehicle 20.

  The imaging unit 120 and the image processing unit 140 constitute an image processing apparatus according to the present invention. Such an image processing apparatus images a scene in front of the vehicle 20, recognizes or discriminates each object or human body in front of the vehicle 20, and generates an object distribution image indicating the position of each object or human body with respect to the vehicle 20. Generate. This will be described in detail later.

  The display unit 200 visually displays the object distribution image generated by the image processing unit 140, and notifies the driver of the vehicle 20 whether there is an obstacle such as an object or a human body in front of the vehicle 20.

  The collision determination unit 310 is a processing function unit that determines whether the vehicle 20 collides with a recognized object based on the object distribution image generated by the image processing unit 140. The speed detection unit 150 is a vehicle speed measurement function unit that detects the traveling speed of the vehicle 20. The collision determination unit 310 further uses the speed information output from the speed detection unit 150 to determine whether the vehicle 20 collides with an obstacle such as an object or a human body that interferes with traveling.

  If the collision determination unit 310 determines that a collision occurs, the audio unit 210 generates an audible audio warning message and / or the vibration unit 220 issues an alarm by vibration to take an emergency response to the driver of the vehicle 20 This will help avoid collision accidents.

  Further, when the collision determination unit 310 determines that the vehicle 20 collides with another object or a human body, the vehicle control unit 400 controls the traveling drive system of the vehicle 20 such as a brake, an accelerator, and / or a steering to control the collision. You may comprise so that an accident may be avoided. The control unit 100 is a system control function unit that controls each part of the collision prevention apparatus 10 and performs these controls.

  Next, the configuration of each part of the collision preventing apparatus 10 will be described in detail. First, FIG. 2 is a schematic side sectional view showing a configuration example of the imaging unit 120. As can be seen from FIG. 2, the imaging unit 120 of the collision prevention apparatus 10 includes an imaging lens 121, a charge coupled device (CCD) sensor 122, and a drive motor 123. The imaging lens 121 is an optical lens configured by a set of a plurality of lenses, for example. Light coming from the object scene enters the lens 121 and forms an image on the imaging surface of the CCD sensor 122, and the CCD sensor 122 forms an image frame of the object scene. The drive motor 123 drives the lens set of the lens 121, that is, moves each lens in the direction of the optical axis 125, thereby focusing each object at different distances from the vehicle 20 sequentially and clearly on the CCD sensor 122. Mechanism.

  FIG. 3 is a diagram illustrating a range in which the imaging unit 120 captures an image. As shown in FIG. 3, the lens 121 images the front scene of the vehicle 20 at a constant imaging angle, that is, the angle of view α. An object or human body within the imaging angle α is imaged on the CCD sensor 122 via the lens 121. In this case, only an object included in a virtual plane located at the focusing distance of the imaging unit 120 and substantially perpendicular to the optical axis 125 can form a clear image on the CCD sensor 122. Therefore, the drive motor 123 continues to change the focusing distance of the imaging unit 120 by moving each lens of the lens 121 forward or backward on the optical axis 125, so that various objects within the imaging range α of the lens 121 or The human body is imaged sharply on the CCD sensor 122 sequentially from a distance to a distance or from a distance to a distance.

  In the present application, an operation in which the focusing distance of the imaging unit 120 is continuously increased or decreased and each object at a different position in front sequentially forms a clear image on the CCD sensor 122 is referred to as “focus sweep”.

  The drive motor 123 may be any drive means such as an electric motor or a piezo element used for an optical lens. For example, the control unit 100 rotates the drive motor 123 to a predetermined rotation angle according to several preset focusing distances, and the CCD sensor 122 clearly connects an object or human body at an actual distance corresponding to the focusing distance. Can be imaged.

  When the driving motor 123 drives the lens 121 to perform a focusing sweep, the CCD sensor 122 captures a plurality of image frames in time series. For example, as shown in FIG. 3, in one focusing sweep operation, the focusing distance is sequentially shifted to 50 meters (m), 40 m, 30 m, 20 m, 10 m, and a total of five images FRAME1 to FRAME5 are obtained. Imaged.

  4A to 4E show a plurality of images FRAME1 to FRAME5 taken in one focusing sweep. FIG. 4A shows an image FRAME1 captured when the focusing distance is 50 m, FIG. 4B shows an image FRAME2 captured when the focusing distance is 40 m, and FIG. 4C shows a focusing distance of 30 m. FIG. 4D shows an image FRAME4 picked up when the focusing distance is 20 m, and FIG. 4E shows an image FRAME5 picked up when the focusing distance is 10 m.

  4A to 4E, a thick line indicates an image of an object on which a clear image is formed on the CCD sensor 122, and a thin line indicates an image of an object on which no clear image is formed on the CCD sensor 122.

  As shown in FIG. 4A, an object on a vertical plane that is in front of the vehicle 20 and within the imaging angle α range and 50 m away from the vehicle 20 is clearly imaged. Here, the object clearly imaged on the CCD sensor 122 includes a road surface marker 30a located at a position 50 m ahead, a part of the road shoulder 30b, and a building 30c near the road. Therefore, an image FRAME1 including a clear image (thick line) of these objects and a blurred image of other objects, that is, an out-of-focus image (thin line) is obtained.

  As shown in FIG. 4B, when the driving motor 123 drives the lens 121 to shorten the focusing distance, an object at a position 40 m away from the front of the vehicle 20 is clearly imaged. Here, examples of the object that is clearly imaged by the CCD sensor 122 include a part of the road marking 30a, a part of the road shoulder 30b, and another vehicle 30d at a position 40m ahead. Therefore, an image FRAME2 including a clear image of these objects and a blurred image of other objects is obtained.

  As shown in FIG. 4C, when the driving motor 123 further drives the lens 121 to further reduce the focusing distance, an object at a position 30 m away from the front of the vehicle 20 is clearly imaged. Here, there are a part of the road surface marker 30a, a part of the road shoulder 30b, a tree 30e near the road, and the like as objects that are clearly imaged on the CCD sensor 122. An image FRAME3 including a clear image of these objects and a blurred image of other objects is obtained.

  As shown in FIG. 4D, when the lens 121 is further driven by the drive motor 123 to further reduce the focusing distance, an object at a position 20 m away from the front of the vehicle 20 is clearly imaged. Here, examples of the object that is clearly imaged by the CCD sensor 122 include a part of the road surface marker 30a, a part of the road shoulder 30b, a bicycle 30f, and the like located 20m ahead. An image FRAME4 including a clear image of these objects and a blurred image of other objects is obtained.

  Further, as shown in FIG. 4E, when the lens 121 is driven by the drive motor 123 to further shorten the focusing distance, an object at a position 10 m away from the front of the vehicle 20 is clearly imaged. Here, examples of the object that is clearly imaged by the CCD sensor 122 include a part of the road surface marker 30a and a part of the road shoulder 30b at a position 10 m ahead. An image FRAME5 including a clear image of these objects and a blurred image of other objects is obtained.

  The images FRAME1 to FRAME5 picked up by the image pickup unit 120 are input to the image processing unit 140 as image signals and processed in the image processing unit 140 as described later.

  Next, the operation of the collision preventing apparatus 10 will be described in detail. In the focusing sweep of the present embodiment, the longest focusing distance is 50 m, the shortest focusing distance is 10 m, and in one focusing sweep, the focusing distance is sequentially shortened to 50 m, 40 m, 30 m, 20 m, 10 m, or 10 m, 20 m, 30 m, 40 m, and 50 m are made longer in this order, and thus image frames are taken at the respective focusing distances, and five images FRAME1 to FRAME5 are imaged in one focusing sweep.

  FIG. 5 is a flowchart showing processing in the collision preventing apparatus 10. As shown in the figure, first, the focus sweep starts in step S501. The drive motor 123 of the imaging unit 120 drives the lens 121 so that an object 50 m ahead is clearly imaged on the CCD sensor 122, that is, the focusing distance is 50 m.

  In step S502, the CCD sensor 122 acquires the image FRAME1 of the object scene and outputs the image FRAME1 to the image processing unit 140.

  In step S503, the image processing unit 140 determines whether there is an image of an object on which a clear image is formed in the image FRAME1 sent from the imaging unit 120. In this example, the image processing unit 140 recognizes an image of an object such as a part of a road surface sign 30a, a part of a road shoulder 30b, a part of a road shoulder 30b, and a building 30c beside a road on which a clear image is formed. Further, in the image FRAME1, other vehicles 30d in front of the vehicle 20, trees 30e, bicycles 30f, and the like are also captured, but are not clearly captured and are not recognized by the image processing unit 140.

  In step S504, the image processing unit 140 calculates the position of the recognized object such as the road marking 30a, the road shoulder 30b, and the building 30c based on the focusing distance when the image FRAME1 is captured. A distribution image indicating the position of the object is generated.

  FIG. 6 shows an object distribution image generated by the image processing unit 140. Since the focusing distance at this point is 50 m, only an object in a vertical plane 50 m away from the front can clearly form an image. Also, the coordinate position of the clearly imaged object in the image FRAME1 is specified by the angular position when the image FRAME1 is taken. As a result, the image processing unit 140 detects objects such as the recognized road marking 30a, the road shoulder 30b, and the building 30c in the object distribution image shown in FIG. A graphic representing the is generated. This figure may be a figure that can grasp the characteristics of the object at a glance, that is, what the object is.

  Therefore, in step S505, it is determined whether or not the above-described processing has been completed for all image frames captured by the imaging unit 120 in one focusing sweep. If not, the process proceeds to step S506.

  In step S506, the drive motor 120 drives the lens 121 to adjust the focusing distance. For example, the focusing distance is shortened, and an object at a position 40 m ahead of the vehicle 20 is clearly imaged on the CCD sensor 122.

  Next, each step after step S502 is executed again. In step S502, the imaging unit 120 acquires the image FRAME2 (FIG. 4B) by the CCD sensor 122, and sends the image FRAME2 to the image processing unit 140.

  In step S503, the image processing unit 140, like the processing for the image FRAME1 described above, from the image FRAME2, a part of the road marking 30a 40m away from the vehicle 20 as a clearly imaged object, the road shoulder 30b. Recognize some and other vehicles 30d.

  In step S504, as described above, the image processing unit 140 corresponds to a distance 40 m away from the vehicle 20 in the object distribution image based on the focusing distance (40 m in this case) when the image FRAME2 is captured. A figure representing an object such as a portion of the road marking 30a, a portion of the road shoulder 30b, and another vehicle 30d determined from the image FRAME2 is generated at the position.

  In step S505, it is determined whether or not the above-described processing has been completed for all image frames captured by the imaging unit 120 in one focusing sweep. In this example, FRAME3 to FRAME5 are continuously captured. Similar processing is performed. Next, processing for the images FRAME3 to FRAME5 will be briefly described.

  The imaging unit 120 images the image FRAME3 with a focusing distance of 30 m. The image processing unit 140 recognizes an image of an object such as a part of the road marking 30a, a part of the road shoulder 30b, and a tree 30e that is clearly imaged 30m away from the vehicle 20. The image processing unit 140 generates a graphic representing these objects at a position corresponding to 30 m ahead in the object distribution image shown in FIG.

  Next, the imaging unit 120 sets the focusing distance to 20 m and captures the image FRAME4. The image processing unit 140 recognizes images of objects such as the road marking 30a, the road shoulder 30b, and the bicycle 30f that are clearly imaged 20m away from the vehicle 20, and in the object distribution image shown in FIG. Then, figures representing these objects are generated at positions corresponding to the front 20 m.

  Further, the imaging unit 120 images the image FRAME5 with the focusing distance set to 10 m. The image processing unit 140 recognizes an image of an object such as a part of the road surface marker 30a and a part of the road shoulder 30b that are separated from each other 10m ahead of the vehicle 20 and is clearly imaged. Figures representing these objects are generated at positions corresponding to 10 m.

  When the image processing unit 140 performs the processing of steps S503 and S504 for all image frames captured in one focusing sweep, the object distribution image shown in FIG. 6 showing the position distribution of the object in front of the vehicle 20 is obtained. Complete. The object distribution image in FIG. 6 includes all objects in the range of 50 m to 10 m ahead of the vehicle 20 imaged by the imaging unit 120.

  If it is determined in step S505 that the image processing has been completed for all image frames captured in one focusing sweep, the control returns to step S501. The imaging unit 120 may perform the focusing sweep again from the previously used focusing distance (10 m in this example). At this time, a plurality of image frames are captured again, and the image processing unit 140 generates the next object distribution image and updates the currently displayed object distribution image.

  The display unit 200 displays the generated object distribution image. The driver of the vehicle 20 can find buildings, trees, vehicles, bicycles and pedestrians in front of the vehicle 20 from this display, and can effectively avoid a collision accident. In addition, the object distribution image can notify the driver of the vehicle 20 of a sign such as a road sign and a road shoulder, and the driver can easily steer the vehicle 20 along the displayed road sign.

  The collision determination unit 310 can determine whether the vehicle 20 collides with an obstacle or the like based on the object distribution image generated by the image processing unit 140.

  When the distance from the vehicle 20 to the obstacle ahead is shorter than the predetermined safety distance, the collision determination unit 310 determines that there is a danger of collision, and the display unit 200 displays the dangerous obstacle in a conspicuous color, for example, red. Is displayed to inform the driver of the vehicle 20. Further, the sound unit 210 may generate a warning sound to warn. Further, the vibration unit 220 may vibrate and warn.

  In addition, the object distribution image generated by the image processor 140 displays roadside structures such as buildings, trees, and plantations, other cars, light vehicles such as bicycles, pedestrians, road signs, and shoulders. Yes. Therefore, the collision determination unit 310 may determine whether or not the own vehicle 20 “collides” with such road markings and road shoulders. When the collision determination unit 310 determines that the vehicle 20 collides with a road surface sign or a road shoulder, the collision determination unit 310 displays that the vehicle 20 has deviated from the current lane. In this case, the display unit 200, the sound unit 210, and / or the vibration unit 220 warn the driver of the vehicle 20 to change the traveling direction of the vehicle 20. As a result, for example, the risk of a driver getting tired and deviating from the lane on an expressway and causing an accident can be avoided, and the safety of the vehicle 20 can be improved.

  The collision prevention apparatus 10 is further provided with a speed detection unit 150 that detects the speed of the vehicle 20. The collision determination unit 310 can calculate the safety distance in real time based on the speed of the vehicle 20 detected by the speed detection unit 150, and can more effectively prevent the collision.

  In the collision prevention apparatus 10, information on the safety distance corresponding to the traveling speed of the vehicle 20 may be stored in advance in a recording medium such as the ROM 230, for example. The collision determination unit 310 may calculate the safety distance in real time from the speed of the vehicle 20 detected by the speed detection unit 150 using a mathematical formula.

  Further, in the collision prevention device 10, when the collision determination unit 310 determines that the vehicle can collide, the vehicle control unit 400 controls the steering drive system of the vehicle 20 under the control of the control unit 100 to brake and accelerate the vehicle 20. And / or steering, i.e. braking, accelerator and handling. Thus, when the collision determination unit 310 determines the possibility of a collision with an obstacle ahead of the vehicle 20 from the object distribution image generated by the image processing unit 140, the vehicle control unit 400 determines a predetermined distance between the vehicle 20 and the preceding vehicle. The vehicle 20 can be braked so as to maintain a safe distance.

  On the other hand, when the vehicle 20 deviates from the lane, the vehicle control unit 400 steers the vehicle 20 and prevents the vehicle 20 from departing from the lane. Of course, the driver of the vehicle 20 may intentionally change the lane. At that time, when the driver operates a little strongly to resist the operation by the vehicle control unit 400, the operation detection unit (not shown) of the device detects this operation, and thereby the control of the vehicle control unit 400 is immediately performed. Thus, the control of the vehicle 20 by the driver is permitted.

  Furthermore, it is desirable that the imaging unit 120 of the vehicle 20 performs high-speed imaging. If the imaging unit 120 is configured to capture images at high speed, the exposure time of each image frame is shortened, so blurring of each image frame due to vibration during high speed traveling of the vehicle 20 is suppressed, and thus the image processing unit 140 is An object can be reliably recognized from the captured image. In addition, by performing high-speed imaging with the imaging unit 120, an object can be accurately recognized within a long distance in the traveling direction of the vehicle 20, and the update speed of the object distribution image is increased. Thereby, the real time property of the object distribution image displayed on the display unit 200 is further improved.

  In the above-described embodiment, the imaging unit 120 captures five image frames in one focusing sweep, the longest focusing distance is 50 m, the shortest focusing distance is 10 m, and the step width of each focusing distance therebetween is It was 10m. This is only an example. For example, if the step distance of the focusing distance between two consecutively captured image frames is shortened so that all objects in front of the vehicle 20 can be recognized, The recognition accuracy of each object is increased accordingly, so that it is possible to recognize and display small objects and objects or human bodies that are adjacent in a direction substantially perpendicular to the traveling direction. Thereby, the load of processing each image frame in the image processing unit 140 increases. However, the recognition accuracy for recognizing such an object is set to be selectable, and the user can select an optimum focusing distance that reduces the load on the image processing unit 140 while maintaining the required accuracy depending on the situation. The step width may be determined. The step width of the focusing distance between two image frames that are continuously captured is preferably 2 m or less, and more preferably 1 m or less.

  On the other hand, the longest focusing distance and the shortest focusing distance may be set to values other than the above-described 50 m and 10 m, respectively. In order to improve the safety of the vehicle 20, it is desirable that the longest focusing distance is 50 m or longer and the shortest focusing distance is 5 m or shorter. Further, the longest focusing distance may be increased according to the traveling speed of the vehicle 20 detected by the speed detection unit 150. In that case, the shortest focusing distance may also be increased correspondingly. In this way, it is possible to adaptively cope with city driving and highway driving.

  In the present embodiment, every time the imaging unit 120 performs a focusing sweep, the image processing unit 140 generates an object distribution image and updates it. The anti-collision device 10 is one of the important factors in real time. In order to improve this, it is desirable that the imaging unit 120 updates the object distribution image by performing a focusing sweep at least once per second. Furthermore, when the vehicle 20 travels at a high speed, it is desirable to improve the real-time property of the collision prevention device 10 as compared to when traveling on a general road. However, the imaging unit 120 performs focusing sweep five times or more per second to distribute the object. It is desirable to update the image. Therefore, it is desirable for the imaging unit 120 to perform high-speed imaging at a speed of 500 times / second. For example, an object within a wide distance range of 100 m to 1 m ahead of the vehicle 20 can be recognized, and the object distribution image can be updated at a frequency of 5 times / second or more. In this way, the range, accuracy and real time performance of object recognition are improved.

  In the embodiment described above, one imaging unit 120 is provided at the front portion of the vehicle 20. However, the present invention is not limited to this. For example, a similar imaging unit may be provided at the rear part of the vehicle 20, whereby the driver of the vehicle 20 can grasp the object distribution and the vehicle situation behind the vehicle. When another vehicle approaches from below the predetermined safety distance while the vehicle is traveling, the display unit 200, the voice unit 210 or the vibration unit 220 informs the driver of the host vehicle 20 and prompts the driver to respond. be able to. This makes it possible to more safely avoid a vehicle tracking accident from behind. Further, the vehicle control unit 400 may be configured to accelerate the vehicle 20 so that the vehicle is not tracked at a close distance from the vehicle from the rear.

  Furthermore, a plurality of imaging units may be provided at a plurality of locations on the side and / or rear of the vehicle 20. As a result, when the vehicle 20 is recognized or detected in the lateral direction and / or behind the vehicle 20 and the vehicle is parked on the side of the road, in a parking lot or a garage, The position of obstacles and parking signs can be grasped, and collision accidents during parking can be avoided. Further, the vehicle control unit 400 may be configured such that the vehicle 20 can be parked without any driver operation or with a minimum operation by performing forward or backward movement, steering, acceleration, or brake control.

  By arranging a plurality of imaging units on the vehicle 20 and imaging each of the imaging units in different directions, the image processing unit 140 generates an image showing the distribution of objects and human bodies in a wider range around the vehicle 20 It is possible to improve the convenience of use.

  The image processing unit 140 may be configured to additionally visually display a grid line indicating the distance from the vehicle 20 in the generated object distribution image. As a result, the driver of the vehicle 20 can know the distance from the own vehicle to the recognized object from the object distribution image displayed on the display unit 200, and can further ensure safety while the vehicle is traveling.

  In the above-described embodiment, the imaging unit 120 includes the CCD sensor 122. However, the present invention is not limited to a specific type of such an imaging device. A complementary metal oxide semiconductor (CMOS) sensor or other optical sensor may be used as the image sensor. Further, when the display unit 200 does not display the color of the object, a monochromatic image sensor may be used. This contributes to a reduction in the cost of the collision prevention device 120, and the processing speed in the image processing unit 140 can be improved.

  Further, an infrared sensor may be used as the imaging element in order to cope with use in a dark environment such as at night or in a tunnel. Thereby, for example, the imaging unit 120 can image an object or a human body in an area where the illumination around the vehicle 20 does not reach even at night, and the image processing unit 140 can discriminate the object or human body from the image frame, thereby preventing collision. The practicality of the device 10 is further improved.

  In the above-described embodiment, the image processing unit 140 generates a planar object distribution image as shown in FIG. However, the present invention is not limited to this. The image processing unit 140 may be configured to generate a three-dimensional (three-dimensional) object distribution image.

  FIG. 7 is a schematic diagram illustrating a method by which the image processing unit 140 generates a three-dimensional object distribution image. In the same figure, the images FRAME1 to FRAME5 shown in FIGS. 4A to 4E are arranged at positions corresponding to the respective focusing distances, and are displayed so as to appear stereoscopically with a frame size corresponding to the imaging angle α of the imaging unit 120. The state is shown.

  As in step S503 shown in FIG. 5, the image processing unit 140 recognizes an object on which a clear image is formed in the images FRAME1 to FRAME5.

  Further, as in step S504 shown in FIG. 5, the image processing unit 140 calculates the positions of these recognized objects based on the focusing distances at the time of imaging of the images FRAME1 to FRAME5, and the positions of these objects. A stereoscopic image that visually represents the distribution is generated. Specifically, the positions of the recognized objects in the images FRAME1 to FRAME5 are determined by the respective angular positions when the images FRAME1 to FRAME5 are taken. Therefore, the image processing unit 140 generates a three-dimensional figure 127 that displays the figures of these recognized objects at the respective focusing distance positions in front of the vehicle 20 in the three-dimensional space. In this way, a three-dimensional (three-dimensional) object distribution image 127 is generated.

  When displaying the three-dimensional object distribution image, the display unit 200 can change the viewing angle for displaying the object distribution image 127 by the operation of the driver of the vehicle 20. For example, the viewing angle can be converted to display at the viewing angle of the bird's eye view. Accordingly, the driver of the vehicle 20 can directly or intuitively grasp the exact position of each object in front of the vehicle 20. Further, if the driver sets the viewing angle for displaying the three-dimensional object distribution image 127 to the limit value, a planar object distribution image can be displayed as in the embodiment described above with reference to FIG.

  In the above-described embodiment, the imaging angle α of the imaging unit 120 is fixed, that is, the lens 121 is a single focus lens. However, the imaging angle α is variable, that is, the lens 121 may be a zoom lens. The zoom lens is driven by a zooming mechanism, which, for example, reduces the focal distance and increases the imaging angle α in the focus sweep, for example when shortening the focusing distance, thereby recognizing as many objects as possible nearby. can do. On the other hand, when the focusing distance is lengthened, the focal angle is lengthened and the photographing angle is decreased, so that a distant object is depicted in a large size in the image frame. In this way, the recognition rate of objects and human bodies is further improved.

  Further, in the above-described embodiment, the vehicle 20 is an example of a moving body. However, the moving body collision prevention apparatus according to the present invention is not limited to the land, but can be moved, in particular, can travel autonomously, or can be produced or mined. You may arrange | position to the conveyance mechanism used on the spot and a campus.

  The best embodiment of the present invention has been described above, but the present invention is not limited to this. It is also possible to obtain the functions and effects of each part with some components in the collision preventing apparatus 10.

  Although the present invention has been described in detail with reference to specific embodiments, the spirit and scope of the present invention such that various substitutions, modifications, and changes in form and detail are defined by the appended claims. It will be apparent to those skilled in the art that this can be done without departing from the above. For example, a part of the configuration of the collision prevention apparatus 10 can be used to obtain the effects of each configuration. Accordingly, the scope of the present invention should not be limited to the description of the above-described embodiment and the accompanying drawings, but should be determined based on the description of the claims and equivalents thereof.

  The image processing apparatus according to the present invention can be applied to position identification and monitoring control of each facility at a production site, switching of a viewing angle in a live broadcast of a sports game, and an interactive entertainment system and a mobile object collision prevention apparatus.

α Imaging angle of the imaging unit
10 Moving object collision prevention device
20 Vehicles that are vehicles
100 Control unit
120 Imaging unit
121 lens
122 CCD sensor
123 Focusing mechanism
140 Image processor
150 Speed detector
200 display
210 Voice
220 Vibration part
230 ROM
310 Collision determination unit
30a road marking
30b shoulder
30c building
30d car
30e wood
30f bicycle
400 Vehicle control unit

Claims (14)

Imaging means;
Image processing means for processing an image picked up by the image pickup means,
The imaging means includes an imaging lens, a focusing mechanism that drives the imaging lens to change a focusing distance, and an imaging element,
The focusing mechanism drives the imaging lens to perform a focus sweep that sequentially changes a focusing distance so that an object at a different position in the optical axis direction of the imaging lens is clearly imaged on the imaging element;
The imaging means captures images according to a plurality of different focusing distances in the focus sweep,
The image processing unit recognizes an object that is clearly formed in each of the plurality of images from the plurality of captured images, and corresponds to a focusing distance at the time of capturing each of the plurality of images. And generating an object distribution image for displaying the position of the recognized object.   2. The image processing apparatus according to claim 1, wherein the focusing distance changes with a step width of 1 m or less in the focus sweep.   The image processing apparatus according to claim 1, wherein the imaging unit images at a frequency of 500 images / second or more. The device according to any one of claims 1 to 3,
The imaging means repeats the focus sweep a plurality of times,
The image processing device generates the plurality of object distribution images corresponding to the plurality of focus sweeps.   5. The image processing apparatus according to claim 4, wherein the imaging unit performs the focus sweep five times or more per second.   The image processing apparatus according to claim 1, wherein the imaging element includes at least one of a charge coupled device (CCD) sensor and a complementary metal oxide semiconductor (CMOS) sensor.   The image processing apparatus according to claim 1, wherein the image sensor includes an infrared image sensor. The apparatus according to claim 1, wherein the imaging lens includes a zoom lens;
The imaging means includes a zooming mechanism that drives the zoom lens and changes its focal length.
When the focusing distance becomes longer than a predetermined value, the zooming mechanism drives the zoom lens to increase the focal length, and when the focusing distance becomes shorter than the predetermined value, the zooming mechanism drives the zoom lens to increase the focal length. An image processing apparatus characterized by shortening the length. Image processing method in an image processing apparatus comprising: an imaging lens; a focusing mechanism that drives the imaging lens to change a focusing distance; an imaging unit having an imaging element; and an image processing unit that processes an image captured by the imaging unit Because
In the imaging unit, the focusing mechanism sequentially changes a focusing distance so as to drive the imaging lens to clearly form an object at a different position in the optical axis direction of the imaging lens on the imaging element. And taking each image according to a plurality of different focusing distances in the focus sweep,
The image processing means recognizes an object that is clearly formed in each of the plurality of images from the plurality of captured images, and corresponds to a focusing distance at the time of capturing each of the plurality of images. And generating an object distribution image for displaying the position of the recognized object. A moving body collision prevention device provided on a moving body,
Imaging means for imaging the periphery of the moving body;
Image processing means for processing an image picked up by the image pickup means;
A collision determination means,
The imaging means includes an imaging lens, a focusing mechanism that drives the imaging lens to change a focusing distance, and an imaging element,
The focusing mechanism drives the imaging lens to perform a focus sweep that sequentially changes a focusing distance so that a plurality of objects separated from the moving body are clearly imaged on the imaging element;
The imaging means captures images according to a plurality of different focusing distances in the focus sweep,
The image processing unit recognizes an object that is clearly formed in each of the plurality of images from the plurality of captured images, and corresponds to a focusing distance at the time of capturing each of the plurality of images. And generating an object distribution image displaying the position of the recognized object,
The collision detection apparatus according to claim 1, wherein the collision determination unit determines whether or not the mobile object and the recognized object collide based on the object distribution image. The apparatus according to claim 10, further comprising display means for displaying the object distribution image.
When the collision determination unit determines that the moving body and the recognized object collide, the display unit displays the object in a different color. The apparatus according to claim 10, further comprising notification means for notifying a pilot of the mobile body.
The mobile body collision prevention apparatus according to claim 1, wherein when the collision determination unit determines that the mobile body and the recognized object collide, the notification unit generates a warning message to the operator. A moving body collision prevention device provided on a moving body,
Imaging means for imaging the periphery of the moving body;
Image processing means for processing an image picked up by the image pickup means;
Display means,
The imaging means includes an imaging lens, a focusing mechanism that drives the imaging lens to change a focusing distance, and an imaging device,
The focusing mechanism drives the imaging lens to perform a focus sweep that sequentially changes a focusing distance so that a plurality of objects separated from the moving body are clearly imaged on the imaging element;
The imaging means captures each image according to a plurality of different focusing distances in the focus sweep,
The image processing unit recognizes an object that is clearly formed in each of the plurality of images from the plurality of captured images, and corresponds to a focusing distance at the time of capturing each of the plurality of images. And generating an object distribution image displaying the position of the recognized object,
The moving object collision preventing apparatus, wherein the display means displays the recorded object distribution image. 14. The mobile object collision preventing apparatus according to claim 13, wherein the display unit displays the object distribution image as a three-dimensional image.

Images