JP2009143385A - Movable body detection device, movable body detection method, and movable body detection program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a movable body existing in a distant position without using a large memory capacity. <P>SOLUTION: In a movable body detection device 100, a vanishing point calculation part 106 calculates a vanishing point when an image (for example, an image photographed by a BCM) is acquired. In the movable body detection device 100, a partial area image extraction part 108 stores a predetermined range of an image including the vanishing point as a partial area image in a partial area image storage 103, and a movable body detection part 109 detects the movable body based on the partial area image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a moving object detection apparatus, a moving object detection method, and a moving object detection program for detecting a moving object from an image photographed by a camera mounted on the moving object, and in particular, without using a large memory capacity and far away. The present invention relates to a moving body detection apparatus, a moving body detection method, and a moving body detection program that can accurately detect a moving body present at a position.

  Conventionally, vehicle accidents have frequently occurred near intersections. This is because the driver has a blind spot in the vicinity of the intersection, and the driver cannot quickly find a vehicle approaching the host vehicle from the side surface that becomes the blind spot.

  Therefore, in recent years, the blind corner monitor (BCM), which captures the image of the driver's blind spot and displays it on the driver's seat monitor, eliminates the driver's blind spot and causes an accident near the intersection. Technology to prevent this is being put into practical use. Patent Document 1 also discloses a technique for detecting a moving body (a vehicle approaching the host vehicle) using spatial parallax or temporal parallax and notifying the driver.

JP 2004-173071 A

  In general, on an image, a moving body that is physically far away is a smaller area. In particular, when a wide area image such as a BCM is image-converted and output so that the image of the blind spot area is contained in one image, the area on the image of the moving body that approaches the host vehicle from a distance becomes small.

  Therefore, for example, when a moving object is detected by applying the technique according to Patent Document 1 to BCM, it is necessary to detect a moving object existing in a small area on the image. However, when a moving body existing in a small area is detected using a technique such as Patent Document 1, it is necessary to accumulate information for several frames and take an image difference. There is a problem that the manufacturing cost becomes high.

  The present invention has been made to solve the above-described problems caused by the prior art, and is capable of accurately detecting a moving body located at a distant position without using a large memory capacity. An object of the present invention is to provide a moving body detection method and a moving body detection program.

  In order to solve the above-described problems and achieve the object, the moving body detection device is a moving body detection device that detects a moving body from an image taken by a camera mounted on the moving body, and is based on the image. Vanishing point specifying means for specifying a vanishing point in the image, a partial area storage means for storing an image of a predetermined range including the vanishing point specified by the vanishing point specifying means as a partial area image, and the part It is a requirement that the apparatus further comprises moving body detection means for detecting a moving body in an image photographed by the camera based on the partial area image stored in the area storage means.

  Further, in this moving body detection device, in the above moving body detection device, the vanishing point specifying unit generates a vanishing point candidate line based on the feature amount of the camera and extracts a straight line from the partial region image. And it is a requirement to specify the intersection of the vanishing point candidate line and the straight line as the vanishing point.

  Further, according to the moving object detection device, in the moving object detection device described above, the vanishing point specifying unit is a first unit that indicates a distance between a candidate vanishing point that is a vanishing point candidate and a straight line extracted from the partial region image. Based on one function and a second function indicating the distance between the candidate vanishing point and the vanishing point candidate line generated based on the feature amount of the camera, the sum of the values of the first and second functions is The requirement is to specify the candidate vanishing point that is the minimum value as the vanishing point.

  In addition, in the moving body detection device, the moving body detection device requires that the vanishing point specifying unit assigns a predetermined weight to the value of the second function.

  Moreover, this moving body detection apparatus requires that the vanishing point specifying means corrects the vanishing point candidate line based on the lens curvature of the camera in the above moving body detection apparatus.

  Further, the moving body detection method is a moving body detection method of a moving body detection device that detects a moving body from an image captured by a camera mounted on the moving body, and the moving body detection device uses the camera. A storage step of storing the captured image in a storage device; a vanishing point specifying step of specifying a vanishing point in the image based on the image stored in the storage device; and a predetermined range including the vanishing point A partial area storage step of storing an image in the storage device as a partial area image; and a moving body detection step of detecting a moving body in the image captured by the camera based on the partial area image stored in the storage device; It is a requirement that

  Further, the moving body detection program stores, in a computer, a storage procedure for storing an image photographed by a camera mounted on a moving body in a storage device, and an image stored in the storage device. A vanishing point specifying procedure for specifying a vanishing point, a partial area storing procedure for storing an image of a predetermined range including the vanishing point specified by the vanishing point specifying procedure in the storage device as a partial region image, and storing in the storage device It is a requirement that a moving body detection procedure for detecting a moving body in an image photographed by the camera is executed based on the partial area image.

  According to this moving object detection device, the vanishing point in the image is specified, only the partial area image in a predetermined range including the specified vanishing point is stored in the storage device, and the moving object is detected based on the stored partial area image. Since the detection is performed, the storage target can be limited to the partial area image, and the moving object existing at a far position can be accurately detected without using a large memory capacity.

  Further, according to this moving object detection device, a vanishing point candidate line is generated based on the feature amount of the camera, a straight line is extracted from the partial area image, and an intersection of the vanishing point candidate line and the straight line is used as a vanishing point. Since it specifies, a vanishing point can be specified easily.

  Further, according to the moving object detection device, the first function indicating the distance between the candidate vanishing point that is a vanishing point candidate and a straight line extracted from the partial area image, the candidate vanishing point, and the feature amount of the camera are used. Since the candidate vanishing point at which the sum of the values of the first and second functions is the minimum value is specified as the vanishing point based on the second function indicating the distance to the vanishing point candidate line generated as described above. Even when the vanishing point is not determined by the method 2 (when the intersection is not uniquely determined), it is possible to calculate the optimal vanishing point.

  Moreover, according to this moving body detection apparatus, since the predetermined weight is given to the value of the second function, the vanishing point calculation accuracy can be improved.

  Moreover, according to this moving body detection apparatus, since the vanishing point candidate line is corrected based on the lens curvature of the camera, the vanishing point calculation accuracy can be improved.

  Exemplary embodiments of a mobile object detection device, a mobile object detection method, and a mobile object detection program according to the present invention will be explained below in detail with reference to the accompanying drawings.

  First, a blind corner monitor (hereinafter referred to as BCM) used in the first embodiment will be described. FIG. 1 is a diagram for explaining a blind spot of a driver. Originally, when a driver has an intersection with poor visibility on the left and right (when entering the T-junction), the driver must leave the vehicle 10 to a considerable extent or be in a very close state. The vehicle 20 does not enter the field of view (see FIG. 1).

  Therefore, by using a monitor (BCM) equipped with a wide-angle camera with a sufficient viewing angle as a supplement to the driver's blind spot, the driver can move the side vehicle (for example, the vehicle 20) with a minimum approach. It becomes possible to discover early.

  FIG. 2 is a diagram showing an example of video by BCM. As shown in the figure, this BCM can display an image of an area that becomes a blind spot of a driver on a single screen. However, since the wide-angle image is output after being converted so that the image of the blind spot area is contained in one image, the moving object that approaches the host vehicle from a distance (particularly, the moving object near the vanishing points 30 and 40). The area on the screen becomes small, and when a mobile object detection is attempted using a well-known technique (for example, Patent Document 1), it is necessary to accumulate information for several frames and obtain an image difference. Is required.

  Next, the outline and features of the moving object detection apparatus according to the first embodiment will be described. FIG. 3 is a diagram for explaining the outline and features of the moving object detection device according to the first embodiment. As illustrated in FIG. 1, the moving body detection apparatus according to the first embodiment specifies vanishing points 30 and 40 in an image when an image (for example, an image photographed by BCM) is acquired. The moving body detection device stores an image of a predetermined range including the specified vanishing points 30 and 40 in the storage device as a partial region image, and detects the moving body based on the stored partial region image.

  As described above, the moving body detection apparatus according to the first embodiment specifies the vanishing point in the image, stores only the partial area image in the predetermined range including the specified vanishing point in the storage device, and stores the stored partial area. Since the moving object is detected based on the image, the storage object can be limited to the partial area image, and the moving object existing at a far position can be accurately detected without using a large memory capacity.

  Further, since the processing relating to the moving object detection is executed only on the partial area image, not on the entire screen, it is possible to reduce the calculation and to increase the speed of the moving object detection process.

  By the way, in the moving body detection apparatus according to the first embodiment, it is important to detect a moving body (or an obstacle) located mainly at a location where the driver is blind spot. Therefore, in the first embodiment, as shown in FIG. 3, the detection target region is limited to a partial region centered around the left and right road vanishing points. This is because a moving object that exists in the front or in the vicinity of the host vehicle should enter the field of view if it actually sees the front without going through the BCM, and warn sufficiently when it exists far away. . On the other hand, since the moving body existing in the distance appears near the vanishing point and approaches, it is necessary to include the vanishing point in the partial area image.

  The vanishing point on the left and right when the vehicle enters the T-shaped road apparently moves along with the approach angle of the vehicle's T-shaped road, the right turn, and the left turn. FIG. 4 is a diagram for explaining the movement of the vanishing point when a right turn is performed.

  As shown in the upper part of FIG. 4, when the vehicle makes a right turn, the images captured by the BCM change in the order of images 60, 61, 62, and the vanishing points 50 on the images 60 to 62 are shown in the lower part of FIG. 4. The vanishing point shift changes as shown in FIG. The point a in the vanishing point moving diagram 70 indicates the position of the vanishing point 50 in the image 60, the point b in the vanishing point moving diagram 70 indicates the position of the vanishing point 50 in the image 61, and the point c in the vanishing point moving diagram 70. Indicates the position of the vanishing point 50 of the image 62.

  The vanishing point locus shown in the vanishing point movement diagram 70 is uniquely determined by the lens curve of the camera. More precisely, this locus draws a curve, but can be approximated to a line-symmetric straight line with the center of the screen as an axis, as shown by lines a and b in FIG. Hereinafter, this straight line (lines a and b) is referred to as a vanishing point candidate line.

  Next, the configuration of the moving object detection apparatus that performs the above-described processing will be described. FIG. 5 is a diagram illustrating the configuration of the moving object detection apparatus 100 according to the first embodiment. As shown in the figure, the moving body detection apparatus 100 includes a reference vanishing point storage unit 101, a vanishing point candidate line calculation unit 102, a partial region image storage unit 103, an image input unit 104, and a straight line detection unit 105. A vanishing point calculation unit 106, a partial region number control unit 107, a partial region image extraction unit 108, a moving object detection unit 109, and a moving object notification unit 110.

  Among these, the reference vanishing point storage unit 101 is a vanishing point set in advance (hereinafter referred to as a reference vanishing point) according to conditions (camera installation position, camera orientation with respect to the vehicle, depression angle, etc.) at the time of camera (BCM etc.) installation. Storage means for storing the coordinates).

  FIG. 6 is a diagram for explaining a reference vanishing point set in an image. In the first embodiment, for convenience of explanation, a reference vanishing point 80a is set at a position 0 ° in front of the camera, a reference vanishing point 80b is set at a position + 90 ° from the front of the camera, and −90 ° from the front of the camera. The reference vanishing point 80c is set at the position.

  FIG. 7 is a diagram illustrating an angle at which the camera 26 installed in the vehicle 25 can shoot, and FIG. 8 is a diagram illustrating a side view of the vehicle 25 in which the camera 26 is installed. The position of the reference vanishing point 80a is set based on the depression angle of the camera (see FIG. 8), and the positions of the reference vanishing point 80b and the reference vanishing point 80c are set to be higher than the horizontal center in the screen. .

  The vanishing point candidate line calculation unit 102 is a means for calculating a vanishing point candidate line based on the coordinates of the reference vanishing point stored in the reference vanishing point storage unit 101. This vanishing point candidate line indicates that a vanishing point exists on this line according to the specifications of the camera.

  FIG. 9 is a diagram for explaining vanishing point candidate lines set in an image. As shown in the figure, the vanishing point candidate line calculation unit 102 calculates a vanishing point candidate line by calculating a straight line passing through the reference vanishing points 80a and 80c and a straight line passing through the reference vanishing points 80a and 80b. The vanishing point candidate line calculation unit 102 outputs the calculated vanishing point candidate line information to the vanishing point calculation unit 106. Note that the vanishing point candidate line calculation unit 102 may correct the vanishing point candidate line based on the lens curve of the camera.

  The partial area image storage unit 103 is a storage unit that stores the vanishing point calculated by the vanishing point calculation unit 106 and the image of the partial area extracted by the partial region image extraction unit 108. FIG. 10 is a diagram illustrating an example of data stored in the partial area image storage unit 103. As shown in the figure, the partial area image storage unit 103 stores a frame identification number for identifying each partial area image, vanishing point coordinates, and partial area image data (for example, binary data) in association with each other. Yes.

  The image input unit 104 is connected to the above-described camera (BCM or the like) and is a unit that receives image information from the camera. The image input unit 104 outputs image information acquired from the camera to the straight line detection unit 105 and the partial region image extraction unit 108.

  The straight line detection unit 105 is a unit that detects a straight line from the partial area image (see FIG. 3) of the acquired image when the image is acquired from the image input unit 104. Specifically, the straight line detection unit 105 sequentially executes processing for specifying a partial region image in the image and processing for detecting a straight line from the partial region image.

  First, when the straight line detection unit 105 specifies a partial region image in the image, the partial region image is specified based on the coordinates of the latest vanishing point stored in the partial region image storage unit 103. That is, the straight line detection unit 105 specifies the coordinates corresponding to the coordinates of the latest vanishing point on the image acquired from the image input unit 104, and specifies the image of a predetermined region including such coordinates as a partial region image. . In the initial stage, the straight line detection unit 105 identifies the partial region image based on the coordinates of the reference vanishing point.

  Subsequently, the straight line detection unit 105 performs straight line detection from the partial region image. This straight line detection may be performed using any known technique. For example, the straight line detection unit 105 binarizes the edge image by creating a differential image of the partial region image, for example. Straight line detection is performed by performing Hough transform using the edge image as a feature point. The straight line detection unit 105 outputs information on the detected straight lines (a plurality of straight lines) to the vanishing point calculation unit 106.

  In addition, when performing straight line detection, the straight line detection unit 105 may perform Hough transformation only on the lower part of the vanishing point candidate line in the binarized image. In this way, it is possible to exclude vertical edges such as buildings and pillars that are not related to vanishing points.

  The vanishing point calculation unit 106 is a means for calculating a vanishing point based on the vanishing point candidate line acquired from the vanishing point candidate line calculation unit 102 and the straight line acquired from the straight line detection unit 105. FIG. 11 is a diagram for explaining the process of the vanishing point calculation unit 106. Here, for convenience of explanation, it is assumed that the straight line detection unit 105 detects three straight lines a, b, and c.

  The vanishing point calculation unit 106 obtains intersections between the vanishing point candidate lines and the straight lines a, b, and c, and calculates the coordinates of the obtained intersections as new vanishing point coordinates. The vanishing point calculation unit 106 outputs information on the coordinates of the vanishing point to the partial region number control unit 107 and the partial region image extraction unit 108.

  The partial area number control unit 107 is a unit that determines the number of partial area images to be extracted based on the coordinates of the vanishing points acquired from the vanishing point calculation unit 106. For example, the partial region number control unit 107 determines that the number of partial region images to be extracted is two when the coordinates of two types of vanishing points are acquired. The partial area number control unit 107 outputs the determination result to the partial area image extraction unit 108.

  The partial region image extraction unit 108 extracts a partial region image in the image input from the image input unit 104 based on the vanishing point coordinates acquired from the vanishing point calculation unit 106 and the determination result of the partial region number control unit 107. It is means to do. Here, for convenience of explanation, it is assumed that the number of vanishing point coordinates acquired from the vanishing point calculation unit 106 is equal to the number of partial region images that are the determination result of the partial region number control unit 107. Do.

  The vanishing point calculation unit 106 specifies the coordinates in the image corresponding to the coordinates of the vanishing point, and inputs an image (input from the image input unit 104) using a certain image region centered on the coordinates of the vanishing point as a partial region image. And is stored in the partial area image storage unit 103 in association with the coordinates of the vanishing point.

  The moving object detection unit 109 is means for detecting a moving object included in the partial area image based on the data of each partial area image stored in the partial area image storage unit 103. A well-known technique may be used as a method for the moving body detection unit to detect the moving body. For example, the moving object detection unit 109 performs moving object detection using temporal parallax or optical flow, and outputs a detection result (such as a position of the moving object on the image) to the moving object notification unit 110.

  The moving body notification unit 110 acquires the detection result of the moving body from the moving body detection unit 109, and uses the driver's seat monitor or a speaker (not shown) based on the acquired detection result to move the moving body to the driver. It is a means to notify existence.

  For example, the moving body notification unit 110 determines the urgent level of warning from the relationship between the area occupied by the detected moving body in the screen and the image position, and surrounds the moving body in the monitor in a rectangle according to the urgency level. Alternatively, when the urgency level is equal to or higher than a predetermined value, a warning type is selected according to the urgency level, such as a warning by voice, and the driver is warned.

  Next, a processing procedure of the moving object detection apparatus 100 according to the first embodiment will be described. FIG. 12 is a flowchart of a process procedure performed by the moving object detection apparatus 100 according to the first embodiment. As shown in the figure, in the moving object detection apparatus 100, the vanishing point candidate line calculation unit 102 acquires information on the reference vanishing point from the reference vanishing point storage unit 101 (step S101), and generates a vanishing point candidate line ( In step S102), the straight line detection unit 105 determines an initial partial region (step S103).

  Then, the straight line detection unit 105 executes a straight line detection process in the partial region image (step S104), and the vanishing point calculation unit 106 calculates the intersection of the straight line group and the vanishing point candidate line to identify the vanishing point. (Step S105), the partial area image extraction unit 108 extracts a partial area image based on the coordinates of the vanishing point (step S106).

  Subsequently, the moving body detection unit 109 executes a moving body detection process in the partial region image (step S107), and the moving body notification unit 110 outputs a detection result (step S108). Note that the processing from step S104 to step S108 is executed for each partial region image.

  As described above, the partial region image extraction unit 108 extracts the partial region image, and the moving body detection unit 109 detects the moving body from the partial region image. Therefore, the data accumulation amount can be greatly reduced, and the moving body can be accurately detected. Can be detected.

  As described above, the mobile body detection device 100 according to the first embodiment identifies the vanishing point in the image, stores only the partial area image in the predetermined range including the identified vanishing point in the storage device, and stores the memory. Since the moving object is detected based on the partial area image, the storage object can be limited to the partial area image, and the moving object existing at a distant position can be accurately detected without using a large memory capacity.

  Although the embodiments of the present invention have been described so far, the present invention may be implemented in various different forms other than the first embodiment described above. Therefore, another embodiment included in the present invention will be described below as a second embodiment.

(1) About vanishing point calculation For example, in the first embodiment, when the vanishing point calculation unit 106 calculates the vanishing point, the vanishing point candidate line created by the vanishing point candidate line calculation unit 102 and the straight line detection unit The vanishing point was calculated by obtaining the intersection with each straight line detected by 105 (see FIG. 11), but the vanishing point calculating method is not limited to this, and for example, the following evaluation function is Can be used to calculate the vanishing point.

によって表すことができる。ここで、ｄｉ（ｘ，ｙ）（ｉ＝１、・・・ｎ）は、直線検出部１０５によって検出された直線と、消失点の候補となる座標（以下、候補座標）（ｘ，ｙ）との距離を示す関数（直線の数に応じてかかる関数の数は増加する）であり、Ｄ（ｘ，ｙ）は、消失点候補線と候補座標（ｘ，ｙ）との距離を示す関数である。また、Ｗは重み係数である。消失点算出部１０６は、反復法などを用いて、評価関数Ｆ（ｘ，ｙ）が最小値をとる候補座標（ｘ，ｙ）を算出し、算出した候補座標（ｘ，ｙ）を新たな消失点の座標として特定する。 Here, the evaluation function F (x, y) is

Can be represented by Here, di (x, y) (i = 1,..., N) is a straight line detected by the straight line detection unit 105 and coordinates (hereinafter, candidate coordinates) (x, y) that are candidates for vanishing points. (The number of such functions increases according to the number of straight lines), and D (x, y) is a function indicating the distance between the vanishing point candidate line and the candidate coordinates (x, y). It is. W is a weighting factor. The vanishing point calculation unit 106 calculates candidate coordinates (x, y) at which the evaluation function F (x, y) has the minimum value using an iterative method or the like, and calculates the calculated candidate coordinates (x, y) as new values. Specify as the vanishing point coordinates.

  Thus, since the vanishing point calculation unit 106 calculates the vanishing point using the evaluation function F (x, y), the vanishing point is not determined by the method shown in the first embodiment (the intersection is not uniquely determined). ), It is possible to calculate an optimal vanishing point.

  The vanishing point calculation unit 106 calculates the vanishing point by multiplying the weight W by D (x, y) instead of di (x, y). This is because di (x, y) is a function that depends on a straight line extracted directly from an image captured by the camera, and thus may be affected by the shooting environment, but D (x, y) is This is because the function depends on the vanishing point candidate line set in advance and is not affected by the shooting environment or the like. Therefore, the precision of the vanishing point can be improved by multiplying D (x, y) by the weight W. The weighting factor W is set in advance in consideration of camera specifications, monitor resolution, the number of straight lines detected by the straight line detection unit 105, and the like.

(2) Number of partial area images For example, in Example 1 described above, the partial area number control unit 107 determines the number of partial areas based on the number of vanishing points, but is not limited thereto. is not. For example, the partial area number control unit 107 can hold map information in advance (or cooperate with the navigation system) and determine the number of partial areas based on the position of the host vehicle and the map information. For example, when the host vehicle enters a T-shaped road, the number of partial areas can be determined as 2.

(3) Partial region image extraction For example, in the above-described first embodiment, the partial region image extraction unit 108 extracts a partial region image with the coordinates of the vanishing point as the center. However, the present invention is not limited to this. Absent. FIG. 13 is a diagram for explaining other partial area image extraction. As shown in the figure, the partial area image extraction unit 108 sets the partial areas 91 and 92 around the coordinates of the vanishing point. As a result, if the partial area exceeds the range of the image 90, the partial area is extracted. 91 and 92 may be moved so as to be included in the range of the image 90, and an image included in the moved partial area may be extracted as a partial area image.

  In the example illustrated in FIG. 13, the partial region image extraction unit 108 sets a partial region 93 in which the partial region 91 is moved to the right in the horizontal direction (for example, the horizontal coordinate at the left end of the partial region image 93 is set to the image 90. Is set in accordance with the horizontal coordinate of the left end of the partial area 92), the partial area 94 is set by moving the partial area 92 to the left in the horizontal direction (for example, the horizontal coordinate of the right end of the partial area image 94 is set to the image 90). The partial area image included in the partial areas 93 and 94 is extracted. As described above, the partial region image extraction unit 108 adjusts the position of the partial region so as to be included in the image, thereby eliminating waste of the detection region of the moving body.

(4) Regarding a camera that captures an image For example, in Embodiment 1 described above, an image is acquired using BCM. However, the present invention is not limited to this. Essentially, any device may be used as long as it is a photographing means capable of photographing an image including a vanishing point. For example, the moving body detection device acquires the image of the side mirror and detects the moving body behind the vehicle, thereby notifying the driver of the existence of a motorcycle or the like existing behind the vehicle and preventing the motorcycle from being involved. I can do it.

(5) System Configuration By the way, among the processes described in the present embodiment, all or a part of the processes described as being automatically performed can be performed manually or manually. All or part of the processing described as being performed can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Each component of the moving object detection apparatus 100 shown in FIG. 5 is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Furthermore, each processing function performed by each device may be realized by a CPU and a program that is analyzed and executed by the CPU, or may be realized as hardware by wired logic.

  FIG. 14 is a diagram illustrating a hardware configuration of a computer configuring the moving object detection apparatus 100 according to the first embodiment. As shown in FIG. 14, the computer (moving body detection device) 200 includes an input device 201, a monitor 202, a camera (BCM) 203, a RAM (Random Access Memory) 204, a ROM (Read Only Memory) 205, and a storage medium. A medium reading device 206 that reads data, a communication device 207 that transmits and receives data to and from other devices, a CPU (Central Processing Unit) 208, and an HDD (Hard Disk Drive) 209 are connected by a bus 210.

  The HDD 209 stores a vanishing point calculation program 209b, a partial area image extraction program 209c, and a moving object detection program 209d that exhibit functions similar to the functions of the moving object detection device 100 described above. When the CPU 208 reads and executes the vanishing point calculation program 209b, the partial area image extraction program 209c, and the moving object detection program 209d, the vanishing point calculation process 208a, the partial area image extraction process 208b, and the moving object detection process 208c are activated. . Here, the vanishing point calculation process 208a corresponds to the vanishing point candidate line calculation unit 102, the image input unit 104, and the straight line detection unit 105 in FIG. The partial area image extraction process 208b corresponds to the partial area number control unit 107 and the partial area image extraction unit 108 in FIG. The moving object detection process 208 c corresponds to the moving object detection unit 109 and the moving object notification unit 110.

  In addition, the HDD 209 stores various data 209 a corresponding to information stored in the reference vanishing point storage unit 101 and the partial area image storage unit 103. The CPU 208 reads out various data 209a stored in the HDD 209, stores it in the RAM 204, specifies the vanishing point using the various data 204a stored in the RAM 204, and detects a moving object from the partial area image.

  Incidentally, the vanishing point calculation program 209b, the partial region image extraction program 209c, and the moving object detection program 209d shown in FIG. 14 do not necessarily need to be stored in the HDD 209 from the beginning. For example, a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card inserted into a computer, or a hard disk drive (HDD) provided inside or outside the computer. Vanishing point calculation program 209b, partial area image extraction in “fixed physical medium”, and “other computer (or server)” connected to the computer via public line, Internet, LAN, WAN, etc. The program 209c and the moving object detection program 209d may be stored, and the computer may read and execute the vanishing point calculation program 209b, the partial area image extraction program 209c, and the moving object detection program 209d.

  The following additional notes are disclosed with respect to the embodiments including Examples 1 and 2 above.

(Supplementary note 1) A moving body detection apparatus for detecting a moving body from an image taken by a camera mounted on the moving body,
Vanishing point identifying means for identifying a vanishing point in the image based on the image;
A partial area storage means for storing an image of a predetermined range including the vanishing point specified by the vanishing point specifying means as a partial area image;
Based on the partial area image stored in the partial area storage means, moving body detection means for detecting a moving body in the image taken by the camera;
A moving body detection apparatus comprising:

(Additional remark 2) The said vanishing point identification means produces | generates a vanishing point candidate line based on the feature-value of the said camera, extracts a straight line from within the said partial area image, and the intersection of the said vanishing point candidate line and the said straight line Is specified as the vanishing point. The moving object detection device according to appendix 1, wherein:

(Additional remark 3) The said vanishing point specific | specification means is a 1st function which shows the distance of the candidate vanishing point used as the candidate of vanishing point, and the straight line extracted from the said partial area image, The said candidate vanishing point, and the characteristic of the said camera Based on the second function indicating the distance to the vanishing point candidate line generated based on the quantity, the candidate vanishing point having the minimum sum of the values of the first and second functions is specified as the vanishing point. The moving body detection apparatus according to Supplementary Note 1, wherein:

(Additional remark 4) The said vanishing point specific | specification means gives a predetermined weight with respect to the value of a said 2nd function, The moving body detection apparatus of Additional remark 3 characterized by the above-mentioned.

(Additional remark 5) The said vanishing point specific | specification means correct | amends the said vanishing point candidate line based on the lens curvature of the said camera, The moving body detection apparatus of Additional remark 2, 3 or 4 characterized by the above-mentioned.

(Additional remark 6) It further has a partial area number determination means for determining the number of partial area images based on the number of vanishing points specified by the vanishing point specifying means, and the partial area storage means includes the partial area number determination means. The moving body detection device according to any one of appendices 2 to 5, wherein the partial region image is stored based on the determination result.

(Additional remark 7) It is a moving body detection method of the moving body detection apparatus which detects a moving body from the image image | photographed with the camera mounted in the moving body,
The mobile object detection device is:
A storage step of storing an image captured by the camera in a storage device;
A vanishing point identifying step for identifying a vanishing point in the image based on the image stored in the storage device;
A partial area storing step of storing an image of a predetermined range including the vanishing point as a partial area image in a storage device;
Based on the partial area image stored in the storage device, a moving body detection step of detecting a moving body in the image captured by the camera;
A moving object detection method comprising:

(Supplementary Note 8) The vanishing point specifying step generates a vanishing point candidate line based on the feature amount of the camera, extracts a straight line from the partial area image, and intersects the vanishing point candidate line with the straight line. The moving object detection method according to appendix 7, wherein: is specified as the vanishing point.

(Supplementary Note 9) The vanishing point specifying step includes a first function indicating a distance between a candidate vanishing point that is a vanishing point candidate and a straight line extracted from the partial region image, and the candidate vanishing point and the camera characteristics. Based on the second function indicating the distance to the vanishing point candidate line generated based on the quantity, the candidate vanishing point having the minimum sum of the values of the first and second functions is specified as the vanishing point. The moving body detection method according to appendix 7, wherein

(Supplementary note 10) The moving object detecting method according to supplementary note 9, wherein the vanishing point specifying step gives a predetermined weight to the value of the second function.

(Additional remark 11) The said vanishing point identification step correct | amends the said vanishing point candidate line based on the lens curvature of the said camera, The moving body detection method of Additional remark 8, 9 or 10 characterized by the above-mentioned.

(Additional remark 12) It further includes the partial area number determination step which determines the number of partial area images based on the number of vanishing points specified by the vanishing point specifying step, and the partial area storage step includes the partial area number determination step. 12. The moving body detection method according to any one of appendices 8 to 11, wherein the partial region image is stored in a storage device based on the determination result.

(Supplementary note 13)
A storage procedure for storing an image taken by a camera mounted on a moving body in a storage device;
A vanishing point identifying procedure for identifying a vanishing point in the image based on the image stored in the storage device;
A partial area storage procedure for storing an image of a predetermined range including the vanishing point identified by the vanishing point identification procedure in a storage device as a partial area image;
Based on the partial area image stored in the storage device, a moving object detection procedure for detecting a moving object in the image captured by the camera;
A moving body detection program for executing

(Additional remark 14) The said vanishing point identification procedure produces | generates a vanishing point candidate line based on the feature-value of the said camera, extracts a straight line from within the said partial region image, and the intersection of the said vanishing point candidate line and the said straight line The moving object detection program according to appendix 13, wherein the moving object detection program is specified as the vanishing point.

(Supplementary Note 15) The vanishing point specifying procedure includes a first function indicating a distance between a candidate vanishing point that is a candidate for a vanishing point and a straight line extracted from the partial region image, and the feature of the candidate vanishing point and the camera. Based on the second function indicating the distance to the vanishing point candidate line generated based on the quantity, the candidate vanishing point having the minimum sum of the values of the first and second functions is specified as the vanishing point. The moving body detection program according to appendix 13, characterized by:

(Supplementary note 16) The moving object detection program according to supplementary note 15, wherein the vanishing point identifying procedure gives a predetermined weight to the value of the second function.

(Supplementary Note 17) The moving object detection program according to Supplementary Note 14, 15 or 16, wherein the vanishing point specifying procedure corrects the vanishing point candidate line based on a lens curvature of the camera.

(Supplementary Note 18) The computer further executes a partial region number determination procedure for determining the number of partial region images based on the number of vanishing points identified by the vanishing point identification procedure, and the partial region storage procedure includes: The moving body detection program according to any one of appendices 14 to 17, wherein the partial region image is stored in a storage device based on a determination result of a number determination procedure.

  As described above, the mobile object detection device, the mobile object detection method, and the mobile object detection program according to the present invention are useful for a mobile object detection system that detects a mobile object on an image, and in particular, while suppressing manufacturing costs. This method is suitable when it is necessary to improve the detection accuracy of the moving object.

It is a figure for demonstrating a driver's blind spot. It is a figure which shows the example of an image | video by BCM. BRIEF DESCRIPTION OF THE DRAWINGS It is a figure for demonstrating the outline | summary and the characteristic of the moving body detection apparatus concerning the present Example 1. FIG. It is a figure for demonstrating the movement of the vanishing point at the time of making a right turn. 1 is a diagram illustrating a configuration of a moving object detection device according to a first embodiment. It is a figure for demonstrating the reference | standard vanishing point set in an image. It is a figure which shows the angle which can be image | photographed with the camera installed in the vehicle. It is a figure which shows the side view of the vehicle in which the camera was installed. It is a figure for demonstrating the vanishing point candidate line set in an image. It is a figure which shows an example of the data which a partial area image memory | storage part memorize | stores. It is a figure for demonstrating the process of a vanishing point calculation part. 3 is a flowchart illustrating a processing procedure of the moving object detection device according to the first embodiment. It is a figure for demonstrating other partial area | region image extraction. It is a figure which shows the hardware constitutions of the computer which comprises the moving body detection apparatus concerning Example 1. FIG.

Explanation of symbols

10, 20, 25 Vehicle 26 Camera (BCM)
30, 40, 50 Vanishing point 60, 61, 62, 90 Image 70 Vanishing point movement diagram 80a, 80b, 80c Reference vanishing point 91, 92, 93, 94 Partial region 100 Mobile object detection device 101 Reference vanishing point storage unit 102 Disappearance Point candidate line calculation unit 103 Partial region image storage unit 104 Image input unit 105 Straight line detection unit 106 Vanishing point calculation unit 107 Partial region number control unit 108 Partial region image extraction unit 109 Mobile object detection unit 110 Mobile object notification unit

Claims (7)

A moving body detection device for detecting a moving body from an image taken by a camera mounted on the moving body,
Vanishing point identifying means for identifying a vanishing point in the image based on the image;
A partial area storage means for storing an image of a predetermined range including the vanishing point specified by the vanishing point specifying means as a partial area image;
Based on the partial area image stored in the partial area storage means, moving body detection means for detecting a moving body in the image taken by the camera;
A moving body detection apparatus comprising:   The vanishing point specifying means generates a vanishing point candidate line based on the feature quantity of the camera, extracts a straight line from within the partial area image, and sets an intersection of the vanishing point candidate line and the straight line as the vanishing point. The mobile object detection device according to claim 1, characterized in that:   The vanishing point specifying means is based on a first function indicating a distance between a candidate vanishing point that is a vanishing point candidate and a straight line extracted from the partial area image, and the candidate vanishing point and a feature amount of the camera. A candidate vanishing point having a minimum sum of values of the first and second functions is identified as a vanishing point based on a second function indicating a distance from the vanishing point candidate line generated in the above manner. The moving body detection apparatus according to claim 1.   The said vanishing point specific | specification means attaches predetermined | prescribed weight with respect to the value of the said 2nd function, The moving body detection apparatus of Claim 3 characterized by the above-mentioned.   5. The moving object detection apparatus according to claim 2, wherein the vanishing point specifying unit corrects the vanishing point candidate line based on a lens curvature of the camera. A moving body detecting method of a moving body detecting device for detecting a moving body from an image photographed by a camera mounted on the moving body,
The mobile object detection device is:
A storage step of storing an image captured by the camera in a storage device;
A vanishing point identifying step for identifying a vanishing point in the image based on the image stored in the storage device;
A partial area storing step of storing an image of a predetermined range including the vanishing point as a partial area image in a storage device;
Based on the partial area image stored in the storage device, a moving body detection step of detecting a moving body in the image captured by the camera;
A moving object detection method comprising: On the computer,
A storage procedure for storing an image taken by a camera mounted on a moving body in a storage device;
A vanishing point identifying procedure for identifying a vanishing point in the image based on the image stored in the storage device;
A partial area storage procedure for storing an image of a predetermined range including the vanishing point identified by the vanishing point identification procedure in a storage device as a partial area image;
Based on the partial area image stored in the storage device, a moving object detection procedure for detecting a moving object in the image captured by the camera;
A moving body detection program for executing

Images