JP5631833B2 - Mobile object equipped with object detection device - Google Patents

Description

  The present invention relates to a moving body equipped with an object detection device that detects an object based on parallax between camera images captured by each of a plurality of cameras.

  Conventionally, a method of measuring a distance to an object using a stereo camera is known (see, for example, Patent Document 1).

  This method finds feature points in each of the two camera images captured by each of the two cameras, clarifies the correspondence between the feature points between the camera images, and positions of the associated feature points. Based on the difference (parallax), the distance to the object forming the feature point is calculated by triangulation.

JP 2000-111321 A

  However, the method described in Patent Document 1 has a large calculation cost for finding a feature point and a calculation cost for clarifying the correspondence between the feature points between the camera images. Is difficult to implement. In addition, since this method is a method for deriving a three-dimensional distance distribution from a two-dimensional camera image, only a specific part of the camera image can be set in advance as a measurement range (a range in which an object to be measured exists). In addition, the calculation cost is applied even to the region where the object to be measured cannot exist, and the efficiency is poor.

  In view of the above-described points, an object of the present invention is to provide a moving body on which an object detection device having a lower calculation cost is mounted.

In order to achieve the above object, a mobile object according to an embodiment of the present invention is a mobile object equipped with an object detection device that detects an object based on a parallax between camera images captured by at least two cameras. The object detection device includes a partial image extraction unit that extracts a partial image of a predetermined size including a pixel corresponding to one of a plurality of inspection points on a virtual straight line passing through the imaging space from each camera image. The evaluation value determining unit that determines an evaluation value related to one of the plurality of inspection points based on the similarity between the partial images of the camera images corresponding to the same inspection point, and the evaluation value determining unit determines An object presence / absence determination unit that determines the presence / absence of an object using an evaluation value to be performed, the moving body includes a swiveling body capable of turning, and the virtual straight line corresponds to the moving body when the moving body moves Part of the sky reachable Together through the region, is parallelly arranged, respectively running parallel to the side end surface of the orbiting body.
A moving body according to an embodiment of the present invention is a moving body including an object detection device that detects an object based on parallax between camera images captured by each of at least two cameras. Corresponds to the same inspection point as the partial image extraction unit that extracts a partial image of a predetermined size including a pixel corresponding to one of a plurality of inspection points on a virtual straight line passing through the imaged space from each camera image. An evaluation value determining unit that determines an evaluation value related to one of the plurality of inspection points based on a similarity between partial images of the camera images to be used, and an evaluation value determined by the evaluation value determining unit An object presence / absence determination unit that determines presence / absence of an object, and the moving body includes a revolving body that can turn, and the virtual straight line can reach a part of the moving body when the moving body moves While passing through the space area Disposed present number, each extending radially from the center of the orbiting body.

  With the above-described means, the present invention can provide a moving body equipped with an object detection device with a lower calculation cost.

It is a block diagram which shows roughly the structural example of the object detection apparatus which concerns on this invention. It is a figure which shows an example of the relationship between an imaging part and a measurement line. FIG. 3 is a diagram illustrating a relationship between a camera image captured by each of two cameras illustrated in FIG. 2 and eight inspection points. It is a figure which shows the camera image which two cameras which image a detection target output. It is a figure which shows the partial image containing the pixel corresponding to an inspection point in each of the two camera images of FIG. It is a flowchart which shows the flow of an evaluation value determination process. It is a figure which shows the ternary relationship of the differential value of the brightness | luminance of a horizontal direction, the differential value of the brightness | luminance of a vertical direction, and a brightness | luminance gradient direction. It is a figure which shows the luminance gradient direction of each pixel in the partial image which makes the pixel corresponding to a test | inspection point a center pixel. It is a figure explaining an example of the quantization method of the brightness | luminance gradient direction by an evaluation value determination part. It is a figure explaining an example of the creation method of the histogram of the value of a luminance gradient direction by an evaluation value determination part. It is FIG. (1) which shows distribution of the evaluation value regarding each of 16 inspection points. It is the figure (the 2) which shows distribution of the evaluation value regarding each of 16 inspection points. It is the figure (the 3) which shows distribution of the evaluation value regarding each of 16 inspection points. It is a flowchart which shows the flow of an object detection process. It is a figure which shows the structural example of the shovel in which an object detection apparatus is mounted. It is FIG. (1) which shows the example of arrangement | positioning of the measurement line which the object detection apparatus mounted in a shovel employ | adopts. It is a figure which shows an example of a detection result display screen. It is FIG. (2) which shows the example of arrangement | positioning of the measurement line which the object detection apparatus mounted in a shovel employ | adopts. It is FIG. (3) which shows the example of arrangement | positioning of the measurement line which the object detection apparatus mounted in a shovel employ | adopts.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram schematically showing a configuration example of an object detection apparatus 100 according to the present invention. The object detection device 100 mainly includes a control unit 1, an imaging unit 2, a display unit 3, and an audio output unit 4.

  The control unit 1 is a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an NVRAM (Non-Volatile Random Access Memory), and the like. For example, the control unit 1 expands, in the RAM, programs corresponding to the functional elements of a partial image extraction unit 10, an evaluation value determination unit 11, and an object presence / absence determination unit 12 described later, which are stored in a ROM or NVRAM, The CPU is caused to execute processing corresponding to each functional element.

  The imaging unit 2 is a device that acquires an image, and is, for example, a stereo vision device including at least two cameras. The imaging unit 2 outputs image data captured at a predetermined cycle to the control unit 1.

  At least two cameras constituting the imaging unit 2 are arranged at a predetermined distance and image the same spatial region. Hereinafter, a common space area captured by at least two cameras is referred to as “imaged space”.

  The display unit 3 is a device for displaying various types of information. For example, the display unit 3 is a liquid crystal display, a projector, or the like, and displays various images output from the control unit 1.

  The audio output unit 4 is a device for outputting various types of information as audio, and is, for example, a buzzer, a speaker, and the like, and outputs various types of information as audio in accordance with control signals output from the control unit 1.

  The partial image extraction unit 10 is a functional element that extracts a partial image from an image captured by the imaging unit 2.

  Specifically, the partial image extraction unit 10 extracts a partial image having a predetermined size including pixels corresponding to the inspection points from the camera images captured by each of the plurality of cameras constituting the imaging unit 2. The partial image is, for example, a rectangular image having a pixel corresponding to the inspection point as a central pixel. The partial image may have other shapes such as a circle and an ellipse.

  The “inspection point” is a point on a virtual straight line (hereinafter referred to as “measurement line”) that passes through the imaging space that is a three-dimensional space. The inspection point is preferably set in advance according to the detection target of the object detection apparatus 100. In this case, the object detection apparatus 100 includes a 3D coordinate of each inspection point in the 3D coordinate system defined in the 3D space and a corresponding 2D coordinate system defined for each of the plurality of camera images. Preliminarily associate and hold the dimension coordinates. However, the inspection point may be dynamically set according to the detection target of the object detection apparatus 100. In this case, the object detection apparatus 100 dynamically sets measurement lines and inspection points, and dynamically calculates corresponding two-dimensional coordinates in each of a plurality of camera images based on the three-dimensional coordinates of each inspection point. .

  FIG. 2 is a diagram illustrating an example of the relationship between the imaging unit 2 and the measurement line VL. The two cameras 2a and 2b that constitute the imaging unit 2 and the measurement line VL passing through the imaging space related to the cameras 2a and 2b. Eight inspection points IP1 to IP8 are shown.

  The measurement line VL is an arbitrary straight line that passes through the space to be imaged. Preferably, the measurement line VL is a spatial region in which an object that is a detection target of the object detection device 100 is likely to exist, or a detection target of the object detection device 100. It is arranged so as to penetrate a spatial region where there is a high possibility that the object will move.

  Further, the distance between the inspection points arranged on the measurement line VL is arbitrary, and may be equally spaced or unequal. Note that the longer the distance between the inspection points, that is, the smaller the number of inspection points, the smaller the calculation load required for the object detection apparatus 100 to detect the object. On the other hand, the shorter the distance between inspection points, that is, the greater the number of inspection points, the higher the resolution of the object detection apparatus 100.

  FIG. 3 shows the relationship between the camera images 2aR and 2bR captured by the two cameras 2a and 2b shown in FIG. 2 and the eight inspection points IP1 to IP8. The camera image 2aR captured by the camera 2a includes partial images 2aR1 to 2aR8 centering on the pixels 2aP1 to 2aP8 corresponding to the inspection points IP1 to IP8, respectively. Similarly, the camera image 2bR captured by the camera 2b includes partial images 2bR1 to 2bR8 centered on the pixels 2bP1 to 2bP8 corresponding to the inspection points IP1 to IP8, respectively. The partial images 2aR1 to 2aR8 and the partial images 2bR1 to 2bR8 have the same size. When the sizes of the partial images 2bR1 to 2bR8 are made common, there is an advantage that the arithmetic processing is easy. Therefore, in FIG. 3, illustration of the range of the partial image in each of the camera images 2aR and 2bR having each of the inspection points IP2 to IP7 as the central pixel is omitted. Note that the size of the partial image may be changed for each inspection point according to the distance from the camera as long as it is common among the partial images corresponding to the same inspection point. In this case, there is an advantage that the size of the partial image can be matched with the size of the reflection of the object in the camera image, which changes according to the distance from the camera. Note that the size of the camera image 2aR is the same as the size of the camera image 2bR. FIG. 3 shows that there is a parallax between the camera image 2aR captured by the camera 2a and the camera image 2bR captured by the camera 2b due to the difference between the installation position of the camera 2a and the installation position of the camera 2b. It shows that.

  FIG. 4 shows camera images 2aR and 2bR output from the two cameras 2a and 2b that capture the detection target 50. Note that the measurement line shown in FIG. 4 has a different extension direction from the measurement line VL shown in FIG. 3, but will be described using the same reference numeral VL for convenience of explanation. The same applies to the inspection points IP1 to IP5.

  In FIG. 4, the cameras 2a and 2b are installed at a height of 500 mm from the floor surface. The measurement line VL extends in parallel with the optical axis direction of the cameras 2a and 2b through the midpoint MP of the line connecting the cameras 2a and 2b on a plane having a height of 500 mm from the floor surface. The five inspection points IP1 to IP5 are arranged at equal intervals (for example, 250 mm intervals) on the measurement line VL, and the inspection point IP1 having the smallest distance from the midpoint MP is located at a point 1000 mm from the midpoint MP. Be placed. Moreover, the detection target 50 is a helmet and is arranged at a point 1750 mm from the midpoint MP.

  FIG. 5 shows two sets of five partial images including pixels corresponding to each of the five inspection points IP1 to IP5 in each of the two camera images 2aR and 2bR in FIG.

  The partial image 2aR1 is a partial image having a pixel 2aP1 corresponding to the inspection point IP1 as a central pixel in the camera image 2aR captured by the camera 2a. The partial image 2bR1 is a partial image having the pixel 2bP1 corresponding to the inspection point IP1 as the central pixel in the camera image 2bR captured by the camera 2b.

  Similarly, each of the partial images 2aR2 to 2aR5 is a partial image in which each of the pixels 2aP2 to 2aP5 corresponding to each of the inspection points IP2 to IP5 in the camera image 2aR is a central pixel. Further, each of the partial images 2bR2 to 2bR5 is a partial image in which each of the pixels 2bP2 to 2bP5 corresponding to each of the inspection points IP2 to IP5 in the camera image 2bR is a central pixel.

  In this way, the partial image extraction unit 10 extracts a partial image of a predetermined size having each pixel corresponding to each of the plurality of inspection points as a central pixel from each camera image.

  The evaluation value determination unit 11 is a functional element that determines an evaluation value related to an inspection point based on the similarity between corresponding partial images.

  Specifically, the evaluation value determination unit 11 determines an evaluation value related to a specific inspection point based on the similarity between partial images having a pixel corresponding to the specific inspection point as a central pixel.

  FIG. 6 is a flowchart showing a flow of processing (hereinafter referred to as “evaluation value determination processing”) in which the evaluation value determination unit 11 determines an evaluation value related to the inspection point.

  First, the evaluation value determination unit 11 calculates the luminance gradient direction of each pixel in the partial image (step S1).

  “Luminance gradient direction” means the direction in which the luminance change is maximum at a certain target pixel. In this embodiment, the luminance differential value in the horizontal direction (horizontal direction of the screen) and the luminance in the vertical direction (vertical direction of the screen). The differential value dIy (i, j) of is formed by the angle θ formed.

Specifically, when the luminance of the pixel at the coordinates (i, j) in each of the camera images 2aR and 2bR is represented by I (i, j), the horizontal luminance differential value dI _x (i, j), vertical The differential value dI _y (i, j) of the luminance in the direction and the luminance gradient direction θ are expressed by the following equations.

FIG. 7 is a diagram showing the relationship between the three values of the horizontal luminance differential value dI _x (i, j), the vertical luminance differential value dI _y (i, j), and the luminance gradient direction θ. , Indicates that the luminance gradient direction θ is an angle between the horizontal luminance differential value dI _x (i, j) and the vertical luminance differential value dI _y (i, j).

  FIG. 8 shows the luminance gradient direction of each pixel in the partial image 2aR1 having the pixel 2aP1 corresponding to the inspection point IP1 as the central pixel in the camera image 2aR. Specifically, FIG. 8A shows an overall image of the partial image 2aR1, and FIG. 8B shows an enlarged view of a broken-line circle portion of FIG. 8A. In FIG. 8A, for convenience of explanation, the partial image 2aR1 is shown in a size of 9 pixels vertically and 9 pixels horizontally. In FIG. 8B, the luminance gradient direction of each pixel is indicated by an arrow.

  In this way, the evaluation value determination unit 11 calculates the luminance gradient direction θ of each pixel in the partial image.

  After calculating the luminance gradient direction θ of each pixel in the partial image, the evaluation value determination unit 11 quantizes the calculated luminance gradient direction θ (step S2).

  FIG. 9 is a diagram for explaining an example of the quantization method of the luminance gradient direction θ by the evaluation value determination unit 11.

  As shown in FIG. 9A, the evaluation value determination unit 11 converts the luminance gradient direction into a value “0” when the luminance gradient direction θ is not less than 0 degrees and less than 45 degrees. Similarly, the evaluation value determination unit 11 has a luminance gradient direction θ of 45 degrees to less than 90 degrees, 90 degrees to less than 135 degrees, 135 degrees to less than 180 degrees, 180 degrees to less than 225 degrees, 225 degrees to less than 270 degrees, When the angle is greater than or equal to 270 degrees and less than 315 degrees and greater than or equal to 315 degrees and less than 360 degrees, the luminance gradient direction is set to value “1”, value “2”, value “3”, value “4”, value “5”, value “6”, respectively. ”And the value“ 7 ”. FIG. 9B shows an example of the result of quantization, and corresponds to FIG.

  In this way, the evaluation value determination unit 11 quantizes the luminance gradient direction θ of each pixel in the partial image having the pixel corresponding to the specific inspection point as the central pixel.

  After quantizing the luminance gradient direction θ, the evaluation value determination unit 11 creates a histogram of the quantized luminance gradient direction values (step S3).

  FIG. 10 is a diagram for explaining an example of a method of creating a histogram of values in the luminance gradient direction by the evaluation value determination unit 11. Note that the partial image 2aR1 in FIG. 10A corresponds to the partial image 2aR1 in FIG.

  As shown in FIG. 10 (A), the evaluation value determining unit 11 includes a left side area composed of four columns on the left side of a column including the central pixel 2aP1, and a column including the central pixel 2aP1. And the right region composed of four columns on the right side of the column including the central pixel 2aP1.

  After that, the evaluation value determination unit 11 counts the appearance frequency of the value in the luminance gradient direction of each pixel included in the left region for each value of “0” to “7”. Similarly, the evaluation value determination unit 11 counts the appearance frequency of the value in the luminance gradient direction of each pixel included in the right region for each value of “0” to “7”.

  FIG. 10B shows the appearance frequency of each of 16 types (dimensions) of values composed of values “0” to “7” for the left region and values “0” to “7” for the right region. It is an example of the histogram which shows distribution.

  In this way, the evaluation value determination unit 11 creates a histogram representing the characteristics of the partial image with the pixel corresponding to the specific inspection point as the central pixel.

After creating a histogram representing each feature of a plurality of partial images related to a specific inspection point, the evaluation value determination unit 11 determines an evaluation value C _GRA related to the specific inspection point (step S4).

Specifically, the evaluation value determining unit 11 captures the histogram representing the characteristics of the partial image 2aR1 in the camera image 2aR captured by the camera 2a and related to the inspection point IP1, and the camera 2b related to the inspection point IP1. based on the histogram representing the characteristic of the partial image 2bR1 in the camera image 2br, determines an evaluation value _{C GRA} an inspection point IP1.

As the evaluation value C _GRA , for example, an evaluation value based on the intensity difference of each element between histograms as represented by the following equation is adopted.

Note that K represents the number of dimensions, and in this embodiment is “16” corresponding to the number of types of values in the luminance gradient direction. Ig (k) represents the k-th value in one partial image. For example, Ig (1) represents the appearance frequency (value in the first dimension) of the value “0” in the luminance gradient direction in the partial image 2aR1. ). Tg (k) represents the k-th value in the other partial image. For example, Tg (1) represents the appearance frequency (value in the first dimension) of the value “0” in the luminance gradient direction in the partial image 2bR1. ).

In this case, a low evaluation value _{C GRA} smaller the higher the similarity of the two partial images 2AR1,2bR1, similarity evaluation value _{C GRA} greater the two partial images 2AR1,2bR1 an inspection point IP1. Note that the relationship between the magnitude of the evaluation value C _GRA and the level of similarity may be opposite depending on the method of calculating the evaluation value C _GRA .

In addition, when there are three or more partial images related to a specific inspection point, that is, when the imaging unit 2 is configured by three or more cameras, the evaluation value determination unit 11 determines the three or more portions. two evaluation value C _GRA between of the images on the calculated for all combinations, to derive a final evaluation value C _GRA based on a plurality of evaluation values C _GRA calculated. In this case, the evaluation value determination unit 11, for example, a plurality of evaluation values C _GRA statistics be derived (average value, maximum value, minimum value, a median value, or the like.) As the final evaluation value C _GRA Good.

In this way, the evaluation value determination unit 11 determines the evaluation value C _GRA related to the specific inspection point based on the similarity of the partial images corresponding to each other with the pixel corresponding to the specific inspection point as the central pixel. To do.

In addition, the evaluation value determination unit 11 performs the above-described evaluation value determination process for each of a plurality of preset inspection points, and determines an evaluation value C _GRA for each of the plurality of preset inspection points.

The object presence / absence determination unit 12 is a functional element that determines the presence / absence of an object based on the evaluation value C _GRA for each of a plurality of inspection points, which is determined by the evaluation value determination unit 11.

11 to 13 are diagrams showing distributions of evaluation values C _GRA for each of the 16 inspection points IP1 to IP16.

  11 to 13, the cameras 2a and 2b are installed at a height of 500 mm from the floor surface. The measurement line VL extends in parallel with the optical axis direction of the cameras 2a and 2b through the midpoint MP of the line connecting the cameras 2a and 2b on a plane having a height of 500 mm from the floor surface. The 16 inspection points IP1 to IP16 are arranged at intervals of 250 mm on the measurement line VL, and the inspection point IP1 having the shortest distance from the midpoint MP is arranged at a point 1000 mm from the midpoint MP. Further, the detection target 50 is a helmet, which is arranged at a point 1500 mm from the midpoint MP in FIG. 11, arranged at a point 2250 mm from the midpoint MP in FIG. 12, and arranged at a point 3000 mm from the midpoint MP in FIG. Is done.

In the case of FIG. 11, the object existence judgment unit 12, distribution et evaluation value _{C GRA} for each inspection point IP1~IP16 aligned on the measurement line VL, evaluation of test points IP3 corresponding to a point 1500mm from the midpoint MP It is determined that the value C _GRA is an extreme value (minimum value). Then, the object presence / absence determination unit 12 determines that the detection target 50 exists at a point 1500 mm from the midpoint MP. In the case of FIG. 12, the object presence / absence determination unit 12 determines that the evaluation value C _GRA related to the inspection point IP6 corresponding to the point 2250 mm from the midpoint MP is an extreme value (minimum value), and from the midpoint MP. It is determined that the detection target 50 exists at a point of 2250 mm. Similarly, in the case of FIG. 13, the object presence / absence determination unit 12 determines that the evaluation value C _GRA related to the inspection point IP9 corresponding to the point 3000 mm from the midpoint MP is an extreme value (minimum value), and the midpoint MP. It is determined that the detection target 50 exists at a point of 3000 mm from the distance.

In this manner, the object existence judgment unit 12 extracts the extreme (minimum value) from a plurality of evaluation values C _GRA, test points having an evaluation value C _GRA to be its extreme (hereinafter, the "adopted point" It is determined that there is an object at the position

  In this determination, the similarity between the partial images regarding the inspection point corresponding to the point where the object exists is relatively high, while the similarity between the partial images regarding the inspection point corresponding to the point where the object does not exist is relatively low. Based on the fact that The partial image obtained by the camera 2a and the partial image obtained by the camera 2b relating to the inspection point corresponding to the point where the object exists include the object as a common subject, whereas the inspection point corresponding to the point where the object does not exist. This is because the partial image obtained by the camera 2a and the partial image obtained by the camera 2b may include different objects (distant objects existing at different points) as subjects.

The object presence / absence determination unit 12 may determine the presence / absence of an object based on the distribution of the plurality of evaluation values _CGRA .

Specifically, the object existence judgment unit 12 may determine the object presence or absence of, based on the relationship between the evaluation value C _GRA inspection points surrounding the evaluation value C _GRA and adoption point of adopting points . In this case, the evaluation value C _GRA of the inspection points around the adopted points is, for example, the average value, maximum value, minimum value, intermediate value, etc. of the evaluation values C _GRA of the plurality of inspection points adjacent to the adopted points.

For example, the object existence judgment unit 12, only when the difference between the evaluation value C _GRA inspection points surrounding the evaluation value C _GRA and adoption point of adoption point is a predetermined value or more, the object to the adoption point position May be determined to exist. That is, the object presence / absence determination unit 12 may determine that there is no object at the position of the adopted point when the difference is less than a predetermined value. This is because, even when the object does not exist at the position of the inspection point, if the luminance of the background is uniform, the evaluation value C _GRA is as low as when the object is present. In addition, when an object exists, the evaluation value C _GRA of one specific inspection point tends to be low, whereas when there is no object and the background brightness is uniform, a plurality of adjacent inspection points This is because the evaluation value C _GRA is easily lowered uniformly.

  In addition, the object detection apparatus 100 transmits the determination result of the object presence / absence determination unit 12 to the operator through the display unit 3 and the audio output unit 4.

  Specifically, the object detection apparatus 100 displays an image captured by one of the cameras 2 a and 2 b on the display unit 3, and the adoption point corresponding to the position where the object is determined to be present by the object presence determination unit 12. The image area around is displayed differently from other image areas. For example, the object detection apparatus 100 displays the color of the image area around the adopted point different from the colors of the other image areas.

  For example, the object detection apparatus 100 outputs an alarm through the sound output unit 4 or outputs a sound message such as “There is an object at a point of X meters” from the sound output unit 4.

  Next, a flow of processing (hereinafter referred to as “object detection processing”) in which the object detection apparatus 100 detects an object will be described with reference to FIG. FIG. 14 is a flowchart showing the flow of the object detection process, and the object detection apparatus 100 repeatedly executes the object detection process at a predetermined cycle. Further, the measurement line VL is set in advance so as to cross a spatial region where there is a high possibility that an object to be detected exists. A predetermined number of inspection points are arranged in advance at equal intervals on the measurement line VL. The imaging unit 2 includes two cameras 2a and 2b, and the cameras 2a and 2b output camera images 2aR and 2bR to the control unit 1.

  First, the control unit 1 of the object detection apparatus 100 causes the partial image extraction unit 10 to generate a partial image having a pixel corresponding to one of a plurality of inspection points as a central pixel from each of the camera images 2aR and 2bR. Extract (step S11).

Then, the control part 1 performs the above-mentioned evaluation value determination process by the evaluation value determination part 11, and determines the evaluation value _CGRA regarding the inspection point (step S12).

Thereafter, the control unit 1 determines whether or not the evaluation values C _GRA for all the inspection points set in advance have been determined (step S13).

When it is determined that the evaluation values C _GRA regarding all the inspection points set in advance have not yet been determined (NO in step S13), the control unit 1 performs step S11 regarding the inspection points for which the evaluation value C _GRA has not yet been determined. And step S12 is performed.

When it is determined that the evaluation values C _GRA relating to all the inspection points set in advance have been determined (YES in step S13), the control unit 1 uses the object existence determination unit 12 to distribute the evaluation values C _GRA relating to all the inspection points. The presence or absence of an object is determined based on (Step S14).

In the present embodiment, the object detection apparatus 100 passes through the one measurement line VL based on the distribution of the evaluation values C _GRA for each of the plurality of inspection points arranged on one measurement line VL. The existence of an object in space is determined. However, the present invention is not limited to this. The object detection apparatus 100 may determine the presence / absence of an object for each measurement line based on the distribution of the evaluation value C _GRA for each of the plurality of inspection points arranged on each of the plurality of measurement lines.

  With the above configuration, the object detection apparatus 100 determines the presence / absence of an object using an evaluation value based on the similarity between partial images in each of a plurality of camera images, with a pixel corresponding to a specific inspection point as a central pixel. To do. As a result, the object detection apparatus 100 can determine the presence / absence of an object without having to find a feature point by image processing such as edge detection, and can reduce the calculation cost required to determine the presence / absence of the object. .

  In addition, since the object detection apparatus 100 can arbitrarily set the position of the inspection point, the measurement range can be limited to the minimum necessary, and the calculation cost required to determine the presence or absence of the object can be further reduced. be able to.

  In addition, since the object detection apparatus 100 can arbitrarily set the position of the inspection point, the resolution of object detection in a specific area can be increased.

  Next, the arrangement of measurement lines and inspection points when the object detection apparatus 100 is mounted on the excavator 60 as a moving body will be described with reference to FIGS.

  15 is a diagram illustrating a configuration example of the excavator 60 on which the object detection device 100 is mounted. FIG. 15A illustrates a side view of the excavator 60, and FIG. 15B illustrates a top view of the excavator 60. .

  The excavator 60 has an upper swing body 63 mounted on a crawler-type lower traveling body 61 via a swing mechanism 62 so as to be rotatable around a swing axis PV.

  The upper swing body 63 includes a cab (operator's cab) 64 on the front left side thereof, a drilling attachment E on the front center thereof, and an imaging unit 2 on the rear surface thereof. In addition, the display part 3 is installed in the position which the driver | operator in the cab 64 is easy to visually recognize. In the cab 64, the control unit 1 and the audio output unit 4 are installed.

  FIG. 16 is a diagram illustrating an example of the arrangement of measurement lines employed by the object detection device 100 mounted on the excavator 60. In FIG. 16, seven measurement lines VLa to VLg are arranged, and four inspection points are arranged at equal intervals in each of the seven measurement lines VLa to VLg. Note that the four inspection points may be arranged at unequal intervals.

  Specifically, four inspection points IPa1 to IPa4 are arranged on the measurement line VLa, four inspection points IPb1 to IPb4 are arranged on the measurement line VLb, and four inspection points IPc1 are arranged on the measurement line VLc. ~ IPc4 is arranged. Further, four inspection points IPd1 to IPd4 are arranged on the measurement line VLd, four inspection points IPe1 to IPe4 are arranged on the measurement line VLe, and four inspection points IPf1 to IPf4 are arranged on the measurement line VLf. The four test points IPg1 to IPg4 are arranged on the measurement line VLg.

The object detection apparatus 100 determines the presence / absence of an object for each measurement line based on the distribution of the evaluation values C _GRA for each of the four inspection points arranged on each of the seven measurement lines VLa to VLf.

  The seven measurement lines VLa to VLg extend radially from the rear of the excavator 60 around the rear part of the upper swing body 63 of the excavator 60 and are arranged so that the angles between two adjacent measurement lines are equal. The Note that the angle between two adjacent measurement lines may be different. The seven measurement lines VLa to VLg extend parallel to the horizontal plane, and the height from the ground is determined according to the detection target.

  With this arrangement of the seven measurement lines VLa to VLg, the object detection device 100 can detect objects existing in various directions behind the excavator 60. Further, due to this arrangement, the arrangement density of inspection points arranged at equal intervals on the measurement line becomes denser as it is closer to the excavator 60. Therefore, this arrangement enables detection of an object existing behind the excavator 60 that is a blind spot of the operator of the excavator 60, and enables detection of a more detailed object closer to the excavator 60.

  In addition, since the seven measurement lines VLa to VLg are defined in a coordinate system based on the upper swing body 63, each inspection point on the measurement lines VLa to VLg is imaged fixed to the upper swing body 63. It moves with the movement of the part 2, that is, with the excavator 60 moving forward and backward, or with the turning of the upper turning body 63. As a result, the object detection apparatus 100 can always detect an object existing behind the upper swing body 63 regardless of whether the excavator 60 moves forward or backward or the upper swing body 63 rotates.

  FIG. 17 is an example of a detection result display screen D1 that displays a detection result of the object detection device 100 mounted on the excavator 60. The object detection device 100 outputs the detection result to the display unit 3 and displays the detection result display screen D1 on the display unit 3.

  The detection result display screen D1 displays a CG (Computer Graphics) image 70 representing the position of the excavator 60 at the upper center of the screen. The CG image 70 is arranged such that an image portion corresponding to the excavation attachment E of the excavator 60 faces the screen upward direction.

  In FIG. 17, the detection result display screen D1 is on the inspection point IPa3 on the measurement line VLa, the inspection point IPb4 on the measurement line VLb, the inspection point IPc3 on the measurement line VLc, and the measurement line VLd shown in FIG. A state in which it is determined that an object exists at the inspection point IPd3 and the inspection point IPf4 on the measurement line VLf and it is determined that no object exists on the measurement line VLe and the measurement line VLg is shown.

  As illustrated in FIG. 17, the object detection apparatus 100 displays a circular mark in a superimposed manner on an image portion at a point where it is determined that an object exists. This is to inform the operator of the presence of the object. Note that the object detection apparatus 100 may notify the operator of the presence of the object by other display methods such as changing the color of the image portion at the point where it is determined that the object is present or blinking the image. . In this way, the object detection apparatus 100 can make the operator easily recognize the determination result of the presence / absence of an object using the imaging unit 2.

  FIG. 18 is a diagram illustrating another example of the arrangement of measurement lines employed by the object detection device 100 mounted on the excavator 60. In FIG. 18, four measurement lines VLh to VLk are arranged, and five inspection points are arranged at equal intervals in each of the four measurement lines VLh to VLk. Note that the five inspection points may be arranged at unequal intervals.

  Specifically, five inspection points IPh1 to IPh5 are arranged on the measurement line VLh, five inspection points IPi1 to IPi5 are arranged on the measurement line VLi, and five inspection points IPj1 are arranged on the measurement line VLj. To IPj5, and five inspection points IPk1 to IPk5 are arranged on the measurement line VLk.

The object detection apparatus 100 determines the presence / absence of an object for each measurement line based on the distribution of the evaluation value C _GRA for each of the five inspection points arranged on each of the four measurement lines VLh to VLk.

  The four measurement lines VLh to VLk extend in parallel to the side end face (the optical axis of the imaging unit 2) of the upper swing body 63 of the shovel 60 behind the shovel 60. Further, the four measurement lines VLh to VLk extend in parallel to the horizontal plane, and the height from the ground is determined according to the detection target. Note that the four measurement lines VLh to VLk may extend parallel to the rear end surface of the upper swing body 63 of the excavator 60 behind the excavator 60.

  With this arrangement of the four measurement lines VLh to VLk, the arrangement density of inspection points arranged at equal intervals on the measurement line is constant regardless of the distance from the shovel 60. Therefore, this arrangement of the four measurement lines VLh to VLk enables detection of an object existing behind the excavator 60 which is a blind spot of the operator of the excavator 60 and exists at a position relatively far from the excavator 60. The object to be detected can be detected with the same resolution as the case of detecting the object existing at a point relatively close to the excavator 60.

  FIG. 19 is a diagram illustrating still another example of the arrangement of measurement lines employed by the object detection device 100 mounted on the excavator 60. In FIG. 19, five measurement lines VLv to VLz are arranged, and four inspection points are arranged at equal intervals in each of the five measurement lines VLv to VLz. Note that the four inspection points may be arranged at unequal intervals.

The object detection apparatus 100 determines the presence / absence of an object for each measurement line based on the distribution of the evaluation value C _GRA for each of the four inspection points arranged on each of the five measurement lines VLv to VLz.

  The five measurement lines VLv to VLz extend radially from the rear of the excavator 60 around the rear part of the upper swing body 63 of the excavator 60, and are arranged so that the angles between two adjacent measurement lines are equal. The Note that the angle between two adjacent measurement lines may be different. Further, the five measurement lines VLv to VLz are arranged on one vertical plane.

  With this arrangement of the five measurement lines VLv to VLz, the object detection device 100 can detect objects of various heights existing behind the excavator 60. Further, due to this arrangement, the arrangement density of inspection points arranged at equal intervals on the measurement line becomes denser as it is closer to the excavator 60. Therefore, this arrangement enables detection of an object existing behind the excavator 60 that is a blind spot of the operator of the excavator 60, and enables detection of a more detailed object closer to the excavator 60.

  Each inspection point is associated with a predetermined two-dimensional coordinate in a two-dimensional coordinate system defined for each of the two camera images. Therefore, the five measurement lines VLv to VLz apparently move with the movement of the imaging unit 2, that is, with the excavator 60 moving forward and backward, or with the turning of the upper swing body 63. As a result, the object detection apparatus 100 can always detect an object existing behind the upper swing body 63 regardless of whether the excavator 60 moves forward or backward or the upper swing body 63 rotates.

  In the above-described embodiment relating to the excavator 60 on which the object detection apparatus 100 is mounted, a plurality of measurement lines extending radially in the horizontal direction and a plurality of measurement lines extending radially in the vertical direction have been described separately. These measurement lines may be arranged in combination. In that case, the measurement line may be arranged so as to extend in a three-dimensional radial manner, for example.

  Further, in the above-described embodiment, the imaging unit 2 is attached to the rear part of the upper swing body 63, and the measurement line has an arrangement suitable for detecting an object behind the excavator 60. However, the present invention is not limited to this. For example, the imaging unit 2 may be additionally or alternatively attached to one or both of the side portions of the upper swing body 63. In such a case, the measurement line is suitable for detecting an object located on the side of the upper swing body 63, and is disposed so as to extend radially to the side of the upper swing body 63, for example.

  In the above-described embodiment, the object detection device 100 is mounted on the excavator 60 including the turning mechanism 62, and the imaging unit 2 moves forward and backward with the lower traveling body 61 and turns with the upper turning body 63. However, the present invention is not limited to this. For example, the imaging unit 2 may move forward and backward with the lower traveling body 61, but may be attached so as not to pivot with the upper swing body 63. In this case, the object detection device 100 can always detect an object existing behind the lower traveling body 61 regardless of the turning of the upper swinging body 63. Further, the object detection device 100 may be mounted on a moving body such as a vehicle that does not have a turning mechanism, or may be mounted on a moving body in which the moving direction is limited.

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

For example, in the above-described embodiment, the object detection apparatus 100 employs an evaluation value based on the intensity difference between the histograms as the evaluation value C _GRA , but the sum of squares of luminance difference SSD (Sum of Squared Difference), luminance A sum of differences SAD (Sum of Absolute Difference), normalized cross-correlation ZNCC (Zero-mean Normalized Cross-Correlation), or the like may be adopted as an evaluation value. Further, the object detection apparatus 100 may use algorithms such as SIFT (Scale-Invariant Feature Transform), HOG (Histograms of Oriented Gradients), etc. (Hirohiro Fujiyoshi, “Gradient-based features”). Extraction-SIFT and HOG- ", Information Processing Society of Japan, Research Report CVIM 160, pp.211-214, 2007.)

  DESCRIPTION OF SYMBOLS 1 ... Control part 2 ... Imaging part 2a, 2b ... Camera 3 ... Display part 4 ... Audio | voice output part 10 ... Partial image extraction part 11 ... Evaluation value determination part 12. .. Object existence / non-existence determining unit 60 ... excavator 61 ... lower traveling body 62 ... turning mechanism 63 ... upper turning body 64 ... cab 70 ... CG image 100 ... object detection device D1 ... Detection result display screen IP1 to IP16 ... Inspection point VL ... Measurement line

Claims (2)

A moving body equipped with an object detection device that detects an object based on parallax between camera images captured by at least two cameras,
The object detection device includes:
A partial image extraction unit that extracts a partial image of a predetermined size including a pixel corresponding to one of a plurality of inspection points on a virtual straight line passing through an imaging space;
An evaluation value determination unit that determines an evaluation value related to one of the plurality of inspection points based on the similarity between the partial images of each camera image corresponding to the same inspection point;
An object presence / absence determination unit that determines the presence / absence of an object using the evaluation value determined by the evaluation value determination unit,
The mobile body includes a revolving body capable of revolving,
The virtual straight line, with a portion of the moving body when the moving body is moving through the spatial region reachable, are parallelly arranged, respectively running parallel to the side end face of the swivel body,
Moving body. A moving body equipped with an object detection device that detects an object based on parallax between camera images captured by at least two cameras,
The object detection device includes:
A partial image extraction unit that extracts a partial image of a predetermined size including a pixel corresponding to one of a plurality of inspection points on a virtual straight line passing through an imaging space;
An evaluation value determination unit that determines an evaluation value related to one of the plurality of inspection points based on the similarity between the partial images of each camera image corresponding to the same inspection point;
An object presence / absence determination unit that determines the presence / absence of an object using the evaluation value determined by the evaluation value determination unit,
The mobile body includes a revolving body capable of revolving,
The imaginary straight line passes through a space area where a part of the moving body can reach when the moving body moves, and a plurality of the virtual straight lines are arranged, each extending radially from the center of the revolving body.
Moving body.