JP2020056627A - Imaging device - Google Patents

Abstract

To enable a far distance to be measured.SOLUTION: An imaging device of the present invention comprises: a first imaging unit having a plurality of pixels for capturing an image; a second imaging unit having a plurality of pixels for capturing an image; and a processing unit for processing a first image captured by the first imaging unit and a second image captured by the second imaging unit and acquiring distance information pertaining to a subject. The base line length B (in m), focus distance F (in mm) and pixel pitch p (in mm) of the first and second imaging units satisfy the relationship B×F/p>700.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an imaging device.

  2. Description of the Related Art Conventionally, there is a technique for measuring a distance to a subject using a compound eye camera such as a stereo camera.

  For example, there is a technology that processes a pair of captured images captured by a stereo camera mounted on an air vehicle to calculate distance information with respect to a lower scenery and an attitude angle of the air vehicle based on the distance information (for example, Patent Document 1). 1). In the technique of Patent Literature 1, a plane is determined based on distance information of a plurality of measurement points, and a tilt with respect to the plane is calculated as an attitude angle of the flying object.

  In the technique of Patent Literature 1, distance information from a flying object to a landscape below is calculated using an image captured by a stereo camera. The distance between the flying vehicle and the lower landscape can be several hundred meters. However, it is difficult for a conventional stereo camera to measure a subject at such a distance.

  Therefore, it is an object of the imaging device of the present disclosure to enable remote distance measurement.

  An imaging device according to an embodiment of the present invention includes a first imaging unit having a plurality of pixels for capturing an image, a second imaging unit having a plurality of pixels for capturing an image, and an image captured by the first imaging unit. A processing unit configured to process the first image and the second image captured by the second imaging unit to obtain distance information of the subject, wherein a base line length B (unit) of the first imaging unit and the second imaging unit : M), focal length F (unit: mm) and pixel pitch p (unit: mm) satisfy the relationship of B × F / p> 700.

  According to the imaging device of the present disclosure, it is possible to measure a far distance.

FIG. 2 is a perspective view illustrating an example of a configuration of an imaging device according to Embodiment 1. FIG. 2 is a plan view illustrating an example of a configuration of an imaging device according to Embodiment 1. FIG. 2 is a block diagram illustrating an example of a functional configuration of the imaging device according to Embodiment 1. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the imaging apparatus according to Embodiment 1. FIG. 2 is a plan view showing an example of an arrangement of components of the imaging device according to the first embodiment. 5 is a flowchart illustrating an example of the operation of the imaging device according to Embodiment 1. FIG. 4 is a diagram showing an example of a distance image generated by the imaging device according to the first embodiment. FIG. 4 is a block diagram showing an example of a functional configuration of an imaging device according to Embodiment 2. 11 is a flowchart illustrating an example of the operation of the imaging device according to Embodiment 2. FIG. 9 is a block diagram showing an example of a functional configuration of an imaging device according to Embodiment 3. 11 is a flowchart illustrating an example of the operation of the imaging device according to Embodiment 3. FIG. 14 is a block diagram illustrating an example of a functional configuration of an imaging device according to Embodiment 4.

  In recent years, it has been studied to apply a distance measurement technique using a stereo camera to detection of a subject in a vehicle, a robot, a monitoring camera, or the like. For example, a stereo camera is used in a vehicle for automatic driving technology and driving support technology, a robot is used for object recognition, and a surveillance camera is used for human detection. In an environment where a current stereo camera is used, if the distance information is up to about 100 m, the distance information can be sufficiently utilized practically.

  For example, Patent Document 2 discloses an imaging system including a plurality of cameras. The imaging system calculates a distance estimated by processing as a monocular camera (hereinafter, also referred to as “monocular processing”) and a distance estimated by processing as a stereo camera (hereinafter, also referred to as “stereo processing”). Combined. When the estimated distance is shorter than 50 m and the blur amount of the image is small, it is described that the distance to the target object can be stably calculated by the stereo processing. Therefore, the estimated distance in the stereo processing is adopted. When the estimated distance is shorter than 50 m and the blur amount of the image is large, the estimated distance in the monocular processing is adopted. When the estimated distance is longer than 50 m, it is described that the distance calculation in the stereo processing is difficult because the parallax value is small, and therefore, the estimated distance in the monocular processing is adopted.

  Patent Literature 2 also describes a case where the estimated distance in stereo processing is used when the estimated distance is longer than 50 m up to about 100 m. In this case, the object is searched from the two images by pattern matching with a template of the object such as a vehicle prepared in advance. Further, the distance to the object is calculated from the parallax of the object between the two images.

  As described above, the conventional stereo camera can calculate a distance up to about 50 m, and can calculate a distance up to about 100 m for an object whose shape or the like is known in advance. For this reason, when the conventional stereo camera is applied to the technique of Patent Literature 1, the distance to the target object may be longer than 100 m, and desired distance information may not be obtained.

  When a stereo camera is mounted on an aircraft, the stereo camera is required to measure a distance from a flying aircraft to an object on an airfield such as a runway and an obstacle. Thereby, it is possible to recognize the object at the airfield instead of the pilot's eyes. For safer navigation of the aircraft, it is desirable that the stereo camera acting as a substitute for the pilot's eye has a distance measuring capability equal to or greater than that of the human eye. Specifically, it is desirable that the distance measurable by the stereo camera is equal to or greater than the limit value of the distance at which both human eyes can sense the depth.

  For example, according to Non-Patent Document 1, regarding parallax between two human eyes, the depth sensitivity at which depth is perceived decreases as the viewing distance, which is the distance from the viewpoint to the target, decreases, and the depth sensitivity decreases between 700 to 750 m. Converge to a value. Therefore, both eyes of a human cannot distinguish a difference in depth between two images that are longer than the distance.

  For this reason, it is desirable that the stereo camera mounted on the aircraft has the ability to measure the distance of an object that is 700 m or more away. However, conventional stereo cameras only have the ability to measure distances that are significantly shorter than the distance at which a person can fully perceive depth sensitivity. In conventional stereo cameras, there is no stereo camera capable of measuring a long distance such as 700 m or more.

  Therefore, the technology of the present disclosure provides an imaging device that enables a long distance measurement such as 700 m or more.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

(Embodiment 1)
<Configuration of imaging device 1>
The configuration of the imaging device 1 according to the first embodiment will be described. FIG. 1 is a perspective view showing an example of the configuration of the imaging device 1 according to the first embodiment. FIG. 2 is a plan view illustrating an example of a configuration of the imaging device 1 according to Embodiment 1. As shown in FIGS. 1 and 2, in the present embodiment, the imaging device 1 will be described as including two imaging units 10 and 20. Note that the imaging device 1 may include two or more imaging units, and may include, for example, three or more imaging units. Further, the imaging device 1 may be configured by one or more devices. When the imaging device 1 is configured by two or more devices, the two or more devices may be arranged in one device, or may be separately arranged in two or more separated devices. . In this specification and in the claims, the term “device” may mean not only one device, but also a system including a plurality of devices.

  The imaging device 1 includes a first imaging unit 10, a second imaging unit 20, a housing 30, and a control device 40. The housing 30 houses the first imaging unit 10, the second imaging unit 20, and the control device 40. The control device 40 controls the operation of the entire imaging device 1. Details of the control device 40 will be described later.

  Each of the first imaging unit 10 and the second imaging unit 20 captures an image in the field of view and outputs two digital images. Examples of the first imaging unit 10 and the second imaging unit 20 are cameras that capture a still image and / or a moving image. The first imaging unit 10 and the second imaging unit 20 may be housed together in one housing 30 and form one compound eye camera, specifically, a two-lens camera, as in the present embodiment. Alternatively, two monocular cameras may be separately housed in two housings. The first imaging unit 10 and the second imaging unit 20 are controlled by the control device 40 to perform an imaging operation in synchronization. That is, the first imaging unit 10 and the second imaging unit 20 capture images simultaneously. The first imaging unit 10 and the second imaging unit 20 output the captured image to the control device 40. In the present specification and claims, the term “image” means not only the image itself but also image data that is data indicating an image and an image signal that is a signal indicating an image.

  The first imaging unit 10 includes a first lens 11 and a first imaging device 12. The second imaging unit 20 includes a second lens 21 and a second imaging device 22. The first lens 11 and the second lens 21 are spaced apart in the same direction, and are exposed outside the housing 30. In the present embodiment, the optical axis center LA1 that is the optical axis of the first lens 11 and the optical axis center LA2 that is the optical axis of the second lens 21 are parallel, but the present invention is not limited to this. The base line length, which is the distance between the optical axis centers LA1 and LA2, is “B”. The base line length B is the distance between the center of the first lens 11 and the center of the second lens 21.

  Here, a direction parallel to the optical axis centers LA1 and LA2 is defined as a z-axis direction, and a direction in which the optical axis centers LA1 and LA2 are arranged is defined as an x-axis direction. Further, a direction perpendicular to the x-axis direction and the z-axis direction is defined as a y-axis direction. The xz plane includes the optical axis centers LA1 and LA2. The base length is the distance between the optical axis centers LA1 and LA2 in the x-axis direction.

  The first imaging element 12 is arranged on the optical axis center LA1 of the first lens 11. The first imaging element 12 receives light incident through the first lens 11 and outputs an image represented by the light. The second imaging element 22 is arranged on the optical axis center LA2 of the second lens 21. The second imaging element 22 receives light incident through the second lens 21 and outputs an image represented by the light. In the present embodiment, the focal length f1 between the first lens 11 and the first image sensor 12 is the same as the focal length f2 between the second lens 21 and the second image sensor 22. The second lens 21 has the same characteristics and capabilities, but is not limited thereto.

  Each of the first imaging element 12 and the second imaging element 22 includes a plurality of light receiving elements arranged in an array. The light receiving element outputs a pixel value indicating the intensity of the received light. An example of a pixel value is a luminance value. The light receiving elements constitute a plurality of pixels that capture images in each of the first imaging element 12 and the second imaging element 22. In the present embodiment, the pixel pitch p1, which is the interval between the light receiving elements of the first imaging element 12, is the same as the pixel pitch p2 of the light receiving elements of the second imaging element 22, but is not limited to this. Examples of the first image sensor 12 and the second image sensor 22 include a charge coupled device (CCD) image sensor and a complementary metal oxide semiconductor (CMOS) image sensor. In the present embodiment, the first image sensor 12 and the second image sensor 22 are the same image sensor having the same resolution and the same image sensor dimensions, but are not limited thereto.

  The functional configuration of the control device 40 will be described. FIG. 3 is a block diagram illustrating an example of a functional configuration of the imaging device 1 according to the first embodiment. As illustrated in FIG. 3, the imaging device 1 includes an input unit 2, an output unit 3, a first imaging unit 10, a second imaging unit 20, and a control device 40. The control device 40 includes a main control unit 41, an imaging control unit 42, and an image processing unit 43. The image processing unit 43 includes an acquisition unit 431, a distance calculation unit 432, an image generation unit 433, a parameter storage unit 434, and a storage unit 435.

  The input unit 2 acquires operations, information, commands, and the like input from an operation input device of the imaging device 1 and an external device connected to an input I / F (interface) of the imaging device 1.

  The output unit 3 outputs information such as an image captured by the first imaging element 12 and the second imaging element 22 and a distance image calculated from the images to a display of the imaging device 1 and an output I / F of the imaging device 1. Output to the connected external device.

  The main control unit 41 controls the operation of the entire imaging device 1. The main control unit 41 controls the operation of each part of the imaging device 1 according to the operation, information, instructions, and the like obtained via the input unit 2.

  The imaging control unit 42 controls an imaging operation of the first imaging unit 10 and the second imaging unit 20 such as a synchronized imaging operation. The imaging control unit 42 outputs the images captured by the first imaging unit 10 and the second imaging unit 20 to the image processing unit 43.

  The image processing unit 43 generates a distance image indicating a distance from the imaging device 1 to a subject using the first image and the second image that are images captured by the first imaging unit 10 and the second imaging unit 20. Output. That is, the image processing unit 43 processes the first image and the second image to acquire distance information of the subject. The first image and the second image are images captured by the first imaging unit 10 and the second imaging unit 20 at the same time. The distance image is an image in which the pixel value of each pixel is the distance from the imaging device 1 to the subject projected by the pixel. The distance information includes information indicating a distance from the imaging device 1 to the subject, and includes, for example, a distance value between the subject and the imaging device 1 projected by each pixel, a distance image, and the like. Here, the image processing unit 43 is an example of a processing unit.

  The parameter storage unit 434 stores parameters related to the first imaging unit 10 and the second imaging unit 20. The above parameters are camera parameters of the first imaging unit 10 and the second imaging unit 20, and include external parameters and internal parameters. Examples of the external parameters are parameters indicating the positions and orientations of the first imaging unit 10 and the second imaging unit 20, and the like. Examples of the internal parameters are parameters indicating the distortion of the lens, the focal length, the size of one pixel of the image sensor, the pixel coordinates of the center of the optical axis, and the like. The pixel coordinates are coordinates in units of pixels, and are two-dimensional coordinates on an image.

  The acquisition unit 431 acquires the first image and the second image captured by the first imaging unit 10 and the second imaging unit 20 via the imaging control unit 42, and causes the storage unit 435 to store the first image and the second image. For example, the acquisition unit 431 acquires an image signal indicating the first image and the second image, performs gain control in accordance with the acquired signal intensity, performs A / D conversion, and converts the first image and the The two images are output to the storage unit 435.

  The storage unit 435 stores various types of information and can retrieve the stored information. The storage unit 435 stores the first image and the second image captured by the first imaging unit 10 and the second imaging unit 20. The first image and the second image include information indicating an image such as a pixel value of each pixel, an imaging time of the image, and identification information of an imaging unit that has acquired the image.

  The distance calculation unit 432 acquires the first image and the second image from the storage unit 435, and uses the first image and the second image and the camera parameters of the parameter storage unit 434 to determine the subject of each pixel to be projected. Calculate the three-dimensional position. That is, the distance calculation unit 432 calculates the distance from the subject to the imaging device 1 for each pixel. The distance from the subject to the imaging device 1 is also the distance between the subject and the first imaging unit 10 and the second imaging unit 20. The distance calculation unit 432 outputs the distance value at the position of each pixel of the first image and the second image to the image generation unit 433.

  Specifically, the distance calculation unit 432 corrects the lens distortion of the first lens 11 and the second lens 21 for each of the first image and the second image. Further, the distance calculation unit 432 performs a stereo matching process for each small area between the first image and the second image. Accordingly, the distance calculation unit 432 calculates, for each pixel, a parallax generated according to the distance to the subject, and calculates the distance between the subject and the imaging device 1 using the parallax. The parallax corresponds to a shift in the position of a pixel that projects the same subject between the first image and the second image.

  For example, for the pixel on the first image, the distance calculation unit 432 calculates the first pixel coordinates that are the pixel coordinates of the pixel and the second image on the second image in which the same subject as the subject represented by the pixel is displayed. A second pixel coordinate which is a pixel coordinate of the corresponding point is detected. The difference between the pixel positions of the first pixel coordinates and the second pixel coordinates is the parallax. The distance calculation unit 432 performs the stereoscopic view using the first pixel coordinates, the second pixel coordinates, and the camera parameters, and the object represented by the pixel corresponding to the first pixel coordinates and the second pixel coordinates, and the imaging device 1 Is calculated. Then, the distance calculation unit 432 performs the above processing for all pixels on the first image. Thereby, the distance calculation unit 432 calculates the distance value at the position of each pixel.

  For example, for each pixel of the first image, the image generation unit 433 generates an image in which the pixel value of the pixel is a distance value corresponding to the position of the pixel, that is, a distance image. In the above description, the distance value and the distance image at the position of each pixel are calculated based on the first image, but the second image may be used as the reference. The image generation unit 433 outputs the distance image to the output unit 3 and / or the storage unit 435. For example, the image generation unit 433 outputs the distance image to the output unit 3 via the main control unit 41.

  The hardware configuration of the control device 40 will be described. FIG. 4 is a block diagram illustrating an example of a hardware configuration of the imaging device 1 according to the first embodiment. As illustrated in FIG. 4, the imaging apparatus 1 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a memory 104, an operation input device 105, It includes an I / F 106, an output I / F 107, a first imaging unit 10, and a second imaging unit 20 as constituent elements. The above components are connected to each other, for example, via a bus. The above components may be connected via any of wired communication and wireless communication.

  The operation input device 105 receives an operation input by a user, and includes input devices such as a button, a dial, a touch panel, and a microphone for voice input.

  The input I / F 106 is connected to an external device of the imaging device 1 and receives inputs of information, commands, and the like from the external device.

  The output I / F 107 is connected to an external device of the imaging device 1 and outputs information and the like to the external device. The input I / F 106 and the output I / F 107 may be arranged separately or may be arranged integrally. The input I / F 106 and the output I / F 107 may be connected to different devices or may be connected to the same device. The input I / F 106 and the output I / F 107 have a function of a communication I / F connected to an external network, and may receive and output information and the like via the external network.

  The CPU 101 implements the functions of the input unit 2, the output unit 3, and the control device 40. Regarding the control device 40, the CPU 101 implements the functions of the main control unit 41, the imaging control unit 42, and the acquisition unit 431, the distance calculation unit 432, and the image generation unit 433 of the image processing unit 43.

  The memory 104 implements the functions of the parameter storage unit 434 and the storage unit 435 of the image processing unit 43. The memory 104 is configured by a storage device such as a semiconductor memory, a hard disk drive (HDD), or a solid state drive (SSD). Note that the memory 104 may include the ROM 102 and / or the RAM 103.

  The CPU 101 is configured by a processor or the like, the ROM 102 is configured by a nonvolatile semiconductor storage device or the like, and the RAM 103 is configured by a volatile semiconductor storage device or the like. A program for operating each component is stored in the ROM 102 or the memory 104 in advance. The program is read from the ROM 102 or the memory 104 or the like to the RAM 103 and expanded by the CPU 101. The CPU 101 realizes its function by executing each coded instruction in the program developed on the RAM 103. The program is not limited to the ROM 102 and the memory 104, and may be stored in a recording medium such as a recording disk. In addition, the program may be transmitted via a wired network, a wireless network, a broadcast, or the like, and may be loaded into the RAM 103.

  Note that when the functions are realized by the CPU 101, the above-described components may be realized by a program execution unit such as the CPU 101, may be realized by a circuit, or may be realized by a combination of the program execution unit and the circuit. Good. For example, these components may be realized as an LSI (Large Scale Integration) which is an integrated circuit. These components may be individually formed into one chip, or may be formed into one chip so as to include some or all of them. As the LSI, an FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, a reconfigurable processor that can reconfigure the connection and / or setting of circuit cells inside the LSI, or a plurality of specific applications. An ASIC (Application Specific Integrated Circuit) in which functional circuits are integrated into one may be used.

<Arrangement of first imaging unit 10 and second imaging unit 20>
Further, the arrangement of the first imaging unit 10 and the second imaging unit 20 will be described. FIG. 5 is a plan view illustrating an example of an arrangement of components of the imaging device 1 according to the first embodiment. As shown in FIG. 5, for the first imaging unit 10 and the second imaging unit 20, the distance Z (unit: m) between the first lens 11 and the second lens 21 and the subject A, and the base line length B (unit: m) ), The focal lengths f1 and f2 (unit: mm), and the pixel pitches p1 and p2 (unit: mm) of the first image sensor 12 and the second image sensor 22 satisfy the following equation 1.
Z = B × F / D = B × F / p | pc1-pc2 | (formula 1)
The distance Z is the distance between the subject A and a plane that passes through the lens centers of the first lens 11 and the second lens 21 and is perpendicular to the optical axis centers LA1 and LA2, and the first imaging unit 10 and the second imaging unit 20 The distance between the image capturing apparatus 1 and the subject A. Also, F = f1 = f2 and p = p1 = p2. Further, the parallax D in the first image sensor 12 and the second image sensor 22 is D = p | pc1-pc2 |. “Pc1” is a pixel coordinate of the subject A in the x-axis direction on the first image captured by the first imaging unit 10, and “pc2” is a pixel coordinate on the second image captured by the second imaging unit 20. Are the pixel coordinates of the subject A in the x-axis direction. | Pc1-pc2 | is the number of pixels corresponding to parallax (hereinafter, also referred to as “pixel parallax”).

In the present embodiment, the imaging apparatus 1 satisfies Z ≧ 700 m in order to enable distance measurement of a subject at a distance of 700 m or more. Further, in order to realize the above-described distance measurement, one or more pixel parallax is required in the distance measurement of a subject at a distance of 700 m or more. That is, pixel parallax | pc1−pc2 | ≧ 1. Based on Equation 1, it is necessary to satisfy the following Equation 2 in order to satisfy these conditions.
700 <B × F / p (Equation 2)
Therefore, in the present embodiment, the first lens 11, the second lens 21, the first imaging device 12, and the second imaging device 22 of the first imaging unit 10 and the second imaging unit 20 are configured to satisfy Expression 2. Is done.

<Operation of imaging apparatus 1>
The operation of the imaging device 1 according to the first embodiment will be described. Specifically, an operation of generating a distance image by the imaging device 1 will be described. FIG. 6 is a flowchart illustrating an example of an operation of the imaging device 1 according to the first embodiment. As shown in FIG. 6, first, the first imaging unit 10 and the second imaging unit 20 simultaneously image the same subject, and convert the two images, the first image and the second image, which are simultaneously imaged, into the image processing unit 43. (Step S1). That is, the captured images of the first imaging unit 10 and the second imaging unit 20 are obtained.

  Next, in the image processing unit 43, the obtaining unit 431 obtains the first image and the second image, performs digitization processing, and outputs the digital image to the storage unit 435 (Step S2).

  Next, the distance calculation unit 432 performs stereo processing on the first image and the second image (Step S3). That is, for each pixel of the first image and the second image, the distance calculation unit 432 calculates the distance between the subject of the pixel and the imaging device 1.

  Next, the image generation unit 433 generates a distance image in which the calculated distance value of each pixel is set as a pixel value of a pixel at a position corresponding to the position of the pixel, and outputs the distance image to the output unit 3 and / or the storage unit 435. (Step S4).

  By performing the processing of steps S1 to S4, the imaging device 1 generates the distance image including the distance information to the subject, which is displayed in the two images captured by the first imaging unit 10 and the second imaging unit 20 at the same time. Generate.

<Example>
An example of the imaging device 1 according to the first embodiment will be described. The imaging apparatus 1 according to the embodiment is configured so that each parameter satisfies Expression 2. When Expression 2 is satisfied, it is desirable that the base line length B and the focal length F of the parameters be longer. However, as the focal length increases, the angle of view decreases. For this reason, in the embodiment, the focal length is set to be longer while maintaining the angle of view required for the imaging device 1. The base line length is limited by the size of the target on which the imaging device 1 is mounted. In the embodiment, the imaging device 1 is mounted on a small unmanned aerial vehicle or a multicopter having a wing length of 5 m or less. For this reason, in the example, the base line length was set to be longer while enabling mounting on a small unmanned aerial vehicle and a multicopter.

  Further, it is desirable that the pixel pitch p of the first image sensor 12 and the second image sensor 22 among the parameters is larger. The pixel pitch corresponds to the pixel size. The larger the pixel size, the smaller the noise in the captured image, and the higher the accuracy of stereo matching. In the embodiment, the pixel pitch is set to be larger.

  It is desirable that the parallax between the images captured by the first imaging unit 10 and the second imaging unit 20 among the parameters is larger. As the parallax increases, the error in the distance between the subject and the imaging device 1 decreases. In the embodiment, each parameter is set such that the pixel parallax | pc1-pc2 | calculated when the base line length B, the focal length F, and the pixel pitch p that satisfy Expression 2 are applied to Expression 1, becomes larger. .

  As described above, in the example, the base line length B was set to 0.4 m, the focal length F was set to 35 mm, and the pixel pitch p was set to 0.00586 mm. Further, as the first image sensor 12 and the second image sensor 22, a CMOS image sensor having a resolution of 2,073,600 pixels of 1920 pixels × 1080 pixels, that is, a so-called 2K resolution was used. Note that an image sensor having a resolution of 4K or more may be used for the first image sensor 12 and the second image sensor 22. Thereby, the detection accuracy of the pixel value of each pixel can be improved, and the pixel parallax can be increased.

  When the base line length B = 0.4 (m), the focal length F = 35 (mm), and the pixel pitch p = 0.00586 (mm) are applied to the left side “B × F / p” of Expression 2, B × F / p = 2389 (m) is obtained. Further, in Expression 1, when B × F / p = 2389 (m) is applied and the distance Z is applied 700 (m), a pixel parallax | pc1−pc2 | = 3.4 (pixel) is obtained. In the embodiment, each parameter is set based on Expressions 1 and 2 so that the pixel parallax is 3 pixels or more. Therefore, the pixel parallax at a predetermined distance of 700 m or more is preferably one pixel or more, and more preferably three pixels or more.

  FIG. 7 shows an example of a distance image generated by the imaging apparatus 1 according to the embodiment configured as described above. According to FIG. 7, the distance to a subject having a distance of 700 m or more is shown.

  The imaging device 1 capable of measuring a long distance can be assumed to be mounted on vehicles, robots, moving objects such as aircraft, multicopters and drones, and monitoring devices outside buildings. You.

<Effects>
As described above, the imaging apparatus 1 according to the first embodiment includes a first imaging unit 10 having a plurality of pixels for capturing an image, a second imaging unit 20 having a plurality of pixels for capturing an image, and a first imaging unit. An image processing unit 43 that processes the first image captured by the unit 10 and the second image captured by the second imaging unit 20 to obtain distance information of the subject; The base line length B (unit: m), focal length F (unit: mm), and pixel pitch p (unit: mm) of the unit 20 satisfy the relationship of B × F / p> 700.

  According to the above configuration, when the first imaging unit 10 and the second imaging unit 20 image a subject at a distance of 700 m from the first imaging unit 10 and the second imaging unit 20, that is, a subject with a visibility of 700 m, the first image The parallax of the subject between the second images is one pixel or more. Therefore, the imaging device 1 can acquire distance information using two images having a parallax of one pixel or more in a visibility of 700 m or more. Such distance information has high accuracy for a distant subject. Therefore, the imaging device 1 enables a far distance measurement that is impossible with a conventional stereo camera.

  Further, in the imaging apparatus 1 according to Embodiment 1, the first imaging unit 10 and the second imaging unit 20 include the first subject, which is a subject at a distance of 700 m shown in the first image, and the second subject, which is shown in the second image. The parallax with the first subject may be configured to be 3 pixels or more. According to the above configuration, the parallax of the same subject between the two images is increased, so that the accuracy of the distance information of the subject is improved. In particular, since the parallax is three or more pixels, it is possible to more reliably prevent the same subject, such as the same point, from being displayed at the same position in the first image and the second image, that is, the pixel at the pixel coordinates. Thereby, the accuracy of the distance information is improved.

(Embodiment 2)
The imaging apparatus 1A according to the second embodiment differs from the first embodiment in that the attitude of a flying object on which the imaging apparatus 1A is mounted is detected using a distance image. Hereinafter, the second embodiment will be described focusing on the points different from the first embodiment, and the description of the same points as the first embodiment will be appropriately omitted.

  Imaging device 1A according to Embodiment 2 is mounted on a flying object. Examples of the flying object are an aircraft on which a pilot is boarded, a drone such as an unmanned aerial vehicle, and the like, and in the present embodiment, an unmanned small aircraft. The imaging device 1A is arranged near the tip of the small unmanned aerial vehicle. For example, the fields of view of the first imaging unit 10 and the second imaging unit 20 are in front of and below the front of the small unmanned aerial vehicle.

  The configuration of the imaging device 1A will be described. FIG. 8 is a block diagram illustrating an example of a functional configuration of an imaging device 1A according to Embodiment 2. As shown in FIG. 8, the imaging device 1A includes a control device 40A. The control device 40A includes an image processing unit 43A. The image processing unit 43A includes an acquisition unit 431, a distance calculation unit 432, an image generation unit 433, a parameter storage unit 434, a storage unit 435, a target point calculation unit 436, a self-position calculation unit 437, and a self-posture. And a calculation unit 438. The hardware configuration of the imaging device 1A is the same as that of the first embodiment. The functions of the target point calculation unit 436, the self-position calculation unit 437, and the self-posture calculation unit 438 are realized by the CPU 101 and the like, like the other components. Here, the target point calculation unit 436, the self-position calculation unit 437, the self-posture calculation unit 438, and the image processing unit 43A are examples of a calculation unit.

  The target point calculation unit 436 determines a target point on the ground, and calculates coordinates of the determined target point. In the present embodiment, the target point is the landing point, but is not limited to this. The target point calculation unit 436 uses one of the first image and the second image (hereinafter also referred to as a “luminance image”) and a distance image of these images. The target point calculation unit 436 may use any of the first image and the second image as the luminance image.

  The target point calculation unit 436 removes, from the luminance image and the distance image, pixels that do not form a planar subject, and extracts a pixel group that forms a subject in a planar area equal to or larger than a predetermined range. For example, the target point calculation unit 436 may extract the pixel group in the same manner as in Patent Document 1.

  Specifically, in the distance image in which the pixel position is defined by the x-axis and y-axis pixel coordinates, the target point calculation unit 436 calculates the x-coordinate and distance value z of each pixel on the horizontal axis parallel to the x-axis. Is generated. The target point calculation unit 436 approximates the regression line on the xz coordinate by applying the least squares method to the data series. The target point calculation unit 436 removes data located outside the predetermined range from the regression line on the xz coordinates from the data series. Since the pixel corresponding to the removed data is a pixel that does not form a planar subject, the target point calculation unit 436 removes the pixel corresponding to the removed data from the distance image.

  Further, the target point calculation unit 436 performs the above processing by scanning the distance image in the y-axis direction. That is, the target point calculation unit 436 generates a data series on the horizontal axis passing through each y coordinate of the distance image, approximates the regression line by applying the least squares method to the data series, and obtains a predetermined value from the regression line. Data located outside the range is removed from the data series. Thus, pixels that do not form a planar subject are removed from the distance image. Further, the target point calculation unit 436 removes a pixel corresponding to the pixel removed from the distance image from the luminance image. In each of the distance image and the luminance image, a pixel group that forms a subject in one or more planar regions can be formed.

  The storage unit 435 stores the outline of the landing area that is a plane area on the ground including the landing point. Examples of landing areas are runways, roads, heliports and drone ports.

  The target point calculation unit 436 extracts an edge of each plane area in the luminance image after the removal. Further, the target point calculation unit 436 acquires the contour of the landing area from the storage unit 435, performs pattern matching between the contour and the edge of the plane area, and specifies the plane area corresponding to the landing area.

  Further, the target point calculation unit 436 searches for an index indicating a landing point included in the landing area in the luminance image after removal. When an index is found, the target point calculation unit 436 determines the index as a target point. If no index is found, the target point calculation unit 436 searches for a landing area in the distance image, and determines the position of the pixel with the smallest distance value as the target point. The target point calculation unit 436 outputs the pixel coordinates of the target point in the luminance image and the distance image to the self-position calculation unit 437.

  The self-position calculating unit 437 acquires a distance value from the imaging device 1A to the target point from the pixel value at the pixel coordinates of the target point in the distance image. The self-position calculating unit 437 calculates the position of the target point with respect to the optical axis center of the first imaging unit 10 or the second imaging unit 20 from the pixel coordinates of the target point in the luminance image and the distance value from the imaging device 1A to the target point. Is calculated. The position of the target point is a position in a direction perpendicular to the center of the optical axis. The self-position calculating unit 437 calculates the position of the imaging device 1A with respect to the target point from the distance and the position of the target point. In the present embodiment, the self-position calculation unit 437 calculates the position of the imaging device 1A with respect to the target point using camera coordinates having the first imaging unit 10 or the second imaging unit 20 as the origin. In the camera coordinates, the center of the optical axis of the first imaging unit 10 or the second imaging unit 20 is the Z axis, and two axes perpendicular to the Z axis and perpendicular to each other are the X axis and the Y axis. The X axis and the Y axis are parallel to the light receiving surface of the first image sensor 12 or the second image sensor 22. Note that the self-position calculation unit 437 may calculate the position of the imaging device 1A with respect to the target point using the coordinates set for the target point.

The self-posture calculation unit 438 calculates the posture of the imaging device 1A with respect to the plane of the landing area, and outputs the calculated posture to the output unit 3 and / or the storage unit 435. Specifically, the self-posture calculation unit 438 calculates a function represented by the following Expression 3 using the luminance image and the distance image. The function of Equation 3 indicates the plane of the landing area at the camera coordinates of the first imaging unit 10 or the second imaging unit 20.
aX + bY + cZ = 1 (Equation 3)
At this time, the self-posture calculation unit 438 calculates the coordinates of the three-dimensional positions of the three subjects projected by at least three pixels included in the landing area using the camera coordinates. The self-posture calculation unit 438 calculates the coefficients a, b, and c of Expression 3 by solving a simultaneous equation using the coordinates of the three three-dimensional positions. For example, the inclination with respect to the Y axis and the Z axis in Expression 3 when X = 0 can indicate the pitch angle of the imaging device 1A. The inclination with respect to the X axis and the Z axis in Expression 3 when Y = 0 may indicate the roll angle of the imaging device 1A. The self-posture calculation unit 438 calculates the posture angle of the imaging device 1A with respect to the plane of the landing area using Expression 3.

  The operation of the imaging device 1A will be described. FIG. 9 is a flowchart illustrating an example of the operation of the imaging device 1A according to the second embodiment. As illustrated in FIG. 9, first, the image processing unit 43A performs the processing of steps S1 to S4 in the same manner as in the first embodiment, thereby obtaining the first image of the first imaging unit 10 and the second image of the second imaging unit 20. A distance image is generated using the two images.

  Next, in the image processing unit 43A, the target point calculation unit 436 uses the luminance image as the first image or the second image and the distance image to generate a plane area formed by the landing area, that is, a plane area corresponding to the landing area. Is specified (step S21). More specifically, the target point calculation unit 436 specifies a subject in a planar area in the luminance image and the distance image by removing pixels that do not form a planar subject. Further, the target point calculation unit 436 specifies the landing area by performing pattern matching between the edge of the plane area and the contour of the landing area in the luminance image.

  Next, the target point calculation unit 436 calculates the coordinates of the target point in the landing area (Step S22). Specifically, the target point calculation unit 436 specifies a target point in the landing area using the luminance image or the distance image. Further, the target point calculation unit 436 calculates the pixel coordinates of the target point in the luminance image and the distance image.

  Next, the self-position calculating unit 437 calculates the position of the imaging device 1A with respect to the target point (Step S23). Specifically, the self-position calculating unit 437 calculates the camera coordinates of the target point using the pixel coordinates and the distance value of the target point in the luminance image and the distance image. The camera coordinates are the camera coordinates of the first imaging unit 10 or the second imaging unit 20 that has captured the luminance image. The self-position calculator 437 calculates the position of the imaging device 1A with respect to the target point from the camera coordinates of the target point.

  Next, the self-posture calculation unit 438 calculates the posture of the imaging device 1A with respect to the plane of the landing area (Step S24). Specifically, the self-posture calculation unit 438 calculates a function indicating the plane of the landing area on the camera coordinates using the luminance image and the distance image. Further, the self-posture calculating unit 438 calculates the posture angle of the imaging device 1A with respect to the plane of the landing region by calculating the inclination of the plane of the landing region on the camera coordinates, and outputs the output unit 3 and / or the storage unit 435. Output to The attitude angle of the imaging device 1A corresponds to the attitude angle of the flying object on which the imaging device 1A is mounted.

  Other configurations and operations of the imaging device 1A according to the second embodiment are the same as those of the first embodiment, and thus description thereof will be omitted. According to the imaging apparatus 1A according to the second embodiment as described above, the same effects as those of the first embodiment can be obtained.

  Furthermore, the imaging device 1A according to Embodiment 2 acquires the position and orientation of the flying object on which the imaging device 1A is mounted, using distance information acquired by processing the first image and the second image. An image processing unit 43A including a target point calculation unit 436, a self-position calculation unit 437, and a self-posture calculation unit 438 may be provided. Further, the image processing unit 43A may acquire the position and the attitude of the flying object with respect to the target point.

  According to the above configuration, the imaging device 1A can calculate the position and attitude of the flying object with respect to the distant target point because the distance measurement can be performed in a distant place. For this reason, the position and attitude of the flying object calculated by the imaging device 1A can be used for control of flying the flying object. For example, when the target point is a landing point, the position and attitude can be used for landing control of the flying object such as automatic landing control. For example, a GPS (Global Positioning System) mounted on an air vehicle may be subject to radio interference due to the ionosphere in the atmosphere. In such a case, the imaging device 1A can be used for detecting the position and attitude of the flying object instead of the GPS. Therefore, the flight safety of the flying object can be improved.

(Embodiment 3)
The imaging device 1B according to the third embodiment differs from the first and second embodiments in that an object is detected using a range image. Hereinafter, the third embodiment will be described focusing on the differences from the first and second embodiments, and the description of the same points as the first and second embodiments will be appropriately omitted.

  The configuration of the imaging device 1B according to the third embodiment will be described. FIG. 10 is a block diagram illustrating an example of a functional configuration of an imaging device 1B according to Embodiment 3. As shown in FIG. 10, the imaging device 1B includes a control device 40B. The control device 40B includes an image processing unit 43B. The image processing unit 43B includes an acquisition unit 431, a distance calculation unit 432, an image generation unit 433, a parameter storage unit 434, a storage unit 435, a clustering unit 439, a determination unit 440, and a position calculation unit 441. Including. The hardware configuration of the imaging device 1B is the same as that of the first embodiment. The functions of the clustering unit 439, the determination unit 440, and the position calculation unit 441 are realized by the CPU 101 and the like, like the other components. Here, the clustering unit 439, the determination unit 440, the position calculation unit 441, and the image processing unit 43B are examples of a first detection unit.

  The clustering unit 439 divides the range image into local regions, and calculates a distance histogram of the local region. Further, the clustering unit 439 calculates the similarity of the distance histogram between the local regions, and classifies a plurality of local regions regarded as similar and located close to each other as one block. Then, the clustering unit 439 determines that the subjects projected to all the local regions included in the classified one lump constitute one object.

  For example, the clustering unit 439 sets an area of 8 pixels × 8 pixels as a local area. Since the resolution of the first image sensor 12 and the second image sensor 22 is 1920 pixels × 1080 pixels, 32400 local regions are formed. The clustering unit 439 calculates a distance histogram indicating the frequency of each distance value from the distance values of 64 pixels in each local region. Further, the clustering unit 439 normalizes the distance histogram so that the sum of the frequencies becomes 1. The clustering unit 439 calculates the similarity of the normalized distance histogram (also referred to as “distance histogram feature amount”) between the two local regions, and calculates the similarity for all combinations of the local regions. The details of the method of calculating the similarity of the distance histogram are disclosed in, for example, Japanese Patent Application Laid-Open No. H10-157, and thus the description thereof is omitted.

  The clustering unit 439 classifies the local regions whose similarity is equal to or larger than the threshold. That is, the clustering unit 439 divides the local area into groups each including a local area whose similarity is equal to or larger than the threshold. The threshold is set in advance and stored in the storage unit 435. Further, the clustering unit 439 extracts a set of local regions formed by local regions adjacent to each other on the range image as one lump for local regions in one group. This block is composed of pixels having a similar distance from the imaging device 1B. A subject projected by a pixel included in one lump can be regarded as constituting an object such as one object. Further, the clustering unit 439 calculates the coordinates of the pixels included in each chunk on the distance image, that is, the pixel coordinates of each chunk, and outputs the calculated coordinates to the determination unit 440 and / or the storage unit 435.

  The determination unit 440 estimates the actual size of each chunk, and determines whether the chunk is a target object based on whether or not the actual dimension falls within the dimension threshold range. The threshold range is a range between the upper threshold and the lower threshold. The target object is an object to be detected by the imaging device 1B. When the imaging device 1B is mounted on a flying object, for example, the target object is an obstacle or the like that hinders flight or landing. The target object may be defined based on its type, or simply based on its dimensions. For example, an object within a predetermined size range may be set as a target object.

  The determination unit 440 calculates the pixel coordinates of each block in the luminance image that is the first image or the second image from the pixel coordinates of each block in the distance image. Further, the determination unit 440 extracts edges around each block on the luminance image, and calculates the coordinates of the pixels included in the edges, that is, the pixel coordinates of the edges. Then, the determination unit 440 obtains a distance value at the pixel coordinates of the edge using the distance image.

  Further, the determination unit 440 converts the pixel coordinates of the edge into camera coordinates using the pixel coordinates of the edge and the distance value at the pixel coordinates of the edge. The coordinates after the conversion are the camera coordinates of the edge of the block object. Furthermore, the determination unit 440 calculates the width of the lump in a direction perpendicular to the optical axis center of the first imaging unit 10 or the second imaging unit 20 using the camera coordinates of the edge of the lump subject. The calculated width is an estimated value of the actual dimension of the width of the block subject. The calculated width may be a maximum width or an average of a plurality of widths.

  Further, the determination unit 440 searches for the distance value of each pixel inside the edge in the distance image, and extracts the minimum distance value. Further, the determination unit 440 calculates a difference between the extracted minimum distance value and the distance value at the pixel coordinates of the edge. The calculated difference is an estimated value of the actual size of the height of the block subject. The distance value at the pixel coordinates of the edge may be a maximum value, a minimum value, an average value, or the like.

  The determination unit 440 determines that the subject is a target object when both the estimated values of the actual dimensions of the width and the height of the lump subject are within the width dimension threshold range and the height dimension threshold range. Otherwise, it is determined that the subject is not a target object. The width dimension threshold range and the height dimension threshold range are determined in advance according to the target object to be set, and are stored in the storage unit 435. The determination unit 440 calculates the estimated value of the actual size for each block as described above, and determines whether or not the block is a target object. The determination unit 440 outputs the determination result and information such as the distance value of the target object, camera coordinates, pixel coordinates, and the like to the position calculation unit 441.

  Upon receiving the information on the target object, the position calculation unit 441 calculates the position of the target object and outputs the calculated position to the output unit 3 and / or the storage unit 435. Specifically, the position calculation unit 441 generates position information indicating the position of the target object using the distance value of the target object, the camera coordinates of the edge of the target object, and the pixel coordinates acquired from the determination unit 440. For example, the position calculation unit 441 may calculate the distance and the direction from the imaging device 1B to the target object as the position information. For example, the position calculation unit 441 may generate, as the position information, an image in which the edge of the emphasized target object is superimposed and displayed on the image captured by the first imaging unit 10 or the second imaging unit 20.

  The operation of the imaging device 1B will be described. FIG. 11 is a flowchart illustrating an example of the operation of the imaging device 1B according to Embodiment 3. As shown in FIG. 11, the image processing unit 43B generates a distance image by performing the processing of steps S1 to S4 in the same manner as in the first embodiment.

  Next, the clustering unit 439 of the image processing unit 43B classifies and extracts clusters of pixels composed of pixels having similar distances from the imaging device 1B in the distance image (step S31). That is, the clustering unit 439 clusters a cluster of pixels formed of pixels having similar distances.

  Next, the determination unit 440 estimates the actual size of each block (Step S32). Specifically, the determination unit 440 estimates the actual dimensions of the width and height of the subject in each block using the distance image, the luminance image, and the pixel coordinates of each block.

  Next, the determination unit 440 determines whether or not the estimated value of the actual size of the block is within the size threshold range (step S33). That is, the determination unit 440 determines whether both the estimated values of the actual width and height of the block object are included in the width dimension threshold range and the height dimension threshold range. When both the estimated values of the actual dimensions of the width and the height are within the respective dimension threshold ranges (Yes in step S33), the determination unit 440 proceeds to step S34. Otherwise (No in step S33), Is completed.

  In step S34, the determination unit 440 determines the lump determined in step S33 as the target object, and outputs information on the target object to the position calculation unit 441.

  Next, the position calculation unit 441 generates position information indicating the position of the target object using the information of the target object (Step S35). The position calculation unit 441 outputs the position information of the target object to the output unit 3 and / or the storage unit 435.

  Other configurations and operations of the imaging device 1B according to the third embodiment are the same as those of the first embodiment, and thus description thereof will be omitted. According to the imaging apparatus 1B according to the third embodiment, the same effects as those of the first embodiment can be obtained.

  Further, the imaging device 1B according to Embodiment 3 includes a clustering unit 439, a determination unit 440, and a position calculation unit 441 that detect an object using distance information acquired by processing the first image and the second image. An image processing unit 43B may be provided. Further, the image processing unit 43B may acquire position information of the detected object using the distance information.

  According to the above configuration, since the imaging device 1B can perform distance measurement in a distant place, it can detect a distant object and its position. Therefore, when the imaging device 1B is mounted on a moving object such as a high-speed railway and an aircraft, the imaging device 1B can detect an object existing in the traveling direction of the moving object before the moving object approaches or contacts the object. Therefore, safe operation of the moving body is possible.

(Embodiment 4)
The imaging device 1C according to the fourth embodiment differs from the third embodiment in that a moving subject is detected with respect to the imaging device 1C using the distance image. Hereinafter, the fourth embodiment will be described focusing on the differences from the first to third embodiments, and the description of the same points as the first to third embodiments will be omitted as appropriate.

  The configuration of an imaging device 1C according to Embodiment 4 will be described. FIG. 12 is a block diagram illustrating an example of a functional configuration of an imaging device 1C according to Embodiment 4. As shown in FIG. 12, the imaging device 1C includes a control device 40C. The control device 40C includes an image processing unit 43C. The image processing unit 43C includes an acquisition unit 431, a distance calculation unit 432, an image generation unit 433, a parameter storage unit 434, a storage unit 435, a clustering unit 439, a determination unit 440, a position calculation unit 441, And a movement detection unit 442. The hardware configuration of the imaging device 1C is the same as that of the first embodiment. The function of the movement detection unit 442 is realized by the CPU 101 and the like, like the other components. Here, the movement detection unit 442 and the image processing unit 43C are examples of a second detection unit.

  The movement detection unit 442 detects, as a moving subject, a target object that moves with respect to the imaging device 1C from among the target objects determined by the determination unit 440.

  The movement detection unit 442 compares pixel coordinates of the same target object extracted from these distance images between two distance images having different imaging times. Two distance images having different imaging times are generated from two sets of the first image and the second image captured by the first imaging unit 10 and the second imaging unit 20 at different imaging times. It is desirable that the imaging times of the two distance images be closer so that the two distance images include the same target object. For example, one distance image is a distance image captured immediately before the other distance image.

  The movement detection unit 442 calculates the movement distance of the target object between the two range images based on the displacement of the pixel coordinates of the target object. Specifically, the movement detection unit 442 converts each of the two pixel coordinates of the target object into camera coordinates, and calculates the movement distance using the two camera coordinates. The movement detection unit 442 determines that the target object is a moving subject when the movement distance is equal to or greater than the movement threshold, and determines that the target object is a non-moving subject when the movement distance is less than the movement threshold. . The movement threshold is determined in advance according to the difference between the imaging times of the two distance images used for comparing the pixel coordinates of the target object, and is stored in the storage unit 435. For example, the movement threshold may be determined according to the frame rates of the first imaging unit 10 and the second imaging unit 20.

  The movement detection unit 442 outputs information such as the distance value, the camera coordinates, and the pixel coordinates of the moving subject detected as described above to the position calculation unit 441.

  When receiving the information of the moving subject, the position calculating unit 441 calculates the position of the moving subject and outputs the calculated position to the output unit 3 and / or the storage unit 435. Further, the position calculating unit 441 may calculate and output the speed and the moving direction of the moving subject with respect to the imaging device 1C.

  Other configurations and operations of the imaging device 1C according to the fourth embodiment are the same as those of the third embodiment, and thus description thereof will be omitted. Then, according to the imaging apparatus 1C according to the fourth embodiment described above, the same effect as that of the third embodiment can be obtained.

  Further, the imaging apparatus 1C according to Embodiment 4 uses the distance information acquired by processing the first image and the second image for the moving object that moves within the field of view of the first imaging unit 10 and the second imaging unit 20. The image processing unit 43 </ b> C including the movement detection unit 442 that detects using the method may be provided. Further, the image processing unit 43C may acquire the position information of the detected moving body using the distance information.

  According to the above configuration, the imaging device 1C can detect a distant moving object and its position because the distance can be measured in a distant place. Further, information on the moving body and its position detected by the imaging device 1C can be used regardless of whether the imaging device 1C is mounted on a stationary object or a moving object. Furthermore, the imaging device 1C can detect the moving direction and speed of the moving object using the position information of the moving object corresponding to a plurality of pieces of distance information having different image capturing times.

  For example, when the imaging device 1C is mounted on a moving object that moves at a high speed such as a high-speed railway, the detection result of the imaging device 1C is used to determine in advance whether the moving object may hinder the progress of the moving object. Can be When the imaging device 1C is mounted on a stationary object, the detection result of the imaging device 1C can be used for monitoring a distant moving body. For example, the imaging device 1C disposed on the ground can be used as one of sensors for monitoring a flying object such as a drone from the ground. For example, the imaging device 1C can be used as one of sensors for monitoring a suspicious object from a distance at sea or at a border guard.

  The imaging device 1C according to Embodiment 4 detects whether or not the target object is a moving subject based on the moving distance of the target object displaced between the two range images, but is not limited thereto. Not done. The imaging device 1C may detect whether or not the target object is a moving subject based on the moving distance of the target object that is displaced between three or more range images. In this case, the sum of the moving distances of the target object among the three or more range images may be compared with the moving threshold. Alternatively, the moving distance of the target object and the moving threshold between each pair of the distance images among the three or more distance images may be compared.

(Other embodiments)
As mentioned above, although the example of the embodiment of the present invention was explained, the present invention is not limited to the above-mentioned embodiment and the modification. That is, various modifications and improvements are possible within the scope of the present invention. For example, various modifications made to the embodiment or the modified example, and forms constructed by combining components in different embodiments and modified examples are also included in the scope of the present invention.

  For example, the imaging device according to the embodiment includes two imaging units, but may include three or more imaging units. In this case, the imaging device may generate the distance image by using two of the images captured by the three or more imaging units. Alternatively, the imaging device may generate a plurality of distance images by using images of each pair in images captured by three or more imaging units. Further, the imaging device may generate one distance image by combining a plurality of distance images, for example, to reduce a measurement error.

  In the imaging device according to the embodiment, the first imaging unit 10 and the second imaging unit 20 have the same focal length f1 and f2, and the pixel pitches p1 and p2 of the first imaging device 12 and the second imaging device 22. Are the same, but are not limited thereto. The focal lengths f1 and f2 may be different. In this case, for example, the shorter one of the focal lengths f1 and f2 may be determined so as to satisfy Expression 2. Further, the pixel pitches p1 and p2 may be different. In this case, for example, the larger one of the pixel pitches p1 and p2 may be determined so as to satisfy Expression 2.

  In the imaging device according to the embodiment, the optical axes of the first imaging unit 10 and the second imaging unit 20 are parallel, but are not limited to this. Between the case where the optical axis is parallel and the case where the optical axis is not parallel, even if the three-dimensional position of the subject is the same, the image coordinates of the subject on the first image and the second image are different, and The parallax of the subject is also different. Therefore, a correction coefficient or the like may be applied to an element that satisfies Expression 2 in order to incorporate the difference in parallax between the two cases. For example, when the correction coefficient is “α”, the corrected pixel pitch “p ′ (= α × p)” obtained by correcting the pixel pitch “p” that satisfies Equation 2 is used as the pixel pitch when the optical axis is not parallel. Is also good.

  Further, the numbers such as the ordinal numbers and the quantities used above are all examples for specifically explaining the technology of the present invention, and the present invention is not limited to the illustrated numbers. Further, the connection relation between the constituent elements is illustrated for specifically explaining the technology of the present invention, and the connection relation for realizing the function of the present invention is not limited to this.

  The division of blocks in the functional block diagram is merely an example, and a plurality of blocks may be implemented as one block, one block may be divided into a plurality of blocks, and / or some functions may be moved to another block. Good. Also, a single piece of hardware or software may process functions of a plurality of blocks having similar functions in parallel or in a time-division manner.

1, 1A, 1B, 1C Imaging device 10 First imaging unit 20 Second imaging unit 43, 43A, 43B, 43C Image processing unit (processing unit, calculation unit, first detection unit, second detection unit)
436 Target point calculation unit (calculation unit)
437 Self position calculation unit (calculation unit)
438 Self-posture calculation unit (calculation unit)
439 Clustering unit (first detection unit)
440 Judgment unit (first detection unit)
441 Position Calculator (First Detector)
442 Movement detection unit (second detection unit)

JP 2002-188917 A JP 2014-238409 A International Publication No. 2010/140613

Shojiro Osada, "Visual depth distance information and its depth sensitivity", Television, The Institute of Image Information and Television Engineers, August 1977, Vol. 31, No. 8, p. 649-655

Claims (5)

A first imaging unit having a plurality of pixels for capturing an image,
A second imaging unit having a plurality of pixels for capturing an image,
A processing unit that processes the first image captured by the first imaging unit and the second image captured by the second imaging unit to obtain distance information of the subject,
Base line length B (unit: m), focal length F (unit: mm), and pixel pitch p (unit: mm) of the first imaging unit and the second imaging unit are:
B × F / p> 700
An imaging device that satisfies the following relationship. The first imaging unit and the second imaging unit may further have a parallax between the first subject, which is a subject at a distance of 700 m, projected on the first image, and the first subject, projected on the second image, is 3 pixels or more. The imaging device according to claim 1, wherein: The image processing apparatus further includes a calculation unit that obtains the position and attitude of the flying object on which the imaging device is mounted using the distance information,
The imaging device according to claim 1, wherein the calculation unit acquires a position and a posture of the flying object with respect to a target point. Further comprising a first detection unit for detecting an object using the distance information,
The imaging device according to any one of claims 1 to 3, wherein the first detection unit further obtains position information of the detected object using the distance information. The moving body moving within the field of view of the first imaging unit and the second imaging unit, further comprising a second detection unit that detects using the distance information,
The imaging device according to any one of claims 1 to 4, wherein the second detection unit further acquires position information of the detected moving body using the distance information.