JP5802516B2 - Image acquisition device - Google Patents

Description

  The present invention relates to an image acquisition device that acquires a light spectrum characteristic of a measurement object and a hyperspectral image.

  At the same time as obtaining three-dimensional data of a measurement object, for example, by photogrammetrically measuring the measurement object, obtaining an image of the measurement object and obtaining three-dimensional data with an image.

  Three-dimensional data with an image obtained by a conventional three-dimensional measuring apparatus is used for map data and the like, and effects such as improving the visibility of the user are obtained.

  On the other hand, the obtained data is the three-dimensional position data of the measurement object, and the obtained information is the three-dimensional position of the measurement object.

  When measuring a measurement object, it is desirable to obtain more information, and it is desirable to obtain not only the position information of the measurement object but also information on the properties of the measurement object. (GIS).

  For example, if information on the growth status of crops can be obtained, it is possible to make appropriate judgments and appropriate treatments for agricultural work, or if the type of mineral exposed on the ground surface can be judged, it is possible to select an appropriate civil engineering method. And so on.

JP 2011-89895 A JP 2006-10376 A

  In view of such circumstances, the present invention provides an image acquisition device capable of acquiring optical spectrum characteristics, particularly hyperspectral images.

  The present invention includes an optical characteristic changing unit, an optical system that includes an objective lens, guides light from the objective lens to the optical characteristic changing unit, and an imaging device that receives light via the optical characteristic changing unit. And the optical property changing unit includes a plurality of dividing units and selectively arranges one of the dividing units in the optical path, and the dividing unit selects a specific wavelength from the light from the optical system. The present invention relates to an image acquisition device having a first area and a second area that does not change the optical characteristics of the light from the optical system.

  The present invention further includes an imaging control device, and an image of an image captured through the second region of one dividing unit and an image captured through the second region of another dividing unit. Based on the matching, an image captured through the first region of the one division unit and an image captured through the first region of the other division unit are combined to produce an optical spectrum composite image. The present invention relates to an image acquisition device to be created.

  The present invention further relates to an image acquisition apparatus further comprising a stop arranged in an optical path, the stop having a stop hole, and the wavelength selected by the optical characteristic changing unit changing by moving the stop. Is.

  The present invention also relates to the image acquisition apparatus, wherein the optical characteristic changing unit further includes another division unit that does not change the optical characteristics in both the first area and the second area.

  In the invention, it is preferable that the imaging control device is based on image matching between an image captured through the second region of a plurality of division units and a still image captured through the further division unit. The image acquisition apparatus which produces | generates a hyperspectral image by synthesize | combining the image imaged through the said 1st area | region of the division part, and the said still image.

  Furthermore, the present invention further includes a GPS device for measuring geocentric coordinates, and the imaging control device acquires a still image at the first point via the further dividing unit, and stops at the first point. A plurality of feature points are extracted from the image, and a moving image composed of frame images continuous in time series is acquired via the further dividing unit while moving from the first point to the second point, The moving image tracking is performed with the moving image moving from the point to the second point, the still image is acquired at the second point via the other dividing unit, and the feature point is specified in the still image at the second point. Based on the feature points, the still image of the first point and the still image of the second point are stereo-matched, and based on the position of the geocentric coordinate system of the first point and the second point measured by the GPS device. Forming a three-dimensional model and Device is intended according to the image capture device to create a four-dimensional model having a synthesizing and said 3-dimensional model and the light spectrum composite image, 3-dimensional position data and optical spectral information.

  According to the present invention, an optical characteristic changing unit, an optical system that includes an objective lens, guides light from the objective lens to the optical characteristic changing unit, and an image sensor that receives light via the optical characteristic changing unit. And the optical property changing unit includes a plurality of division units and selectively arranges one of the division units in the optical path, and the division unit transmits a specific wavelength from the light from the optical system. Since the first area to be selected and the second area in which the optical characteristic of the light from the optical system is not changed, the optical characteristic is changed by the optical characteristic changing unit and the optical characteristic is changed by one camera. Both of the optical spectrum images thus obtained can be acquired simultaneously, and the structure can be simplified and the cost can be reduced.

  According to the invention, it further includes an imaging control device, and an image captured through the second region of one division unit and an image captured through the second region of the other division unit, Based on the image matching, an image captured through the first region of the one dividing unit and an image captured through the first region of the other dividing unit are combined to generate an optical spectrum. Since the image is created, the matching condition between the images captured through the second region can be applied to the image captured through the first region as it is.

  According to the present invention, the optical system further includes a stop disposed in the optical path, the stop having a stop hole, and the wavelength selected by the optical characteristic changing unit is changed by moving the stop. An optical spectrum in a predetermined wavelength range can be acquired without exchanging the optical property changing unit.

  According to the invention, the optical property changing unit further includes another division unit that does not change the optical property in both the first region and the second region, so that only the real image that does not change the optical property is obtained. Thus, moving image tracking can be easily performed when moving the image acquisition apparatus.

  According to the invention, the imaging control device is based on image matching between an image captured through the second region of a plurality of division units and a still image captured through the further division unit. Since the hyperspectral image is created by synthesizing the image captured through the first region of the plurality of division units and the still image, the hyperspectral image can be easily acquired with one camera. it can.

  Furthermore, according to the present invention, a GPS device for measuring geocentric coordinates is further provided, and the imaging control device acquires a still image at the first point via the further dividing unit, and the first point. A plurality of feature points are extracted from the still image, and a moving image composed of time-sequential frame images is acquired via the other dividing unit while moving from the first point to the second point, and The moving image tracking is performed with the moving image moving from the first point to the second point, and a still image is acquired at the second point via the other dividing unit, and the feature point is added to the still image at the second point. , And stereo-matching the still image of the first point and the still image of the second point based on the feature point, and the position of the geocentric coordinate system of the first point and the second point measured by the GPS device To form a three-dimensional model based on The image control device synthesizes the optical spectrum composite image and the three-dimensional model to create a four-dimensional model having three-dimensional position data and optical spectrum information, so that the four-dimensional model can be easily acquired with one camera. Thus, it is possible to easily obtain arbitrary three-dimensional position data and optical spectrum information of the measurement object.

It is the schematic which shows the small air vehicle in which the imaging device which concerns on this invention is mounted. It is a schematic block diagram of the camera part and imaging control apparatus of the imaging device which concern on 1st Example of this invention. It is a front view of the interference filter provided with the some permeation | transmission interference film | membrane from which the characteristic which concerns on a 1st Example differs. 4A and 4B are explanatory views showing an optical system of a camera having a transmission type interference filter used in an embodiment of the present invention, in which FIG. 5A shows a state in which the aperture is aligned with the optical axis, and FIG. The state separated from the shaft is shown. It is a graph which shows the relationship between an incident angle and the peak wavelength which permeate | transmits. It is a graph which shows the wavelength transmission characteristic corresponding to the incident angle to an interference filter. It is explanatory drawing which shows the state which acquires a hyperspectral image in a present Example. It is explanatory drawing which shows the state between images at the time of acquiring multiple images in a hovering state. It is a flowchart explaining the effect | action of the 1st Example of this invention. 10 is a flowchart for explaining details of STEP: 03 to STEP: 06 in FIG. 9. It is explanatory drawing which shows the optical system which has a reflection type interference filter with the camera used for the 2nd Example of this invention, (A) shows the state in which the aperture hole corresponded with the optical axis, (B) A state in which the aperture hole is separated from the optical axis is shown. It is a front view of the interference filter provided with the some reflective interference film from which the characteristic in a 2nd Example differs. It is explanatory drawing which shows the optical system which has a reflective interference filter in the example of a change of the camera used for the 2nd Example of this invention, (A) shows the state in which the aperture hole corresponded with the optical axis, B) shows a state where the aperture hole is separated from the optical axis. It is a schematic block diagram of the camera part and imaging control apparatus of the imaging device which concern on 3rd Example of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  An image acquisition apparatus according to an embodiment of the present invention is mounted on a small unmanned aerial vehicle, for example, a small helicopter capable of remote control or a small helicopter capable of autonomous flight.

  FIG. 1 shows a small aircraft 1 equipped with an image acquisition device according to this embodiment.

  In FIG. 1, 2 is a base control apparatus installed on the ground. The base control device 2 is capable of data communication with the flying object 1 and stores and manages the control of the flying object 1, setting and changing the flight plan, and information collected by the flying object 1.

  The flying object 1 is a helicopter as a small flying object that autonomously flies, for example. The helicopter 1 is operated by remote control from the base control device 2, or a flight plan is set from the base control device 2 to a control device (not shown) of the helicopter 1, and the control device is operated by a navigation means (described later). ) And fly autonomously according to the flight plan. Further, the control device can control the navigation means to control the helicopter 1 to fly at a predetermined speed and a predetermined altitude, and to control the helicopter 1 in a hovering (stationary flight state) at a predetermined position.

  The helicopter 1 includes a fuselage 3 and a required number of propellers provided in the fuselage 3, for example, four sets of propellers 4, 5, 6, and 7, for example, front, rear, left, and right. Each of the motors is individually connected to a motor (not shown), and the driving of the motors is controlled independently. The propellers 4, 5, 6, 7 and the motor constitute a flying means of the flying object.

  The airframe 3 is provided with a GPS device 9 that measures the reference position of the helicopter 1 (for example, the center of the airframe 3).

  An imaging device 11 is mounted on the body 3 of the helicopter 1. The imaging device 11 has an optical axis 12, the optical axis 12 extends downward, and the imaging device 11 captures an image below the helicopter 1.

  Next, a schematic configuration of the imaging apparatus 11 according to the first embodiment of the present invention will be described with reference to FIG.

  The imaging device 11 includes a camera unit 13 and an imaging control device 21, and the imaging control device 21 is a photogrammetry of a measurement object based on image data captured by the camera unit 13 and position information from the GPS device 9. Or processing such as synthesis of the image data and optical spectrum image data acquired by the camera unit 13.

  First, the camera unit 13 will be described.

  The camera unit 13 includes a camera 14, an interference filter 15 that is an optical characteristic changing unit described later, and a motor 16 as a switching unit for the interference filter 15. The camera 14 is provided on the optical axis 12 and can acquire an image (real image) and an optical spectrum image of an object to be measured.

  The camera 14 captures an image of a measurement location and outputs digital image data so that a still image can be captured at predetermined time intervals and a moving image that continuously captures images can be captured. It has become.

  The camera 14 has a CCD or CMOS sensor as a collection of pixels as the image sensor 17, and the center of the image sensor 17 (coordinate center of the light receiving surface) is the optical axis. The relationship between the optical axis 12 and the image sensor 17 is set so that 12 passes vertically. Therefore, each pixel of the image sensor 17 can specify the position (coordinates) on the image sensor 17, and the angle of view of each pixel (angle with respect to the optical axis 12) can be known.

  The camera 14 has an interference filter 15 provided on the optical axis 12 and having an interference film formed in part. The interference filter 15 is rotatably supported and can be rotated by the motor 16. ing. The interference filter 15 has a plurality of divided portions having different optical characteristics, and each divided portion has a region where the interference film is formed and a region not having optical characteristics. The division part is selected by rotating the interference filter 15 by the motor 16. The selected splitting unit coincides with the optical axis of the camera 14, the light passes through the selected splitting unit, and light of a predetermined wavelength and light of all wavelengths according to the splitting unit that passes further. At the same time, a light spectrum image of a predetermined wavelength and a real image are simultaneously acquired by the camera 14.

  Next, the details of the interference filter 15 will be described with reference to FIG.

  The interference filter 15 is a transmission type interference filter, and has a disk shape as shown in FIG. 3, and the transmission surface is equally divided into a required angle in the circumferential direction (in the figure, divided into six equal parts). The part is formed as divided transmission surfaces 60a to 60f.

  A second region, a first region, and a second region are formed concentrically on the divided transmission surfaces 60a to 60e, which are a plurality of divided portions, among the divided transmission surfaces 60a to 60f. In the first region, transmission interference films having different selection wavelength characteristics of the selection wavelengths λ1 to λ5 are formed for each of the divided transmission surfaces 60a to 60e. Further, the divided transmission surface 60f, which is another divided portion, is formed in the first region and the first region. The two regions (60f ′, 60f ″) do not have optical characteristics, and transmit all wavelengths.

  For example, in the first region, the selective wavelength λ1 is 400 nm to 450 nm on the divided transmission surface 60a, the selective wavelength λ2 is 450 nm to 525 nm on the divided transmission surface 60b, and the selective wavelength λ3 is 525 nm to 650 nm on the divided transmission surface 60c. A transmission interference film having a selection wavelength λ4 of 650 nm to 750 nm and a selection wavelength λ5 of 750 nm to 870 nm is formed on the split transmission surface 60d, respectively.

  The divided transmission surfaces 60a to 60e have outer peripheral portions 60a ′ to 60e ′ on the outer peripheral side of the first region and inner peripheral portions 60a ″ to 60e ″ on the inner peripheral side as the second region. These second regions pass all wavelengths. The image 58 formed on the interference filter 15 extends over the second region, the first region, and the second region, and a part of the outer periphery of the image 58 is the outer peripheral portions 60a ′ to 60a ′. 60e 'and 60f' overlap, and a part on the inner peripheral side overlaps the inner peripheral portions 60a "to 60e" and 60f ".

  When the interference filter 15 is used, when the divided transmission surface 60f is selected and the optical axis is matched with the divided transmission surface 60f, the entire transmission is performed without selecting the wavelength, and only real image data is acquired. . Further, when any one of the divided transmission surfaces 60a to 60e is selected, for example, when the divided transmission surface 60e is selected, the first region that transmits the transmission interference film of the divided transmission surface 60e in the imaging 58. The part 58a is wavelength-selected, and the second region parts 58b and 58c that pass through the outer peripheral part 60e ′ and the inner peripheral part 60e ″ are totally transmitted, and optical spectrum image data obtained in the first area part 58a; Mixed image data obtained by integrating the real image data as the second region portions 58b and 58c is acquired.

  When the wavelength range of the desired optical spectrum ranges from 400 nm to 870 nm, the divided transmission surfaces are sequentially switched from the divided transmission surface 60a to the divided transmission surface 60e, and the optical spectrum is switched for each of the switched divided transmission surfaces 60. Images are acquired.

  Thus, by the rotation of the interference filter 15, a wavelength is selected in the range of 400 nm to 870 nm, and an image is acquired by the imaging device 17 for each selected wavelength, and predetermined light in the range of 400 nm to 870 nm is obtained. A spectrum can be acquired.

  In addition, the number of divisions is determined according to the wavelength of the optical spectrum to be obtained, and if the wavelength of the optical spectrum to be obtained is limited, the optical spectrum is acquired by selecting the divided transmission surface having the corresponding selective wavelength characteristic. do it.

  The interference filter 15 is a disc and is rotatable, but has a long rectangular shape. The interference filter 15 is divided in the longitudinal direction to form a divided transmission surface, and the interference filter 15 is slid in the longitudinal direction. The divided transmission plane may be switched.

  Next, the imaging control device 21 will be described.

  The imaging control device 21 includes an arithmetic control unit (CPU) 22, an image data recording unit 23, a camera controller 24, a camera control unit 25, a filter controller 26, a spectrum data storage unit 27, an image composition unit 28, an image processing unit 29, A feature extracting unit 31, a matching unit 32, a measuring unit 33, a model image forming unit 34, a display unit 35, and a storage unit 36 are provided.

  The camera control unit 25 controls the camera 14, the interference filter 15, and the diaphragm 55 in synchronization. The diaphragm 55 will be described later. The filter controller 26 drives the motor 16 based on a command signal from the camera control unit 25, and rotates and positions the interference filter 15 so that a light beam passes through a predetermined divided transmission surface of the interference filter 15. .

  The camera controller 24 acquires a signal emitted from the image sensor 17 based on a command signal from the camera control unit 25. When the light beam passes through the divided transmission surfaces 60a to 60e of the interference filter 15, the acquired mixed image data is converted into real image data according to the first region portion 58a and the second region portions 58b and 58c that are transmitted. And optical spectrum image data of a predetermined wavelength. The separated real image data is stored in the image data recording unit 23 in association with the imaging time, and the optical spectral image data is stored in the spectral data storage unit 27 in association with the imaging time.

  When the light beam passes through the divided transmission surface 60f of the interference filter 15, both the first region and the second region are all transmission surfaces, so that still image data consisting only of a real image is acquired. The acquired still image data is stored in the image data recording unit 23 in association with the imaging time.

  The image composition unit 28 synthesizes the optical spectrum image data stored in the spectrum data storage unit 27 based on the real image data of the second region stored in the image data recording unit 23, and further, an optical spectrum composite image. And the still image are combined to create a hyperspectral image having spectral information in all pixels (pixels) of one image.

  The image processing unit 29 includes the feature extraction unit 31 and the matching unit 32. The image processing unit 29 extracts at least five or more feature points (pass points) from one frame of image data, and image data different in time or different imaging. The image data acquired from the position is tracked or matched based on the feature points.

  For image tracking and image matching, an SSDA method (Sequential Similarity Detection Algorithm), a normalized cross-correlation method, a least square matching method, or the like is used.

  The measurement unit 33 performs photogrammetry based on two pieces of image data acquired by the camera 14 from different imaging positions and position data of the imaging positions.

  The model image forming unit 34 associates the three-dimensional data of each pixel measured by the measuring unit 33 with the hyperspectral image, and forms a model image having four-dimensional data of three-dimensional data + light spectrum image data.

  The storage unit 36 includes a program necessary for camera control, a program necessary for motor control, a program necessary for synthesis of image data and optical spectrum image data, a program for image tracking, a program necessary for image processing, and a measurement. Various programs such as a program necessary for forming the model image, a program necessary for forming the model image, and a program for controlling the display unit 35 are stored. The image data recording unit 23 and the spectrum data storage unit 27 may be formed in part of the storage unit 36.

  Next, an example of the camera 14 used in the present embodiment will be described with reference to FIGS. The camera 14 described below is configured to obtain a light spectrum obtained by further subdividing the light spectrum obtained by the divided transmission surfaces 60a to 60e of the interference filter 15.

  FIG. 4 shows the optical system 45 of the camera 14 and the interference filter 15 provided in the optical path of the optical system 45.

  In FIG. 4, reference numeral 46 denotes an optical axis of the optical system 45. On the optical axis 46, an objective lens 47, a first relay lens 48, a second relay lens 49, a third relay lens 50, an imaging lens 51, an image pickup lens. An element 52 is provided. 4, 53 indicates an image formed by the objective lens 47, and f indicates the focal length of the second relay lens 49.

  A diaphragm 55 is disposed on the second relay lens 49 side of the first relay lens 48, and the objective lens 47, the first relay lens 48, the second relay lens 49, the third relay lens 50, and the connection. The optical system 45 is configured by the image lens 51, the imaging element 52, and the diaphragm 55.

  The diaphragm 55 has a slit-shaped diaphragm hole 55a extending in a direction perpendicular to the paper surface in the drawing. The diaphragm 55 is disposed at the object-side focal position or substantially the object-side focal position of the second relay lens 49, and the diaphragm 55 is perpendicular to the optical axis 46 (perpendicular to the diaphragm hole 55a). The diaphragm 55 is supported so as to be movable in the direction), and the position of the diaphragm 55 is appropriately changed by position displacement means such as a linear motor.

  Here, the diaphragm 55 and the second relay lens 49 constitute a telecentric optical system 56. The light beam that has passed through the first relay lens 48 is divided by the telecentric optical system 56 into a large number of parallel light beams (principal rays 57).

  A transmission type interference filter in which a plurality of transmission type interference films having different selection wavelength characteristics are formed at a condensing position of the principal ray 57 (an imaging position by the second relay lens 49 or a substantially imaging position). 15 is disposed. The interference filter 15 is supported so as to be rotatable about a rotation shaft 59, and is further rotatable by a rotating means such as a motor. The interference filter 15 functions as a wavelength selection filter, and a light beam having a specific wavelength that has passed through the interference film of the interference filter 15 is coupled onto the image sensor 52 by the third relay lens 50 and the imaging lens 51. Imaged. The formed image is formed at a specific wavelength and becomes a two-dimensional image.

  The interference filter 15 has a property that the selected wavelength characteristic changes depending on the incident angle of the light beam incident on the interference filter 15. FIG. 5 shows the relationship between the incident angle and the transmitted peak wavelength (dependence of the peak wavelength on the incident angle), and it can be seen that the peak wavelength is changed by changing the incident angle.

  4A, the aperture hole 55a of the aperture 55 is positioned on the optical axis 46, and in this case, the principal ray 57 is parallel to the optical axis 46. FIG. Next, when the diaphragm 55 is moved as shown in FIG. 4B, for example, as shown in the drawing, the principal ray 57 is inclined with respect to the optical axis 46. That is, the incident angle with respect to the interference filter 15 changes. Therefore, the wavelength transmitted through the interference filter 15 can be changed by moving the diaphragm 55.

  For example, referring to FIG. 5, when the incident angle with respect to the interference filter 15 is changed from 0 ° to 50 °, the peak of the transmission wavelength changes from 600 nm to 520 nm. That is, the interference filter 15 has a wavelength selection range W of 600 nm to 520 nm. FIG. 6 shows a wavelength transmission characteristic corresponding to the incident angle to the interference filter 15 when the angle of the interference filter 15 is changed to 0 °, 10 °, 20 °, 30 °, and 40 °. An example of the obtained optical spectrum is shown.

  In FIG. 4, the interference filter 15 is inclined with respect to the optical axis 46, but as shown in FIG. 5, the incident angle dependency becomes linear when the incident angle exceeds 10 °. Therefore, by tilting the interference filter 15 in advance, a change in the selected wavelength with respect to the displacement of the diaphragm 55 can be effectively obtained.

  Therefore, each time the diaphragm 55 is displaced, an image is acquired from the image sensor 17 and, for example, the divided transmission surface of the interference filter 15 having the wavelength transmission characteristics shown in FIG. , An optical spectrum in the wavelength range of 600 nm to 520 nm can be obtained. When acquiring an optical spectrum in a wavelength range exceeding 600 nm to 520 nm, the interference filter 15 can be rotated so that a split transmission surface having a different wavelength selection range W ′ is disposed in the optical path of the principal ray 57. That's fine. As described above, by combining the diaphragm 55 with the interference filter 15, it is possible to obtain a light spectrum obtained by further subdividing the light spectrum obtained by the interference filter 15 itself.

  Hereinafter, the operation in this embodiment will be described with reference to FIGS. In the following, the case where the optical system 45 having the interference filter 15 is used as the camera 14 and the camera 14 is mounted on the helicopter 1 that is a flying object will be described.

  In a state where an image is acquired in the hovering state, the posture of the camera 14 always fluctuates and cannot be said to be completely stationary. Therefore, there is a slight shift in the image acquired for each wavelength. For this reason, if the optical spectrum images stored in the spectrum data storage unit 27 are synthesized as they are, problems such as errors or blurring occur.

  FIG. 8 shows a state in which an optical spectrum image having selected wavelengths λ1, λ2, λ3, and λ4 is acquired in the hovering state. In S1 of FIG. 8A, the imaging device 11 is completely stationary. S2 is a hovering state and the imaging device 11 is changing. FIG. 8B is a diagram in which the optical spectrum images of λ1, λ2, λ3, and λ4 acquired in a state where the imaging device 11 is fluctuated are developed with time. In the figure, black circles are feature points extracted from images (real images) transmitted through the transmission regions of the outer peripheral portions 60a ′ to 60e ′ and the inner peripheral portions 60a ″ to 60e ″, which are the second regions. As can be seen from FIG. 8A and FIG. 8B, when the images are synthesized as they are, the feature points do not match between the images, and errors or blurring can be seen.

  Therefore, it is necessary to match (relatively align) the feature points in the second region so that the optical spectrum image acquired for each wavelength in the hovering state can be synthesized.

  In the present embodiment, by rotating the interference filter 15, the principal ray 57 is transmitted through the divided transmission surface 60f, whereby a still image consisting only of a real image can be obtained, and the principal ray 57 is transmitted to the divided transmission surface 60a. ˜60e is transmitted to obtain a light spectrum image, and further, a real image is obtained in which the principal ray 57 is transmitted through the transmissive regions of the outer peripheral portions 60a ′ to 60e ′ and the inner peripheral portions 60a ″ to 60e ″. Therefore, mixed image data in which these are integrated can be acquired.

  First, after the helicopter 1 is hovered at the point O1 and stopped, the interference filter 15 is rotated to select the divided transmission surface 60f. Thereafter, the camera 14 acquires a still image (left image 42) at the O1 point via the divided transmission surface 60f, and the GPS device 9 measures the position of the O1 point. The obtained still image is stored in the image data recording unit 23, and the image processing unit 29 further has at least 5 feature points (preferably many) from the image data of the second region portions 58b and 58c of the still image. ) Extracted.

  After the still image is acquired at the point O1, the interference filter 15 is rotated at the point O1, and the divided transmission surfaces 60a to 60e are sequentially switched. Further, the position of the diaphragm 55 is displaced at predetermined time intervals on each divided transmission surface, a mixed image is acquired every time the position of the diaphragm 55 is changed, and each divided transmission is transmitted with respect to a portion that has passed through the first region portion 58a. An optical spectrum image of λ1 to λn is acquired for each surface.

  For example, when the split transmission surface 60a is selected, optical spectrum images of λa1 to λan are acquired, and when the split transmission surface 60b is selected, optical spectrum images of λb1 to λbn are acquired and the split When the transmission surface 60c is selected, an optical spectrum image of λc1 to λcn is acquired. When the divided transmission surface 60d is selected, an optical spectrum image of λd1 to λdn is acquired, and the divided transmission surface 60e is obtained. When selected, optical spectrum images of λe1 to λen are acquired.

  By acquiring all the above optical spectrum images, optical spectrum images at all wavelengths are acquired. During the acquisition of the optical spectrum image, at least five feature points are extracted from the real image obtained by the second region portions 58b and 58c, and tracking between real images temporally adjacent based on the feature points is performed. Is done. In addition, after the acquisition of the real image, if an optical axis tilt or the like occurs between the two real images before acquiring the next real image, coordinate conversion is executed based on the feature points between the two real images. Matching is performed.

  Note that the real image separated from the mixed image has a narrow imaging range, but since the real image is a captured image in a stationary state, the amount of displacement between the images is small, and the real image in a narrow range Tracking is possible.

  Furthermore, since the positional relationship between the real image and the optical spectrum image separated from the mixed image is always constant, the conditions obtained between the real images can be applied to matching between the optical spectrum images as they are.

  Thus, when optical spectrum images at all wavelengths at the O1 point are acquired and tracking between all real images is completed, all the optical spectrum images are aligned and error-free based on the tracking information of the real images. Can be synthesized.

  Also, a hyperspectral image is created by synthesizing the optical spectrum composite image and the first region 58a of the still image at the point O1.

  When the movement from the O1 point to the O2 point is started, the interference filter 15 is first rotated to switch to the divided transmission surface 60f. After switching the interference filter 15, at least five feature points are first extracted from the still image acquired at the O1 point, and movement from the O1 point to the O2 point is started. During the movement from the O1 point to the O2 point, a moving image (frame image) is acquired by the camera 14 through the divided transmission surface 60f, and each frame image is tracked based on the extracted feature points. Since the split transmission surface 60f can acquire a real image in the entire region, image tracking can be performed in response to a large displacement.

  When the O2 point is reached, a still image (right image 43) at the O2 point is acquired by the camera 14 via the divided transmission surface 60f, and the position of the O2 point is measured by the GPS device 9. Next, the image processing unit 29 specifies at least five feature points in the still image at the O2 point, and performs matching between the still image at the O1 point and the still image at the O2 point based on the feature points. Further, the measurement unit 33 performs photogrammetry based on the still image acquired at the O1 point, the still image acquired at the O2 point, and the position data of the O1 point and the O2 point measured by the GPS device 9, and for each pixel. Acquire 3D data.

  Finally, the three-dimensional data obtained at the O1 point and the hyperspectral image obtained at the O1 point are synthesized to create a four-dimensional image including the spectral data.

  In the above description, the hyperspectral image is acquired at the O1 point. However, the hyperspectral image can also be acquired at the O2 point by performing the same processing as the O1 point. Therefore, a hyperspectral image may be acquired at the O2 point.

  Therefore, it is possible to recognize the size of the growth state of the crop from the light spectrum or from the three-dimensional data of the crop. Alternatively, information on the type of the substance whose surface condition is exposed can also be acquired.

  The above-described photogrammetry, optical spectrum image, hyperspectral image acquisition synthesis and the like will be further described with reference to FIGS.

  (Step 01) Hovering of the helicopter 1 is started at the point O1, the interference filter 15 is rotated to select the divided transmission surface 60f, and the left image 42 is acquired by the camera 14 at the point O1. Further, the position of the helicopter 1 (that is, the O1 point) is measured by the GPS device 9.

  (Step 02) The feature point is extracted by subjecting the acquired left image 42 to edge processing or corner extraction processing.

  (Step 03) Then, the interference filter 15 is intermittently rotated so that the principal ray 57 passes through a predetermined region of the divided transmission surfaces 60a to 60e, and the position of the diaphragm 55 is changed for each divided transmission surface. A mixed image having a light spectrum image and a real image is acquired by the image camera 14 for each position of each aperture 55. In addition, the optical spectrum image and the real image are separated from the acquired mixed image, and image tracking at the same position (hereinafter, the same position tracking) is performed by the separated real image, and each optical spectrum is based on the result of the same position tracking. Image position correction between images is executed.

  Acquisition of the optical spectrum image at the O1 point and image position correction will be described in STEP: 21 to STEP: 27.

  STEP: 21 When spectrum measurement is started, a mixed image having an optical spectrum image portion in a predetermined wavelength region (λ1 to λn) is acquired at predetermined time intervals for each wavelength.

  (Step 22) The acquired mixed image is separated into an optical spectrum image portion and a real image portion by the camera controller 24, and the optical spectrum image portion is stored in the spectrum data storage portion 27 in time series, and the real image Are stored in the image data recording unit 23 in time series.

  STEP: 23 Image tracking (co-location tracking) extracts at least five feature points from the real image portion (first real image) acquired simultaneously with the separated optical spectrum image portion (λ1), and is temporally adjacent. The feature point is specified in the real image portion (second real image) synchronized with the next optical spectrum image portion (λ2).

  STEP: 24 Matching between the first real image portion and the second real image portion based on the obtained feature points of the first real image portion and the second real image portion, the first real image portion and the second real image portion Coordinate conversion with the image portion is executed.

  (Step 25) Since the real image portion and the optical spectrum image portion are acquired at the same time, the positional relationship with the optical spectrum image portion corresponding to the real image portion is always constant. These conditions are applied to the optical spectrum image portions that are temporally adjacent.

  STEP: 26 It is determined whether or not the optical spectrum image parts of all wavelengths are acquired. If not acquired, the process returns to STEP: 21 and the acquisition of the optical spectrum image part and the same position tracking are performed.

  (Step 27) When the optical spectrum image portions for all wavelengths in the predetermined wavelength region (λ1 to λn) are acquired, all the optical spectrum image portions are synthesized under the conditions obtained by tracking by the real image portion to thereby obtain the O1 point. The first optical spectrum composite image having the optical spectrum of the predetermined wavelength region (λ1 to λn) at can be acquired. Furthermore, a hyperspectral image can be acquired by synthesizing the first optical spectrum composite image and the still image.

  STEP: 04 to STEP: 06 When the still image and the hyperspectral image at the O1 point are acquired, the helicopter 1 moves to the O2 point. In the movement, the interference filter 15 is first rotated to select the divided transmission surface 60f. During movement, a moving image is acquired by the camera 14 and image tracking (movement tracking) is executed. The movement tracking may be executed based on the feature points extracted from the left image 42, or image tracking during movement may be performed using the feature points finally obtained by the same position tracking in the hovering state.

  (Step 07) When the helicopter 1 reaches the O2 point and the movement tracking is finished, the hovering is started and the right image 43 which is a still image is acquired.

  (Step 08) In the hovering state, the interference filter 15 is rotated to select the divided transmission surfaces 60a to 60e, the diaphragm 55 is further displaced, and a mixed image is acquired by the camera 14 for each position of the diaphragm 55. To do. Further, the optical spectrum image portion and the real image portion are separated from the acquired mixed image, and image tracking at the same position (hereinafter, the same position tracking) is performed by the separated real image portion, and based on the result of the same position tracking. Image position correction is performed between the optical spectrum image portions.

  STEP: 21 to STEP: 27 are executed, optical spectrum image portions for all wavelengths in the predetermined wavelength region (λ1 to λn) are acquired at the point O2, and all the obtained optical spectrum image portions are synthesized. A second optical spectrum composite image having an optical spectrum of a predetermined wavelength region (λ1 to λn) at the O2 point is acquired, and further, the second optical spectrum composite image and the right image 43 are combined to form a hyperspectral image. Is acquired.

  STEP: 09, STEP: 10, STEP: 11 Matching is performed based on the feature points specified in the right image 43 by image tracking and the feature points specified in the left image 42, and the left image Coordinate conversion (relative orientation) based on either 42 or the right image 43 is performed, and further, coordinate conversion (absolute orientation) to the geocentric coordinates of the GPS device 9 is performed.

  (STEP: 12, STEP: 13) Stereo matching between the left image 42 and the right image 43 is performed based on the absolute orientation result, and a three-dimensional model of terrain having three-dimensional position data is obtained.

  (Step 14) As described above, since the mixed image and the still image are captured coaxially and correspond to 1: 1, three-dimensional position data at the position where the optical spectrum is obtained is obtained, and the optical spectrum image By synthesizing the three-dimensional model, a four-dimensional model having three-dimensional position data of the terrain and optical spectrum information can be created.

  As described above, in this embodiment, the camera 14 is provided with the disc-shaped interference filter 15, and the interference filter 15 is formed with a divided transmission surface that is divided at a required angle in the circumferential direction. Since one of the surfaces is an all-transmission surface that transmits all wavelengths, both the real image part and the optical spectrum image part can be acquired with one camera, so the optical spectrum image is aligned based on the real image. Even when the image acquisition device is provided on a moving object such as the flying object 1, it is possible to acquire a highly accurate optical spectrum composite image or hyperspectral image. Further, it is not necessary to separately provide a spectrum camera for acquiring an optical spectrum image in addition to a camera for acquiring a real image, so that the structure can be simplified and the cost can be reduced.

  Further, by providing the diaphragm 55, it is possible to obtain one or more light spectra of the split transmission surface.

  In addition, since the second region where the principal ray 57 is totally transmitted is formed on a part of the divided transmission surface on which the interference films having different selection wavelength characteristics are formed, the optical spectrum image portion and the real image portion are simultaneously acquired. Even if a shift occurs between the images over time, the optical spectrum image portion can be easily synthesized by matching the real image portion.

  Further, since the divided transmission surface 60f is a total transmission surface through which the principal ray 57 is totally transmitted, only a real image can be acquired in a wide range, and processing that requires extraction of a large number of feature points such as moving image tracking. Even if it exists, the imaging device 11 in a present Example is applicable.

  In the optical system 45, the incident angle with respect to the interference filter 15 can be changed by changing the position of the diaphragm 55, and the wavelength can be changed within a predetermined range. If the number is fixed and there are few types, the diaphragm 55 may be omitted.

  FIGS. 11 and 12 show an optical system 45 ′ in the second embodiment of the present invention.

  In the optical system 45 shown in FIG. 3, the transmission type interference filter 15 is used. However, as shown in FIG. 11, the reflection type interference filter 61 which is an optical characteristic changing unit is used to constitute the optical system 45 ′. You can also The interference filter 61 is formed by forming a reflective interference film on a reflecting mirror, is rotatably supported around a rotating shaft 59, and is further rotatable by a rotating means such as a motor. In the optical system 45 ′, the wavelength is selected by the interference filter 61 reflecting the principal ray 57.

  In FIG. 11, the same components as those shown in FIG. 4 are denoted by the same reference numerals, and the description thereof is omitted.

  Also in the second embodiment, as shown in FIG. 11B, by moving the diaphragm 55, the incident angle of the principal ray 57 to the interference filter 61 changes, and a predetermined wavelength selection range W is achieved. A specific wavelength inside is selectively reflected.

  As shown in FIG. 12, the interference filter 61 has a disk shape, and is equally divided into a required angle in the circumferential direction (six equal parts in the drawing) on the reflecting surface, and the dividing portion is divided reflecting surface 63a. To 63f.

  In the interference filter 61, a second region, a first region, and a second region are formed concentrically on the divided reflection surfaces 63a to 63e, which are a plurality of divided portions among the divided reflection surfaces 63a to 63f, respectively. In the first region, a reflection interference film that is a first region having different selection wavelength characteristics of the selection wavelengths λ1 to λ5 is formed for each of the divided reflection surfaces 63a to 63e, and the divided reflection surface 63f that is another divided portion is The first region and the second region (63f ′, 63f ″) do not have optical characteristics and reflect all wavelengths.

  Further, each of the divided reflection surfaces 63a to 63e has an outer peripheral portion 63a ′ to 63e ′ on the outer peripheral side of the first region and an inner peripheral portion 63a ″ to 63e ″ on the inner peripheral side as the second region. These second regions reflect all wavelengths. The image 58 formed on the interference filter 61 extends over the second area, the first area, and the second area, and a part of the outer periphery of the image 58 is the outer peripheral portions 63a ′ to 63a ′. 63e 'and 63f' overlap, and a part on the inner peripheral side overlaps the inner peripheral parts 63a "to 63e" and 63f ".

  The light beam reflected by the interference filter 61 is imaged on the image sensor 52 through the imaging lens 51. When the interference filter 61 is used, when the divided reflection surface 63f is selected and the optical axis 46 is made to coincide with the divided reflection surface 63f, total reflection is performed without selecting the wavelength, and only the static image data is stationary. Image data is acquired. Further, when any one of the divided reflection surfaces 63a to 63e is selected, for example, when the divided reflection surface 63e is selected, the wavelength is selected by being reflected by the first region portion 58a of the divided reflection surface 63e and the above-described connection is made. Of the image 58, the second region portions 58b and 58c are partially totally reflected to obtain mixed image data.

  As described above, the optical system 45 ′ can be made compact by using the reflective interference filter 61 as the interference filter.

  FIG. 13 shows a modification of the optical system 45 ′ shown in FIG.

  In the modification shown in FIG. 13, a reflective interference filter 62 is used, and the interference filter 62 has the same configuration as the interference filter 61.

  An objective lens 47, a first relay lens 48, and an aperture 55 are disposed on the optical axis 46, and a second relay lens 49 is disposed on the optical axis that is parallel to the optical axis 46 and separated by a predetermined amount. The interference filter 62 is provided to face the lens 49. The light beam reflected by the interference filter 62 is deflected by the reflecting mirror 64, and the deflected light beam is imaged on the image sensor 52 through the imaging lens 51.

  In this modified example, since the first relay lens 48 and the diaphragm 55 are located at positions deviated from the optical axis 46 of the second relay lens 49, the principal ray 57 divided by the telecentric optical system 56 is inclined to the interference filter 62. Then enter. Further, as shown in FIG. 13B, if the diaphragm 55 is moved away from the optical axis 46, the incident angle of the principal ray 57 is further increased. Accordingly, the selected wavelength can be changed by moving the diaphragm 55.

  In the above modification, since the reflection type interference filter 62 is used, the optical system 45 ″ can be made compact, and the third relay lens 50 is also used as the second relay lens 49. The number of parts can be reduced, and the cost can be reduced.

  Next, a third embodiment of the present invention will be described with reference to FIG.

  FIG. 14 shows a schematic configuration of the image pickup apparatus 11 in the third embodiment. In the third embodiment, the mixed image data picked up by the camera 14 is converted into the optical spectrum image data by the camera controller 24. It is stored in the image data recording unit 23 without being separated into real image data.

  In the mixed image data stored in the image data recording unit 23, at least three feature points are extracted from the real image portion, that is, the second region portions 58b and 58c (see FIG. 3), and temporally based on the feature points. The mixed image data adjacent to is matched.

  The mixed image data is image data in which optical spectrum image data and real image data are integrated, and the positional relationship between the optical spectrum image data and real image data in the mixed image data is always constant. As a result of matching the mixed image data based on the optical spectrum image portion, the optical spectrum image portion is also automatically matched.

  Finally, from the matched mixed image data, the image separation unit 30 separates the real image part, thereby obtaining an optical spectrum composite image having an optical spectrum in a predetermined wavelength region (λ1 to λn).

  As described above, in the third embodiment, the feature point extraction and matching process of the real image portion in the mixed image data also serves as the synthesis process of the optical spectrum image data, thereby reducing the processing steps and reducing the processing burden. Can be achieved.

DESCRIPTION OF SYMBOLS 1 Helicopter 2 Base control apparatus 3 Airframe 9 GPS apparatus 11 Imaging apparatus 12 Optical axis 13 Camera part 14 Camera 15 Interference filter 21 Imaging control apparatus 22 Arithmetic control part 28 Image composition part 29 Image processing part 33 Measurement part 33 Model image formation part 35 Display unit 36 Storage unit 45 Optical system 47 Objective lens 48 First relay lens 49 Second relay lens 50 Third relay lens 51 Imaging lens 52 Imaging element 55 Aperture 56 Telecentric optical system 57 Main light beam 58 Imaging 60 Dividing transmission surface 61 Interference filter 62 Interference filter 63 Split reflection surface 64 Reflector

Claims (5)

An optical characteristic changing unit; an optical system including an objective lens; and an optical system that guides light from the objective lens to the optical characteristic changing unit; an image sensor that receives light via the optical characteristic changing unit; and an imaging control device; And the optical property changing unit includes a plurality of dividing units and selectively arranges one of the dividing units in the optical path, and the dividing unit selects a specific wavelength from the light from the optical system. a first region of an image have a second region which does not change the optical properties of the light, the imaging control device that is captured through the second region of the one division portion from the optical system, other Based on the image matching with the image captured through the second region of the dividing unit, the image captured through the first region of the one dividing unit and the first of the other dividing unit Combine the image captured through the area and combine the optical spectrum. Image acquisition apparatus characterized by creating an image. The image acquisition apparatus according to claim 1, further comprising a stop disposed in an optical path, the stop having a stop hole, and the wavelength selected by the optical characteristic changing unit changing by moving the stop. The optical characteristic change unit, image acquisition apparatus according to claim 1 or claim 2 further further comprising other divided portion does not change the optical characteristics in the first region and the second region both. The imaging control device is configured to match the images of the plurality of division units based on the image matching between the images captured through the second regions of the plurality of division units and the still images captured through the further division unit. The image acquisition apparatus according to claim 3 , wherein a hyperspectral image is created by synthesizing the image captured through the first region and the still image. A GPS device for measuring geocentric coordinates is further provided, and the imaging control device acquires a still image at the first point via the further division unit, and a plurality of feature points from the still image at the first point. To obtain a moving image composed of frame images continuous in time series through the further dividing unit while moving from the first point to the second point, and further from the first point to the second point. In addition to performing moving image tracking with the moving image being moved, a still image is acquired at the second point via the further dividing unit, the feature point is identified in the still image at the second point, and the feature point Based on the stereo matching between the still image of the first point and the still image of the second point, a three-dimensional model is formed based on the position of the geocentric coordinate system of the first point and the second point measured by the GPS device. In addition, the imaging control device includes the optical scanner. To synthesize a spectrum synthesized image and the three-dimensional model, an image acquisition apparatus according to claim 4 for creating a four-dimensional model having a three-dimensional position data and optical spectral information.