JP6959029B2 - Multi-line image sensor device and imaging device - Google Patents

Description

The present invention relates to a multi-line image sensor device and a photographing device for detecting a moving object from an image (single image) for one scene.

Traditionally, traffic volume surveys such as fixed-point observations have been conducted, but these surveys are costly and require time for surveys and analysis.

Further, in the field of image analysis, moving objects can be extracted by performing difference processing between images using images for two scenes with a minute time difference, but pair images taken at about the same time (for example, stereo). (Pair photography) is required, and there is a problem that a complicated process is included in the analysis process.

On the other hand, many earth observation satellites (artificial satellites) in recent years are equipped with a focal plane array (FPA) sensor camera, for example, by using a push broom sweep method. It is designed to take color (composite) images for observing the earth while moving on it.

However, due to the nature of the sensor arrangement, the focal plane array sensor camera causes a slight difference in the imaging and signal processing time of each band (for each color type). In particular, when the object to be photographed is a moving object that moves faster than the exposure time of the sensor, registration between bands is not performed correctly, causing a phenomenon of color shift.

That is, since the focal plane array sensor camera is a frame sensor (also called an area image sensor) capable of two-dimensional imaging, the light receiving surface is wide and the entire subject can be photographed relatively easily, but the resolution is low. .. Further, when photographing a spherical surface such as the earth, post-processing of the image becomes more complicated.

On the other hand, a line sensor (also referred to as a linear image sensor) capable of one-dimensional imaging can obtain a high-resolution image. Further, since a plurality of light receiving elements (photoelectric conversion elements) are arranged in a horizontal row, it is also suitable for photographing a spherical surface. Furthermore, technology for miniaturizing the optical system is also being researched.

Therefore, multi-line sensors capable of capturing color (composite) images are often mounted not only on earth observation satellites but also on printers and the like.

As such a multi-line sensor, a multi-spectral camera (also referred to as a multi-band one-dimensional line sensor camera) capable of simultaneously obtaining color images of a plurality of bands is known (see, for example, Patent Document 1).

Further, an imaging device capable of suppressing variation in image sharpness due to a difference in light receiving direction has also been proposed (see, for example, Patent Document 2).

Japanese Unexamined Patent Publication No. 2012-60411 International Publication (W0) 2011/010641 (PCT / JP2010 / 062184)

Here, the problem to be solved by the present invention will be described in more detail by taking as an example a case where an earth observation satellite equipped with a multispectral camera as disclosed in Patent Document 1 takes a picture while moving in orbit. However, here, a CCD line sensor camera will be described as an example.

The exposure time of the CCD line sensor camera mounted on the earth observation satellite is generally the time for the satellite to travel one line of scanning. For example, if the satellite has a spatial resolution (or image resolution) of 1 meter, the exposure time is the time required for the satellite to travel a distance of 1 meter.

Therefore, when a satellite equipped with a camera having CCD line sensors for three lines of R, G, and B shoots a moving object while moving in orbit, the moving object moving faster than the exposure time is between the line sensors. Due to the small physical separation distance, the reading of the signal charge cannot keep up with the speed of the moving object, and as a result, it appears as a color shift on the color (composite) image.

For example, if the color images of the bands taken at the same point on the same date and time and corrected so as to overlap geometrically are superimposed with the same size, the color composite image shows a stopped vehicle, road, building, etc. Still objects generally overlap, but moving objects such as moving vehicles, ships, or aircraft are accompanied by positional deviations ranging from a few pixels (also called pixels) to a dozen or so pixels.

That is, in the case of a CCD line sensor camera capable of acquiring a color (composite) image, a certain number of photodiodes (PDs) are arranged in a horizontal row, especially in an interline type structure, and the same number of VDCs (in parallel with this) It includes an R image line sensor, a G image line sensor, and a B image line sensor in which a vertical transfer unit) is arranged. Further, the line sensors for these three lines are arranged in parallel with a predetermined interval (physical separation distance), respectively.

Then, in each line sensor, for example, light from the optical system is exposed by the photodiode in response to a drive pulse transmitted from the sensor drive unit at intervals of time when the sensor moves a distance of one pixel. .. Further, in one exposure (line scanning), the signal charges photoelectrically converted by the photodiodes of each line sensor are, for example, simultaneously read out to the VCCD corresponding to each photodiode, sequentially and vertically transferred, and then. It is output via the horizontal transfer unit (HCCD).

When the signal charges of all the pixels in one exposure are output to the VCCD, the drive pulse for the next exposure is given to the R image line sensor, the G image line sensor, and the B image line sensor. Be done.

As described above, the CCD line sensor camera having the interline type structure has a slight physical separation distance between the R image line sensor, the G image line sensor, and the B image line sensor, respectively. Therefore, in the case of a moving body that moves faster than the exposure time, the signal charge cannot be read out in time, resulting in a large color shift.

Even in the case of a stationary object, the color images of each band taken at the same point on the same date and time will have some color shift due to the physical separation distance between the line sensors, so in the process of obtaining a color composite image. , It is necessary to perform alignment by processing such as line delay correction.

However, in the case of a moving body that moves to another pixel during the transfer of signal charge in one exposure, the color image of each band can be sufficiently obtained even if processing such as line delay correction is performed. It becomes difficult to align.

That is, when the color images of the bands obtained by aligning them so as to overlap geometrically are overlapped with the same size, the stationary objects (backgrounds) such as roads and buildings overlap with each other with almost no deviation, but they are moving. In vehicles, ships, aircraft, etc., the amount of movement between pixels ranges from several pixels to a dozen or so pixels depending on the moving speed, so color shift cannot be eliminated by line delay correction alone.

In addition, a method of detecting a moving body using this color shift has also been studied, but in the conventional method, when the color images of each band are combined, the moving body and the stationary object other than the moving body are the same. In order to be expressed as such a colored image, a complicated image processing technique is required to extract only moving objects from the color composite image.

The present invention has been made in view of the above problems, and obtains a multi-line image sensor device and a photographing device capable of clearly distinguishing a moving object from a stationary object and efficiently detecting the moving object. With the goal.

The multi-line image sensor device according to the present invention is
Three primary color image line sensors including an R image line sensor, a G image line sensor, and a B image line sensor, which are arranged in parallel at intervals from each other.
A first that is arranged in parallel with an image line sensor of any of the three primary color image line sensors at the interval and detects a wavelength of light of the same color as the image line sensor of any of the three primary color image line sensors. and a line sensor for the same wavelength component image,
The gist is that a color image is acquired by the three primary color image line sensors, and a black and white image is acquired by the image line sensor of any of the three primary color image line sensors and the first line sensor for the same wavelength component image. ..

The photographing apparatus according to the present invention is
A flying object having a predetermined height with respect to the shooting area and moving in one direction at a predetermined speed is provided, and as the flying object moves, a color image of the three primary colors and a color image of the three primary colors are used. An imaging device that obtains a color image having the same wavelength as any color image and a color image having the same wavelength of light.
Equipped with a multi-line image sensor device
The multi-line image sensor device is
Three primary color image line sensors including an R image line sensor, a G image line sensor, and a B image line sensor, which are arranged in parallel at intervals from each other.
The same wavelength that is arranged in parallel with the image line sensor of any of the three primary color image line sensors at the interval and detects the wavelength of light of the same color as the image line sensor of any of the three primary color image line sensors. Equipped with a line sensor for component images
Moreover,
An optical system for forming an image of light from the photographing area on the three primary color image line sensors and the same wavelength component image line sensor, and
A sensor drive unit that drives the three primary color image line sensors and the same wavelength component image line sensor,
Each output from the three primary color image line sensors output by driving the sensor drive unit is obtained as a first image, a second image, and a third image and horizontally transferred, and the line sensor for the same wavelength component image. It has a horizontal transfer unit that obtains the output from the same wavelength component as the same wavelength component image and transfers it horizontally .
The gist is that a color image is acquired by the three primary color image line sensors, and a black and white image is acquired by the image line sensor of any of the three primary color image line sensors and the same wavelength component image line sensor.

As described above, according to the multi-line image sensor device of the present invention, it is possible to simultaneously obtain a three-primary-color image and a same-wavelength component image having the same wavelength of light of any one of the three-primary-color image line sensors.

Further, according to the photographing device of the present invention, a multi-line image device for simultaneously obtaining a three-primary color image and an image having the same wavelength component which is the wavelength of light of the same color of any of the three primary color image line sensors is built in, and a photographing area is incorporated. Is provided on an air vehicle having a predetermined height and moving in one direction at a predetermined speed, an image is taken, and this is transmitted. Therefore, the receiving side can immediately detect the change of the moving body.

Further, since the color of the moving body is emphasized, the moving body information such as the moving speed, the moving direction, and the number of moving bodies can be calculated with high accuracy.

It is a schematic block diagram of the moving body detection system using the multi-spectral sensor device (multi-line image sensor device) which concerns on this embodiment. It is a schematic block diagram of the earth observation satellite equipped with the artificial satellite camera to which the multispectral sensor apparatus which concerns on this embodiment is applied. It is a schematic block diagram of the artificial satellite camera to which the multispectral sensor apparatus which concerns on Embodiment 1 is applied. FIG. 3 is a schematic cross-sectional view of a multispectral sensor device along the line I-I of FIG. It is a timing chart explaining the basic operation of an artificial satellite camera, and is (a) a schematic waveform diagram of a drive pulse PD, (b) a schematic waveform diagram of a control pulse TG, (c) a schematic waveform diagram of a control pulse BP, and (d). ) schematic waveform diagram of the control pulse GP, (e) schematic waveform diagram of the control pulse _{R 1} P, (f) a schematic waveform diagram of the control pulse _{R 2} P. It is a schematic block diagram of the mobile body detection device in the mobile body detection system using the multispectral sensor device which concerns on Embodiment 1. It is a schematic block diagram of the operation part in a moving body detection device. It is the schematic (image image) explaining the display example of the moving body detection image. It is a flowchart explaining the flow of processing in the area extraction part of a moving body detection apparatus. It is a flowchart explaining the flow of processing in a moving body detection apparatus. It is an operation explanatory diagram for explaining the moving body detection process, (a) the figure which shows the accumulation example of the photographed image eGi for each line in the database for photographed images, (b) the creation of mesh Mi in the memory for a color-enhanced image. The figure which shows an example. (A) A schematic diagram (image image) showing a display example of a moving object detection image (before color enhancement), and (b) a schematic diagram (image image) showing a display example of a moving object detection image (after color enhancement). be. The display characteristics in the moving object detection image are shown, and (a) a characteristic diagram showing a running vehicle as an example, and (b) a characteristic diagram showing a stopped vehicle as an example. It is a schematic diagram explaining the arrangement example of the multispectral sensor apparatus which concerns on Embodiment 1. (A) is a schematic view showing a stopped vehicle as an example, and (b) is a schematic view explaining the display example. (A) is a schematic view showing a vehicle traveling at a low speed as an example, and (b) is a schematic view explaining the display example. (A) is a schematic view showing a vehicle traveling at high speed as an example, and (b) is a schematic view explaining the display example. The moving body detection image is shown in comparison with (a) a schematic view (image image) showing the moving body detection image before color enhancement, and (b) a schematic view showing the moving body detection image after color enhancement (b). Image image). It is the schematic which shows the other configuration example of the multispectral sensor apparatus which concerns on this embodiment. It is a schematic diagram which shows the further structural example of the multispectral sensor apparatus which concerns on this embodiment. It is a schematic block diagram of the multispectral sensor apparatus which concerns on 1st modification of Embodiment 1. It is a schematic block diagram of the multispectral sensor apparatus which concerns on the 2nd modification of Embodiment 1. It is a schematic block diagram of the multispectral sensor apparatus which concerns on the 3rd modification of Embodiment 1. It is a schematic diagram (image image) explaining the case where the moving body is a cloud as an example of the moving body detection image. A case in which the moving body is a wave illustrates an example, (a) a schematic diagram of R ₁ image (image picture), a schematic view of (b) G image (image picture), a schematic diagram of (c) B image ( Image image), (d) Schematic diagram (image image) of the moving body detection image. It is a schematic block diagram of the artificial satellite camera to which the multispectral sensor apparatus (multiline image sensor apparatus) which concerns on Embodiment 2 is applied. FIG. 6 is a schematic cross-sectional view of a multispectral sensor device along line II-II of FIG. It is the schematic explaining the arrangement example of the multispectral sensor apparatus which concerns on embodiment 2. It is a schematic diagram explaining the method of designating a specific area in a moving body detection device. It is a timing chart in the case of line scanning a moving body, which shows a stopped vehicle as an example, (a) a schematic diagram showing the relationship between the multispectral sensor device 214II and the vehicle Ts, and (b) a schematic waveform of an imaging command. Figure, a schematic diagram showing an exposure example of (c) photodiode PD group 214R ₁ p, (d) a schematic diagram showing an example of transfer of the CCD group 214R ₁ f, (e) waveform shaping output example when transfer, (f) Schematic diagram showing an exposure example of the photodiode PD group 214Gp, (g) schematic diagram showing a transfer example of the CCD group 214Gf, (h) waveform molding output example during transfer, (i) exposure example of the photodiode PD group 214Bp. schematic, (j) schematic view showing an example of transfer of the CCD group 214Bf, (k) waveform shaping output example when transfer, (m) photodiode PD group 214R ₂ p schematic diagram showing an exposure example shown, (n) schematic diagram showing an example of transfer of the CCD group 214R ₂ f, (p) waveform shaping output example when transfer. It is a timing chart in the case of line scanning a moving body, which shows a vehicle traveling at a low speed as an example, (a) a schematic diagram showing the relationship between the multispectral sensor device 214II and the vehicle Tma, and (b) an outline of an imaging command. waveform, (c) a schematic diagram showing an exposure example of the photodiode PD group 214R ₁ p, (d) a schematic diagram showing an example of transfer of the CCD group 214R ₁ f, (e) waveform shaping output example when transfer, (f () Schematic diagram showing an exposure example of the photodiode PD group 214Gp, (g) a schematic diagram showing a transfer example of the CCD group 214Gf, (h) an example of waveform molding output during transfer, (i) an exposure example of the photodiode PD group 214Bp. schematic diagram showing, (j) schematic view showing an example of transfer of the CCD group 214Bf, (k) waveform shaping output example when transfer, (m) photodiode PD group 214R ₂ p schematic diagram showing an exposure example, (n ) Schematic diagram showing a transfer example of the CCD group 214R ₂ f, (p) Example of waveform molding output during transfer. It is a timing chart in the case of line scanning a moving body, which shows a vehicle traveling at high speed as an example, (a) a schematic diagram showing the relationship between the multispectral sensor device 214II and the vehicle Tmb, and (b) an outline of an imaging command. waveform, (c) a schematic diagram showing an exposure example of the photodiode PD group 214R ₁ p, (d) a schematic diagram showing an example of transfer of the CCD group 214R ₁ f, (e) waveform shaping output example when transfer, (f () Schematic diagram showing an exposure example of the photodiode PD group 214Gp, (g) a schematic diagram showing a transfer example of the CCD group 214Gf, (h) an example of waveform molding output during transfer, (i) an exposure example of the photodiode PD group 214Bp. schematic diagram showing, (j) schematic view showing an example of transfer of the CCD group 214Bf, (k) waveform shaping output example when transfer, (m) photodiode PD group 214R ₂ p schematic diagram showing an exposure example, (n ) Schematic diagram showing a transfer example of the CCD group 214R ₂ f, (p) Example of waveform molding output during transfer. The display characteristics in the moving object detection image are shown, and (a) a characteristic diagram showing a vehicle traveling at high speed as an example, and (b) a characteristic diagram showing a vehicle traveling at low speed as an example. The purpose is to explain a display example of a moving body, (a) a schematic diagram illustrating the relationship between the vehicle and the multispectral sensor device, and (b) a display example when the traveling direction of the vehicle is the east side. (C) Display example when the traveling direction of the vehicle is the west side, (d) Display example when the traveling direction of the vehicle is the north side, (e) Display example when the traveling direction of the vehicle is the south side. .. An example of the process for obtaining the moving object detection image is shown, and (a) a schematic diagram (image image) of the extracted image, (b) a schematic diagram (image image) of the designated image, and (c) a moving object detection. Schematic diagram of the image (image image). Other examples of the process for obtaining the moving object detection image are shown, and (a) a schematic diagram (image image) of the extracted image, (b) a schematic diagram (image image) of the designated image, and (c) a moving object. Schematic diagram (image image) of the detected image. It is an enlarged view (image image) of the moving body detection image shown in FIG. 36 (c). It is a schematic block diagram of the multispectral sensor apparatus which concerns on 1st modification of Embodiment 2. It is a schematic block diagram of the multispectral sensor apparatus which concerns on the 2nd modification of Embodiment 2. It is a schematic block diagram of the multispectral sensor apparatus which concerns on the 3rd modification of Embodiment 2. It is a schematic block diagram of the multispectral sensor apparatus which concerns on the 4th modification of Embodiment 2. It is a schematic block diagram of the mobile body detection device in the mobile body detection system using the multispectral sensor device which concerns on Embodiment 3. It is the schematic which shows an example of the processing in a moving body detection apparatus. It is the schematic which shows the other example of the processing in a moving body detection apparatus.

The present embodiment shown below exemplifies an apparatus or method for embodying the technical idea (structure, arrangement) of the invention, and the technical idea of the present invention is specified as follows. It's not something. The technical idea of the present invention may be modified in various ways within the scope of the matters described in the claims. It should also be noted that the drawings are schematic and the configurations of the devices and systems are different from the actual ones.

The multi-line image sensor device of the present embodiment will be described, for example, as being built in a photographing device (also referred to as an artificial satellite camera) mounted on an ultra-small artificial satellite.

Then, this multi-line image sensor device can clearly distinguish a moving object (including a vehicle, a ship, an aircraft, a cloud, a fluid such as a tsunami), which is a shooting target (subject), from a stationary object at the time of image processing. It is a line array.

Further, in the following description, the ground center obtains a moving body detection image in which the color display of the moving body is emphasized by superimposing the spectrum of the color images of a plurality of color types (each band) acquired by this multi-line image sensor device. A mobile detection system further equipped with a mobile detection device (also referred to as a ground station) will be described as an example.

In addition, the case where the moving body exists (earth, paper, sky, sea, river, mountain, ...) is the earth, and the moving body is a moving vehicle (including a motorcycle, a motorcycle, etc.) or a tsunami will be described. ..

<Embodiment 1>
FIG. 1 is a schematic configuration diagram of a mobile body detection system using a multispectral sensor device (MSS) to which the multiline image sensor device according to the first embodiment is applied. Here, in the moving object detection system, at the time of one shooting (for one scene) created based on the line-by-line captured image eGi captured by the artificial satellite camera 21 mounted on the earth observation satellite (flying object) 12. For example, a case where a moving vehicle Tm is detected from a moving object detection image (single image) CP will be described as an example. Further, the flying object is an airplane (including a helicopter), a drone, an airship, an artificial satellite, or the like, but in the embodiment, it will be described as an artificial satellite.

As shown in FIG. 1, this moving object detection system includes an artificial satellite (hereinafter referred to as earth observation satellite 12) that observes the earth 10 while moving in one direction (hereinafter referred to as SD direction) in the orbit of the earth 10. It is composed of a moving body detection device 16 that exchanges various data, signals, information, etc. between the earth observation satellite 12 and the ground center via the antenna facility 14. The mobile body detection device 16 is a server system provided in the ground center.

The earth observation satellite 12 is a so-called platform on which an artificial satellite camera 21 (also referred to as a photographing device) is mounted, and orbits in an orbit at a predetermined height while moving in the direction of the arrow SD shown in the figure at a predetermined speed. It is configured as follows.

That is, the earth observation satellite 12 is preferably an ultra-small artificial satellite that flies at an altitude of about 400 km (kilometers) to about 700 km of the earth 10 at a substantially constant speed according to the altitude.

Then, as the arrow moves in the SD direction, the on-board artificial satellite camera 21 continuously captures an image of the surface of the earth 10 with a width of about 2.5 m (meter) × 50 km.

In the artificial satellite camera 21 described above, a multispectral sensor device 214 for line scanning in an arbitrary shooting area TEG by, for example, a push bloom method is built-in.

Here, the movement (attitudes ω, κ, θ) of the earth observation satellite 12 is controlled so that the angle of the imaging surface (horizontal direction) of the artificial satellite camera 21 with respect to the ground surface direction (vertical direction) is always 90 degrees. NS. Further, in the earth observation satellite 12, the artificial satellite camera 21 is adjusted so that the angle of the built-in multispectral sensor device 214 with the SD direction in which the satellite 12 flies is approximately 90 degrees (approximately right angle). That is, the multispectral sensor device 214 is arranged so that the horizontal direction (line arrangement) of the line sensor, which will be described later, is substantially orthogonal to the SD direction.

Then, the earth observation satellite 12 transmits the transmission signal SGi including the line-by-line captured image eGi captured by the multispectral sensor device 214 toward the antenna facility 14 connected to the moving object detection device 16 at the ground center. Here, the transmission signal SGi described above is mainly composed of, for example, a line-by-line captured image eGi and header information EHi.

The line-by-line captured image eGi is a frame signal including the gradation (corresponding to a pixel value) of each band constituting the pixel. For example, the multi-spectral sensor device 214 periodically or continuously for each line during imaging. It is a sensor output (signal charge for one line scan) accompanied by pixel data.

The header information EHi is, for example, meta data at the time of shooting added to the shot image eGi for each line by a C & DH (Command and Data Handling) system or the like. The header information EHi includes, for example, information (ID) for identifying the earth observation satellite 12, information such as latitude, longitude, altitude, and attitude at the time of shooting of the earth observation satellite 12, spatial resolution, shooting date and shooting time, and so on. Information on shooting start time (Collection Start Time) and end time (Collection End Time), information on band type (color type), pixel coordinates Pzai (i, j) indicating pixel positions, destination information, etc. are included. ..

Further, it is desirable that the header information EHi is added in units of the captured image eGi for each line in the transmission unit (27) described later.

The earth observation satellite 12 may have various other functions including basic functions as an artificial satellite (for example, C & DH system, propulsion system, etc.), and detailed description thereof is omitted here. do.

FIG. 2 is a schematic configuration diagram of the vicinity of the artificial satellite camera 21 inside the earth observation satellite 12.

As shown in FIG. 2, the earth observation satellite 12 controls the artificial satellite camera 21, the receiving unit 23 that receives an imaging command or the like sent from the moving object detection device 16 via the antenna facility 14, and the artificial satellite camera 21. It is configured to include at least a sensor control unit 25 and a transmission unit 27 that transmits a transmission signal SGi including a line-by-line captured image eGi to the moving object detection device 16.

At least one of the sensor control unit 25 and the transmission unit 27 may be provided in the artificial satellite camera 21.

As shown in FIG. 2, the artificial satellite camera 21 includes, for example, a multispectral sensor device 214, a sensor drive unit 210 that drives the multispectral sensor device 214 under the control of the sensor control unit 25, and an optical system unit (optical). A system) 212 and a transfer unit (HCCD) 218 are provided.

The sensor drive unit 210 is, for example, a drive pulse (exposure timing signal) at intervals of time when the earth observation satellite 12 travels a distance of one line scan to four types of image line sensors described later in the multispectral sensor device 214. ) Is sent to expose the light from the optical system unit 212.

The optical system unit 212 includes a condensing lens (not shown) that condenses light on the multispectral sensor device 214.

The transfer unit 218 is, for example, the signal charge for each one line scan acquired by the photoelectric conversion action in each photodiode (photoelectric conversion element) PD group in the four types of image line sensors in the multispectral sensor device 214. Is obtained and transferred as a first image (R ₁ ) for each line, a second image (G) for each line, and a third image (B) for each line, and a first line sensor for the same wavelength component image (hereinafter, described below) described later. , The sensor output from the line sensor for the same wavelength component image (for example, 214R ₂ ) is obtained as an image (for example, R ₂ (hereinafter, referred to as the same wavelength component image Ri (R ₂ ) for each line)), and sequentially, Transfer to the transmission unit 27.

The first image (R ₁ ) for each line, the second image (G) for each line, the third image (B) for each line, and the same wavelength component image Ri (R ₂ ) for each line are collectively taken as an image taken for each line. It is called eGi.

(Configuration of Multispectral Sensor Device 214)
FIG. 3 is a schematic configuration diagram of the multispectral sensor device 214. The multispectral sensor device 214 built into the artificial satellite camera 21 includes _{any one of the three primary color image line sensors (R 1} image line sensor 214R ₁ , G image line sensor 214G, B image line sensor 214B). _{The line sensor 214R 2} for the same wavelength component image that detects the wavelength of light of the same color as the line sensor is provided. The three primary color image line sensors and the same wavelength component image line sensor 214R ₂ are collectively referred to as an image line sensor.

That is, the multispectral sensor device 214 of the present embodiment further includes four types of image line sensors including the same wavelength component image line sensor 214R _{2 in addition to the three primary color image line sensors (214R 1, 214G, 214B).} It is a so-called 4-line type multi-line image sensor.

Further, each image line sensor (214R ₁ , 214G, 214B, 214R ₂ ) of the multispectral sensor device 214 has a predetermined interval (physical separation distance BD: for example, 10 μm (micrometer)) BD. They are arranged in parallel.

Further, the same wavelength component image line sensor 214R _{2 is provided for any one of the R 1} image line sensor 214R ₁ , the G image line sensor 214G, and the B image line sensor 214B constituting the three primary color image line sensors. However, in the present embodiment, it is an example provided _{for the R 1} image line sensor 214R _1.

However, the multispectral sensor device 214 of the interline type structure, each of the image line sensor _{(214R 2, 214R 1, 214G} , 214B) of the unit by each pair of pixels at the same position in the line direction cell (UC1, UC2) is configured.

The multispectral sensor device 214 having such a configuration has a function as a multiline image sensor for detecting a moving object.

In FIG. 3, the multispectral sensor device 214 includes a fixed number of unit cells UC1 and UC2 arranged in a line. Each unit cell UC1 is composed of three pixels, for example, three primary color image line sensors (214R ₁ , 214G, 214B), and each unit cell UC2 has two pixels, for example, line sensors 214R ₂ , R _{1 for the same wavelength component image.} It is composed of an image line sensor 214R _1.

Here, the line sensor 214R ₂ for the same wavelength component image and the three primary color image line sensors (214R ₁ , 214G, 214B) are the photodiode PD group 214R ₂ p, 214R ₁ p, 214Gp, 214Bp, and the CCD group 214R _{2, respectively.} It has f, 214R ₁ f, 214Gf, 214Bf, and the like.

The photodiode PD group 214R ₂ p, 214R ₁ p, 214 Gp, 214 Bp consist of a fixed number of photodiode PDs arranged in series, and the CCD group 214R ₂ f, 214R ₁ f, 214 Gf, 214 Bf are each photodiode PD. It consists of a fixed number of vertical CCDs (VCCDs) arranged in parallel with each other.

Each of the vertical CCDs of the CCD group 214R ₂ f, 214R ₁ f, 214Gf, and 214Bf functions as an analog frame memory.

That is, a fixed number of unit cells UC1 and UC2 are configured by each pixel set of each image line sensor of the same wavelength component image line sensor 214R ₂ and the three primary color image line sensors (214R _{1, 214G, 214B).} ..

More specifically, each unit cell UC1 is composed of the photodiode PD groups 214R ₁ p, 214 Gp, 214 Bp and the CCD group 214R ₁ f, 214 Gf, 214 Bf of the three primary color _{image line sensors (214R 1, 214 G, 214 B).} Will be done. Each unit cell UC2 is composed of the photodiode PD group 214R ₂ p, 214R ₁ p and the CCD group 214R ₂ f, 214R ₁ f of the same wavelength component image line sensor 214R ₂ and R ₁ image line sensor 214R _1. Will be done.

Each unit cell UC1 is a cell for acquiring pixel data for forming a line-by-line color composite image aGEi in one line, and each unit cell UC2 is a pixel for forming a line-by-line black-and-white composite image gEAi in one line. This is a cell for acquiring data.

On the other hand, an image for the transfer unit 218R ₂ in the above-mentioned CCD group 214R ₂ f, an image for transfer portion 218R ₁ to CCD group 214R ₁ f is an image for the transfer unit 218G in CCD group 214Gf, the CCD group 214Bf the image for the transfer unit 218B is respectively connected, CCD group _{214R 2 f, 214R 1 f,} 214Gf, the sensor output from 214Bf adapted to be transferred to the transmitter 27.

In the first embodiment, the image transfer unit 218R ₂ , 218R ₁ , 218G, and 218B constitute the transfer unit 218 shown in FIG.

That is, the same wavelength component image line sensor 214R _2, the three primary colors image line sensor _(214R 1, 214G, 214B) in each photodiode PD group _{214R 2 p, 214R 1 p,} 214Gp, by photoelectric the first exposure by 214Bp _{After the converted signal charge is simultaneously read out to the CCD groups 214R 2} f, 214R ₁ f, 214 Gf, and 214 Bf via the transfer gate (read gate) TG, each image transfer unit of the transfer unit 218 is used. It is swept to the transmission unit 27 via 218R ₂ , 218R _{1, 218G, and 218B.}

At the time of this sweep, for example, the signal charge per line scan becomes the line-by-line captured image eGi corresponding to one line, and the pixel coordinates corresponding to the position of the corresponding photodiode PD are added as the header information EHi. Therefore, it becomes a transmission signal SGi.

That is, for each line, the first image (R ₁ ) for each line, the second image (G) for each line, the third image (B) for each line, and the same wavelength component image Ri (R ₂ ) for each line. Will be transmitted as a captured image eGi for each line.

The transfer unit 218 may be provided with a function of converting / amplifying the signal charge into a voltage when the signal charge is transferred. Further, the transfer unit 218 may _{have a horizontal CCD (HCCD) configuration in which signal charges from each CCD group 214R 2} f, 214R ₁ f, 214Gf, and 214Bf are sequentially transferred horizontally (so-called progressive reading method). ..

By using the multi-spectral sensor device 214 having such a configuration, for example, while playing a role as an existing three primary color image line sensor capable of capturing an RGB color (composite) image, only the change of the moving body is immediately detected. It becomes possible to easily create a moving object detection image CP in which a stationary object is displayed in color and other stationary objects are displayed in monochrome (monotone or grayscale).

However, if it is limited to simply displaying only the change in the moving object in color, a monochrome line sensor (not shown) in which sensors with the same sensitivity are arranged for two or more lines is added, and the image is combined after processing to make it stationary. It is possible to display the object in monochrome and to display the change in the moving body in color.

_{Here, the line sensor 214R 2} for the same wavelength component image and the three primary color image line sensors (214R ₁ , 214G, 214B) each have a sensor length SL and a sensor width SW, and each has a physical separation distance BD. It has and is arranged in parallel.

_{The line sensor 214R 2} for the same wavelength component image, the sensor length SL of the three primary color image line sensors (214R ₁ , 214G, 214B), the sensor width SW, the physical separation distance BD, etc. are the earth on which the artificial satellite camera 21 is mounted. It differs for each observation satellite 12, and is appropriately designed according to the altitude, speed, resolution, spatial resolution, imaging target (moving object), etc. of the earth observation satellite 12, and is, for example, about several μm to several ten and several μm. Become.

Therefore, in the case of this multispectral sensor device 214, it is caused by the physical separation distance BD between the _{line sensor 214R 2} for the same wavelength component image and the three primary color image line sensors (214R _{1, 214G, 214B) in one exposure.} Therefore, the images at the same point (on the same line) are taken with a slight time lag (exposure timing difference).

The exposure timing difference due to the physical separation distance BD between the _{line sensor 214R 2} for the same wavelength component image and the three primary color image line sensors (214R _{1, 214G, 214B) at the time of shooting is a color shift on the moving object detection image CP.} Appears as. In particular, the color shift of the change of the moving body on the moving body detection image CP corresponds to the moving speed and the moving direction of the moving body, and becomes larger as the moving body moves faster.

Therefore, in the moving body detection device 16, in the moving body detection image CP, the change in the moving body can be displayed as a colored image by utilizing this color shift, and only the colored image can be further emphasized and expressed. I have to. This makes it possible to make the change in the moving body even more conspicuous on the moving body detection image CP. Therefore, the moving body can be easily detected visually from the moving body detection image CP for one scene, and the change amount of the moving body can be efficiently and automatically extracted even in the computer processing.

FIG. 4 schematically shows the cross-sectional structure of the multispectral sensor device 214, and here, the cross-section of the B image line sensor 214B along the I-I line of FIG. 3 will be illustrated and described.

In the B image line sensor 214B, for example, the light receiving region (n type) of the photodiode PD which is the photodiode PD group 214Bp formed on the surface portion of the p-type silicon (Si) substrate 230 or the surface portion of the p-type well 232. ^{The register portion (n +} type layer) 214Bt of the vertical CCD which becomes the CCD group 214Bf formed on the surface portion of the p-type Si substrate 230 or the surface portion of the p-type well 232 separated from the layer) 234 and the light receiving region 234. And.

Further, the B image line sensor 214B has a poly-Si electrode 238 and a poly-Si electrode 238 provided on the p-type Si substrate 230 or the p-type well 232 via an insulating film 236, excluding the light receiving region 234. The light-shielding film 240 is provided via an insulating film 236 and has an opening in a part of the light receiving region 234.

Further, the B image line sensor 214B has a transparent resin layer 242 provided on the entire surface and a B image color filter (primary color filter) provided on-chip on the upper surface of the resin layer 242 corresponding to the light receiving region 234. ) 244 and a microlens 246 provided on-chip on the resin layer 242 corresponding to the B image color filter 244.

That is, the B image line sensor 214B is configured to include, for example, a B image color filter 244 that transmits light in a band having a wavelength of 450 to 495 nm.

Incidentally, the same wavelength component image line sensor 214R _{2, R 1} image line sensor 214R _1, and G image line sensor 214G are both for the line sensor 214B and the basic configuration for the B image are the same, wherein The detailed description of is omitted.

In short, the same wavelength component image line sensor 214R ₂ and _{R 1} image line sensor 214R _1, as described later, for example, a color for R image filter (primary filter) which transmits light of a band of wavelengths 620~750nm It is configured to prepare.

Further, as will be described later, the G image line sensor 214G is configured to include, for example, a G image color filter (primary color filter) that transmits light in a wavelength band of 495 to 590 nm.

The wavelength band described above is an example, and the half-value width and the like may be different for each of _{the same wavelength component image line sensor 214R 2} and the three primary color image line sensors (214R _{1, 214G, 214B).} Further, in the case of a line sensor (not shown) that acquires a monochrome image independently, a color filter is not required.

FIG. 5 shows a timing chart at the time of shooting as a basic operation of the artificial satellite camera 21 mounted on the earth observation satellite 12.

That is, the earth observation satellite 12 equipped with the artificial satellite camera 21 incorporating the multi-spectral sensor device 214 as described above moves in the SD direction in the orbit, and is arbitrarily moved by the artificial satellite camera 21 on the ground surface portion of the earth 10. Shooting area The inside of the TEG is photographed in a line.

In the artificial satellite camera 21, for example, in the line sensor 214R ₂ for the same wavelength component image and the three primary color image line sensors (214R ₁ , 214G, 214B), the drive pulse of the photodiode PD from the sensor drive unit 210 (FIG. 5A). during the reference) is high, the photodiode PD group _{214R 2 p, 214R 1 p,} 214Gp, exposure by 214Bp performed.

The exposure time is generally the time that the earth observation satellite 12 travels for one line scan (for example, if the satellite has a spatial resolution of 1 meter, the time required for the satellite to fly a distance of 1 meter). Is set to.

When the exposure for a certain period is completed (the drive pulse of the photodiode PD becomes low level), the control pulse of the transfer gate TG from the sensor drive unit 210 (see FIG. 5B) becomes high level. The signal charges accumulated in the photodiode PD groups 214R ₂ p, 214R ₁ p, 214 Gp, and 214 Bp are _{transferred to the CCD groups 214R 2} f, 214R ₁ f, 214 Gf, and 214 Bf at almost the same time.

CCD group _{214R 2 f, 214R 1 f,} 214Gf, the signal charges transferred to 214Bf, respectively (see FIG. 5 (c)) the control pulses BP from the sensor driving unit 210, GP (FIG. 5 (d) see ), R ₁ P (see FIG. 5 (e)), R ₂ P (see FIG. 5 (f)) at high levels, respectively, image transfer units 218R ₂ , 218R ₁ , 218G, 218B of the transfer unit 218. Transferred to.

When the signal charges of all the pixels in one exposure are output to the transfer unit 218, the drive pulse (PD) for the next exposure is transmitted from the sensor drive unit 210 to the line sensor 214R ₂ for the same wavelength component image. It is output to the three primary color image line sensors (214R ₁ , 214G, 214B).

In this way, in one exposure, the signal charge transferred to the transfer unit 218 is regenerated as a transmission signal SGi including the captured image eGi for each line by the transmission unit 27, and then moves toward the antenna facility 14. It is transmitted to the body detection device 16.

The timing chart of FIG. 5 illustrates an example of the operation, and in the actual operation, _{the physical operation between the line sensor 214R 2} for the same wavelength component image and the three primary color image line sensors (214R ₁ , 214G, 214B) is performed. The timing of the start of exposure may be specified in consideration of the difference in exposure timing due to the separation distance BD.

On the other hand, the mobile detection device 16 is a computer system (server system). Although details will be described later, the moving object detection device 16 accumulates Earth Observation Satellite 12 lines each captured image included in the transmission signal SGi transmitted from _{eGi (R 1, G, B} , Ri (R 2)) the Then, the input extraction area is set as the specific area EWi.

Then, the number of captured images eGi for each line corresponding to the specific area EWi is extracted. The extracted line-by-line captured image eGi line-by-line first image (R ₁ ), line-by-line second image (G), and line-by-line third image (B) are combined with a color composite image (multispectral image) of the specific area EWi. Also referred to as GWi). _{Further, the same wavelength component image Ri (R 2} ) for each line of the extracted line-by-line captured image eGi _{and the first image (R 1} ) for each line constituting the color composite image GWi of the specific area EWi are superimposed. A black-and-white composite image (also referred to as a panchromatic image) GEAi corresponding to the specific area EWi is generated. Then, the color composite image GWi is superposed on the black-and-white composite image GEAi, and the amount of change (change) according to the movement speed of the moving body can be immediately recognized as a color shift image (for example, pan sharpening). Image) It is configured to create CP and so on.

Here, the multispectral image is a color image of each band obtained for each wavelength band (R _{1, G,} B) are superposed, a color composite image of so-called full.

A panchromatic image is a so-called monochrome image (panchromatic image) in which only _{a color image (for example, R 1} , Ri (R ₂ )) of a specific wavelength band is observed as a specified image type (for example, a black and white image). Also called).

The pan-sharpened image is an image obtained by combining a multispectral image and a panchromatic image (pan-sharpened processing).

Color composite image GWi specific area EWi described above, for example, the region corresponding to the imaging area TEG, geometrically overlap so aligned (e.g., mosaicking) color image (R ₁ of each band obtained by, It is formed by superimposing G and B) with the same size.

Similarly, the black-and-white composite image GEAi of the specific area EWi described above has at least the same color band obtained by aligning (for example, mosaic processing) the regions corresponding to the shooting area TEG so as to geometrically overlap each other. It is formed by superimposing color images (for example, R ₁ , Ri (R ₂ )) having the same size.

Further, the moving body detection device 16 detects a moving body from the moving body detection image CP for one scene subjected to color enhancement processing, which will be described later, and when the moving amount, moving direction, moving speed, number, or moving body is a vehicle. Is designed to calculate moving object information such as the distance between vehicles. _{In the moving body detecting device 16, for example, when a color shift of R 1} , G, and B over a predetermined number of consecutive pixels (for example, several pixels to a dozen or so pixels) is detected, the moving body detecting device 16 is determined to be a moving body. .. Further, the moving speed of the moving body is calculated from the degree of the color shift, and the moving direction of the moving body is calculated from the direction of the color shift.

The processing in the moving body detection device 16 will be described below.

FIG. 6 shows a schematic configuration (functional block) of the mobile body detection device 16.

The mobile body detection device 16 is mainly composed of, for example, a general-purpose computer system. More specifically, the acquisition unit 161, the captured image database (also referred to as the captured image reception storage unit) 163, the area extraction unit 165, the operation unit 167, the extracted image memory 169, the overlay unit 171 and the color-enhanced image. Memory (also referred to as black-and-white composite image storage unit or pan-sharpened image storage unit) 173, color enhancement unit 175, pixel moving body determination unit 177, display control unit 181 having image memory 181M, display unit 183, and movement. It is configured to include a body information calculation unit 185.

The above-mentioned extracted image memory 169 includes a color image memory 169a and a panchromatic image generation memory (same component image memory) 169b. The captured image database 163 includes an acquired image database (color composite image storage unit) 163a and a header information database 163b.

The acquisition unit 161 captures the transmission signal SGi from the earth observation satellite 12 received at the antenna facility 14. Then, each contained in the taken transmitted signal SGi line captured image _eGi to (R 1, G, B, Ri (R 2)), for example, line delay correction, plane rectangular coordinate transformation, or orthorectified etc. After the above processing, the mosaic processing is performed and the images are sequentially stored in the acquired image database 163a. The acquired image database 163a may include, for example, only the components of the captured image eGi for each line at a predetermined level or higher.

At this time, each of the same wavelength component image Ri (R ₂ ) for each line, the first image (R ₁ ) _for each line, the second image (G) for each line, and the third image (B) for each line is classified by color type. I remember them one by one. For example, the first image (R ₁ ) _for each line, the second image (G) for each line, and the third image (B) for each line are stored in the color composite image area (not shown) in the acquired image database 163a. .. Further, the same wavelength component image Ri (R ₂ ) for each line is stored in the same wavelength component image region (not shown) in the acquired image database 163a.

Further, the header information EHi included in the transmission signal SGi is associated with the captured image eGi for each line and stored in the header information database 163b.

The area extraction unit 165 may, for example, include the same wavelength component image Ri (R ₂ ) for each line, the first image (R ₁ ) _for each line, the second image (G) for each line, and the third image for each line (G) over the shooting area TEG. _{After B) is stored, the first image (R 1} ) _for each line, the second image (G) for each line, and the third image for each line corresponding to the arbitrary specific area EWi specified by the operation unit 167. The image (B) is read from the acquired image database 163a. The above-mentioned specific area EWi may be the same as the shooting area TEG, but if it is detected in real time, the specific area EWi can be set to, for example, 10 m, 30 m, 100 m, and so on. Is preferable.

Then, the number of lines corresponding to the specific area EWi, the first image (R ₁ ) _for each line, the second image (G) for each line, and the third image (B) for each line are combined with the color composite image GWi (R _1, G, B). : Multispectral image) is stored in the color image memory 169a, and the corresponding header information EHi in the header information database 163b is read and stored in the color image memory 169a in association with the color composite image GWi.

Further, the area extraction unit 165 reads the same wavelength component image Ri (R ₂ ) for each number of lines corresponding to the above-mentioned specific area EWi from the acquisition image database 163a and stores it in the panchromatic image generation memory 169b. At this time, the corresponding header information EHi is also stored.

Further, the area extraction unit 165 reads, for example, the color composite image GWi of the specific area EWi from the color image memory 169a, and displays the color-enhanced image memory (pan-sharpened image storage unit) 173. It may be stored in the image memory 181M of the control unit 181 and displayed on the screen of the display unit 183.

The operation unit 167 has an operation panel 167a and a display panel 167b, for example, as shown in FIG.

The operation unit 167 allows the operator to display the color composite image GWi (R1 _, G, B: multispectral image) or the map over the shooting area TEG in advance on the screen of the display unit 183, for example, on the operation panel 167a. Is operated to specify the specific area EWi. Further, various information may be displayed on the screen of the display panel 167b so that the operator can select or input the information.

Information for designating the specific area EWI includes, for example, plane rectangular coordinates, latitude, longitude, area name and address (area name) to be photographed, shooting date, shooting time (shooting start time and end time), and the like. Be done.

The display panel 167b of the operation unit 167 can also be used as the display unit 183. That is, the operation unit 167 may be configured to operate the operation panel 167a while looking at the screen display of the display unit 183.

As shown in FIG. 6, for example, the superimposing unit 171 reads the same wavelength component image Ri (R ₂ ) for each line corresponding to the specific area EWi from the operation unit 167 from the panchromatic image generation memory 169b, and colors the overlay unit 171. The first image (R ₁ ) for each corresponding number of lines is read from the image memory 169a and superimposed.

Then, the superimposed image is stored in the color-enhanced image memory 173 as a panchromatic image (also referred to as a black-and-white composite image GEAi).

Then, the black-and-white composite image GEAi and the color composite image GWi of the color image memory 169a are superposed, and this is used as a moving object detection image before color enhancement (hereinafter referred to as pan sharpened image CP1 before color enhancement). It is stored in the pan-sharpened image storage unit (not shown) of the emphasized image memory 173, and the display control unit 181 is activated.

The display control unit 181 uses, for example, the pan-sharpened image CP1 before color enhancement stored in the color-enhanced image memory 173 or the pan-sharpened image CP2 after color-enhanced, which will be described later, as a moving body detection image CP. It is read into the memory 181M and displayed on the screen of the display unit 183.

For example, when the pan-sharpened image CP2 after color enhancement is displayed as the moving object detection image CP, the color enhancement unit 175 changes the amount of change according to the speed of the moving object with respect to the pan-sharpened image CP1 before color enhancement. As a result, it is intended to emphasize the color expressiveness.

Details will be described later, but for example, in the pan-sharpened image CP1 before color enhancement, as a result of determination by the pixel moving body determination unit 177, any of the pixel values indicating the gradation degree of each pixel color type is "0". If not, the non- "0" pixel value on the mesh Mi coordinates Mi (i, j) in the color-enhanced image memory 173 is changed to the maximum value.

Then, the pan-sharpened image (hereinafter, also referred to as a color-enhanced image) CP2 whose pixel value is changed to the maximum value is stored in the image memory 181M of the display control unit 181 as a moving object detection image CP. It is displayed on the screen of the display unit 183.

Here, as shown in FIG. 8, for example, the moving object detection image CP is a color image (R ₁ , G, B or R 1, G, B or B, G, is highlighted by R _1), else, becomes the background, including vehicles Ts and roads or buildings stopped (stationary object) is displayed as a monochrome image.

That is, at the time of shooting _{, among the detected components of the acquired R 1} image line sensor 214R ₁ , G image line sensor 214G, and B image line sensor 214B, only the moving body has a color corresponding to the change corresponding to the speed. It will be displayed as a color image with a shift. However, in FIG. 8, the color image is emphasized and shown on the vehicle Tm.

Note that FIG. 8 illustrates a moving object detection image CP when the vicinity of an interchange (IC) on an expressway is photographed, and highlights the vehicle Tm that is actually traveling.

In FIG. 6, the pixel moving body determination unit 177 determines, for example, whether or not each pixel of the pan-sharpened image CP1 before color enhancement belongs to a moving body based on the pixel value (pixel value). Details will be described later, but for example, when any of the pixel values indicating the gradation degree of each pixel color type is not "0", it is determined that the pixel belongs to a moving body. On the contrary, when all the pixel values indicating the gradation degree of each pixel color type are "0 (or maximum value)", it is determined that the pixel belongs to a stationary object.

The moving body information calculation unit 185 detects the moving body in the moving body detection image CP based on the moving body detection image CP stored in the color-enhanced image memory 173 or the image memory 181M, and the moving body thereof, for example. It calculates information.

That is, by the mobile information calculating section 185, R _1, G of the color image in a mobile detection image CP, from the degree of unlike and color shift of the sequence of B, for example, the moving speed of the vehicle Tm traveling, moving direction, The distance between vehicles or the number of vehicles is calculated as necessary. The calculation result of the moving body information calculation unit 185 is displayed on the screen of the display unit 183, for example, as a figure such as a numerical value, a character, a color, an arrow having a different thickness or a length, and superimposed on the moving body detection image CP. You may.

The color-enhanced image (CP2) is not limited to being displayed as it is as a moving object detection image CP, and may be combined with, for example, an actual map or existing road network data (R value).

Here, an example of a method of calculating the moving speed v (t) of the moving body will be described.

The moving distance ds of the moving body can be calculated from the amount of deviation of the moving body (number of pixels × spatial resolution). For example, _{assume that the time lag dt between the line sensors 214R 2} , 214R ₁ , 214G, and 214B for each image is 0.1 second, and the spatial resolution is 1 m. When the moving body is displaced by 1 pixel (moving about 10 m per second), the speed v (t) of the moving body is about 36 km / h according to the following equation (1).

v (t) = ds / dt ... (1)
However, v is the speed of the moving body, s is the position of the moving body, and t is the time (the shooting time of the captured image eGi for each line).

As the shooting time of the shot image eGi for each line, for example, the shooting start time and the shooting end time recorded in a metadata file (not shown) are used.

Incidentally, the velocity (Vkm / sec) of the artificial satellite differs depending on the altitude from the ground surface, and the velocity V of the artificial satellite moving in a circular orbit over Hkm from the ground surface can be obtained from the following equation (2).

V = (398600 / (6378 + H)) ^1/2 ... (2)
However, 398600 (km ³ / sec ² ) is the definition of the earth's gravity, and 6378 (km) is the radius of the earth's equatorial line.

Therefore, for example, the velocity V of an artificial satellite moving in a circular orbit 600 km above the surface of the earth is about 7.56 km / sec.

Further, the moving body information calculation unit 185 may estimate not only the speed of the moving body but also the CO ₂ emission amount (transportation weight × mileage × CO ₂ emission basic unit).

(Operation explanation)
Next, the operation of the mobile body detection device 16 of the mobile body detection system having the above configuration will be described.

First, the process of the area extraction unit 165 will be described with reference to the flowchart of FIG. It will be described that the captured image database 163 sequentially stores the captured image eGi for each line scan corresponding to the captured area TEG, which is captured by the artificial satellite camera 21 of the earth observation satellite 12.

In the captured image database 163, for example, as shown in FIG. 11A, the same wavelength component image Ri (R ₂ ) for each line and the first line for each line, in which an X width of 2.5 m × a Y width of 50 km is set as one line. The specific area EWi associated with at least the header information EHi such as the shooting date and time of the line-by-line shot image eGi consisting of the image (R _{1), the line-by-line second image (G), and the line-by-line third image (B).} It will be explained assuming that the number of images passed to is stored.

The area extraction unit 165 reads the latitude, longitude or time corresponding to the specific area EWi designated by the operation unit 167 (S1).

Next, the area extraction unit 165 secures, for example, a planar (two-dimensional) area WRi corresponding to the specific area EWi in the color-enhanced image memory 173 (S2). As the area WRi, for example, as shown in FIG. 11 (b), one mesh has a definable size of mesh Mi at intervals of 2.5 m.

Then, in the specific area EWi, the same wavelength component image Ri (R ₂ ) for each line of the specified image type and the extracted color composite image GWi are read from the captured image database 163 and stored in the extracted image memory 169 (. S3).

The above-mentioned extracted image memory 169 includes a color image memory 169a and a panchromatic image generation memory 169b, and the extracted color composite image GWi is stored in the color image memory 169a, and the same wavelength component image Ri (for each line) R ₂₎ is stored in the panchromatic image generation memory 169b.

_{After the first image (R 1} ) for each line and the same wavelength component image Ri (R ₂ ) for each line of the corresponding number of color composite images GWi stored in the memory for extracted images 169 are combined by the superimposing unit 171. Further, by superimposing on the color composite image GWi, it is stored in the color-enhanced image memory 173 as the pan-sharpened image CP1 before color enhancement.

Alternatively, the pan-sharpened image CP1 before color enhancement may be written in the image memory 181M in the display control unit 181 and displayed on the screen of the display unit 183.

In the following description, the image type is designated as a black-and-white image, and the same number of same-wavelength component images Ri (R ₂ ) corresponding to the specific area EWi and the same number of line-by-line first images (R ₁ ) are superimposed in black and white. The composite image GEAi is stored in the color-enhanced image memory 173, and the pan-sharpened image CP1 before color enhancement in which the black-and-white composite image GEAi and the color composite image GWi of the color image memory 169a are superimposed is the color. It will be described as being stored in the pan-sharpened image storage unit of the emphasized image memory 173.

Next, the moving object detection process will be described with reference to the flowchart of FIG. The moving body detection process is a process performed by the color enhancing unit 175 and the pixel moving body determination unit 177 of the moving body detecting device 16.

As shown in FIG. 10, first, the color enhancement unit 175 specifies the pixel coordinates Pzai (i, j) of the pan-sharpened image CP1 before color enhancement stored in the color enhancement image memory 173, for example (. S05).

Then, the pixel data of the designated pixel coordinates Pzai (i, j) ( _{components of R 2,} R ₁ , G, and B in one pixel: each pixel value) is read (S06). Then, it is determined whether or not all the pixel values of the read pixel data are the same (S07).

In step S07, when all the pixel values are not the same (R ₁ ≠ G ≠ B), it is determined that the image data is the color composite image GWi. That is, when all the pixel values are not equal, it is determined that the pixel belongs to a moving body (for example, Tm2).

Further, for example, as shown in FIG. 18A, when all the pixel values are equal, the pixel is a black-and-white image of the background (for example, if R = G = B = 108, black PK, R = G = If B = 200, it is determined to be white PW) (however, in the case of 256 gradations).

When it is determined in step S07 that the pixel data is the pixel data of the color composite image GWi (NO), the color type of the pixel corresponding to the designated pixel coordinates Pzai (i, j) is determined (S08).

For example, when each pixel value is R ₁ = R ₂ = G ≠ B (however, B> R ₁ = R ₂ = G), it is determined as “blue”. If each pixel value is R ₁ = R ₂ = B ≠ G (however, G> R ₁ = R ₂ = B), it is determined to be “green”. Further, when each pixel value is B = G ≠ (R ₁ = R ₂ ) (however, R ₁ = R ₂ > G = B), it is determined as “red”. As an example, in the case of 256 gradations, for example, as shown in FIG. 18A, if R = 178 and G = B = 129, then red (PR ₁ ), if G = 178, and R = B = 100. For example, if green (PG) and B = 178 and R = G = 102, it is determined to be blue (PB).

Then, the color type determined in step S08 is read, and this color type is emphasized (S10).

For example, when each pixel value is B = G <R, only the pixel value of R is maximized and red (PR ₁ ) is emphasized. When each pixel value is R = B <G, only the pixel value of G is maximized and green (PG) is emphasized. Further, when each pixel value is R = G <B, only the pixel value of B is maximized and blue (PB) is emphasized. The maximum value includes, for example, a range of 240 to 255 (in the case of 256 gradations). Further, for example, as shown in FIG. 18B, each pixel value other than the pixel value to be the maximum value may be set to "0", and the color type can be further emphasized.

That is, in the color enhancement unit 175, for example, as shown in FIG. 18B, if the highest gradation is red (PR ₁ ), R = 255 (G = B = 0), and green (PG). , G = 255 (R = B = 0) and blue (PB), then only the color type determined in step S08 approximates the maximum value so that B = 255 (R = G = 0). It is corrected (emphasized).

By the color enhancement processing as described above, the pan-sharpened image CP1 before color enhancement becomes the pan-sharpened image CP2 after color enhancement. For example, the moving body which moves at a predetermined speed or faster is, R _{1, G,} will be highlighted as a color image with a color shift B.

Next, the color enhancement unit 175 determines whether or not there is another pixel coordinate Pzai (i, j) (S12).

If it is determined in step S12 that there is another pixel coordinate Pzai (i, j) (YES), the pixel coordinate Psai (i, j) is updated and the process returns to step S06 (S13).

On the other hand, if it is determined in step S07 that the pixel values are substantially the same (R ₁ = G = B) (YES), it is determined that the pixels are monochrome images, and the process proceeds to step S12.

Further, when the pixel moving body determination unit 177 determines in step S12 that there is no other pixel coordinate Pzai (i, j) (NO), the pixel moving body determination unit 177 sequentially designates the pixel coordinate Pzai (i, j) (S14, S15). ), Step S06 ~ is repeated until the value of “j” reaches the max value (for example, max = 768).

In this way, when each pixel indicates, for example, "blue", "green", or "red", it is determined to be a color image indicating a moving object (see FIG. 11B).

When the value of "j" is the maximum value (S14: YES), the pan-sharpened image CP2 after color enhancement is written as the moving object detection image CP in the image memory 181M of the display control unit 181 (S16). ), Displayed on the screen of the display unit 183 (S17).

After that, in a state where the moving body detection image CP is displayed on the screen of the display unit 183, for example, the moving body information calculation unit 185 calculates the moving speed and the like as necessary.

FIG. 12 shows an example of a moving body detection image CP, taking a moving vehicle (moving body) as an example. Note that FIG. 12A is a pan-sharpened image CP1 before the color enhancement process is applied, and FIG. 12B is a pan-sharpened image CP2 after the color enhancement process is applied.

In the pan-sharpened image CP2 after color enhancement, for example, as shown in FIG. 12B, the background (stationary object) including a stopped vehicle, a road, a building, etc. is a monochrome image, and only the moving vehicle is used. _{tm1 (B, G, R 1} ) or _{Tm2 (R 1, G, B} ) while being displayed as a color image as in, than for color enhancement previous pan-sharpened image CP1 of FIG. 12 (a), The color image of the display is more emphasized.

The difference in the order of the color types (R ₁ , G, B or B, G, R ₁ ) in the color image depends on the difference in the moving directions of the moving vehicles Tm1 and Tm2.

Further, an arrow indicating the traveling direction (moving direction) may be displayed in the vicinity of the traveling vehicles Tm1 and Tm2, or a numerical value (speed) indicating the moving speed may be displayed.

Regarding the image composition process, after performing processing such as line delay correction in the overlay section 171, the color images of the bands obtained by aligning them so as to be geometrically overlapped are superimposed with the same size. Since it can be created by, detailed description here is omitted.

13 (a) and 13 (b) show the display characteristics of the moving body in the moving body detection image CP.

That is, in the case of a vehicle Tm traveling at a speed equal to or higher than a certain speed, as shown in FIG. even combined, movement between the pixels of the vehicle Tm is because the spanning ten pixels from several pixels in accordance with the moving speed, R _1, in the moving object detecting image CP G, the color shift of B increases.

On the other hand, in the case of the stopped vehicle Ts, as shown in FIG. 13B, for example, the color images of the bands almost overlap each other due to a simple line delay correction or the like, so that R _{1 in the moving object detection image CP} , G, B color shift becomes small. That is, the stopped vehicle Ts is displayed as a monochrome image in the same manner as the background (stationary object) including a road, a building, or the like.

FIG. 14 schematically shows an arrangement example (line arrangement) of the _{line sensors 214R 2} , 214R ₁ , 214G, and 214B for each image in the multispectral sensor device 214.

_{In the first embodiment, the R 1} image line sensor 214R ₁ for generating an RGB color composite image, the G image line sensor 214G, and the B image line sensor 214B, for example, the R ₁ image line sensor 214R A multi-spectral sensor device 214 in which _{line sensors 214R 2} for images having the same wavelength component _{are arranged in parallel so as to be adjacent to 1 is adopted.} That is, in the case of the multispectral sensor device 214, the R ₁ image line sensor 214R ₁ and the same wavelength component image line sensor 214R ₂ are monochrome image line sensors for acquiring a panchromatic image.

In the multispectral sensor device 214, the R ₁ image line sensor 214R ₁ and the same wavelength component image line sensor 214R ₂ are not adjacent to each other, and the R ₁ image line sensor 214R ₁ and the same wavelength component image line sensor are not adjacent to each other. The G image line sensor 214G and the B image line sensor 214B may be arranged between the _{214R 2 (for example, R 1} , G, B, R ₂ ).

Further, in any case, it is possible to replace _{the R 1} image line sensor 214R ₁ with the same wavelength component image line sensor 214R _2.

Regarding the multispectral sensor device 214II in which the G image line sensor 214G and the B image line sensor 214B are arranged between the R ₁ image line sensor 214R ₁ and the same wavelength component image line sensor 214R _2. , Will be described later as the second embodiment.

Here, a display example in which a moving body is displayed in color by using the multispectral sensor device 214 having the above configuration will be further described.

Figure 15 (a), (b) is shows a display example thereof with the vehicle Ts suspended, if the vehicle Ts suspended, R _1, G, color deviation of B in color display殆Ndo Does not occur.

16 (a) and 16 (b) show a vehicle Tma traveling at low speed and a display example thereof, and as the vehicle Tma traveling at low speed moves from the position shown by the solid line in the figure to the position shown by the broken line in the figure. color display of _R 1, G, slight color shift to B occurs.

17 (a) and 17 (b) show a vehicle Tmb traveling at high speed and a display example thereof, and as the vehicle Tmb traveling at high speed moves from the position of the solid line in the figure to the position of the broken line in the figure. , _R 1, G, a large color shift B occurs in a color display.

Therefore, in the moving object detection image CP, the moving vehicle Tm (Tma, Tmb) can be more easily identified by emphasizing the color display, and the vehicle is running according to the degree of color shift of the color display. It is possible to easily determine whether the vehicle Tm of the vehicle Tm is a vehicle Tmb traveling at high speed or a vehicle Tma traveling at low speed.

Further, _{since the arrangement of R 1} , G, and B in the color display depends on the moving direction of the running vehicle Tm and the line arrangement of the multispectral sensor device 214, which is the running vehicle Tm (Tma, Tmb)? It can automatically recognize whether it is moving in the direction of.

As described above, according to the first embodiment, the traveling vehicle can be detected more efficiently, and the detected speed and direction of the traveling vehicle can be accurately and automatically grasped.

That is, when acquiring the captured image eGi for each line using the multispectral sensor device having an interline type structure, two or more line sensors of the same band are arranged, and a slight difference in signal charge reading time between the lines. Is used to acquire moving object detection images for one scene.

As a result, the stationary object can be displayed in grayscale and the moving body can be displayed as a colored image on the moving body detection image, so that only the moving vehicle can be displayed in color on the moving body detection image. It becomes.

Moreover, since it is possible to emphasize the running vehicle and display it, the running vehicle can be reliably detected, and the detected speed, moving direction, number, etc. of the running vehicle can be detected. Can be easily calculated.

Therefore, a multi-line image sensor that can clearly distinguish a moving vehicle from a stationary object and efficiently detect only a moving vehicle without preparing a pair image such as a front view image or a direct view image as in the conventional case. Devices, imaging devices, moving body detection systems, moving body detection devices, and moving body detection programs can be provided.

In the first embodiment described above, an artificial satellite camera 21 having a built-in multispectral sensor device 214 is mounted on an artificial satellite such as the earth observation satellite 12, and the captured image eGi is captured for each line. As described above, the platform is not limited to artificial satellites, and may be, for example, an aircraft, an unmanned aircraft, a CCTV (Closed-Circuit television), a printer, a microscope, or the like.

Needless to say, the artificial satellite is not limited to the earth observation satellite 12.

Further, the multispectral sensor device 214 may have a configuration as shown in FIG. 19 or FIG.

FIG. 19 shows another exemplary configuration of the multi-spectral sensor device according to this embodiment, the multi-spectral as the sensor device 214 _1, For example, R ₁ image line sensor 214R _1, line for the same wavelength component image The sensor 214R ₂ , the G image line sensor 214G, and the B image line sensor 214B may each include a line sensor divided into three parts.

That is, the R ₁ image line sensor 214R ₁ is formed by the line sensors 214R _1a , 214R _1b , 214R _1c , and the same wavelength component image line sensor 214R ₂ is formed by the line sensors 214R _2a , 214R _2b , 214R _2c. , G image line sensor 214G is _{formed by line sensors 214G a} , 214G _b , 214G _c , and B image line sensor 214B is formed by line sensors 214B _a , 214B _b , 214B _c .

FIG. 20 shows still another configuration example of the multi-spectral sensor device according to the present embodiment, and the multi-spectral sensor device 214 ₂ is formed by, for example, line sensors 214R _1a , 214R _1b , and 214R _1c. R ₁ and an image line sensor, line sensor _{214R 2a,} 214R 2b, a line sensor for the same wavelength component image formed by 214R _2c, the line sensor 214G _a, 214G _b, the line sensor G image formed by 214G _c A line sensor for B image formed by the line sensors 214B _a , 214B _b , and 214B _{c may be provided.}

Further, the additional line sensor may be a _{G image line sensor or a B image line sensor regardless of the same wavelength component image line sensor 214R 2} , and two or more lines of the same type may be added. (For example, R ₁ , R ₂ , R ₃ ).

FIG. 21 is a first modification when the multispectral sensor device 214a is configured. That is, in the case of this multispectral sensor device 214a, for example, the B ₁ image line sensor 214B ₁ and the B ₂ image line sensor (same wavelength component image line sensor) 214B ₂ acquire a panchromatic image. It is a line sensor for monochrome images.

FIG. 22 is a second modification when the multispectral sensor device 214b is configured. That is, in the case of this multispectral sensor device 214b, for example, R ₁ image line sensor 214R ₁ and R ₂ image line sensor (first same wavelength component image line sensor) 214R ₂ and R ₃ image line sensor. (Second line sensor for same wavelength component image) 214R ₃ is a line sensor for monochrome image for acquiring a panchromatic image.

FIG. 23 is a third modification in the case where the line sensor 214W for infrared (IR) or near infrared (NIR) is provided to form the multispectral sensor device 214c. That is, in the case of this multispectral sensor device 214c, for example, the R ₁ image line sensor 214R ₁ and the R ₂ image line sensor (second line sensor for the same wavelength component image) 214R ₂ produce a panchromatic image. It is a line sensor for monochrome images for acquisition.

As described above, the multispectral sensor devices 214, 214a, 214b, and 214c are added to the existing CCD line sensor capable of acquiring color images for one line scan at one time at the same time, and further to any of R, G, and B. By newly adding sensors of the same band for at least one line, the moving body (moved during reading of the image) was created with the moving body detection image CP created by the moving body detection device 16. It will be possible to express the color shift of things) more clearly.

Further, the moving body is not limited to the vehicle, and the exposure time in the multispectral sensor device can be arbitrarily controlled from, for example, the moving body detecting device 16, thereby including a ship and an aircraft (Unmanned Aerial vehicle; UAV). ), Clouds, people, wildlife, or waves, which can detect moving objects moving at any speed.

FIG. 24 shows a case where cloud Ta as a moving body is detected from the moving body detection image (image image) CPa as an example.

When detecting cloud Ta, for example, it is possible to calculate the altitude, moving speed, growth speed or size of cloud Ta from the size of shadows and the like. In particular, by making it possible to calculate the growth speed and size of cloud Ta, it can be applied to the case of specifying a warning zone such as a thunderstorm or a torrential rain (guerrilla rainstorm) due to the occurrence of cumulonimbus clouds.

25 (a) to 25 (d) show an example of detecting a wave Tb as a moving body from a moving body detection image (image image) CPb.

When detecting the wave Tb, for example, the velocity and magnitude of the wave Tb can be calculated. In particular, by making it possible to calculate the velocity and magnitude of the wave Tb, it can be applied to the observation of the water level and flow velocity of the tsunami and the sea surface, or the monitoring of the phase of the wave.

A specific example of a detection system that detects a wave (tsunami) as a moving body will be described later as the third embodiment.

<Embodiment 2>
Next, a configuration of a mobile detection system to which the multispectral sensor device (multiline image sensor device) according to the second embodiment is applied will be described. The second embodiment is an example in which the line arrangement of the multispectral sensor device has a smaller influence on the resolution. The same parts as those in the first embodiment described above are designated by the same or similar reference numerals, and detailed description thereof will be omitted.

FIG. 26 schematically shows the configuration of the artificial satellite camera 21 equipped with the multispectral sensor device (MSS) 214II, and is mounted on the earth observation satellite 12 in the mobile detection system shown in FIGS. 1 and 2. It will be explained as being done.

The artificial satellite camera 21 of the second embodiment has a line _{for the same wavelength component image in addition to the three primary color image line sensors (R 1} image line sensor 214R ₁ , G image line sensor 214G, B image line sensor 214B). It is provided with a so-called 4-line multispectral sensor device 214II further comprising a sensor (for example, R ₂ image line sensor 214R _2).

However, this multi-spectral sensor device 214II is _{an existing 3-line multi-spectral system in which the R 1} image line sensor 214R ₁ , the G image line sensor 214G, and the B image line sensor 214B are arranged in this order, for example, in a plan view. _{The same wavelength component image line sensor 214R 2} is arranged in parallel with a predetermined interval (physical separation distance) BD so as to be adjacent to the outside of the B image line sensor 214B of the sensor.

That is, the multi-spectral sensor device 214II according to the second embodiment, as shown in FIG. 26, the same wavelength component image line sensor 214R ₂ at one end of the _{R 1} image line sensor 214R ₁ and the other end The configuration is different from that of the multispectral sensor device 214 according to the first embodiment in that the G image line sensor 214G and the B image line sensor 214B are arranged between the two.

According to the multi-spectral sensor device 214II having such a configuration, by providing a function as a multi-line image sensor for detecting a moving object, for example, the existing three primary colors capable of capturing an RGB color composite image while maintaining the same resolution. While playing the role of an image line sensor, it becomes possible to easily acquire a moving object detection image CP in which only a moving object is highlighted by a color image and other stationary objects are displayed in monochrome.

This makes it possible to make only the moving body stand out even more on the moving body detection image CP. Therefore, the moving body (change) can be easily detected visually from the moving body detection image CP for one scene, and the moving body can be efficiently and automatically extracted even in computer processing.

FIG. 27 schematically shows a cross-sectional structure of the multispectral sensor device 214, and here, a cross-sectional view taken along the line II-II of FIG. 26 will be illustrated and described.

In the multispectral sensor device 214II, the R ₁ image line sensor 214R _{1 is, for example, a photodiode serving as a photodiode PD group 214R 1} p formed on the surface portion of the p-type Si substrate 230 or the surface portion of the p-type well 232. and PD of the light-receiving region (n-type layer) 234, the light receiving region 234 spaced apart from, p-type Si substrate 230 surface portion or p-type well 232 surface portion of the vertical CCD to be formed CCD group 214R ₁ f to the It includes a register unit (n ⁺ type layer) 214R ₁ t.

Similarly, the G image line sensor 214G is _{arranged adjacent to, for example, one end side of the R 1} image line sensor 214R _{1 with} a physical separation diode BD, and is arranged on the surface of the p-type Si substrate 230 or the p-type. The surface portion of the p-type Si substrate 230 or the p-type well separated from the light receiving region (n-type layer) 234 of the photodiode PD forming the photodiode PD group 214 Gp formed on the surface portion of the well 232 and the light receiving region 234. ^{A vertical CCD register portion (n +} type layer) 214 Gt, which is formed on the surface portion of 232 and serves as a CCD group 214 Gf, is provided.

Similarly, the B image line sensor 214B is arranged adjacent to, for example, one end side of the G image line sensor 214G with a physical separation diode BD, and is arranged on the surface of the p-type Si substrate 230 or the p-type well 232. The light receiving region (n-type layer) 234 of the photodiode PD forming the photodiode PD group 214Bp formed on the surface portion of the p-type Si substrate 230 and the p-type well 232 of the surface portion of the p-type Si substrate 230 separated from the light receiving region 234. ^{A vertical CCD register portion (n +} type layer) 214Bt, which is formed on the surface portion and serves as a CCD group 214Bf, is provided.

Similarly, the same wavelength component image line sensor 214R ₂ is arranged adjacent to, for example, one end side of the B image line sensor 214B with a physical separation distance BD, and is arranged on the surface of the p-type Si substrate 230 or p. a photodiode light receiving region (n-type layer) of the PD 234 as a photodiode PD group 214R ₂ p formed on the surface of the mold well 232, spaced apart from the light-receiving region 234, the surface portion of the p-type Si substrate 230 or A vertical CCD register portion (n ⁺ type layer) 214R _{2 t, which is a CCD group 214R 2} f formed on the surface portion of the p-type well 232, is provided.

Further, each of the image line sensors 214R ₁ , 214G, 214B, and 214R ₂ is provided on a p-type Si substrate 230 or a p-type well 232, excluding the light receiving region 234, via an insulating film 236. It includes a G238, a light-shielding film 240 provided on the poly-Si electrode 238 via an insulating film 236 and having an opening in a part of the light receiving region 234, and a transparent resin layer 242 provided on the entire surface.

The R ₁ image line sensor 214R ₁ is provided on-chip on the upper surface of the resin layer 242 corresponding to the light receiving region 234, and for example, an R ₁ image color that transmits light in a wavelength band of 620 to 750 nm. further comprising a filter 244R _1, a microlens 246 is provided on-chip on the resin layer 242 corresponding to _{R 1} image for the color filter 244R _1, a.

Further, the G image line sensor 214G is provided on-chip on the upper surface of the resin layer 242 corresponding to the light receiving region 234, and is, for example, a G image color filter 244G that transmits light in a wavelength band of 495 to 590 nm. Further, a microlens 246 provided on-chip on the resin layer 242 corresponding to the G image color filter 244G is further provided.

Further, the B image line sensor 214B is provided on-chip on the upper surface of the resin layer 242 corresponding to the light receiving region 234, and is, for example, a B image color filter 244B that transmits light in a wavelength band of 450 to 495 nm. , A microlens 246 provided on-chip on the resin layer 242 corresponding to the B image color filter 244B is further provided.

Further, the same wavelength component image line sensor 214R ₂ is provided on-chip on the upper surface of the resin layer 242 corresponding to the light receiving region 234, and has _{the same wavelength as the above-mentioned R 1} image color filter 244R _1. further comprising an _{R 2} image for a color filter 244R ₂ that transmits light of a band of 620～750Nm, the microlens 246 provided on-chip on the resin layer 242 corresponding to the color filters 244R ₂ for _{R 2} image, the ..

Also in the case of the multispectral sensor device 214II having the above configuration, the band of each wavelength is an example, and the half width and the like may be different for each of _{the image line sensors 214R 1} , 214G, 214B, and 214R _2.

FIG. 28 schematically shows an arrangement example (line arrangement) of the _{line sensors 214R 1} , 214G, 214B, and 214R ₂ for images in the multispectral sensor device 214II according to the second embodiment.

In the second embodiment, as shown in FIG. 28, for example, the R ₁ image line sensor 214R ₁ for generating an RGB color composite image, the G image line sensor 214G, and the B image line sensor 214B, for example. A multi-spectral sensor device 214II _{in which the same wavelength component image line sensor 214R 2} is arranged in parallel so as to be adjacent to the B image line sensor 214B is adopted. That is, also in the case of this multispectral sensor device 214II, the R ₁ image line sensor 214R ₁ and the same wavelength component image line sensor 214R ₂ are monochrome image line sensors for synthesizing panchromatic images. ..

Here, in the multispectral sensor device 214II according to the second embodiment, at least the G image line sensor 214G or B is between _{the R 1} image line sensor 214R ₁ and the same wavelength component image line sensor 214R _2. One of the image line sensors 214B is arranged. By _{not adjoining the R 1} image line sensor 214R ₁ and the same wavelength component image line sensor 214R _2, it is possible to prevent the resolution from changing (for example, doubling).

Except that the R ₁ image line sensor 214R ₁ and the same wavelength component image line sensor 214R ₂ are not arranged adjacent to each other, each image line sensor 214R _{1 is} provided with respect to a line direction substantially orthogonal to the SD direction. The 214G, 214B, and 214R ₂ can be arranged freely.

The information for designating the specific area EWi includes plane rectangular coordinates, latitude, longitude, area name and address of the shooting target, shooting date, shooting time (shooting start time and end time), or all-color composite image. Coordinates on AEGi (for example, XaYa to XfYf) and the like can be mentioned.

Further, as shown in FIG. 29, a specific area EWi may be designated by an area name such as Tokyo (JMa), Minato Ward (MM), or Daiba area (DM).

Next, the description of the timing of line scanning by the multispectral sensor device 214II will be supplemented with reference to FIGS. 30, 31, and 32.

30 (a) to 30 (p) show an example in which the moving body is a stopped vehicle Ts. Here, as shown in FIG. 30 (a), the multispectral sensor device 214II is , (A), (B), (C), (D) in the figure, line scanning while moving at a constant speed in the SD direction.

In the case of the stopped vehicle Ts, _{from the end of the exposure by the R 1} image line sensor 214R ₁ to the start of the exposure by the G image line sensor 214G, from the end of the exposure by the G image line sensor 214G to the B image. From the end of the exposure by the B image line sensor 214B to _{the start of the exposure by the same wavelength component image line sensor 214R 2} , the time t1 and the time t2, respectively, until the start of the exposure by the line sensor 214B for the same wavelength component. It is assumed that the time t3 exists (t1 = t2 = t3).

As shown in FIG. 30 (b), the stopped vehicle Ts first follows a shooting command from, for example, the moving body detection device 16, at the timing shown in FIG. It is exposed by _first photodiode image line sensor 214R ₁ PD group 214R ₁ p. Then, as shown in FIGS. 30 (d) and 30 (e), the signal charge of the photodiode PD group 214R ₁ p is used for the _{R 1 image at a time L1 corresponding to the waveform molding output as the exposure ends.} It is transferred to the CCD group 214R ₁ f of the line sensor 214R _1.

Next, the stopped vehicle Ts is exposed at the timing shown in FIG. 30B by the photodiode PD group 214Gp of the G image line sensor 214G, as shown in FIG. 30F. Then, the signal charge of the photodiode PD group 214Gp is transferred to the CCD group 214Gf of the G image line sensor 214G as shown in FIGS. 30 (g) and 30 (h) at the end of the exposure.

Next, the stopped vehicle Ts is exposed by the photodiode PD group 214Bp of the B image line sensor 214B at the timing shown in FIG. 30 (i), as shown in FIG. 30 (i). Then, the signal charge of the photodiode PD group 214Bp is transferred to the CCD group 214Bf of the B image line sensor 214B as shown in FIGS. 30 (j) and 30 (k) at the end of the exposure.

Then, the vehicle Ts suspended at the timing shown (D), as shown in FIG. 30 (m), are exposed by the photodiode PD group 214R ₂ p of the same wavelength component image line sensor 214R _2. Then, the signal charges of the photodiode PD group 214R ₂ p, along with the end of the exposure, Fig. 30 (n), as shown in (p), CCD group having the same wavelength component image line sensor 214R ₂ 214R ₂ Transferred to f.

31 (a) to 31 (p) show an example in which the moving body is a vehicle Tma traveling at a low speed. Here, as shown in FIG. 31 (a), the multispectral sensor device 214II Is line-scanned while moving at a constant speed in the SD direction in the order of (A), (B), (C), and (D) shown in the figure.

Further, when the vehicle Tma during low-speed running, the vehicle when Tma is assumed to have moved to the arrow MD direction by a predetermined speed, the line sensor G image from the end of exposure by _{R 1} image line sensor 214R ₁ 214G From the end of the exposure by the G image line sensor 214G to the start of the exposure by the B image line sensor 214B, from the end of the exposure by the B image line sensor 214B to the same wavelength component image line. Before the start of exposure by the sensor 214R _2, there are time t1'(t1'<t1), time t2'(t2'<t2), and time t3'(t3'<t3), respectively, depending on the speed. (T1'= t2'= t3').

As shown in FIG. 31 (b), the vehicle Tma traveling at a low speed first follows a shooting command from, for example, the moving object detection device 16, at the timing of FIG. 31 (A), as shown in FIG. 31 (c). It is exposed by the photodiode PD group 214R ₁ p of R ₁ image line sensor 214R _1. Then, as shown in FIGS. 31 (d) and 31 (e), the signal charge of the photodiode PD group 214R _{1 p becomes the R 1} image at a time L1'corresponding to the waveform molding output as the exposure ends. The line sensor 214R ₁ is transferred to the CCD group 214R _{1 f.}

Next, the vehicle Tma traveling at a low speed is exposed at the timing shown in FIG. 31 (B) by the photodiode PD group 214 Gp of the G image line sensor 214 G, as shown in FIG. 31 (f). Then, the signal charge of the photodiode PD group 214Gp is transferred to the CCD group 214Gf of the G image line sensor 214G as shown in FIGS. 31 (g) and 31 (h) at the end of the exposure.

Next, the vehicle Tma traveling at a low speed is exposed by the photodiode PD group 214Bp of the B image line sensor 214B at the timing shown in FIG. 31 (i), as shown in FIG. 31 (i). Then, the signal charge of the photodiode PD group 214Bp is transferred to the CCD group 214Bf of the B image line sensor 214B as shown in FIGS. 31 (j) and 31 (k) at the end of the exposure.

Then, the vehicle Tma in low speed at the timing shown (D), as shown in FIG. 31 (m), are exposed by the photodiode PD group 214R ₂ p of the same wavelength component image line sensor 214R _2. Then, the signal charges in the photodiode PD group 214R ₂ p, along with the end of the exposure, Fig. 31 (n), as shown in (p), CCD group having the same wavelength component image line sensor 214R ₂ 214R ₂ Transferred to f.

32 (a) to 32 (p) show an example in which the moving body is a vehicle Tmb traveling at high speed. Here, as shown in FIG. 32 (a), the multispectral sensor device 214II Is line-scanned while moving at a constant speed in the SD direction in the order of (A), (B), (C), and (D) shown in the figure.

Further, when the vehicle Tmb during high-speed driving, assuming that the vehicle Tmb is moving in the arrow MD direction the speed equal to or greater than a predetermined speed, the G image from the end of exposure by R ₁ image line sensor 214R ₁ From the end of exposure by the line sensor 214G for G image to the start of exposure by the line sensor 214B for B image, from the end of exposure by the line sensor 214B for B image to the start of exposure by the line sensor 214G, the same wavelength component Until the start of exposure by the image line sensor 214R ₂ , the time t1 ″ (t1 ″ <t1 ′), the time t2 ″ (t2 ″ <t2 ′), and the time t3, respectively, depending on the speed. ''(T3''<t3') is assumed to exist (t1'' = t2'' = t3'').

As shown in FIG. 32 (b), the vehicle Tmb traveling at high speed first follows a shooting command from, for example, the moving object detection device 16 at the timing of FIG. 32 (A), as shown in FIG. 32 (c). It is exposed by the photodiode PD group 214R ₁ p of R ₁ image line sensor 214R _1. Then, the signal charges in the photodiode PD group 214R ₁ p, along with the end of the exposure, Fig. 32 (d), in (e), the time corresponding to the waveform shaping the output L1 '', _{R 1} It is transferred to the CCD group 214R ₁ f of the image line sensor 214R _1.

Next, the vehicle Tmb traveling at high speed is exposed at the timing shown in FIG. 32B by the photodiode PD group 214Gp of the G image line sensor 214G, as shown in FIG. 32F. Then, the signal charge of the photodiode PD group 214Gp is transferred to the CCD group 214Gf of the G image line sensor 214G as shown in FIGS. 32 (g) and 32 (h) at the end of the exposure.

Next, the vehicle Tmb traveling at high speed is exposed by the photodiode PD group 214Bp of the B image line sensor 214B at the timing shown in FIG. 32 (i), as shown in FIG. 32 (i). Then, the signal charge of the photodiode PD group 214Bp is transferred to the CCD group 214Bf of the B image line sensor 214B as shown in FIGS. 32 (j) and 32 (k) at the end of the exposure.

Then, the vehicle Tmb in high speed at the timing shown (D), as shown in FIG. 32 (m), are exposed by the photodiode PD group 214R ₂ p of the same wavelength component image line sensor 214R _2. Then, the signal charges in the photodiode PD group 214R ₂ p, along with the end of the exposure, Fig. 32 (n), as shown in (p), CCD group having the same wavelength component image line sensor 214R ₂ 214R ₂ Transferred to f.

The multispectral sensor device 214II has fixed physical lengths (sensor length SL) and exposure time of the _{line sensors 214R 1} , 214G, 214B, and 214R _{2 for each image, but the speeds of the moving vehicles Tma and Tmb.} The timing of exposure in each of the image line sensors 214R ₁ , 214G, 214B, and 214R _{2 changes accordingly.}

That is, in the case of a vehicle Tmb traveling at a speed equal to or higher than a certain speed, for example, as shown in FIG. 33A, the color images of the bands obtained by aligning them so as to geometrically overlap are displayed in the same size. even superimposed movement between the pixels of the vehicle Tmb is, therefore spanning ten pixels in accordance with the moving speed, R ₁ of a color _display, G, large color shift occurs in B.

On the other hand, in the case of a vehicle Tma traveling at a speed of a certain speed or less, for example, as shown in FIG. 33 (b), the color images of the bands obtained by aligning them so as to geometrically overlap are displayed in the same size. even superimposed movement between the pixels of the vehicle Tma is, for up to several pixels in accordance with the moving speed, the color display R _1, G, a small color shift B occurs.

Here, the relationship between the moving direction of the moving body and its color display (color shift) will be described.

FIG. 34 (a) illustrates the relationship between the moving direction (MD) of the moving vehicle Tm and the moving direction (SD) of the earth observation satellite 12, and corresponds to, for example, FIGS. 31 and 32. ..

Further, FIG. 34 (b) shows the moving direction of the vehicle Tm on the ground surface as the east side, FIG. 34 (c) shows the moving direction of the vehicle Tm on the ground surface as the west side, and FIG. 34 (d) shows. , The moving direction of the vehicle Tm on the ground surface is the north side, and FIG. 34 (e) is an example in which the moving direction of the vehicle Tm on the ground surface is the south side.

As shown in FIGS. 34 (b) to 34 (e), the moving vehicle Tm always has the color display for the moving direction in the order of _{B, G, R 1 regardless of the moving direction (the color display for the moving direction is always B, G, R 1). Since R 2} is not particularly related to the color display, it is not shown).

That is, in the case of a moving vehicle Tm, the order of B, G, R _{1 depends on the order of arrangement of the line sensors 214R 1} , 214G, and 214B for each image with respect to the moving direction (SD) of the earth observation satellite 12. Color shift occurs. Therefore, by detecting this color shift on the moving object detection image CP of the single scene, it is possible to specify not only the moving vehicle Tm but also the moving direction (MD) of the vehicle Tm.

Further, since the color shift is substantially proportional to the speed of the running vehicle Tm, the speed of the running vehicle Tm can be easily estimated (calculated) from the amount of the shift.

Next, the mobile body detection image CP that can be acquired by the mobile body detection device 16 having the above configuration will be described.

FIGS. 35 (a) to 35 (c) exemplify a case where the vicinity of the IC on the expressway is actually photographed and the amount of change of the moving body is displayed in color by the moving body detecting device 16. As a result, for example, a multispectral image (color composite image GWi of the specific area EWi) as shown in FIG. 35 (a) and a panchromatic image (black and white composite of the specific area EWi) as shown in FIG. 35 (b). By aligning the image GEAi) so as to geometrically overlap, a moving body detection image (before color enhancement) as shown in FIG. 35 (c), which is a pan sharpened image CP1 before color enhancement, is obtained. Be done.

Further, FIGS. 36 (a) to 36 (c) exemplify a case where the vicinity of the connecting road connecting the IC of the expressway and the general road is actually photographed. Note that FIG. 36 (a) is a multispectral image (color composite image GWi of the specific area EWi) near the connecting road, and FIG. 36 (b) is a panchromatic image (black and white composite of the specific area EWi) near the connecting road. Image GEAi), FIG. 36 (c) is a pan-sharpened image CP1 before color enhancement as a moving object detection image (before color enhancement) near the connecting road.

FIG. 37 is an enlarged view of the pan-sharpened image CP1 before color enhancement near the connecting road shown in FIG. 36 (c).

As is clear from the pan-sharpened image CP1 before color enhancement, the plurality of vehicles Tmo traveling on the connecting road DO gradually stop at the red light as they approach the intersection direction from the IC direction. As the speed is reduced, the color display changes from a color display with a large color shift to a monochrome display with a small color shift.

On the contrary, the vehicle Tmi heading toward the IC gradually accelerates, and as the speed increases, the color display changes to a color display having a large color shift.

It should be noted that this pan-sharpened image CP1 is before color enhancement, and by performing the above-mentioned color enhancement processing on the traveling vehicles Tmo and Tmi, the display of the traveling vehicles Tmo and Tmi becomes clearer. Become.

That is, for example, whether or not each pixel of the pan-sharpened image CP1 before color enhancement belongs to the moving vehicle Tm is determined based on the pixel value. For example, if any of the pixel values indicating the gradation degree of each pixel color type is not "0", that pixel is determined to belong to the moving vehicle Tm, and conversely, the gradation degree of each pixel color type is determined. When all of the indicated pixel values are "0", the pixel is determined to be that of a stationary object.

Then, the pixel value indicating the gradation degree of each color type of each pixel determined to be that of the moving vehicle Tm is changed to the maximum value (for example, 240 to 255).

In this way, the pan-sharpened image CP2 after color enhancement in which the gradations of the pixels in which all the pixel values indicating the gradations of the color types are not "0" are changed to the maximum values are displayed as the moving object detection image CP. (Not shown).

As described above, according to the second embodiment as well, the moving body can be detected more efficiently, and the speed and direction of the detected moving body can be accurately and automatically grasped.

By making it possible to automatically detect the moving vehicle Tm, for example, it is possible to identify a straight line area, a curved area, a mountain area, and the slope on the road from the speed.

In particular, in developing countries, it can be applied to the case where the road map is updated by constructing a new road that is not on the map by automatically detecting the moving vehicle Tm.

The multispectral sensor device 214II according to the second embodiment can also be applied to the _{multispectral sensor device 214 1 having the configuration shown in FIG. 19, the multispectral sensor device 214 2} having the configuration shown in FIG. 20, and the like. be.

FIG. 38 shows the multispectral sensor device 214II according to the second embodiment, further including a line sensor 214W for infrared (IR) or near infrared (NIR), and configured as the multispectral sensor device 214d. This is a first modification of the case where

Figure 39 does not depend on the same wavelength component image line sensor 214R _2, as an additional line sensor comprises a _{G 2} image line sensor (line sensor for the same wavelength component image) 214G _2, the multispectral sensor device 214e This is a second modification when the configuration is made. That is, in the case of the multispectral sensor device 214e, the G ₁ image line sensor 214G ₁ and the G ₂ image line sensor 214G ₂ are arranged with the B image line sensor 214B in between to acquire a panchromatic image. It is a line sensor for monochrome images.

Figure 40 does not depend on the same wavelength component image line sensor 214R _2, as an additional line sensor comprises a _{B 2} image line sensor (line sensor for the same wavelength component image) 214B _2, the multispectral sensor device 214f This is a third modification when the configuration is made. That is, in the case of the multispectral sensor device 214f, the B ₁ image line sensor 214B ₁ and the B ₂ image line sensor 214B ₂ sandwich the R image line sensor 214R and the G image line sensor 214G in between. Arranged, it is a line sensor for monochrome images for acquiring panchromatic images.

FIG. 41 is a fourth modification in the case where two or more lines of the same type of line sensor are added, and illustrates a case where the multispectral sensor device 214 g is configured. That is, in the case of the multi-spectral sensor device 214g, the R ₁ image line sensor 214R ₁ and the R ₂ image line sensor (first same wavelength component image line sensor) 214R ₂ and R ₃ image line sensor (second). The same wavelength component image line sensor) 214R ₃ is arranged with the G image line sensor 214G or the B image line sensor 214B sandwiched between them, respectively, to obtain a monochrome image line sensor. It has become.

As described above, in the multispectral sensor devices 214II, 214d, 214e, 214f, and 214g, the sensors in the same band as any of "R", "G", and "B" are newly added with an interval of at least one line. By adopting the configuration added to, in the creation of the moving body detection image CP by the moving body detecting device 16, the color shift of the moving body (the one that moved during the reading of the image) can be more reliably distinguished and expressed. Will be.

<Embodiment 3>
Next, a specific configuration of the mobile body detection device 16 of the mobile body detection system to which the multi-spectral sensor device (multi-line image sensor device) according to the third embodiment is applied will be described with reference to FIG. 42.

Here, an example will be described in which the earth observation satellite 12 equipped with the artificial satellite camera 21 incorporating the multispectral sensor devices (MSS) 214 and 214II senses in real time on the Pacific PO off Sanriku in the Japanese archipelago JM, for example. do.

Further, a case where there are no clouds off Sanriku and the resolution at the time of observation by the satellite 12 is 2.5 m × 300 km will be described.

The mobile body detection device 16 of the ground center may be, for example, the one of the first embodiment, but the case where the mobile body detection device 16 of the ground center has a different configuration will be described as an example.

The acquisition unit 161 sequentially stores the captured image eGi for each line included in the transmission signal SGi received at the antenna facility (not shown) in the memory for each image 163a _{1 to 163a 4.}

At this time, the first image (R ₁ ) for each line is stored in _{the first} R image memory 163a 1, the second image (G) for each line is stored in _{the first G image memory 163a 2, and each line.} The third image (B) is stored in _{the first B image memory 163a 3.}

On the other hand, it is assumed that the same wavelength component image Ri (R ₂ ) for each line is stored in the first memory for the same wavelength component image 163a _4.

Further, the header information EHi included in the transmission signal SGi is stored in the header information database 163b in association with the line-by-line captured image eGi, but is not shown in FIG. 42.

At this time, the R ₁ compositing unit 370a included in the acquisition unit 161 corrects the line delay each time the first image (R ₁ ) for each line is stored in the first R image memory 163a _1. , Plane orthogonal coordinate conversion, orthophoto correction, etc., and then mosaic processing is performed and stored in _{the second R image memory 163a 11.}

Further, every time the G synthesis unit 370b included in the acquisition unit 161 _{stores the second image (G) for each line in the first G image memory 163a 2} , line delay correction and plane orthogonal coordinate conversion are performed. Orthophoto correction or the like, and then mosaic processing is performed and the image is stored in _{the second G image memory 163a 22.}

Further, every time the B synthesis unit 370c included in the acquisition unit 161 _{stores the third image (B) for each line in the first B image memory 163a 3} , line delay correction and plane rectangular coordinate conversion are performed. Orthophoto correction or the like, and then mosaic processing is performed and the image is stored in _{the second B image memory 163a 33.}

Further, every time the same-wavelength component image synthesis unit 370d included in the acquisition unit 161 _{stores the same-wavelength component image Ri (R 2} ) for each line in _{the first same-wavelength component image memory 163a 4, the line is lined up.} It is assumed that after performing processing such as delay correction, plane orthogonal coordinate conversion, orthophoto correction, mosaic processing is performed and the image _{is stored in the second memory for the same wavelength component image 163a 44.} That is, the acquisition unit 161 includes a _{synthesis unit 370a for R 1} , a synthesis unit 370b for G, a synthesis unit 370c for B, and a synthesis unit 370d for the same component image.

That is, the acquired image database 163a includes the first R image memory 163a ₁ , the first G image memory 163a ₂ , the first B image memory 163a _3, and the first same wavelength component image memory 163a ₄ A first acquired image database, a second R image memory 163a ₁₁ , a second G image memory 163a ₂₂ , a second B image memory 163a _33, and a second same wavelength component image. It is composed of a second acquired image database including a memory 163a _44.

The acquired image database 163a may include, for example, only the components of the captured image eGi for each line at a predetermined level or higher.

As shown in FIG. 42, the color image memory 169a is composed of an R ₁ image area memory 169a ₁ , a G image area memory 169a _2, and a B image area memory 169a ₃ . Further, the panchromatic image generation memory 169b is also referred to as a memory for the same component image area.

As shown in FIG. 42, the area extraction unit 165 includes an R ₁ image area extraction unit 365a, a G image area extraction unit 365b, a B image area extraction unit 365c, and an area extraction unit 365d for the same component image. It has.

(Explanation of area extraction unit 165)
R ₁ image for the area extraction section 365a has operation portions entered by (not shown) Resolution (1 dot per 2.5m or 10m screen, · · 200 meters or 1km, · ·) reads the fi. Then, the mesh Mi having the size of this resolution fi is defined in the memory 169a _{1 for the R 1 image area.} Further, an area having a size corresponding to the coordinate system of an arbitrary specific area (coordinates XaYa to XfYf) EWi is defined in _{the second R image memory 163a 11.}

Then, every time the first image (R ₁ ) for each line of the specific area EWi reaches the number corresponding to the specific area EWi specified by the operation unit, the first image E (R ₁ ) for each specific area is reached. Mesh Mi is defined sequentially in. Then, the pixel data contained in the mesh Mi, is stored in the corresponding pixel of the R ₁ image area memory 169a _1.

Further, the G image area extraction unit 365b reads the resolution fi input by the operation unit. Then, the mesh Mi having the size of this resolution fi is defined in _{the G image area memory 169a 2.} Further, an area having a size corresponding to the coordinate system of the arbitrary specific area EWi is defined in _{the second G image memory 163a 22.}

Then, every time the second image (G) for each line of the specific area EWi reaches the number corresponding to the specific area EWi specified by the operation unit, the mesh Mi is displayed on the second image E (G) for each specific area. Are defined sequentially. Then, the pixel data included in the mesh Mi is stored in the corresponding pixel of _{the G image area memory 169a 2.}

Further, the B image area extraction unit 365c reads the resolution fi input by the operation unit. Then, the mesh Mi having the size of this resolution fi is defined in the _{memory 169a 3 for the B image area.} Further, an area having a size corresponding to the coordinate system of the arbitrary specific area EWi is defined in _{the second B image area memory 163a 33.}

Then, every time the third image (B) for each line of the specific area EWi reaches the number corresponding to the specific area EWi specified by the operation unit, the mesh Mi is displayed on the third image E (B) for each specific area. Are defined sequentially. Then, the pixel data included in the mesh Mi is stored in the corresponding pixel of _{the B image area memory 169a 3.}

Further, the area extraction unit 365d for the same component image reads the resolution fi input by the operation unit. Then, the mesh Mi having the size of this resolution fi is defined in the panchromatic image generation memory 169b. Further, a region having a size corresponding to the coordinate system of the arbitrary specific area EWi is defined in _{the second memory for the same wavelength component image 163a 44.}

Then, every time the same wavelength component image Ri (R ₂ ) for each line of the specific area EWi reaches the number corresponding to the specific area EWi specified by the operation unit, the same wavelength component image ERi (R 2) for each specific area is reached. Mesh Mi is defined in order in _2). Then, the pixel data included in the mesh Mi is stored in the corresponding pixel of the panchromatic image generation memory 169b.

The superimposing unit 171 _{designates each pixel of the R 1} image area memory 169a ₁ , the G image area memory 169a ₂ , the B image area memory 169a _3, and the panchromatic image generation memory 169b in order. Then, the first image E (R ₁ ) for each specific area, the second image E (G) for each specific area, the third image E (B) for each specific area, and the same wavelength component image for each specific area of these designated pixels. ERi the _{(R 2),} (more precisely, a pan-sharpened image CP1 before color enhancement) sequentially stores moving object detection image CP in the color enhancement image memory 173 obtained.

Note that the line for each captured image EGI, for each line for each captured image EGI, satellite ID, photographing date and time, the color type _{(R 1, G, B,} R 2), resolution (2.5 m), latitude ( Header information EHi such as X), longitude (Y), and attitude is associated.

Here, the detection parameters are set by the operator, for example, at the time of tsunami detection, the coordinates of the four corners corresponding to the tsunami detection area (XY to XY) TEi of 200 km × 200 km and the tsunami according to the resolution (100 m × 100 m). The range for change detection (WA) and information on the epicenter SC (Ei) can be mentioned.

That is, for the tsunami detection area TEi, a mesh Mi in which one mesh (vertical length × horizontal width) is an interval (for example, 100 m × 100 m) of the tsunami change detection range (WA) is set.

In the moving object detection image CP, it is preferable to perform smoothing processing and to remove clouds, ships, and the like as appropriate because the general shapes are known.

FIG. 43 virtually shows a case where information (Ei) regarding the epicenter SC is provided by the Japan Meteorological Agency or the like in connection with the occurrence of an earthquake on the display screen of the moving object detection image CP.

In the case of this example, the moving object detection image CP exceeds, for example, the epicenter SC of the earthquake that occurred off Sanriku in the Japanese archipelago JM and the tsunami parameter that seems to have occurred in the vicinity of the epicenter SC. Waves Tb and are included. Accordingly, as shown in FIG. 43, the color image of the wave Tb (B image, G image, R ₁ image), and by storing the mesh Mi is set to the tsunami detection area TEi, it can be detected as a tsunami.

In the mobile body detection device 16, for example, the process is started when the information (Ei) regarding the epicenter SC is supplied to the area extraction unit 165 (365a, 365b, 365c, 365d) as a detection parameter. You may. In this case, the area extraction unit 165 (365a, 365b, 365c, 365d) is used for color image memory 169a (169a ₁ , 169a ₂ , 169a ₃ ) and pancro image generation based on the information (Ei) regarding the epicenter SC. It is possible to have the memory 169b define the epicenter SC.

FIG. 44 virtually shows a case where the information (Ei) regarding the epicenter SC is not provided by the Japan Meteorological Agency or the like due to the occurrence of an earthquake on the display screen of the moving object detection image CP.

In the case of this example, in the moving body detection device 16, it is necessary to constantly monitor the moving body detection image CP, but the color image of the wave Tb exceeding the tsunami parameter is set in the tsunami detection area TEi. By storing it in, it can be detected as a tsunami.

As described above, according to the third embodiment, it can also be applied to the detection of a moving body such as a tsunami.

In addition to the sea, it is also used for river management such as monitoring the water level and flow velocity of rivers with flash floods, and for monitoring large-scale natural disasters such as landslides and avalanches (energy estimation, etc.). Available.

Further, by using an infrared sensor, an X-ray sensor, or the like, it can be applied to check the capacity in the tank, and to check the loading and unloading of trucks and containers at night.

Further, the line sensor is not limited to a sensor such as a 3-line system, and may be, for example, a sensor such as a 3CCD system using a dichroic prism.

Furthermore, it goes without saying that it can be applied not only to a CCD line sensor but also to a line sensor having a CMOS structure.

10 Earth 12 Earth observation satellite (artificial satellite / flying object)
14 Antenna facility 16 Mobile detector (ground station)
21 Artificial satellite camera 23 Receiver 25 Sensor control 27 Transmitter 161 Acquisition section 175 Color enhancement section 214, 214II Multispectral sensor device (multiline image sensor device)

Claims (13)

Three primary color image line sensors including an R image line sensor, a G image line sensor, and a B image line sensor, which are arranged in parallel at intervals from each other.
A first that is arranged in parallel with an image line sensor of any of the three primary color image line sensors at the interval and detects a wavelength of light of the same color as the image line sensor of any of the three primary color image line sensors. and a line sensor for the same wavelength component image,
A color image is acquired by the three primary color image line sensors, and a black and white image is acquired by the image line sensor of any of the three primary color image line sensors and the first line sensor for the same wavelength component image. Multi-line image sensor device. The line sensor for R image is
A group of R image filters in which a fixed number of R image filters are arranged in series,
It has an R image photoelectric conversion element group provided under the R image filter group and in which a fixed number of photoelectric conversion elements are arranged in series.
The line sensor for G image is
A G image filter group arranged in parallel with the R image filter group at a certain interval and a fixed number of G image filters arranged in series, and a G image filter group.
It has a G-image photoelectric conversion element group provided under the G-image filter group and in which a fixed number of photoelectric conversion elements are arranged in series.
The B image line sensor is
A B image filter group arranged in parallel with the G image filter group at a certain interval and a fixed number of B image filters arranged in series, and a B image filter group.
It is provided in the lower layer of the B image filter group, and is configured to include a B image photoelectric conversion element group in which a fixed number of photoelectric conversion elements are arranged in series.
The first line sensor for the same wavelength component image is
The same fixed number of image filters as any of the R image filter, the G image filter, or the B image filter is the R image filter group, the G image filter group, or the B image filter. A first filter group for the same wavelength component image arranged in parallel with the above-mentioned interval with respect to the filter group,
A first photoelectric conversion element group for the same wavelength component image, which is provided under the first layer of the same wavelength component image filter group and in which a fixed number of photoelectric conversion elements are arranged in series,
The multi-line image sensor device according to claim 1, wherein the multi-line image sensor device is provided. The multi-line image sensor device according to claim 1 or 2, wherein the first line sensor for the same wavelength component image is arranged adjacent to the line sensor for an image of any one of the three primary color image line sensors. .. According to claim 1 or 2, the first line sensor for the same wavelength component image is arranged adjacent to an image line sensor different from the image line sensor of any one of the three primary color image line sensors. The multi-line image sensor device described. The multi-line image sensor device according to claim 1 or 2, wherein the first line sensor for the same wavelength component image is arranged at the rearmost stage via the three primary color image line sensors. Moreover,
The same fixed number of image filters as the first same wavelength component image filter group is the R image filter group, the G image filter group, the B image filter group, or the first same. A second group of filters for the same wavelength component image arranged in parallel with the same interval with respect to the group of filters for the wavelength component image, and a group of filters for the same wavelength component image.
A second photoelectric conversion element group for the same wavelength component image, which is provided under the second layer of the same wavelength component image filter group and in which a fixed number of photoelectric conversion elements are arranged in series,
The multi-line image sensor device according to claim 2 , further comprising a second line sensor for the same wavelength component image having the above. Moreover,
A certain number of infrared image filters or near-infrared image filters are the R image filter group, the G image filter group, the B image filter group, or the first same wavelength component image filter. A group of filters for infrared images arranged in parallel with a certain interval with respect to the group,
An infrared image photoelectric conversion element group provided under the infrared image filter group and a fixed number of photoelectric conversion elements arranged in series, and an infrared image photoelectric conversion element group.
The multi-line image sensor device according to claim 2 , further comprising an infrared image line sensor having the above. Moreover,
The R image photoelectric conversion element group, the G image photoelectric conversion element group, the B image photoelectric conversion element group, and the first same wavelength component image photoelectric conversion element group are arranged in parallel, respectively. The multi-line image sensor device according to claim 2 , further comprising a vertical transfer unit. A flying object having a predetermined height with respect to the shooting area and moving in one direction at a predetermined speed is provided, and as the flying object moves, a color image of the three primary colors and a color image of the three primary colors are used. An imaging device that obtains a color image having the same wavelength as any color image and a color image having the same wavelength of light.
Equipped with a multi-line image sensor device
The multi-line image sensor device is
Three primary color image line sensors including an R image line sensor, a G image line sensor, and a B image line sensor, which are arranged in parallel at intervals from each other.
The same wavelength that is arranged in parallel with the image line sensor of any of the three primary color image line sensors at the interval and detects the wavelength of light of the same color as the image line sensor of any of the three primary color image line sensors. Equipped with a line sensor for component images
Moreover,
An optical system for forming an image of light from the photographing area on the three primary color image line sensors and the same wavelength component image line sensor, and
A sensor drive unit that drives the three primary color image line sensors and the same wavelength component image line sensor,
Each output from the three primary color image line sensors output by driving the sensor drive unit is obtained as a first image, a second image, and a third image and horizontally transferred, and the line sensor for the same wavelength component image. It has a horizontal transfer unit that obtains the output from the same wavelength component as the same wavelength component image and transfers it horizontally .
An imaging device characterized in that a color image is acquired by the three primary color image line sensors and a black and white image is acquired by the image line sensor of any of the three primary color image line sensors and the same wavelength component image line sensor. The line sensor for R image is
A group of R image filters in which a fixed number of R image filters are arranged in series,
A group of photoelectric conversion elements for R images provided under the filter group for R images and in which a fixed number of photoelectric conversion elements are arranged in series.
It has a vertical transfer unit arranged in parallel with the photoelectric conversion element group for R image.
The line sensor for G image is
A G image filter group arranged in parallel with the R image filter group at regular intervals and a fixed number of G image filters arranged in series, and a G image filter group.
A group of photoelectric conversion elements for G images provided under the G image filter group and in which a certain number of photoelectric conversion elements are arranged in series, and a group of photoelectric conversion elements for G images.
It has a vertical transfer unit arranged in parallel with the G-image photoelectric conversion element group, and has.
The B image line sensor is
A B image filter group arranged in parallel with the G image filter group at regular intervals and a fixed number of B image filters arranged in series, and a B image filter group.
A group of photoelectric conversion elements for B images provided under the filter group for B images and in which a fixed number of photoelectric conversion elements are arranged in series, and a group of photoelectric conversion elements for B images.
It has a vertical transfer unit arranged in parallel with the photoelectric conversion element group for B image.
The line sensor for the same wavelength component image is
The same fixed number of image filters as any of the R image filter, the G image filter, or the B image filter is the R image filter group, the G image filter group, or the B image filter. A group of filters for the same wavelength component image arranged in parallel with a certain interval with respect to the filter group,
A group of photoelectric conversion elements for the same wavelength component image provided under the filter group for the same wavelength component image and in which a fixed number of photoelectric conversion elements are arranged in series, and a group of photoelectric conversion elements for the same wavelength component image.
It has a vertical transfer unit arranged in parallel with the photoelectric conversion element group for the same wavelength component image.
The vertical transfer unit of the R image line sensor, the vertical transfer unit of the G image line sensor, the vertical transfer unit of the B image line sensor, and the vertical transfer unit of the same wavelength component image line sensor. The imaging device according to claim 9, wherein the image is connected to the horizontal transfer unit. The line sensor for the same wavelength component image is
The imaging device according to claim 9 or 10, wherein the image pickup device is arranged adjacent to any of the three primary color image line sensors. The air vehicle includes a transmitter and
The transmitter
The first image, the second image, the third image, and the same wavelength component image, which are horizontally transferred via the horizontal transfer unit, are set as line-by-line shot images for each shooting timing, and the line-by-line shooting is performed. A means for generating and transmitting a transmission signal in which the identification information of the flying object, the date and time of shooting, the latitude and longitude at the time of shooting, the color type, and the destination information are added to the image. The photographing apparatus according to claim 9, wherein the image pickup apparatus is configured to have. The imaging device according to claim 9 or 12, wherein the flying object is an artificial satellite.

Images