JP2005086279A - Imaging apparatus and vehicle provided with imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an inexpensive imaging apparatus capable of obtaining excellent image quality by using a fisheye lens. <P>SOLUTION: An optical system is set so that part not needing photographing is projected on the outside of an imaging element when a fisheye image is projected on the imaging element. That is, an imaging region 6a is a region to be supervised and observed in the fisheye image 6a, and a non-imaging region 6b is a region not needing supervision and observation. The optical system is set so that the imaging region is projected on the imaging element 3 and the non-imaging region 6b is projected on the outside of the imaging element 3. Since the projected image of a required part is made greater than that of prior arts (all of fisheye image has been projected on the imaging element) by not projecting the unnecessary part on the imaging element. Thus, the number of pixels of the region on which the required part is projected is increased and the image quality of the image after converting the fisheye image into a normal image can be enhanced. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus and the like, for example, an imaging apparatus used as a back monitor of a vehicle.

  In recent years, surveillance cameras and observation cameras using fisheye lenses have come to be used. Since the fisheye lens has a wide angle of view of around 180 degrees, a single camera can shoot a wide range of subjects.

FIG. 12A shows a fish-eye image projected on the image sensor of the camera in a conventional camera using a fish-eye lens. In FIG. 12, the projected subject is schematically indicated by characters “A” to “D”.
The image sensor 103 is formed in a rectangular shape (rectangle), and a circular fisheye image 105 is projected on the image sensor 103 by a fisheye lens.
Since the fisheye lens has a hemispherical field of view with an angle of view of around 180, the fisheye image is formed by forming a circular image and projecting a subject in a hemispherical space in front of the fisheye lens.
In the conventional camera, since the entire range of the fisheye image is a subject to be imaged, the optical design is made so that the fisheye image is inscribed in the image sensor 103.

A fish-eye image is a distorted image due to its nature such that the periphery of the image is compressed in order to project a subject in a hemispherical field to a circular projection image.
On the other hand, a normal image (an image without distortion) can be restored from a distorted image using the optical relationship between the subject and the fisheye image. In general, in a system using a fisheye lens, in order to improve visibility, etc. The distorted image is converted into a normal image and displayed on the display device.
In the image sensor 103, normal image conversion areas 107a to 107d for correcting distortion are set, and a portion of the fisheye image 105 projected onto this area is extracted and subjected to normal image conversion.

FIG. 12B shows a state where the image after normal image conversion is displayed on the display device. The display screen is divided into four display areas 108a to 108d, and the fish-eye images projected on the normal image conversion areas 107a to 107d are converted into normal images and displayed.
In the figure, as an example, a portion projected from the fish-eye image 105 onto the normal image conversion areas 107a to 107d is extracted and subjected to normal image conversion. However, in general, one or a plurality of normal image conversion areas are arbitrarily selected according to the object to be imaged. Can be specified.
As described above, the invention using the fisheye lens includes the following “full-field still camera orientation system”.

JP-T 6-501585

According to the present invention, a portion of a designated area is extracted from a fish-eye image and subjected to normal image conversion, thereby displaying a wide range of photographing objects in real time.
Since this invention projects a subject in a hemispherical field of view using a fisheye lens, a wide range can be taken with a single camera. For example, when installed in the center of the ceiling of a store, The entire store can be photographed with this camera.

By the way, in the conventional system using a fisheye lens, there is a problem that the image quality of a portion that has been compressed and projected (for example, the peripheral portion) is deteriorated when correcting the distortion of the fisheye image (correct image conversion). It was.
In order to improve the image quality, it may be possible to use a high-resolution camera with a large number of pixels of the image sensor, but the high-resolution camera is expensive, and there is a problem that the cost for producing the imaging device increases. is there. In addition, when a high-resolution camera is used, the time required for image processing increases.

Accordingly, a first object of the present invention is to provide an inexpensive imaging apparatus that can obtain a good image quality using a fisheye lens.
It is a second object of the present invention to provide a vehicle including an inexpensive imaging device that can obtain a good image quality using a fisheye lens.

In the invention described in claim 1, an imaging device is provided with a fisheye lens that forms a circular fisheye image and an imaging element having a rectangular imaging region on which the fisheye image is projected, The diameter of the fish-eye image formed on is larger than the length of the short side of the imaging region, thereby achieving the first object.
According to a second aspect of the present invention, the imaging apparatus according to the first aspect is characterized in that the center point of the fish-eye image and the center point of the imaging region are arranged so as not to coincide with each other.
According to a third aspect of the present invention, in the imaging apparatus according to the first or second aspect, the fisheye lens includes a changing unit that changes a range of a fisheye image projected on the imaging element. .
According to a fourth aspect of the present invention, there is provided a vehicle including the imaging device according to the first, second, or third aspect, wherein the imaging device uses a center point of the circular fisheye image as the center point. The second object is achieved by disposing the image pickup region above the center point of the imaging region.
According to a fifth aspect of the present invention, in a vehicle including the imaging device according to the fourth aspect, an image acquisition unit that acquires a fisheye image projected on the imaging element as an image, and a predetermined value for the acquired image. Distortion correcting means for correcting the distortion in the given area, and image display means for displaying an image of the area in which the distortion is corrected.
In addition, in the invention according to claim 5, comprising an area designating unit for designating an area of the image acquired by the image acquisition unit, the distortion correction unit corrects the distortion in the region specified by the region designating unit, You may do it.
Further, in the invention described in claim 4 or claim 5, a changing means for changing a range of an imaging target projected by the fisheye lens may be provided.

  According to the present invention, an image with good image quality can be obtained using a camera using a fisheye lens.

Hereinafter, preferred embodiments of an imaging device and a vehicle according to the present invention will be described with reference to FIGS.
(1) Outline of Embodiment When projecting a fish-eye image on an image sensor, an optical system is set so that a portion that does not need to be photographed is projected outside the image sensor.
By not projecting the unnecessary portion onto the image sensor, the projected image of the necessary portion can be made larger than in the past (projecting all fisheye images onto the image sensor).
Therefore, the number of pixels in a region where a necessary portion is projected increases, and the image quality of the image after normal image conversion can be improved.

In FIG. 2, the imaging region 6 a is a region where a subject requiring observation is projected in the fish-eye image 5, and the non-imaging region 6 b is a region where a subject not requiring observation is projected.
As shown in the drawing, the optical system is set so that the imaging area 6 a is projected onto the imaging element 3 and the non-imaging area 6 b is projected outside the imaging element 3.
In the imaging device 3, a conversion area for converting to a normal image is set in advance as a normal image conversion area 7. A portion of the imaging area 6a that is projected on the normal image conversion area 7 is extracted to become a normal image. Converted.

In this way, by projecting the non-imaging region 6b outside the imaging device 3, the projected image projected on the normal image conversion region 7 is made larger than when the entire fisheye image 5 is projected on the imaging device 3. Can do.
As a result, the number of pixels in the normal image conversion area 7 is increased, and the image quality of the image after normal image conversion is improved.
In addition, the number of pixels in the normal image conversion area 7 is set to 1 / integer of the number of pixels of the image displayed on the display device. As a result, a load related to image processing such as interpolation between pixels during normal image conversion is reduced. Can be reduced.

(2) Details of Embodiment (a) First Embodiment As an example, an imaging device described below is mounted on a vehicle as a back monitor, a side monitor, or the like, and is used by a driver to observe a blind spot or the like. And
Note that the imaging apparatus according to the first embodiment is not limited to being mounted on a vehicle but can be widely used, and the case of using it for indoor monitoring will be described later.
In the following, a portion to be monitored / observed among subjects projected by the fisheye lens will be referred to as an observation target.

FIG. 1 is a block diagram illustrating an example of the configuration of the imaging apparatus 1 according to the first embodiment.
The imaging apparatus 1 includes a fisheye lens 10, an imaging camera 12 as an image acquisition unit, a normal image conversion unit 14 as a distortion correction unit, an image display unit 16 as an image display unit, and an area designation unit 13 as an area designation unit. It is configured.
The imaging device 1 is used as, for example, an in-vehicle monitor (back monitor, side monitor, etc.).

The fisheye lens 10 is a lens system configured by combining a plurality of lenses, and can obtain an angle of view of 180 degrees. For this reason, the fisheye lens 10 can form and project a subject in front of the fisheye lens 10 in the optical axis direction as a circular fisheye image.
Here, the angle of view refers to the range of the subject to be projected expressed as an angle.

  In the present embodiment, the fisheye lens 10 with 180 degrees is used as the angle of view, but a fisheye lens having another angle of view may be used. That is, in general, a fisheye lens can project a range of about 180 degrees of view (having a hemispherical field of view), and a fisheye lens whose angle of view is variable from 90 degrees to 185 degrees has been put to practical use. May be used.

The imaging camera 12 is a conversion device that converts a fisheye image projected by the fisheye lens 10 into a digital signal (fisheye image).
The imaging camera 12 includes, for example, an imaging element having a rectangular (rectangular) imaging area configured by a CCD (Charge Coupled Device) element, and a fish-eye image is projected onto the imaging element.
The imaging element is configured by arranging a plurality of pixels, and each pixel converts light into electric charge, thereby converting a projected image into a digital signal (image).

The normal image conversion device 14 has an extraction function for extracting a fisheye image projected on a preset area (hereinafter referred to as a normal image conversion area) from the fisheye image converted into a digital signal, and the extracted fish It has a normal image conversion function for converting an eye image into a normal image without distortion.
Note that the position of the normal image conversion area can be reset using an area designating device 13 described later.

More specifically, the normal image conversion device 14 stores information for specifying a region for normal image conversion on the image sensor, and the normal image conversion device 14 converts a normal image from a fisheye image converted into a digital signal. Extract the part included in the conversion area.
Note that the positions of the fisheye lens 10 and the imaging camera 12 are adjusted so that the observation target is projected onto the normal image conversion area 7.

Next, the normal image conversion device 14 removes the distortion of the extracted fisheye image, converts it into a normal image, and outputs it to the image display device 16.
A geometric conversion formula between a fisheye image and a normal image is known, and a normal image can be restored from a fisheye image using this conversion formula. This conversion formula is described in, for example, the above-mentioned Patent Document 1.

By the way, there are two types of fisheye lenses, an all-around fisheye lens and a diagonal fisheye lens. Each lens can produce a circular projection image, but the projected fisheye image is different.
In the case of the all-around fisheye lens, an angle of view of 180 degrees is obtained in all directions, and in the case of the diagonal fisheye lens, an angle of view of 180 degrees is obtained in the diagonal direction.
Conversion formulas for normal image conversion are known for any fish-eye lens, and the imaging apparatus 1 can be configured using any fish-eye lens.
In this embodiment, a case where an all-around fisheye lens is used will be described, but a diagonal fisheye lens may be used. In this case, a normal image conversion formula corresponding to the fisheye image by the diagonal fisheye lens is used.

  The normal image conversion device 14 operates based on the image processing program that realizes the extraction function and the normal image conversion function, an arithmetic device that calculates a normal image from a fish-eye image, and these programs and various A storage device for storing data is built in, and the above processing is performed by computer processing.

The image display device 16 is a display device including a display device such as a CRT (Cathode-ray Tube), a liquid crystal display, or a plasma display, and uses the image signal output from the normal image conversion device 14 on the display device. The image after normal image conversion is displayed on.
Note that the imaging device 1 displays the observation target in real time by photographing the subject at predetermined time intervals and displaying the subject on the image display device 16.

The area specifying device 13 is a device for resetting the position of the normal image conversion region set by the normal image conversion device 14 with respect to the image sensor, and the normal image conversion region is set at an arbitrary position on the image sensor. It can be reset.
By making it possible to set the normal image conversion area over the entire image sensor, it is possible to display the entire area of the fisheye image projected on the image sensor on the image display device 16.

Next, a fish-eye image projected on the image sensor will be described. Terms used in this specification are defined as follows.
“Vertical” means the direction of gravity acting on the subject, and “Down” and “Up” mean the direction in which gravity acts and vice versa.
“Horizontal” means a direction perpendicular to the direction in which gravity acts, and “right” and “left” mean the right side and the left side of the subject, respectively.
Further, in the fisheye image and the image sensor shown below, the upper side and the lower side are set in the vertical direction, respectively, and the right side and the left side are set in the right direction and the left direction, respectively.

FIG. 2 is a diagram illustrating a state in which the fisheye image 5 is projected onto the imaging element 3 in the imaging apparatus 1.
The number of horizontal pixels of the image sensor 3 is 640 pixels, and the number of vertical pixels is 480 pixels. Note that the outer dimensions of the image sensor 3 in the horizontal and vertical directions are amm × bmm (a> b).
The imaging element 3 is generally used in an in-vehicle imaging device. As described above, in the first embodiment, since a general-purpose imaging device generally used for in-vehicle use can be used, the cost can be reduced.

The diameter of the fisheye image 5 is Rmm. The horizontal length a of the image sensor 3 and the diameter R of the fisheye image 5 are set to be equal, and both ends of the fisheye image 5 are inscribed in both ends of the image sensor 3.
For this reason, the diameter R of the fish-eye image 5 is set to be larger than the length b of the shorter side of the image sensor 3.

Further, the lower end of the fisheye image 5 is in contact with the lower end of the image sensor 3. For this reason, the imaging range from the left end to the right end of the image sensor 3 corresponds to an angle of view of 180 degrees, and the lower end corresponds to a position directly below the fisheye lens 10.
Since the vertical length b of the image sensor 3 is smaller than the diameter R of the fisheye image 5, the upper end portion of the fisheye image is outside the image sensor 3, and the upper part of the fisheye image is on the image sensor 3. Is not shown.

As described above, the angle of view of the fish-eye image 5 projected in the direction of the longer side of the image sensor 3 is 180 degrees.
In the present embodiment, the horizontal angle of view in the fisheye lens is 180 degrees, but is not limited to this and may be 180 degrees or less, or may be greater than 180 degrees.

Thus, the diameter R of the fish-eye image 5 formed on the image sensor is set larger than the vertical length b (short side) of the image sensor 3, and the center point of the fish-eye image and the image area ( The fish-eye image 5 is placed on the image sensor 3 by disposing the center points of the image sensors) to be inconsistent (in this case, the center point of the fish-eye image is moved above the center point of the imaging region). The projected imaging area 6a and the non-imaging area 6b that has not been projected onto the imaging device 3 are divided.
The range of the subject projected in the imaging region 6a in the fisheye image 5 is a range that can be monitored and observed by the imaging device 1 (hereinafter, the observable range), but the subject that is projected on the non-imaging region 6b. The range is a range that cannot be monitored or observed (hereinafter, an observation unnecessary range).

In the imaging device 1, a portion projected onto the imaging area 6a is made to correspond to a range necessary for the driver to observe the outside of the vehicle, and to a range unnecessary to observe the non-imaging area 6b.
For example, as shown in FIG. 3, when the imaging device 1 is installed at the rear part of the vehicle and used as a back monitor, the driver wants to confirm the space necessary for the operation of the vehicle, that is, the observable range 22 ( The horizontal field angle is 180 degrees), and it is not necessary to photograph the observation unnecessary range 21 where no obstacle exists.
In this way, when driving the vehicle, a range in which an obstacle present in the blind spot can be observed may be projected onto the imaging region 6a.

  Therefore, in the imaging apparatus 1, the optical system is set so that, in the fisheye image 5, the observable range 22 is projected onto the imaging region 6a, and the observation unnecessary range 21 is projected onto the non-imaging region 6b. Note that adjusting the positional relationship of the optical system is sometimes referred to as alignment.

  By setting the observation unnecessary range 21 in the non-imaging region 6b, it is possible to project the image of the observation target on the image sensor 3 by that much. For this reason, the number of pixels in the portion onto which the observation target is projected increases, and the image quality of the image after normal image conversion can be improved. The reason why a high resolution cannot be obtained when the number of pixels is small is that the corresponding pixels in the image after normal image conversion must be used in an overlapping manner.

Returning to FIG. 2, the image sensor 3 is provided with a normal image conversion area 7 having a substantially rectangular shape, and a fish-eye image projected on this area is extracted by the normal image conversion device 14, and further the normal image is displayed. Is displayed on the image display device 16.
The number of horizontal pixels in the normal image conversion area 7 is about 320 pixels, and the number of vertical pixels is about 240 pixels.

By the way, in the conventional system using the fisheye lens, the optical system is set so that the entire fisheye image 5 is accommodated in the image sensor 3. This is effective in the case of monitoring or the like, which is a use application utilizing the characteristics of a fisheye lens.
However, when the optical system is designed in this way, the outer size of the normal image conversion area is about 213 pixels in the horizontal direction and 160 pixels in the vertical direction, and in order to obtain a normal image, interpolation between pixels must be performed. The image quality is inferior to the image obtained by the imaging device 1.
Therefore, when a fisheye image includes a portion that does not need to be photographed for a purpose, a clearer image can be obtained by designing the optical system so that the portion is not projected like the imaging device 1. Also, the image pickup device 3 can be effectively used.

  Although an example of the normal image conversion area 7 has been described above, the shape and outer dimensions of the normal image conversion area 7 are not limited to those described above, and various settings can be made as necessary. This may be set in advance at the manufacturing factory, or may be set by the user.

Next, resetting of the normal image conversion area 7 will be described.
The imaging apparatus 1 moves the position of the normal image conversion area 7 on the image sensor 3 to position the observation target at the center of the normal image conversion area 7.
Thereby, the observation object can be positioned at the center of the normal image conversion area 7 without adjusting the optical system.

Now, as shown in FIG. 4A, it is assumed that the subject 4 to be observed is located at a location deviated from the center of the normal image conversion area 7.
In this case, if the normal image conversion area 7 is set at the position of the area 7 b, the subject 4 can be projected on the center of the normal image conversion area 7.
In order to perform this process, the imaging apparatus 1 resets the position of the normal image conversion area 7 stored in the normal image conversion apparatus 14 (FIG. 1) to the position of the area 7b in the following manner.

The area specifying device 13 includes area specifying means for specifying the position of the area 7b for the user.
The area designation device 13 acquires the position of the area 7b set by the user using this, and sends this to the normal image conversion device 14.
Then, the normal image conversion device 14 replaces the information specifying the range of the normal image conversion area 7 currently stored with the area 7b.
As the normal image conversion device 14 resets the position of the normal image conversion area 7, the numerical value input to the conversion equation for converting the fisheye image to the normal image also conforms to the reset position of the normal image conversion area 7. Change to what you did.
Thus, the normal image conversion device 14 includes conversion area setting means.

FIG. 4B is a diagram showing the reset normal image conversion area 7. As shown in the figure, the subject 4 is projected on the center of the normal image conversion area 7.
The normal image conversion device 14 extracts a portion of the fish-eye image 5 projected on the reset normal image conversion area 7 and converts it into a normal image.

FIG. 5 is a diagram showing an example of the operation unit 41 that designates the position of the normal image conversion area 7 by the area designation device 13.
In FIG. 1, the area specifying device 13 and the image display device 16 are illustrated as separate devices. However, in the imaging device 1, these are configured as one device, and the region when used as the region specifying device 13 is used. The designation mode and the image display mode when used as the image display device 16 can be switched.
On the screen 46, an area designation screen is displayed in the area designation mode, and an image after normal image conversion is displayed in the image display mode.
In the following description, it is assumed that the operation is performed in the area designation mode.

The operation unit 41 includes a marker movement button 43, a position determination button 44, a screen 46, and the like.
The screen 46 is composed of a display device such as a CRT, a liquid crystal, or a plasma display, for example, and the area projected on the image sensor 3 in the fisheye image 5 is displayed as a fisheye image.
A region that is not projected onto the image sensor 3 is indicated by being surrounded by a dotted line.

Further, a marker 47 is displayed on the screen 46 so as to overlap the fisheye image 5.
The marker 47 is a diagram having the same shape as the outer peripheral frame of the normal image conversion area 7, and is an element corresponding to the area 7b in FIG.
The marker moving button 43 is composed of buttons with up, down, left and right arrows displayed. When one of these buttons is pressed, the marker 47 is moved in the direction corresponding to the arrow on the screen 46. Move.

The user operates the marker moving button 43 so that the observation target, that is, the subject 4 is positioned at the center of the marker 47.
The position determination button 44 is a button for determining the position of the marker 47. When the position determination button 44 is pressed after the marker 47 is moved, the area specifying device 13 transmits the position of the marker 47 to the normal image converting device 14.

In this way, by resetting the normal image conversion area 7, the user can designate an area to be observed in the observable range.
Since the degree of distortion of the projected image varies depending on the position of the fish-eye image 5, the shape of the marker 47 also changes according to the distortion of the projected image depending on the position.

In the above, the shape of the normal image conversion area 7 is set in accordance with the shape of the image after normal image conversion. However, the shape is not limited to this, and various shapes can be adopted.
For example, as shown in FIG. 6, it is possible to set a normal image conversion area 7 over a horizontal field angle of 180 degrees.
In this example, the right end portion of the normal image conversion area 7 is in contact with the right end portion of the image sensor 3, and the left end portion is in contact with the left end portion of the image sensor 3.
For this reason, if the fish-eye image projected on the normal image conversion area 7 is subjected to normal image conversion, an area of 180 degrees in the horizontal direction can be observed from the fish-eye lens 10. However, in this case, a converted normal image having a horizontally long band shape is displayed on a part of the screen 46 in the vertical direction.
2 may be switched between a mode for displaying the area shown in FIG. 2 on the entire screen 46 and a mode for displaying the horizontal angle of view by 180 degrees.

The shape of the normal image conversion area 7 is set according to the shape of the image displayed after the normal image conversion.
That is, in the fisheye image 5 in the present embodiment, the projected image is compressed toward the outer periphery, so that the outer peripheral portion of the normal image conversion region 7 is narrower than the central portion of the fisheye image 5. .

When the imaging apparatus 1 according to this example is mounted on the side surface of the vehicle, the subject on the side surface can be displayed over an angle of view of 180 degrees.
Therefore, when this imaging device 1 is mounted on the front and rear sides and the left and right side surfaces of the vehicle, the entire periphery of the vehicle can be observed with the four imaging devices 1.
In this case, the normal image conversion device 14 and the image display device 16 can be shared by the four fisheye lenses 10 and the imaging camera 12.
The shape and arrangement of the normal image conversion area 7 described above are merely examples, and a plurality of normal image conversion areas 7 having an arbitrary shape are provided at arbitrary positions on the image sensor 3 according to the use of the image pickup apparatus 1. It is possible to provide.

Incidentally, a conventional in-vehicle imaging device generally uses a wide-angle lens having a horizontal field angle of about 100 degrees, and it is necessary to install two imaging devices on one side and eight imaging devices on the entire circumference. For this reason, the conventional imaging apparatus has a large cost burden, a mounting space has to be secured, and processing time is required for eight units.
On the other hand, according to the imaging device 1 of this embodiment, if one is installed on each side of the vehicle, the entire periphery of the vehicle can be photographed.

Although the case where the imaging device 1 is mounted on a vehicle has been described above, it can also be used for other purposes.
Here, the case where the imaging device 1 is used for indoor monitoring will be described as an example.
When the room is monitored using the image pickup apparatus 1, the entire room can be photographed by installing the image pickup apparatus 1 at a location described in FIGS. 7 (a) and 7 (b) below.

FIG. 7A is a diagram showing a plan view 31a and a side view 31b of a room having a hexahedral shape.
Since the imaging apparatus 1 has an angle of view of 180 degrees in the horizontal direction and an angle of view of 90 degrees or more from vertically downward to vertically upward, the entire indoor space is made an observable range when installed as shown in the figure. be able to.
That is, the horizontal direction of the room and the horizontal direction of the image pickup apparatus 1 are aligned on the ceiling of the upper center of the wall 32 of the image pickup apparatus 1, and the vertical downward direction of the image pickup apparatus 1 is aligned with the vertical downward direction of the room.

FIG. 7B shows a side view 35a of a room having a hexahedral shape and a side view 35b perpendicular to the side view 35a.
When the imaging apparatus 1 is installed as shown in the figure, the entire indoor space can be set as an observable range.
That is, the imaging device 1 is installed on either wall of the ceiling 36 so that the vertical direction of the imaging device 1 is aligned with the depth direction of the room so that the optical axis of the fisheye lens 10 is perpendicular to the ceiling 36.

As described above, when the imaging device 1 is installed at the ceiling of the wall or near the ceiling, the entire indoor space can be set within the observable range.
Even in the case of outdoor monitoring as well as indoors, the effect can be exhibited if the position of the subject to be monitored satisfies the observable range of the imaging apparatus 1.
The imaging device 1 is limited in the monitoring / observation range in principle, but is effective for applications that are not restricted by the generated limitation.

Next, a modification of the positional relationship between the image sensor 3 and the fisheye image 5 will be described with reference to FIG.
In the example of FIG. 8A, the diameter R of the fish-eye image 5 is longer than the horizontal length a and the vertical length b of the image sensor 3, and the center of the fish-eye image 5 and the image sensor 3 The center of the match.
Therefore, the central part of the fish-eye image 5 is an imaging region 6a, and the upper end portion, the lower end portion, and the left and right end portions are non-imaging regions 6b.

In this example, it is not possible to obtain an angle of view of 180 degrees in the horizontal direction, but a higher resolution can be obtained because the size of the fish-eye image 5 is larger than that described with reference to FIG.
The projection method based on this positional relationship is a method suitable for observing the vicinity of the center of the fish-eye image 5 with high resolution, although an angle of view of 180 degrees in the horizontal direction is not necessary.

In the example of FIG. 8B, the upper end portion and the lower end portion of the fisheye image 5 are the non-imaging area 6b, and the central portion of the fisheye image 5 is the imaging area 6a. In addition, a field angle of 180 degrees is secured in the horizontal direction.
This projection method based on the positional relationship is a method that is suitable for a case where an angle of view of 180 degrees is ensured in the horizontal direction and the upper end portion and the lower end portion of the fisheye image 5 do not need to be observed.
In the example of FIG. 8C, the lower end portion of the fisheye image 5 is a non-imaging region 6b. In addition, a field angle of 180 degrees is ensured in the horizontal direction, and a subject directly above the fisheye lens 10 can be projected in the vertical direction.
This projection method based on the positional relationship is a method suitable for securing a field angle of 180 degrees in the horizontal direction and observing a portion from the vicinity of the central portion of the fish-eye image 5 to the upper end portion.

The following effects can be obtained by the first embodiment described above.
(1) In order to project a portion of the fish-eye image 5 necessary for monitoring and observation onto the image sensor 3, the size of the normal image conversion area 7 is projected onto the entire image sensor 3. Can be larger. Therefore, the number of pixels in the normal image conversion area 7 increases, and the image quality of the image after normal image conversion can be improved.
(2) By changing the positional relationship between the fish-eye image 5 and the image sensor 3, the resolution after normal image conversion can be improved while using the conventional image sensor 3, and the manufacturing cost of the image pickup apparatus 1 is reduced. can do.
(3) By setting the number of pixels in the normal image conversion area 7 to be approximately the same as the number of pixels displayed on the image display device 16, it is possible to reduce the load required for image processing such as interpolation between pixels, and to perform high-speed image processing. Processing can be performed.
(4) Since only the portion of the fish-eye image 5 projected on the normal image conversion area 7 needs to be subjected to normal image conversion, the calculation load is reduced compared to the case where the entire area projected on the image sensor 3 is subjected to image processing. The image processing can be performed at high speed.
(5) By inscribing the fish-eye image 5 at both ends of the image sensor 3 in the horizontal direction, an angle of view of 180 degrees in the horizontal direction can be obtained. Therefore, when the imaging device 1 is mounted on a vehicle, it is possible to observe one entire side surface with one imaging device 1. Further, the entire circumference of the vehicle can be monitored and observed by installing one imaging device 1 on each of the front and rear sides and the left and right side surfaces of the vehicle.
(6) Since the position of the normal image conversion region 7 can be reset on the image sensor 3, the observation target can be specified from the entire fisheye image projected on the image sensor 3.

(B) Second Embodiment In the first embodiment, the observable range is fixed, but in the second embodiment, the positional relationship between the center of the fisheye image and the center of the image sensor can be changed. By adding a positional relationship adjustment mechanism to the imaging apparatus 1, the observable range can be changed.
When the observation object is not projected on the normal image conversion area 7, the observation object can be projected on the normal image conversion area 7 by moving the fisheye image 5 by the positional relationship adjusting mechanism.

Moreover, the part which is not projected on the image pick-up element 3 among the fish-eye images 5 can be projected on the image pick-up element 3 by moving the fish-eye image 5 with a position adjustment mechanism.
As a result, the entire object projected by the fisheye image can be set as the observable range as in the conventional camera.
In addition, since the projected image to be projected is larger than that of the conventional camera, an image with better image quality than that of the conventional camera can be obtained.

FIG. 9 is a block diagram illustrating an example of a configuration of the imaging apparatus 1a according to the second embodiment.
Note that the same reference numerals are given to the same components as those of the imaging device 1 described in the first embodiment, and description thereof is omitted as appropriate.

The positional relationship adjustment mechanism 11 is a mechanism that adjusts the positional relationship between the fisheye lens 10 and the imaging camera 12.
As a method of adjusting the positional relationship, for example, there is a method of mechanically changing the optical system using an electrostrictive element such as a servo mechanism or a piezo element.
More specifically, for example, a method of shifting the optical axis of the fisheye lens 10 by moving the fisheye lens 10 with a driving device such as a motor, or a prism is provided between the fisheye lens 10 and the imaging camera 12, and this prism is moved with the driving device. There is a method of bending the optical axis of the fisheye lens 10 by doing so.

Alternatively, the position of the fish-eye image 5 projected on the image sensor 3 may be changed by moving the image sensor 3 by attaching the image sensor 3 to a movable mechanism with the optical system fixed.
Alternatively, both the optical system and the position of the image sensor 3 may be changed.
In addition, since the fish-eye image 5 can be moved on the image sensor 3 by moving the optical system or the image sensor 3, the positional relationship adjusting mechanism 11 constitutes a moving unit.
Further, when the angle of the optical axis of the optical system is changed, the range of the subject projected by the fisheye lens 10 is also changed. Thus, the positional relationship adjustment mechanism 11 also constitutes a changing unit.

The parameter input device 17 is a device for inputting a movement amount for moving the fisheye image 5.
From the parameter input device 17, the user can designate an observation target while observing the fisheye image 5 projected on the image sensor 3.
The parameter input device 17 calculates the movement amount of the fish-eye image 5 from the positional relationship between the designated observation target and the normal image conversion area 7 and outputs this to the positional relationship control device 18 as digital information.

The positional relationship control device 18 is a device that acquires the movement amount of the fisheye image 5 input from the parameter input device 17 and controls the positional relationship adjustment mechanism 11 so that the fisheye image 5 moves by this movement amount. .
More specifically, for example, the movement amount acquired as digital information from the parameter input device 17 is converted into analog information, which is amplified and output to the positional relationship adjustment mechanism 11.

Next, an example of moving the fisheye image 5 will be described with reference to FIG.
FIG. 10A shows a state before the movement, and FIG. 10B shows a state after the movement.
In FIG. 10A, the projected image of the subject 4 (observation target) is projected on the upper side of the image sensor 3. For this reason, only a part of the subject 4 is projected onto the normal image conversion area 7.
In order to project the subject 4 onto the center of the normal image conversion area 7, the area 7 a indicated by the dotted line may be matched with the normal image conversion area 7.
Now, assuming that the area 7a is above the normal image conversion area 7 by the length L, if the fisheye image 5 is moved downward by L, the subject 4 is moved to the normal image conversion area as shown in FIG. 7 can be projected on the center.
Note that, in FIG. 10B, a portion where the fisheye image 5 was originally projected is indicated by a dotted line.

  Therefore, when the positional relationship between the fisheye image 5 and the image sensor 3 is the positional relationship shown in FIG. 10A, parameters are input so that the fisheye image 5 is moved downward by L in the imaging device 1a. Then, the positional relationship control device 18 operates the positional relationship adjustment mechanism 11 to move the fisheye image 5 downward by L. Thereby, the subject 4 can be projected on the center of the normal image conversion area 7.

  In the above example, since the subject 4 is above the normal image conversion area 7, the fisheye image 5 has been moved downward. In the case of being in the direction or in the downward direction, the fish-eye image 5 can be similarly moved to project the subject 4 onto the center of the normal image conversion area 7.

The above operation can be performed on the parameter input device 17 using an operation unit similar to the operation unit 41 shown in FIG.
Here, the configuration of the operation unit 41 will be described.
The user can move the marker 47 on the display device 47 by operating the position determination button 44.

The parameter input device 17 stores the position of the normal image conversion area 7 in advance, and when the user presses the position determination button 44, the movement amount of the fisheye image 5 is calculated from the positional relationship between the marker 47 and the normal image conversion area 7. The calculation result is output to the positional relationship control device 18.
As described above, the parameter input device 17 stores a movement amount calculation program for calculating the movement amount of the fisheye image 5, a central processing unit that executes the program, a movement amount calculation program, and other data. Equipment.

Since the diameter R of the fish-eye image 5 is larger than the length b of the image sensor 3 in the vertical direction, a part of the fish-eye image 5 is not displayed on the screen 46, but the marker 47 moves to an uncovered portion. can do.
Therefore, although only a part of the fisheye image 5 is projected onto the image sensor 3, the entire fisheye image 5 can be made an observable range.
Further, when the positional relationship adjusting mechanism 11 operates and the fisheye image 5 moves, the fisheye image 5 displayed on the fisheye screen 46 also moves accordingly.

Next, a procedure until the imaging device 1a (FIG. 9) adjusts the position of the fisheye image 5 and displays a normal image will be described with reference to the flowchart of FIG.
First, the parameter input device 17 acquires the position of the marker 47 (step 5).
This is to acquire the position where the marker 47 is displayed on the screen 46 when the user presses the position determination button 44.
Next, the parameter input device 17 determines whether or not the stored position of the normal image conversion area 7 matches the position of the marker 47 acquired in step 5 (step 10).

When the position of the normal image conversion area 7 matches the position of the marker 47 (step 10; Y), the normal image conversion apparatus 14 corrects the portion of the fish-eye image 5 that is projected on the normal image conversion area 7. An image is converted and output to the image display device 16 (step 25).
Then, the image display device 16 displays the image after normal image conversion on the display device (step 30).

On the other hand, when the position of the normal image conversion area 7 and the position of the marker 47 do not match (step 10; N), the parameter input device 17 acquires the position of the normal image conversion area 7 stored in advance and the position acquired in step 5 The amount of movement of the fisheye image 5 is calculated from the position of the marker 47 and output to the positional relationship control device 18 (step 15).
Next, the positional relationship control device 18 generates a control signal for driving the positional relationship adjustment mechanism 11 by the amount of movement input from the parameter input device 17 and outputs the control signal to the positional relationship adjustment mechanism 11.

The positional relationship adjustment mechanism 11 operates based on a control signal from the positional relationship control device 18 and adjusts the positional relationship between the fisheye image 5 and the image sensor 3 (step 20).
After the positional relationship adjusting mechanism 11 adjusts the positional relationship, the normal image conversion device 14 converts the portion of the fish-eye image 5 projected onto the normal image conversion region 7 into a normal image (step 25), and displays this as an image display. The device 16 displays (step 30).

In the second embodiment described above, the following effects can be obtained in addition to the same effects as those of the imaging device 1 described above.
(1) By moving the fisheye image 5 on the image sensor 3, a subject that is not located in the normal image conversion area 7 can be positioned in the normal image conversion area 7.
(2) By moving the fish-eye image 5 on the image pickup device 3, even though only a part of the fish-eye image 5 is projected on the image pickup device 3, it is the same as a conventional camera using a fish-eye lens. The entire fisheye image 5 can be set as an observable range. Moreover, the image quality is better than the conventional camera.
(3) Although the observable area is fixed in the imaging apparatus 1, the observable area can be changed in the imaging apparatus 1a.

The first embodiment and the second embodiment have been described above, but it is also possible to combine them.
That is, the imaging device 1 includes the parameter input device 17 of the imaging device 1a, and the observation target can be specified by combining the resetting of the position of the normal image conversion area 7 and the movement of the fisheye image 5.

It is a block diagram showing an example of composition of an imaging device concerning a 1st embodiment. In 1st Embodiment, it is explanatory drawing showing the state by which the fisheye image was projected on the image pick-up element. In 1st Embodiment, it is explanatory drawing which showed the example of installation to the vehicle of an imaging device. It is explanatory drawing about the reset of the normal image conversion area | region which concerns on 1st Embodiment. It is explanatory drawing which showed an example of the operation part which concerns on 1st Embodiment. In 1st Embodiment, it is explanatory drawing in the case of setting the normal image conversion area | region over a field angle of 180 degree | times. In 1st Embodiment, it is explanatory drawing about the case where the room is monitored with an imaging device. In 1st Embodiment, it is a figure for demonstrating the modification of the positional relationship of an image pick-up element and a fish-eye image. It is the block diagram which showed an example of the structure of the imaging device which concerns on 2nd Embodiment. In 2nd Embodiment, it is explanatory drawing about the example of a movement of the fish-eye image on an image pick-up element. In 2nd Embodiment, it is a flowchart for demonstrating the procedure until an imaging device adjusts the position of a fish-eye image and displays a normal image. It is explanatory drawing about the conventional camera using a fisheye lens.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 3 Imaging element 5 Fish eye image 10 Fish eye lens 11 Positional relationship adjustment mechanism 12 Imaging camera 13 Area | region designation | designated device 14 Normal image converter 16 Image display device 17 Parameter input device 18 Positional relationship control device

Claims (5)

A fisheye lens that forms a circular fisheye image;
An imaging device having a rectangular imaging region on which the fisheye image is projected;
Comprising
The diameter of the fish-eye image imaged on the said image pick-up element is larger than the length of the short side of the said imaging area, The imaging device characterized by the above-mentioned. The imaging apparatus according to claim 1, wherein a center point of the fish-eye image and a center point of the imaging region are arranged so as not to coincide with each other.   The imaging apparatus according to claim 1, further comprising a changing unit that changes a range of a fisheye image that the fisheye lens projects onto the imaging device. A vehicle comprising the imaging device according to claim 1, claim 2, or claim 3,
The vehicle having an imaging device, wherein the imaging device arranges a center point of the circular fish-eye image above a center point of the imaging region. Image acquisition means for acquiring a fisheye image projected on the image sensor as an image;
Distortion correcting means for correcting distortion in a predetermined region of the acquired image;
Image display means for displaying an image of a region in which the distortion is corrected;
A vehicle comprising the imaging device according to claim 4.

Images