JP2005175852A - Photographing apparatus and method of controlling photographing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photographing apparatus having a plurality of photographing optical systems with different photographing magnifications and capable of efficiently moving the direction of the line of sight of the photographing optical system with high photographing magnification into a direction in which an object is located. <P>SOLUTION: The photographing apparatus includes: a first photographing optical system (101) with first photographing magnification; and a second photographing optical system (102) with a second photographing magnification greater than the first photographing magnification. The second photographing optical system can be turned. The incidence pupil positions of the first and second photographing optical systems are located on the turning axis (301a) of the second photographing optical system. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for photographing a specific area using a plurality of imaging optical systems having different imaging magnifications. In particular, the present invention relates to a technique for alleviating a shift of a shooting area caused by a spatial distance between a plurality of different imaging optical systems.

  As a photographic device that shoots a specific place in the photographic area with different photographic magnifications using multiple cameras with different photographic magnifications, it is mounted on one camera with a low photographic magnification and a pan / tilt pan head. A method using a single camera with a high photographing magnification is known (for example, see Patent Document 1).

  FIG. 17 shows that a wide-angle camera 1701 having a wide-angle optical system captures a wide area (wide-area monitoring area 1704), and a specific subject 1703 existing in the wide-area monitoring area 1704 has a higher imaging magnification than the wide-angle camera 1701. A system for enlarging an image with a zoom camera 1702 capable of double operation is shown.

  The wide-angle camera 1701, for example, always photographs the wide area monitoring area 1704, which is a restricted access area, and detects a specific subject 1703 that enters the wide area monitoring area 1704.

  The position and size information of the specific subject 1703 is detected, and the detection result is transmitted to the zoom camera 1702. Then, the zoom camera 1702 is driven in the pan direction and the tilt direction, and zooms in to an appropriate zoom magnification to capture an enlarged image of the specific subject.

  The wide-angle camera 1701 detects a specific subject 1703 for each image, detects the subject position and size, and transmits the image to the zoom camera 1702, and the zoom camera 1702 is a position sequentially transmitted in time series. The information is converted into the pan / tilt angle, the magnification is determined from the size information, and the enlarged image of the specific subject, that is, the intruder, is tracked by controlling the camera platform and the zoom mechanism (not shown).

  The position information in the real space from a specific point in the image photographed by the camera is generally given by a matrix (hereinafter this matrix is referred to as a camera parameter), and each element of the camera parameter is an absolute coordinate in the real space, the camera's It is given by the camera coordinate stretched by the optical axis direction and its vertical plane, and the focal length of the camera.

  In addition, parameter determination requires at least six reference points (hereinafter referred to as landmarks) that are not on the same plane. In practice, parameters are optimized from many landmarks to improve accuracy. Yes.

  Although this method can obtain coordinates in a three-dimensional space with high accuracy, it requires a lot of effort to obtain camera parameters for both the wide-angle camera and the zoom camera, and the camera parameters are obtained again each time the camera is moved. There must be.

  However, when used for the purpose of monitoring as shown in FIG. 17, it is desirable that a camera is installed above the ground regardless of whether it is indoors or outdoors so that an intruder moving on the ground surface can be seen from a wide range. In this case, the position information of the intruder, that is, the specific subject 1703 can often be approximated as a coordinate point on the two-dimensional space as a point on the ground plane.

  That is, the landmarks A, B, and C are set at three points on the real space that are not on the same straight line, and by checking the absolute coordinates of each landmark in the real space and the camera coordinates of the camera photographed image in advance, Position information can be obtained even if there is an intruder at an arbitrary place on the ground plane.

  The position information acquired in this way is transmitted to the zoom camera 1702, and the zoom camera performs pan / tilt so that the intruder position coordinates are arranged in the optical axis direction. Further, if the zoom camera is limited to panning and tilting operations on a specific subject detected by a wide-angle camera, there is a method that does not involve absolute coordinates of the subject in real space.

  FIG. 18 shows two cameras arranged at a distance D such that the wide-angle camera 1701 does not interfere with the pan / tilt of the zoom camera 1702. In the initial state of the zoom camera 1702, the zoom camera viewing direction 1802 (optical axis direction) is set to be parallel to the optical axis direction 1801 of the wide-angle camera 1701.

  When the specific subject 1703 is detected by the wide-angle camera 1701, an angle θ formed by the wide-angle camera optical axis 1801 and a straight line extending from the wide-angle camera 1701 to the specific subject 1703 is obtained.

  This angle θ is projected onto a predetermined orthogonal axis (provisionally called a pan axis and a tilt axis), and a pan angle θp and a tilt angle θt are obtained and transmitted to the zoom camera 1702. The zoom camera 1702 moves the line-of-sight direction by the pan angle θp and the tilt angle θt with respect to the zoom camera initial line-of-sight direction 1802 and captures the specific subject 1703.

As described above, for a device that uses a plurality of cameras with different imaging magnifications to capture a specific area in space at different shooting magnifications, the camera gaze direction is controlled once using the absolute coordinates of the specific area in real space. And a method for determining the angle between the camera and the specific area from the video in which the specific area is detected and controlling the camera viewing direction.
JP 2000-295600 A (paragraph number 0019,0022, FIG. 22)

  However, in the conventional method, many problems described below remain. First, when the camera viewing direction is controlled using the absolute coordinates of the specific subject in the real space as described with reference to FIG. 17, at least three landmarks are determined, and in the wide-angle camera, the specific subject 1703 photographed by the camera is determined. An operation for obtaining the absolute coordinates of the intruder by obtaining the internal division ratio between the position and the three landmarks on the image and obtaining the points that divide the absolute coordinates of the three landmarks in the real space by the internal division ratio. Is necessary.

  In a zoom camera, it is necessary to calculate the pan / tilt angle of the zoom camera from the absolute coordinates of the intruder.

  In addition, these calculations are sufficiently shorter than the scene sampling time of each scene image obtained in time series like a moving image.

  Furthermore, there is a problem that requires a lot of calculation load and labor, for example, when all the initial settings are required when the installation location is changed in either the wide-angle camera or the zoom camera.

  FIG. 20 is a diagram showing a state where an obstacle 2003 having a size larger than the specific subject 1703 exists in the monitoring area shown in FIG.

  The obstacle 2003 causes a blind spot 2001 and a blind spot 2002 in the wide-angle camera 1701 and the zoom camera 1702, respectively, but the blind spot differs depending on the installation location of the camera.

  For example, in FIG. 20, the wide-angle camera 1701 can detect the specific subject 1703 and calculates the position information thereof and transmits it to the zoom camera 1702, but the specific subject 1703 is positioned within the blind spot 2002 area due to the obstacle 2003. Therefore, the zoom camera 1702 cannot photograph the specific subject 1703 due to the obstacle 2003 being shaded.

  Such a problem is more likely to occur as the installation positions of the wide-angle camera 1701 and the zoom camera 1702 are farther apart, and an enlarged image of the intruder can be captured only by installing a new zoom camera in another place where the obstacle 2003 does not become a blind spot. There is no way to do it.

  Next, as shown in FIG. 18, the wide-angle camera optical axis 1801 and the specific subject 1703 are separated by two cameras arranged at a distance D that does not interfere with the pan / tilt of the zoom camera 1702. A problem in the case of controlling the zoom camera 1702 by transmitting the formed angle θ will be described.

  FIG. 19 is a view of FIG. 18 as viewed from above. As described above, the wide-angle camera 1701 and the zoom camera 1702 are separated from each other by a sufficient distance D that does not interfere with the rotation of the zoom camera 1702 (hereinafter, D is referred to as the baseline length of the wide-angle camera 1701 and the zoom camera 1702). In the initial state, the wide-angle camera optical axis 1801 and the zoom camera initial line-of-sight direction 1802 (optical axis) are set in parallel.

  Now, when the specific object 1703 that is an intruder in the monitoring range area is detected by the wide-angle camera 1701, a line (wide-angle camera viewing direction 1804) extending from the center (principal point position) of the wide-angle lens 1901 to the specific object 1703 and the optical axis. Assume that an angle θ formed by 1801 is obtained.

  The angle θ is transmitted to the zoom camera 1702 and rotates about the rotation axis 1902 by the angle θ with respect to the optical axis direction 1802 in the initial state. However, the optical axis (zoom camera line-of-sight direction 1803) of the zoom camera 1702 after the rotation of the angle θ is separated from the wide-angle camera line-of-sight direction 1804 by d (= Dcos (θ)), and the zoom camera field of view 1903 by zooming is photographed. The higher the magnification, the narrower.

  For example, when the zoom camera 1702 having a length of 40 cm in the long side direction and the wide-angle camera 1701 are rotated at intervals of 60 cm (D = 60 cm) so that they do not collide with each other, d is the maximum value (= D). In the front direction (θ = 0 deg.), A shift of approximately one person's shoulder width (about 60 cm) occurs, and there is a problem that an optimal enlarged image cannot be obtained by the zoom camera.

  Further, FIG. 21 is a view in which a wide-angle camera 1701 and a zoom camera 1702 are installed vertically, and FIG. 22 is a view of FIG. 21 as viewed from above. This figure is different from FIG. 18 in that the camera is installed vertically and that the pan rotation axis 1902 of the zoom camera is located at a distance E from the principal point position of the wide-angle lens 1901 on the horizontal plane.

  The operation is the same as described above. When an intruder in the surveillance area is detected by the wide-angle camera, the angle θ formed by the viewing direction of the wide-angle camera and the optical axis is obtained, and the zoom camera uses the optical axis in the initial state around the rotation axis. Rotate by an angle θ relative to the direction.

  However, as indicated by e in the figure, the wide-angle camera line-of-sight direction 1804 and the zoom camera line-of-sight direction 1803 are shifted by a distance e = Esin (θ). Hereinafter, the relationship between the shift amount e and the zoom camera field of view by zooming is the same as described above, and there is a problem that an optimal enlarged image cannot be obtained by the zoom camera.

  In order to solve the above-described problem, the first configuration of the photographing apparatus of the present invention is a first photographing optical system (for example, a wide-angle optical system unit) having a first photographing magnification, and is larger than the first photographing magnification. A second imaging optical system (for example, a zoom optical system unit) having a second imaging magnification, the second imaging optical system is rotatable, and the first and second imaging optical systems The entrance pupil position is arranged on the rotation axis of the second photographing optical system.

  Here, it is preferable that the second photographing optical system can be rotated around an axis orthogonal to the rotation axis.

  Further, the second photographic optical system may be variable in magnification, and the second photographic optical system may be moved in the optical axis direction in accordance with the variable magnification.

  Control means for determining a shooting direction of the second shooting optical system based on a shot image obtained by using the first shooting optical system and rotating the second shooting optical system in the determined shooting direction. It is preferable to provide it.

  In addition, it is preferable that the control unit detects a specific subject area from the photographed image and determines a photographing direction of the second photographing optical system so as to include the specific subject area.

  Further, there is provided control means for determining a second photographing magnification based on a photographed image obtained using the first photographing optical system and setting the second photographing optical system to the determined second photographing magnification. Further, it is preferable that the control means detect a specific subject area from the photographed image and determine the second photographing magnification according to the specific subject area.

  The first configuration of the control method of the photographing apparatus according to the present invention includes a first photographing optical system (for example, a wide-angle optical system unit) having a first photographing magnification, and a second photographing magnification larger than the first photographing magnification. A second imaging optical system (for example, a zoom optical system unit), the second imaging optical system is rotatable, and the entrance pupil positions of the first and second imaging optical systems are A method for controlling an imaging apparatus disposed on a rotation axis of a second imaging optical system, the method comprising: initializing the direction of the second imaging optical system; and using the first imaging optical system Detecting a specific subject area from within the captured image, determining a direction of the second imaging optical system so as to include the detected specific object area, and a second imaging optical system in the determined direction And a step of rotating around the rotation axis.

  Here, the second photographing optical system is capable of zooming, a step of determining the second photographing magnification so as to include the detected specific object region, and the second photographing optical system determined by the second photographing optical system. It is preferable to have a step of setting the magnification and a step of moving the second imaging optical system in the optical axis direction in accordance with the magnification of the second imaging optical system.

  It is preferable that the steps other than the step for initializing the direction are sequentially repeated.

  According to the first configuration of the imaging apparatus of the present invention described above, since the entrance pupil positions of the first and second imaging optical systems are arranged on the rotation axis of the second imaging optical system, the first The position information related to the relative position between the photographing optical system and the subject can be applied to the second photographing optical system without correction, and the second photographing optical system can be quickly and accurately rotated to the photographing position. be able to. In addition, since the photographing magnification of the second photographing optical system is higher than the photographing magnification of the first photographing optical system, a detailed image of the subject can be obtained.

  Further, when the second imaging optical system can be changed in magnification, the second imaging optical system is advanced and retracted in the optical axis direction by the position changing means, and the entrance pupil position is arranged on the rotation axis. The second photographic optical system can be quickly and accurately moved to the photographing position, and the subject can be photographed at a variable magnification according to the photographer's preference.

  According to the first configuration of the control method of the imaging apparatus of the present invention, the entrance pupil positions of the first and second imaging optical systems are arranged on the rotation axis of the second imaging optical system, A step of detecting a specific subject region from a captured image obtained by using the first photographing optical system, a step of determining a direction of the second photographing optical system so as to include the detected specific subject region, and the determination Since the photographing apparatus is controlled by the step of rotating the second photographing optical system around the rotation axis in the direction as described above, the movement of the second photographing optical system in the line-of-sight direction can be controlled efficiently and accurately. In addition, since the photographing magnification of the second photographing optical system is higher than the photographing magnification of the first photographing optical system, a detailed image of the subject can be obtained.

  In addition, when the second photographing optical system can be changed in magnification, there is a step in which the second photographing optical system advances and retracts in the optical axis direction and the entrance pupil position is on the rotation axis. It is possible to efficiently control the movement of the system in the line-of-sight direction and to photograph the subject at a variable magnification according to the photographer's preference.

  Embodiments of the present invention will be described below with reference to the drawings.

  A first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a sectional view in the optical axis direction of a wide-angle optical system unit (first imaging optical system) and a zoom optical system unit (second imaging optical system) in an imaging apparatus that is Embodiment 1 of the present invention.

  The arrows in the figure are two of the three coordinate axes and the XYZ axes in the actual three-dimensional space. The two orthogonal axes on the plane horizontal to the ground are the XY axes, and the axis perpendicular to the ground is the Z axis. Of the two axes (XY axes) horizontal to the ground, the axis parallel to the optical axis of the wide-angle optical system unit 101 of the photographing apparatus installed on the horizontal plane is defined as the X axis, and the axis orthogonal to the optical axis is defined as Y. The axis.

  FIG. 1 shows an XZ plane projection image of the photographing apparatus among the XYZ axes defined as described above. That is, a projection view when the photographing apparatus is viewed from the side is shown. However, in order to clarify the internal configuration in FIG. 1, the outer frame of the apparatus is not shown, but a plurality of optical units to be described later are mounted with the common substrate of the apparatus as an absolute coordinate system.

  In FIG. 1, reference numeral 101 denotes an optical system having a low photographing magnification and a wide-angle optical system unit having a wide-angle visual field range. The imaging magnification of the wide-angle optical system unit 101 is determined optimally depending on the size of the monitoring area and the installation location of the apparatus. Note that the wide-angle optical system unit 101 may be a single-focus optical system or a variable-magnification optical system that changes the focal length according to the range of the monitoring area and performs a shooting operation at an optimal shooting magnification. However, in this embodiment, it is desirable to fix the photographing magnification during the monitoring operation, and in this embodiment, a single focus optical system is used for the sake of simplicity.

  The wide-angle optical system unit 101 may be mounted on a pan head so that it can be driven in the pan direction and the tilt direction, or may be a fixed type. However, in this embodiment, it is desirable to fix the direction of the optical axis during the monitoring operation, and in order to simplify the explanation, this embodiment does not have a pan / tilt mechanism, and the optical axis of the wide-angle optical system unit. The direction is set by changing the installation direction of the apparatus.

  The wide-angle optical system unit 101 includes, in order from the object side, a wide-angle lens 1011 including a super-wide-angle lens such as a fisheye lens, an aperture 103, and an image sensor 104 (for example, a CCD or CMOS image sensor). In general, components necessary for taking a digital moving image, such as a lens barrel, an image sensor driving circuit, and an image processor are also included. Further, the entrance pupil 1012 of the wide-angle optical system unit 101 is located closer to the object side than the stop 103.

  Reference numeral 102 denotes a zoom optical system unit. In relation to the single-focus optical system in the wide-angle optical system unit 101 described above, the zoom optical system unit 102 is a single-focal length optical system as long as it can display a photographed image having a photographing magnification higher than that of the wide-angle optical system unit 101. However, in order to obtain an optimal magnified image corresponding to the distance from the apparatus to the specific subject and the size of the specific subject, a variable focal length optical system is desirable.

  Therefore, the zoom optical system unit 102 of this embodiment is configured by a variable focal length optical system. The zoom optical system unit 102 has almost the same component configuration as the wide-angle optical system unit 101, and includes a zoom lens 1021, an aperture 103, and an image sensor 104 in order from the object side. It also includes a processor. Note that the zoom optical system unit has a control device that drives and controls the zoom lens 1021 and a drive mechanism that drives the zoom optical system unit in the pan direction and the tilt direction. Not shown. Further, the entrance pupil 1022 of this zoom optical system unit is located closer to the object side than the stop 1022.

  Now, the wide-angle optical system unit 101 and the zoom optical system unit 102 are arranged apart from each other by an interval H in the Z-axis direction (vertical direction). The interval H drives the zoom optical system unit 102 in the pan direction and the tilt direction. Even in this case, the minimum distance that does not interfere with the wide-angle optical system unit 101 is set, and in this embodiment, the wide-angle optical system unit 101 is disposed at the upper part in the vertical (Z-axis) direction, and the zoom optical system unit 102 is disposed at the lower part. In the previous initial state, the optical axes of both units are set to be parallel.

On the other hand, the arrangement relationship between the two units in the X-axis direction in FIG. 1 is such that the entrance pupil positions 1012 and 1022 are at the same position (the X coordinate is X ₀ ).

  Here, the positional relationship of both units in a plane including a horizontal plane (XY axis plane) will be described with reference to FIG. FIG. 2 is a projection view of the wide-angle optical system unit 101 and the zoom optical system unit 102 when viewed in the Z-axis direction.

The photographing apparatus is arranged so that the entrance pupil positions of the wide-angle optical system unit 101 and the zoom optical system unit 102 coincide with each other on the coordinates (X ₀ , Y ₀ ) on the XY plane, and both units in the initial state. These optical axes are made to coincide with each other on the XY plane as shown in the figure.

  As described above, the arrangement relationship between the wide-angle optical system unit 101 and the zoom optical system unit 102 is such that the entrance pupil positions of both units coincide on the XY plane and are arranged as close as possible in the direction perpendicular to the XY plane. The optical axes of both units coincide with each other on the XY plane and are parallel with each other on the XZ plane.

  Next, a panning mechanism for driving the zoom optical system unit 102 in the pan direction and the tilt direction will be described with reference to FIG. FIG. 3 is a view of the zoom optical system unit 102 and the panning mechanism as seen on the XZ plane.

  In the figure, reference numeral 301 denotes a panning motor for panning the viewing direction of the zoom optical system unit 102. Actually, the motor is controlled by a stepping motor or the like with high precision, and a gear mechanism that transmits the rotation by the motor to the zoom optical system unit at an appropriate rotation ratio or a zoom optical system lens barrel A member such as a tripod base that is fixed and connected to a motor or gear mechanism, a fixing member that fixes the motor to the imaging device substrate, etc. are also indispensable. Although a vibration mechanism may be mounted, in this embodiment, all of them including 301 are referred to as a panning motor.

  As shown in FIG. 3, the panning motor 301 is attached to the zoom optical system unit 102 so that the entrance pupil 1022 is located on a rotation axis 301a extending in a direction orthogonal to the optical axis (on a straight line including the rotation axis 301a). The zoom optical system unit 102 rotates in the direction of the arrow in the figure when the panning motor 30 is driven.

  That is, panning by the panning motor 301 is performed in parallel with the XY plane, with the entrance pupil position as the rotation center, and the viewing direction of the zoom optical system is also translated in the XY plane.

  FIG. 4 is a view of FIG. 3 as viewed from the Z-axis direction. The zoom optical system unit 102 and the tilt mechanism will be described with reference to FIG. A circle indicated by a dotted line in FIG. 4 represents the panning motor 301 described above, and the panning motor 301 is arranged below the zoom optical system unit 102 in this figure, and its rotation axis is the XY coordinates (X0, Y0) of the entrance pupil position. ).

  Reference numeral 401 in the figure denotes a tilting motor for tilting the viewing direction of the zoom optical system unit 102. The configuration of the tilting motor 401 is almost the same as that of the panning motor 301.

  As shown in FIG. 4, the tilting motor 401 is attached to the zoom optical system unit 102 so that the entrance pupil 1022 is positioned on a rotation axis 401a extending in the direction orthogonal to the optical axis. The tilting motor 401 is driven to rotate in the direction of the arrow in the figure. That is, the tilt by the tilting motor 401 is centered on the entrance pupil position, and the plane including the tilt direction is orthogonal to the XY plane.

  Here, the operation of the imaging apparatus of the present embodiment will be described with reference to FIG. FIG. 5 is a top view of the photographing apparatus, and corresponds to FIGS. 2 and 4 described above. First, in the initial state, the optical axes of the wide-angle optical system unit 101 and the zoom optical system unit 102 coincide with each other in a direction indicated by a dotted line in the figure, and a reference direction for obtaining an angle θ which will be described later. And

  Next, when the wide-angle optical system unit 101 detects a specific subject 1703 that is an intruder in the monitoring visual field region, the imaging apparatus detects the incident angle from the object to the optical system, that is, the optical axis with the entrance pupil position as the origin. Find the angle θ to make.

  For ease of explanation, it is assumed that the angle θ is represented only by the panning angle, and the panning angle θ is immediately transmitted to the zoom optical system unit 102, and the zoom optical system unit 102 receives the panning angle θ and receives the zoom optical system line-of-sight direction. Is rotated by θ, and the rotation speed of the panning motor 301 is determined to rotate about the rotation axis 301a including the entrance pupil position by the angle θ.

  Assuming that the field of view by zooming is the same as that of the above-described conventional example, in this embodiment, the viewing direction of the wide-angle optical system unit 101 is in the zoom optical system field of view 501 (the imaging region of the zoom optical system unit 102) indicated by hatching in the figure. It can be seen that the specific subject 1703 is completely captured.

  Next, the signal processing mode of the present invention will be described with reference to FIG. FIG. 6 is a block diagram for explaining the flow of signal processing of the present invention. In the figure, reference numeral 601 denotes a wide-angle optical unit having a wide monitoring area as a visual field range, and includes a wide-angle lens including a super-wide-angle lens such as a fisheye lens, and optical components such as a diaphragm and a focus adjustment mechanism as described above.

  Reference numeral 602 denotes an image sensor, which is arranged so that the optical axis of the wide-angle optical unit 601 is the center of the image sensor or the center of the imaged image area from the image sensor and is a plane perpendicular to the optical axis.

  The wide-angle optical unit 601 forms an image of the monitoring area on the imaging surface of the image sensor 602, and performs signal processing such as A / D conversion, exposure correction, auto white balance, gradation correction, and color correction by an image signal processing unit (not shown). Then, a video signal is sent to the image memory 603 and the subject extraction unit 604.

  The image memory 603 is a video signal used for subject extraction processing, which will be described later. The image memory 603 stores a video image at a time t past as a still image, or stores a moving image from the past τ time to the present. Depends on the method.

  The subject extraction unit 604 is a signal processing unit that detects a specific subject from the captured image in the monitoring area. A well-known subject extraction method will be briefly described below.

  FIG. 7A shows a state in which a certain office is monitored by an imaging device. Here, the area shown in FIG. 7 (a) is usually an area that is not allowed for persons other than those involved, and when an object or person enters the area, an alarm is issued to alert the monitoring staff and the like, and a video is recorded. Etc. are necessary.

  Therefore, subject extraction described below is performed for intruders in the area. The first well-known method is usually called a background difference method, and the area in the monitoring area is stored in a normal state as a still image. This still image is often referred to as a “background image” and is also referred to in this embodiment.

  Therefore, FIG. 7A shows a state in which the monitoring area is in a normal state, and represents a “background image” taken at a certain time t at the same time. Also, since the background image is not constant due to environmental fluctuations such as illumination light, for example, it is often updated at regular intervals.

  Now, after obtaining the background image as shown in FIG. 7A, the moving image video in the monitoring area is photographed in time series and enters the monitoring state. For the monitoring area video (hereinafter referred to as the current image) that is sequentially captured, the difference from the “background image” corresponding to the frame coordinates is calculated for each frame. When the monitoring area is in a normal state, the current image is not different from the background image, and the difference in the frame is almost equal to zero if the influence of noise or the like is excluded.

This can be expressed as an expression:
d (x, y) = (Ir (x, y) −Ib (x, y)) ² −I)
When d (x, y) <Th, d (x, y) = 0−II)
When d (x, y)> = Th, d (x, y) = 1
Here, Ir (x, y) is a signal value at the coordinates (x, y) of the current image, Ib (x, y) is a signal value at the coordinates (x, y) of the background image, and Th is a predetermined threshold value. is there.
In equation (I), the square of the difference between the original image and the background image is calculated. The difference signal is aligned with a positive sign, and the absolute value of the difference | Ir (x, y) −Ib (x, y) | may be used instead of the square.

  In order to eliminate the influence of noise or the like contained in the image in the formula II), the difference is set to 0 when the difference is equal to or smaller than a certain value. In addition, the formula II) also has the tolerance until the next background image is acquired even when the environment changes due to illumination or the like.

In other words, even if there is some intensity of sunlight due to cloud visibility, etc., and the current image becomes slightly darker than the background image, it plays the role of not counting as an intruder by the threshold Th.
The formula I) II) is calculated for all pixels of the image, and for each pixel, the pixels above the threshold and the pixels below the threshold are represented by a bitmap plane.

  FIG. 7 (b) shows a situation where a human has entered the monitoring area, and the image signal of FIG. 7 (b) and the image signal of FIG. The difference is calculated every time.

  FIG. 7 (c) is a bitmap plane in which each pixel in the image is represented by white in the pixel above the threshold and black in the pixel below the threshold according to the formula II).

  Now, the operation of the photographing apparatus when the background difference method is used will be described. The wide-angle optical system unit 601 that sets the visual field range in the determined monitoring area first acquires a background image before entering the monitoring operation. For example, when a start switch or the like is turned on from a user interface (not shown), the wide-angle optical system unit 101 captures the monitoring area as a moving image for several seconds, and all frame images output from the image sensor 602 are temporarily stored in the image memory 603. Remembered.

  Next, the frame image stored in the image memory 603 is read again by a CPU (not shown) or the like, and an average image of each frame image taken for several seconds is created to average noise mixed in each frame, and a background image is created. Is stored in the image memory 603 again.

  When the recording of the background image in the image memory 603 is completed, monitoring starts, and the photographed image by the wide-angle optical system unit 101 is input to the subject extraction unit 604, and subject extraction is performed by the method described above.

The subject extraction unit also outputs an intruder detection result at the same time. For example, by accumulating d (x, y) in the formula II) for all pixels in the frame, the total number of pixels different from the background can be obtained.
s = ΣyΣxd (x, y)
If this is divided by the total number of pixels in the frame, the ratio to the entire screen is known, but it is determined that an intruder exists only when there is an intruder of a certain size or larger in the monitoring area. May be.

  The above is the basic of subject extraction by the background difference method. In addition, the illumination light and the person are discriminated from the distribution of the difference square value obtained by the formula (I), the background image average density is the current image average density, etc. The difference square value of the formula (I) is obtained after correction, and isolated points and minute areas are deleted from the white point connectivity of the difference bitmap, and the maximum connected area is determined as an intruder. Attempts have also been made to devise.

  In addition to the background difference method, the inter-frame difference method is a method often used in subject extraction, and will be described briefly. Similar to the background difference method, the inter-frame difference method is also a method for taking a difference between a past video and a current time video, but refers to a monitoring region image in a normal state in advance such as a background image. Instead of this, it is a method in which an image past a certain time from the current time is stored and a difference is obtained.

Expressed as a formula:
d (x, y) = (I (x, y, t) −I (x, y, t−τ)) ² −I)
When d (x, y) <Th, d (x, y) = 0−II)
When d (x, y)> = Th, d (x, y) = 1
Here, I (x, y, t) is a signal value at the coordinates (x, y) of the image at the current time (t), and I (x, y, t−τ) is a time τ from the current time (t). Only the signal value at the coordinates (x, y) of the past image, Th is a predetermined threshold value. The advantage of inter-frame difference is that it does not require the effort to acquire a background image in advance, and the reference image for which the difference is taken is a relatively short time (τ), so that it is not easily affected by environmental fluctuations due to the illumination described above. It is.

  On the other hand, the disadvantage of the interframe difference is the difference from the past video with a short time (τ) interval, so if the intruder is stationary in the same posture for more than time (τ), the difference is zero and the intruder area cannot be detected. There are also problems such as.

  However, even for this point, a template is created for the region once extracted by the inter-frame difference, and even if the extracted region disappears due to the intruder's stillness, if the same region matches the image stored in the template, it is still. Attempts have also been made to determine that the region is the subject region.

  Regarding the operation of this method, monitoring is started immediately when a start switch or the like is turned on from a user interface (not shown). The frame image is stored in the image memory 603 for the first τ time.

  After τ time, an image at the current time is input to the subject extraction unit 604 and stored in the image memory 603, and a frame image before τ time is read from the image memory 603 and input to the subject extraction unit 604. The difference between the image and the image before τ time is taken.

  As described above, the subject extraction by the background difference method and the inter-frame difference method has been described. However, in this embodiment, the method is not limited as long as the intruder can detect from the photographed image from the wide-angle optical system unit. When there is no intruder in the normal monitoring area, that is, when no intruder is detected by the subject extracting unit 604, the captured image of the monitoring area is displayed on the image display unit 614 via the image composition unit 613.

  On the other hand, when an intruder is detected by the subject extraction unit 604, the aforementioned bitmap image is sent to the subject direction detection unit 606 and the scaling factor determination unit 605. Hereinafter, the subject direction detection unit 606 and the scaling factor determination unit 605 will be described with reference to FIG.

  FIG. 8 (a) is a bitmap image (see FIG. 7 (c)) of the intruder in which the subject is detected, and the specific subject region is shown in black and the background region is shown in white for easy viewing. A bitmap image (see FIG. 8A) input to the subject direction detection unit 606 and the scaling factor determination unit 605 creates a rectangular region including a specific subject region as indicated by 801. In practice, XY coordinates as shown in the figure may be defined in the bitmap plane, and the upper left corner coordinates (Xs, Ys) and the lower right corner coordinates (Xe, Ye) may be stored.

  Next, the subject direction detection unit 606 obtains the X-axis and Y-axis components (Dx, Dy) of the distance D from the image center o to the rectangular center c as shown in FIG. 8B.

  Here, the image center o coincides with the optical axis of the wide-angle optical unit, the image Y coordinate direction is parallel to two points connecting the entrance pupil of the imaging apparatus, and the X axis direction connects the entrance pupil of the imaging apparatus. It is set to be perpendicular to the two points. Furthermore, the Y-axis direction is parallel to the plane including the tilt direction of the zoom optical system unit, and the X-axis direction is parallel to the plane including the panning direction.

  This relationship is achieved by adjusting the arrangement of the optical unit and the area sensor when the apparatus is manufactured. Therefore, the distance from the image center o to an arbitrary point in the image corresponds to the angle formed when viewing the object point with the entrance pupil position as the origin, and is unambiguous depending on the imaging magnification (or focal length) of the wide-angle optical system. When (Dx, Dy) is obtained as shown in FIG. 5B, the pan angle θp is uniquely obtained from Dx, and the tilt angle θt is obtained from Dy.

  The subject direction detection unit 606 obtains the pan angle θp and tilt angle θt from (Dx, Dy) using, for example, mathematical formulas, a look-up table, and the like, and outputs them to the optical system rotation control unit 609 in FIG. .

  The optical system rotation control unit 609 controls the necessary rotation speed, rotation direction, rotation speed, etc. from the pan angle θp and tilt angle θt for the optical system rotation drive unit 610 composed of the panning motor and tilting motor described above. The zoom optical system unit performs panning and tilting.

Next, the scaling factor determination unit 605 will be described. First, as shown in FIG. 8B, a width w and a height h of a rectangular area 801 that includes a specific subject area on a bitmap plane in which a subject is detected are obtained. This uses the above-mentioned (Xs, Ys) and (Xe, Ye).
w = Xe-Xs
h = Ye-Ys
Next, the magnifications (Mw, Mh) of the width and height are calculated using the width W and the height H when a predetermined specific subject is enlarged and displayed by the zoom optical system unit. The width W and the height H may be the entire image sensor effective area of the zoom optical system unit, or may be within the effective area with some margin.

  FIG. 8C shows an enlarged photographed image of an intruder by the zoom optical system unit. In the figure, the solid line frame is the image sensor effective area, and the enlarged display area (W, H) indicated by the dotted line frame inside the effective area. ).

Then, when the zoom optical system unit is enlarged, the magnification M when zooming with the actual zoom optical system unit is set to the smaller one of Mw and Mh so that the subject does not protrude from the display area in the vertical or horizontal direction.
Mw = W / w
Mh = H / h
M = Min (Mw, Mh)
The variable magnification M is sent to the optical system magnification control unit 609. The optical system magnification control unit 609 uses the focal length fw of the known wide-angle optical system unit in advance to use the focal length fz (= M · fw) is calculated, and the optical system magnification driving unit 608 is instructed to have the focal length fz.

  Next, a method for controlling the photographing apparatus will be described with reference to FIG. Here, FIG. 25 is a flowchart showing a procedure of operation control of the photographing apparatus.

  First, in step 1, the optical axis direction of the zoom camera optical system unit 102 is initialized (the optical axis is overlapped with the zoom camera initial line-of-sight direction shown in FIG. 5), and in step 2, the wide-angle optical system unit 101 is driven. Take a picture of the surveillance area.

  In step 3, the presence or absence of an intruder in the monitoring area is detected. This intruder detection is performed by the subject extraction unit 604 using the background difference method or the inter-frame difference method, and if an intruder is detected, the process proceeds to step 4.

  In step 4, the pan angle, tilt angle, and zoom magnification of the zoom optical system unit 102 are determined. Specifically, the subject direction detection unit 606 and the scaling factor determination unit 605 determine based on the above-described bitmap image input from the subject extraction unit 604.

  In step 5, the optical system rotation drive unit 610 is driven based on the pan angle and tilt angle calculated by the subject direction detection unit 606, and the zoom optical system unit 102 is within the zoom optical system field area 501 (see FIG. 5). Move to a position (shooting position) where an intruder is included. The zoom optical system unit 102 that has moved to the shooting position starts the shooting operation after the zoom lens 1021 is driven to a position where the zoom ratio calculated by the zoom ratio determination unit 605 is obtained.

  While the intruder is moving in the monitoring area, steps 3 to 5 are repeatedly performed, and the zoom optical system unit 102 is driven so as to follow the movement of the intruder.

  As described above, when the zoom optical system unit is directed toward the specific subject and photographed at an appropriate magnification, the photographed image is sent to the image composition unit 613. The image composition unit mixes the image captured by the wide-angle optical system unit and the image captured by the zoom optical system unit. As a type of mixing, for example, a case as shown in FIG. 9 can be considered.

  FIG. 9A shows only the entire monitoring area on the full screen. (B) displays the entire surveillance area on the entire screen and displays an enlarged image of the intruder in a small window. (C) displays a magnified image of an intruder on the entire screen and displays the entire monitoring area in a small window. (D) displays only the magnified image of the intruder in full screen.

  These display methods can be selected by a monitor using a window display operation unit (not shown). Although not shown in FIG. 6, each video may be connected to an external storage device and recorded.

  A second embodiment of the present invention will be described. In the first embodiment, by arranging the entrance pupil position on the rotation axis of the zoom optical system unit, the panning angle to the specific subject obtained by the wide-angle optical system unit completely matches the panning angle of the zoom optical system unit. In this embodiment, the subject direction detection unit 606 of the first embodiment is improved.

  FIG. 10 is a diagram showing the present embodiment, where O represents the origin of the coordinate system of the place where the photographing apparatus is installed. The photographing devices are arranged on a straight line extending upward in the figure from the origin O, and are arranged in the order of the wide-angle optical system unit 101 and the zoom optical system unit 102 from the origin 0 side.

  Here, the distance from the origin 0 to the entrance pupil position of the wide-angle optical system unit 101 is H, and the entrance pupil position of the zoom optical system unit 102 is at a distance h from the entrance pupil position of the wide-angle optical system unit 101. H is measured when the photographing apparatus is installed. The order of the optical system units may be reversed.

  Here, it is assumed that the intruder is at a distance L from the origin O, and the wide-angle optical system unit 101 detects the intruder at the tilt angle θw with respect to the optical axis direction indicated by the dotted line in the figure. Since the pan angle is the same as that in the first embodiment, the description in this embodiment is omitted.

On the other hand, a tilt angle necessary for the zoom optical system unit 102 at a distance H + h from the origin 0 to capture the intruder as a center is defined as θz. here,
H = Ltan (θw)
(H + h) = Ltan (θz)
Therefore, L that cannot actually be measured is deleted,
tan (θz) = (H + h / H) tan (θw)
θz = tan ⁻¹ {(H + h / H) tan (θw)}
It becomes.

Here, a block diagram of the photographing apparatus of this embodiment is shown in FIG. 6 differs from FIG. 6 in the first embodiment in addition to FIG. 6, an interface unit (UI) 1101 that inputs the installation height when the apparatus is installed, and a lookup table that quotes tan ⁻¹ θ in the subject direction detection unit. 1102 has been added.

This apparatus inputs the height H from the floor surface from the input interface UI1101 when the apparatus is installed. The CPU (not shown) uses the input H and the known entrance pupil position interval h, and θz = tan ⁻¹ {(H + h / H) tan ( θw)} is calculated, and a table having θw as an input and θz as an output is created in a lookup table 1102 made of RAM.

  The above only needs to be performed once at the time of installation of the apparatus, and it is only necessary to input the installation height, so no labor is required.

  When an intruder in the monitoring area is detected during actual monitoring and the angle θ formed from the entrance pupil position of the wide-angle optical system unit 101 is obtained, the pan angle θwp becomes the panning angle of the zoom optical system unit 102 as it is, and the tilt angle The angle θwt is corrected by the look-up table 1102 and the tilt angle θzt is sent to the optical system rotation control unit 609.

  As described above, this embodiment is effective when the specific subject is at the height of the floor O when the apparatus is installed, and the floor height is usually constant and sufficient for indoor monitoring such as that shown in FIG. The method is effective.

  Moreover, there is no practical problem at all when the apparatus is installed at a sufficient height outdoors (H >> h) or when an intruder is sufficiently away from the apparatus (L >> H).

In the present embodiment, θz = tan ⁻¹ {(H + h / H) tan (θw)} is calculated in advance, stored in the lookup table, and corrected by the monitoring lookup table. However, it is needless to say that there is no problem even if the sequential calculation is performed when there is a sufficient margin in processing speed due to sufficiently high-speed arithmetic processing.

  FIG. 12 shows a third embodiment of the present invention. In this embodiment, a mirror optical component 1201 is added to the wide-angle optical system unit of Embodiment 1, and an optical system generally called an omnidirectional camera is used. Here, the mirror optical component 1201 (hereinafter simply referred to as a mirror) will be briefly described.

  The mirror 1201 is, for example, a rotating body (hereinafter referred to as a hyperboloid mirror) whose cross section is formed into a hyperboloid shape, and the imaging device is installed at a specific position below the mirror 1201 to remove the mirror 1201 from the mirror 1201. By receiving the reflected light, an angle of view of 360 ° around the mirror is imaged.

  In general, the hyperbola has two focal points, and by adjusting the principal point of the taking lens to one focal point of the hyperbola, the acquired image is converted into a panoramic image having a real space as a cylindrical projection surface, for example, by an appropriate conversion formula. It can be converted into various projected coordinates.

  In this embodiment, as shown in FIG. 12, this mirror is used for the wide-angle optical system unit 101, and a 360 ° circumference (omnidirectional) around the mirror is imaged as a monitoring region, and the specific detected by the wide-angle optical system unit 101 is detected. The direction θ of the subject is determined using a conversion formula described later, and the zoom optical system unit is pan-tilt zoomed in the direction θ.

  FIG. 13 illustrates the positional relationship between the wide-angle optical system unit 101 and the zoom optical system unit 102 in the present embodiment. The hyperbolic surface rotation axis of the mirror 1201 and the lens optical axis of the wide-angle optical system unit 102 are matched. , Where the lens principal point is aligned with the lens side focal point of the hyperboloid.

  Further, as in the first embodiment, the entrance pupil position of the wide-angle optical system unit 101 and the entrance pupil position of the zoom optical system coincide with each other in the vertical direction, that is, the hyperboloid rotation axis of the mirror 1201 and the entrance of the wide-angle optical system unit 101. The pupil position and the entrance pupil of the zoom optical system unit 102 are arranged on a straight line.

  Next, angle detection of a specific subject will be described using these optical systems. FIG. 14 is a diagram for explaining the relationship when a subject in real space reflected by the mirror 1201 forms an image on the imaging surface (herein referred to as an image surface) 1401 of the wide-angle optical system unit 101.

  The two focal points of the rotating hyperboloid are Om and Oc, and the lens principal point is placed at the focal point Oc outside the hyperboloid as described above. The image plane is virtually above the focal point Oc by the focal length f.

  As shown in the figure, the origin of the coordinate axes is placed at the centers of the focal points Om and Oc (O in the figure), and XYZ axes orthogonal to each other are defined. The mirror rotation axis coincides with the Z axis, and the image plane 1401 is parallel to the plane including the XY axes.

  Also, a two-dimensional axis is defined on the image plane 1401 with the intersection point with the Z axis as the origin, the axis parallel to the X axis as the x axis, and the axis parallel to the Y axis as the y axis. The Z coordinate of the origin is (Z = fc).

Now, the object point P ₀ (X ₀ , Y ₀ , Z ₀ ) in the real space is reflected by the mirror surface P (X, Y, Z) and imaged at the position p (x, y) on the camera imaging surface. Then. The angle formed by the object point P ₀ (X ₀ , Y ₀ , Z ₀ ) as viewed from the focal point Om is defined as a pan angle θ parallel to the XY plane and a tilt angle φ perpendicular to the XY plane as shown in the figure.

  Therefore, the object pan angle θ and the tilt angle φ are obtained using the point p (x, y) on the image plane defined as described above.

First, the pan angle θ will be described. FIG. 15 is a diagram showing the XY plane projection plane of FIG. 14, and the pan angle θ can be obtained from tan ⁻¹ (x / y). Next, the tilt angle φ will be described.

  FIG. 16 is a cross-sectional view including the Z-axis and the object mirror reflection point P (XYZ) in FIG. Now, in this two-dimensional plane, a coordinate axis and an R coordinate axis taken perpendicular to the Z axis are defined, and the description will be made on the RZ plane.

Here, the coordinates of the object point P ₀ , the mirror reflection point P, and the image plane imaging point p are P ₀ (R ₀ , Z ₀ ), P (R, Z), and p (r, fc), respectively. .

  Also, the mirror cross section is hyperbola as shown in the figure.

Formula 1

... I)

The straight line connecting the object point P ₀ and the focal point Om is represented by Z = kR + c (II)
And
Now, at the mirror reflection point P, I) and II) are established.

On the other hand, the R coordinate R of the mirror reflection point P from the triangle Oc-P-ZP and the triangle Oc-p-Zp is R = r (c + Z) / f (III)
However,

Formula 2

By the way, the tilt angle φ is nothing but the inclination of the straight line II).
φ = tan ⁻¹ (k) ・ ・ ・ IV)
Therefore, if k is obtained from the formulas I) II) III) and substituted into the formula IV), the tilt angle φ can be obtained.

II) From the formula, k = (Zc) / R (V)
I) R is eliminated from the formula III) and b ² r ² −a ² f ² ) Z ² + 2cb ² r ² Z + (b ² c ² r ² + a ² b ² f ² ) = 0... VI)
VI) Solve the quadratic equation for Z in equation (3) to obtain Z, III) substitute Z into equation (3) to obtain R, and substitute Z and R into equation (V) to obtain k.

When k is obtained, the tilt angle φ can be obtained by using a lookup table of y = tan ⁻¹ (x) prepared in advance as described in the second embodiment. The pan angle θ and the tilt angle φ obtained as described above are transmitted to the optical system rotation control unit 609, and the zoom optical system unit rotates.

  FIG. 23 shows a fourth embodiment of the present invention. In this embodiment, a mechanism is added so that the positional relationship between the lens barrel and tripod seat of the zoom optical system unit shown in FIG. 3 of the first embodiment moves back and forth in the optical axis direction. In the zoom optical system, the entrance pupil position changes according to the magnification.

  Therefore, as described above, in order for the wide-angle optical system unit entrance pupil position, the zoom optical system unit entrance pupil position, and the zoom optical system unit panning rotation axis to coincide, the zoom optical unit according to the zoom magnification of the zoom optical system Need to move adjust.

  FIG. 23 represents one embodiment of such a mechanism. FIG. 23A shows a slider 2302 attached to the zoom optical system unit barrel 2301, and grooves are carved in the slider 2302 at extremely small intervals.

  FIG. 23B shows a tripod seat for supporting the panning motor 301 and the lens barrel. A tripod seat position adjusting motor 2303 for moving the lens barrel support position of the tripod seat is arranged in the vicinity of the tripod seat.

  At the tip of the output shaft of the tripod seat position adjustment motor 2303, there is a rotating plate, and the rotating plate is also grooved at extremely small intervals like the slider 2302. FIG. 23 (c) is a view from the inside of the ring of the tripod seat. As shown in the figure, the aforementioned groove is also carved on the inside of the ring of the tripod seat.

  In this embodiment, the slider 2302 and the inside of the tripod seat ring are shown with grooves in a part thereof for easy understanding of the figure. However, in order to support and move the barrel more stably, the barrel and tripod seat Grooves may be carved on the entire inner surface of the ring.

  By such a mechanism, when the zoom optical system unit is zoomed, the tripod seat position adjustment motor 2303 rotates and the entrance pupil position of the lens barrel is in the center of the panning rotation axis (FIG. 23 (c) 2304). The tube is moved back and forth in the optical axis direction. This is shown in FIG.

  In this way, the entrance pupil position corresponding to the imaging magnification is stored in advance, and the entrance pupil position is always maintained even during zooming by moving the lens barrel back and forth so that the panning rotation axis comes to the pupil position corresponding to the imaging magnification. Is obtained.

  In Embodiment 1 of the present invention, a plurality of imaging optical systems with different imaging magnifications are used, a specific subject is extracted from a captured image by an imaging optical system with a low imaging magnification, and the line-of-sight direction of the imaging optical system with a high imaging magnification is extracted from the extraction result. In a moving compound eye imaging device, by aligning the entrance pupil position in the vertical direction of the panning rotation plane, it is possible to adapt to the zoom optical system without correcting the pan angle obtained from the wide-angle optical system, and quickly and accurately. It is possible to turn the line of sight toward the target.

  Further, in the second embodiment, it is possible to quickly and accurately turn the line of sight to the target object simply by inputting the installation state when installing the photographing apparatus.

  Further, in the third embodiment, a wide-angle optical system can be obtained by adding an optical member that captures all directions of 360 ° around the image capturing apparatus to obtain a monitoring area free from all-around blind spots, and can accurately detect intruders. The zoom optical system moves its line of sight, and an enlarged image can be taken.

The figure which looked at this apparatus structure from the side in the apparatus structural example of Example 1 of this invention is represented. The figure which looked at this apparatus structure from the top by the apparatus structural example of Example 1 of this invention is represented. It is a figure explaining the pan rotating shaft of this apparatus by the apparatus structural example of Example 1 of this invention. It is a figure explaining the tilt rotation axis | shaft of this apparatus by the apparatus structural example of Example 1 of this invention. It is a figure showing the operation example of this apparatus of Example 1 of this invention. It is a block diagram explaining the signal processing of Example 1 of this invention. It is a figure explaining the to-be-photographed object extraction process of this apparatus of Example 1 of this invention. It is a figure explaining the process which determines the direction of a to-be-photographed object and determines a magnification from the result of the to-be-extracted object of this apparatus of Example 1 of this invention. It is a figure explaining the example of a display of the display apparatus of this apparatus of Example 1 of this invention. It is a figure explaining the to-be-photographed object direction detection of Example 1 of this invention. It is a block diagram explaining the signal processing of Example 2 of this invention. The bird's-eye view of this apparatus structure is represented by the apparatus structural example of Example 3 of this invention. The figure which looked at this apparatus structure from the side in the apparatus structural example of Example 3 of this invention is represented. It is a figure explaining the relationship between the hyperboloid mirror of Example 3 of this invention, and an image plane. It is a figure explaining the relationship between the hyperboloid mirror of Example 3 of this invention, and an image plane, and FIG. 14 was projected on XY plane. It is a figure explaining the relationship between the hyperboloid mirror of Example 3 of this invention, and an image plane, or 1 sectional drawing of FIG. It is a figure explaining the imaging device of a prior art, and it is a figure explaining utilization of the landmark on the ground. It is a figure explaining the imaging device of a prior art, and is a figure explaining utilization of the pan / tilt angle of a wide-angle camera. It is a figure explaining the imaging device of a prior art, and is the figure which looked at FIG. 18 from the top. It is a figure explaining the imaging device of a prior art, and is a figure explaining the problem which becomes a blind spot from the installation position of a camera. It is a figure explaining the imaging device of a prior art, and is a figure explaining the example which piled up the camera vertically. It is a figure explaining the imaging device of a prior art, and is a figure explaining the problem by the shift | offset | difference of a rotating shaft. It is the figure which showed the slide of Example 4. It is operation | movement explanatory drawing of the zoom optical system unit which advances / retreats in an optical axis direction according to the position of an entrance pupil. It is a flowchart explaining the operation control of an imaging device.

Claims (10)

A first imaging optical system having a first imaging magnification;
A second imaging optical system having a second imaging magnification larger than the first imaging magnification;
The second imaging optical system can be rotated,
An imaging apparatus, wherein the entrance pupil positions of the first and second imaging optical systems are arranged on a rotation axis of the second imaging optical system.   The imaging apparatus according to claim 1, wherein the second imaging optical system is rotatable even about an axis orthogonal to the rotation axis. The second photographing optical system is capable of zooming,
The photographing apparatus according to claim 1, further comprising a position changing unit that moves the second photographing optical system in an optical axis direction in accordance with the zooming.   Control means for determining a photographing direction of the second photographing optical system based on a photographed image obtained using the first photographing optical system and rotating the second photographing optical system in the determined photographing direction. The imaging apparatus according to claim 1, further comprising:   5. The photographing according to claim 4, wherein the control means detects a specific subject area from the photographed image and determines a photographing direction of the second photographing optical system so as to include the specific subject area. apparatus.   Control means for determining the second photographing magnification based on a photographed image obtained by using the first photographing optical system and setting the second photographing optical system to the determined second photographing magnification. The photographing apparatus according to claim 3, further comprising:   The photographing apparatus according to claim 6, wherein the control unit detects a specific subject area from the photographed image and determines the second photographing magnification according to the specific subject area. A first photographing optical system having a first photographing magnification; and a second photographing optical system having a second photographing magnification larger than the first photographing magnification. The second photographing optical system includes: A method of controlling an imaging apparatus that is rotatable and the entrance pupil positions of the first and second imaging optical systems are arranged on a rotation axis of the second imaging optical system,
Initializing the direction of the second imaging optical system;
Detecting a specific subject area from within a photographed image obtained using the first photographing optical system;
Determining the direction of the second imaging optical system so as to include the detected specific object region;
And a step of rotating the second imaging optical system about the rotation axis in the determined direction. The second photographic optical system is capable of zooming,
Determining the second imaging magnification so as to include the detected specific subject area;
Setting the second imaging optical system to the determined second imaging magnification;
9. The method of controlling an imaging apparatus according to claim 8, further comprising a step of moving the second imaging optical system in an optical axis direction in accordance with zooming of the second imaging optical system. 10. The method of controlling an imaging apparatus according to claim 8, wherein the steps other than the step of initializing the direction are sequentially repeated.