JP2018169488A - Imaging apparatus, imaging system, and imaging control program - Google Patents

Abstract

To track a moving subject in a wide range and perform super wide-field imaging, even when the direction of an imaging apparatus is not changed.SOLUTION: The imaging apparatus acquires a plurality of images continuously connected to each other by imaging through a plurality of variable power optical systems 2A to 2H directed in different azimuths. Each of the plurality of variable power optical systems includes a shift optical system 233 capable of performing a shift operation with respect to an optical axis. When it is determined that a moving subject P is included in the angle of view of the first variable power optical system 2A, control means 306 causes the first variable power optical system to perform a variable power operation to a telephoto side and to perform the shift operation, so as to track the subject and further, and causes at least one of the variable power optical systems 2B to 2F different from the first variable power optical system to perform at least one of the variable power operation and the shift operation, so as to compensate the change of the angle of view caused by the variable power operation and the shift operation in the first variable power optical system.SELECTED DRAWING: Figure 15

Description

  The present invention relates to an imaging apparatus suitable for ultra-wide field imaging such as omnidirectional imaging and panoramic imaging.

  As a method for performing super-wide-field imaging as described above, a technique is known in which a plurality of captured images obtained by a plurality of cameras are connected to form a continuous image. When super-wide-field imaging is used for surveillance purposes such as crime prevention, a specific subject image is often enlarged and observed in detail from the captured wide landscape image. Conventionally, in such a case, a part of the captured image is cut out and enlarged by performing electronic zoom. However, degradation of image quality due to electronic zoom becomes a problem. There is also a need for a camera that can track a moving subject while enlarging it.

  Japanese Patent Laid-Open No. 2004-26883 includes two cameras: a wide-angle camera that can image almost the entire monitoring area, and a movable camera that has a pan head mechanism and can change the imaging direction and zoom magnification according to an external signal. A configured imaging system is disclosed. In this imaging system, the position of the subject to be tracked is specified from the wide-field image acquired by the wide-angle camera, and the imaging direction and zoom magnification of the movable camera are controlled so that a tracking image of the subject with high resolution can be obtained. Japanese Patent Application Laid-Open No. 2004-228561 discloses an optical tracking for tracking a subject by shifting an image stabilizing lens for reducing image blur caused by camera shake or the like, and an electronic for tracking the subject by cutting out a region of the subject from a captured image. A camera that can perform type tracking is disclosed.

JP2015-194901A Japanese Patent Application Laid-Open No. 2006-046580

  However, the imaging system disclosed in Patent Document 1 uses a wide-angle camera and a movable camera arranged at positions separated from each other, and an ultra-wide-field image and an enlarged image of a specific subject with one camera. And can not get. In addition, the camera disclosed in Patent Document 2 keeps tracking a subject with a large amount of movement from the imaging angle of view and the limit of the amount of shift of the anti-vibration lens as long as it is directed in a certain direction. I can't.

  The present invention provides an imaging apparatus that can track a moving subject in a wide range and can perform imaging with an ultra-wide field of view without changing the orientation of the imaging apparatus.

  An imaging apparatus according to an aspect of the present invention includes an imaging unit that acquires a plurality of images that are continuously connected to each other by imaging through a plurality of variable magnification optical systems oriented in different directions, Control means for controlling the operation of the double optical system. Each of the plurality of variable magnification optical systems includes a shift optical system capable of performing a shift operation with respect to the optical axis of the variable magnification optical system. When the control means determines that an object to be moved is included in the angle of view of the first variable power optical system among the plurality of variable power optical systems, the first variable power optical system includes a telephoto side. And a shift operation to track the subject, and at least one other variable magnification optical system different from the first variable magnification optical system is used in the first variable magnification optical system. It is characterized in that at least one of a scaling operation and a shifting operation is performed so as to compensate for a change in the angle of view due to the scaling operation and the shifting operation.

  An imaging apparatus according to another aspect of the present invention includes an imaging unit that acquires a plurality of images that are continuously connected to each other by imaging through a plurality of variable magnification optical systems oriented in different directions. And control means for controlling operations of the plurality of variable magnification optical systems. Each of the plurality of variable magnification optical systems includes a shift optical system capable of performing a shift operation with respect to the optical axis of the variable magnification optical system. Then, the control means determines that an object moving toward the field angle of the second variable power optical system is included in the field angle of the first variable power optical system among the plurality of variable power optical systems. In such a case, the first variable power optical system is caused to perform a zooming operation toward the telephoto side, a shift operation is performed so as to track the subject, and the second variable power optical system is also subjected to a zooming operation toward the telephoto side. And a shift operation so as to take over the tracking of the subject.

  Note that an imaging system that includes the imaging apparatus and an instruction apparatus that inputs an instruction to control the zooming operation of a plurality of zooming optical systems to the control unit also constitutes another aspect of the present invention.

  An imaging control program as a computer program that causes the computer of the imaging apparatus to execute the above-described processing for control also constitutes another aspect of the present invention.

  According to the present invention, it is possible to perform imaging with an ultra-wide field of view while tracking a moving subject within a wide range without changing the orientation of the imaging device.

1 is an external perspective view of an omnidirectional camera that is an embodiment of the present invention. The perspective view of the lens-barrel in an Example. The disassembled perspective view of the lens barrel in an Example. The disassembled perspective view of the 2 group lens barrel in an Example. The front view of the 2nd group barrel in an Example. Sectional drawing (collapse position) of the lens barrel in an Example. Sectional drawing (WIDE position) of the lens-barrel in an Example. Sectional drawing (TELE position) of the lens-barrel in an Example. The perspective view which shows the state which extended the tripod of the camera of an Example. The perspective view which shows the starting state of the camera of an Example. The conceptual diagram which shows the view angle distribution in an Example. The figure which shows the lens angle of view (horizontal and vertical line direction) in the starting state in an Example. The figure which shows the lens angle of view (horizontal and vertical line direction) in the telephoto state in an Example. The figure explaining the optical axis shift using the anti-vibration mechanism in an Example. The figure which shows the angle-of-view change for enlarging and tracking a to-be-photographed object seen from a perpendicular direction in an Example. The figure which shows the case where the change of a tracking angle of view is compensated by wide angle side zoom, and the figure which shows the case where the change of a tracking angle of view is compensated by optical axis shift. Another figure which shows the view angle change for enlarging and tracking a to-be-photographed object in an Example, seeing from a perpendicular direction. The figure which shows the case where the change of a tracking angle of view is compensated by wide angle side zoom and optical axis shift, and the figure which shows the tracking taking over between lens-barrels. FIG. 10 is still another view showing the change in the angle of view for enlarging and tracking the subject in the embodiment as seen from the vertical direction. The figure which shows a motion vector when a to-be-photographed object approaches a camera with a low speed and a light beam in an Example. The figure which shows a motion vector when a to-be-photographed object crosses a camera with a low speed and a light beam in an Example. The block diagram which shows the structure of the omnidirectional imaging system containing the camera of an Example. 6 is a flowchart illustrating subject tracking processing in the camera of the embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows the appearance of an omnidirectional camera as an image pickup apparatus that is an embodiment of the present invention. This omnidirectional camera connects a plurality of images (partial images) obtained by imaging through a plurality of imaging optical systems oriented in different directions to obtain an image in an imaging range of 360 ° horizontally and 360 ° vertically. A certain omnidirectional image (composite image) is generated.

  Reference numeral 1 denotes a camera body. The camera body 1 has a spherical exterior member 1a. The lens barrel 2 (2A to 2H) containing the imaging lens faces the outer azimuths that are different from each other at six locations in the horizontal plane of the camera body 1 and two locations in the vertical direction (upper surface portion and lower surface portion). Is held (arranged) as follows. Specifically, the lens barrels 2A to 2F are oriented in different directions in the horizontal plane, and the lens barrels 2G and 2H are oriented in the upper and lower sides, which are different orientations in the vertical direction.

  The vertical line direction is a direction in which a straight line extends along the direction of receiving gravity when the camera body 1 is fixed to a horizontal camera installation surface using a built-in tripod 5 to be described later, and is also referred to as a vertical direction. A horizontal plane is a plane orthogonal to the vertical direction, and a direction (azimuth) included in the horizontal plane is referred to as a horizontal direction. The imaging lenses in the total eight lens barrels 2 are variable power optical systems capable of a variable power operation (optical zoom) for changing the focal length. Although the zooming optical system actually corresponds to an imaging lens, in the following description, a lens barrel including the imaging lens will be described as equivalent to a zooming optical system.

  A lens barrier 20 that opens and closes the front surface of the imaging lens is provided at the front end of each lens barrel 2. The lens barrier 20 is opened and closed between a retracted state in which the lens barrel 2 protrudes from the camera body 1 and a protruding state in which the lens barrel 2 protrudes from the camera body 1 in accordance with ON and OFF of the power button 7 disposed on the upper surface of the camera body 1. It is performed in conjunction with the operation of moving out or moving in.

  In addition, between the lens barrel 2 (2G) on the top surface of the camera body 1 and the lens barrels 2 (2A to 2F) at six locations in the horizontal plane, illumination light is evenly applied to the subject around the camera. A ring-shaped illumination unit 3 is provided. In addition, the three movable illumination units 4 and the camera body 1 are stabilized between the lens barrel 2 (2H) on the lower surface of the camera body 1 and the lens barrels 2 (2A to 2F) at six locations in the horizontal plane. A built-in tripod (hereinafter simply referred to as a tripod) 5 is provided for fixing horizontally. The movable illumination unit 4 can adjust the position to an arbitrary illumination position protruding from the camera body 1 from the storage position shown in the figure. In the tripod 5, the user can arbitrarily change the length and angle of the leg portion from the storage state shown in the figure, and thereby the height of the camera body 1 from the camera installation surface can be adjusted. The movable lighting unit 4 and the tripod 5 protruding from the storage position correspond to protruding members.

  A battery chamber 6 is provided on the lower surface of the camera body 1, and power is supplied to the camera body 1 from the battery stored in the battery chamber 6. Furthermore, a zoom button 8 is provided on the upper surface of the camera body 1 to select a lens barrel 2 that performs optical zooming from among the eight lens barrels 2 and to instruct the optical zooming. In addition, a release button 9 for starting an imaging preparation operation (focus adjustment operation and photometry operation) and an imaging operation (exposure of an image sensor described later) is also provided.

  The optical zoom, the imaging preparation operation, and the imaging operation can also be remotely controlled by communication from an external instruction device such as a personal computer or a smartphone.

  FIG. 2 shows one of the eight lens barrels 2 (2 </ b> A to 2 </ b> H) provided in the camera body 1. The lens barrel 2 is a collapsible barrel, and the lens barrel 2 is stored in the camera body 1 when the camera is in a power-off state. Further, as described above, since the lens barrier 20 covers the front surface of the imaging lens in the power OFF state, scratches on the front surface, intrusion of dust into the lens barrel, and the like are prevented.

  FIG. 3 shows the lens barrel 2 in an exploded manner. The imaging lens includes a first group lens, a second group lens, and a third group lens (not shown). Reference numeral 21 denotes a first group barrel that holds the first group lens and includes the lens barrier 20 described above. Reference numeral 22 denotes an aperture unit that adjusts the amount of light. Reference numeral 23 denotes a second group barrel that holds the second group lens and includes an anti-vibration mechanism and a shutter.

  A moving cam ring 24 is provided with a cam groove portion for moving the first group lens barrel 21, the aperture unit 22 and the second group lens barrel 23 in the optical axis direction on the inner peripheral portion thereof. On the outer periphery of the moving cam ring 24, a gear portion to which the driving force from the driving motor 29 is transmitted is provided. The movable cam ring 24 is held so as to be rotatable around the optical axis with respect to the first group lens barrel 21, the aperture unit 22, the second group lens barrel 23, and a rectilinear tube 25 and a fixed tube 27 to be described later. Is driven to rotate by receiving the driving force at the gear portion. The rectilinear cylinder 25 guides the first group lens barrel 21, the aperture unit 22, and the second group lens barrel 23 in the optical axis direction, and prevents rotation around these optical axes. The third group lens barrel 26 holds a third group lens. The fixed barrel 27 accommodates the first group barrel 21, the aperture unit 22, the second group barrel 23, the movable cam ring 24, and the third group barrel 26 so as to be movable in the optical axis direction. A cam groove portion for moving the movable cam ring 24 in the optical axis direction is provided on the inner peripheral portion of the fixed cylinder 27.

  Reference numeral 28 denotes an image sensor holding unit that holds an image sensor (not shown). The fixed cylinder 27 is fixed, and the drive motor 29 described above is held. A lens barrel flexible wiring board 28 a is attached to the image sensor holding unit 28. The lens barrel flexible wiring board 28a is connected to the diaphragm flexible wiring board 221 connected to the driving unit of the diaphragm unit 22 and the second group flexible wiring board 231 connected to the shutter driving unit of the second group lens barrel 23. Connected with.

  The configuration of the lens barrel 2 described above is merely an example, and may be a lens barrel that is not retractable but that performs optical zooming only by moving the lens inside.

  FIG. 4 is an exploded view of the vibration isolating mechanism in the second group barrel 23 shown in FIG. FIG. 5 shows a vibration isolation mechanism (except for a second group flexible substrate 231 and a sensor holder 232 described later) as viewed from the subject side in the optical axis direction. Reference numeral 239 denotes a second group base which is a base of the second group barrel 23. The second group base 239 is provided with three follower pins 239a that engage with the cam groove portion of the movable cam ring 24 described above. A shutter yoke 239b is attached to a partial area around the optical axis of the second group base 239. The cover member 239c covers the shutter yoke 239b from the subject side and is fixed to the second group base 239. A shutter actuator including a shutter yoke 239b and a rotor (not shown) is a two-point switching actuator in which the stop position of an arm provided on the rotor is switched by switching the energization direction.

  Reference numeral 235 denotes a second group lens holder. A second group lens 233 (LS2) is fixed to the second group lens holder 235 together with a mask member 234 for cutting harmful light by adhesion or the like. The second group lens holder 235 holds magnets 235A and 235B. In the following description, reference numerals A and B correspond to the A direction (pitch direction) and the B direction (yaw direction) shown in FIG. The magnet 235A is held by the second group lens holder 235 so that the N pole and the S pole are arranged in the A direction, and the magnet 235B is held so that the N pole and the S pole are arranged in the B direction. Hooks 235a are provided at two locations of the second group lens holder 235, and one end of a tension spring 236 is hung on each hook 235a.

  Reference numerals 237A and 237B denote coil units including coils and bobbins. The coil units 237A and 237B are fixed to a concave portion provided at a position different from the shutter yoke 239b in the second group base 239 and facing the magnets 235A and 235B in the optical axis direction by fixing means such as adhesion. Yes. Power feeding to the coil is performed by connecting the second group flexible substrate 231 to each terminal portion embedded in the bobbin and electrically connected to the coil.

  The other end of the spring 236 hung on the second group lens holder 235 is hung on a hook 239d of the second group base 239. Three non-magnetic balls 238 are sandwiched between the second group base 239 and the second group lens holder 235. The spring 236 biases the second group lens holder 235 toward the second group base 239 in the optical axis direction. Thereby, the second group lens holder 235 is pressed against the second group base 239 via the ball 238 in the optical axis direction. The second group lens holder 235 can move (shift) smoothly in a plane perpendicular to the optical axis while rolling the ball 238. By shifting the second group lens holder 235 within the plane according to camera shake such as camera shake, it is possible to perform image stabilization control that suppresses image blur on the image sensor.

  The sensor holder 232 is disposed so as to cover the magnets 235 </ b> A and 235 </ b> B and their peripheral portions in the second group lens holder 235 from the subject side, and is fixed to the second group base 239. The sensor holder 232 positions and holds Hall elements 231A and 231B described later.

  Reference numeral 231 denotes a second group flexible substrate to which the aforementioned coil units 237A and 237B and a shutter actuator are connected. In addition, Hall elements 231A and 231B are mounted on the second group flexible substrate 231. By detecting a change in the magnetic field from the magnets 235A and 235B shifted together with the second group lens holder 235 by the Hall elements 231A and 231B, the amount of movement, that is, the position of the second group lens holder 235 from the neutral position on the optical axis is determined. Can be detected. Thereby, the feedback control of the position of the second group lens holder 235 can be performed in the image stabilization control.

  FIG. 6 shows a cross section of the lens barrel 2 in the retracted state (collapsed position). In the retracted state, the first, second and third group lenses LS1, LS2 and LS3 are arranged so that the distance between them is minimized, and the lens barrier 20 is closed.

  FIG. 7 shows a cross section of the lens barrel 2 in the wide-angle end state (Wide position). By driving the drive motor 29 from the retracted state, the first group lens barrel 21 and the movable cam ring 24 are extended so as to protrude from the fixed cylinder 27 (camera body 1), and the first, second and third group lenses LS1, LS2, LS3. Is placed in a position as shown in the figure and is ready for imaging. At this time, the angle of view of the lens barrel 2 is maximized, and the image area obtained through one lens barrel 2 is the widest in an omnidirectional image described later.

  FIG. 8 shows a cross section of the lens barrel 2 in the telephoto end state (Tele position). By driving the drive motor 29 from the wide-angle end state, the first group lens barrel 21 and the moving cam ring 24 are further extended to the fixed cylinder 27 from the wide-angle end state, and the first, second and third group lenses LS1, LS2, LS3 are moved. It arrange | positions in a position as shown in a figure. At this time, the angle of view of the lens barrel 2 is minimized, and the image area obtained through one lens barrel 2 in the omnidirectional image is the narrowest. In this embodiment, a case where a lens barrel having a variable power optical system having three lens groups is used will be described. However, the number of lens groups may be a number other than three.

  When it is desired to enlarge the image area of a specific subject in the omnidirectional image, the optical image of the subject is placed on the image sensor by performing an optical zoom to the telephoto side of the lens barrel 2 capturing the subject. Can be magnified and projected. For this reason, it is possible to obtain an enlarged subject image without image quality degradation like the electronic zoom. Each of the eight lens barrels 2 provided in the camera can continuously change the angle of view within the range from the wide-angle end state to the telephoto end state.

  FIG. 9 shows how the user manually pulls out the tripod 5 and fixes it to the camera installation surface as preparation of the camera body 1 before imaging. The three legs 5a of the tripod 5 can be extended in the C direction from the storage state shown in FIG. 1, and the camera body 1 is supported by the three grounding parts 5b. The angle between the three legs 5a can be changed from α to β in the D direction, and each leg 5a can be expanded and contracted, so that the height of the camera body 1 can be freely set by combining them. Can be adjusted.

  The camera body 1 has a built-in mechanism for matching the angles of the three legs 5a with respect to the vertical direction. As a result, if the camera installation surface is horizontal, the lens barrels 2A to 2F provided in the camera body 1 are accurately directed in the horizontal direction, and the lens barrels 2G and 2H are respectively accurately positioned above the vertical line (the sky). Side) and lower side (ground side).

  FIG. 10 shows a state in which the power supply button 7 provided on the upper surface of the camera body 1 is turned on by the user and the camera is activated. All the lens barrels 2 (2A to 2H) are extended from the power-off state, and the lens barrier 20 is opened, so that imaging is possible. In this state, the field angles of the eight lens barrels 2 are set so that omnidirectional imaging can be performed with the vertical direction added to the horizontal 360 °.

  FIG. 11 shows the distribution of image areas (hereinafter referred to as partial images) for the eight lens barrels 2A to 2H in an omnidirectional image that is a captured image obtained by omnidirectional imaging. Partial images obtained through each of the lens barrels 2A to 2F in the horizontal 360 ° are denoted by ImgA to ImgF. Further, partial images obtained through the top lens barrel 2G and the ground lens barrel 2H are indicated by ImgG and ImgH, respectively.

  The boundary line between the partial images is a stitch portion obtained by connecting adjacent partial images to obtain one continuous omnidirectional image. In the present embodiment, an image larger than the partial image is acquired through each of the lens barrels 2A to 2H. Then, a common feature point is detected from the surrounding area larger than the partial image in two of the eight acquired images, and the two acquired images are connected so that the feature points overlap each other. A partial image connected so that there is no image is obtained. An omnidirectional image is generated by performing such a joining process on all acquired images.

  FIG. 12A shows the angle of view of the lens barrel 2 (2A to 2F) in the horizontal plane when the camera body 1 is viewed from above. The angle of view of each of the six lens barrels 2A to 2F arranged at equal circumferential intervals in the horizontal plane is θ1. In addition, 360-degree omnidirectional imaging is possible in a continuous region outside the circle of radius R1 where parts of the field angles of the lens barrels 2 adjacent to each other in the circumferential direction overlap each other. R1 is a distance from the center of the camera body 1 (optical axis position of the lens barrel 2G) to a position where continuous omnidirectional imaging is possible, and L1 is a distance from the center of the camera body 1 to each of the lens barrels 2A to 2F. It is the distance to the tip. L2 indicates the omnidirectional imaging close distance obtained by subtracting L1 from R1, that is, the shortest imaging distance at which no discontinuity occurs between the respective viewing angles of the lens barrels 2A to 2F.

  In the present embodiment, the angle of view of the lens barrel 2 will be described as a simple conical angle of view, not an angle of view in consideration of the short side, long side, and diagonal of the image sensor. Further, the amount of extension of the lens barrel 2 changes due to zooming, but the amount of change is sufficiently small and negligible with respect to the omnidirectional imaging close distance L2, so that the lens barrel can be viewed from the center of the camera body 1 regardless of the zoom position. The distance to the tip of each of 2A to 2F is L1.

  12B shows the lens barrels 2A, 2D, 2G, and 2H when the camera body 1 in the state shown in FIG. 12A is viewed from the horizontal direction (the arrow direction in FIG. 12A). The angle of view in the vertical plane is shown. The field angles of the lens barrels 2A and 2D are each θ1. On the other hand, the field angles of the lens barrels 2G and 2H are θ2 (> θ1). Continuous omnidirectional imaging is possible in a region outside the circle of radius R1 where a part of the field angles of the lens barrels 2A, 2D, 2G, and 2H overlap.

  In order to obtain the omnidirectional imaging close distance L2 as in FIG. 12A, the distance from the center of the camera body 1 to a position where continuous omnidirectional imaging is possible needs to be R1. However, since the number of lens barrels 2G and 2H used for omnidirectional imaging is smaller in the vertical plane than in the horizontal plane, the angle of view of these lens barrels 2G and 2H is set to θ2 larger than θ1.

  At the time of starting the camera, for example, the angle of view θ1 of the lens barrels 2A to 2F and the angle of view θ2 of the lens barrels 2G and 2H are set so that the shortest omnidirectional imaging close distance L2 is obtained. However, narrower angles of view θ1 and θ2 may be set so that a longer omnidirectional imaging close distance L2 can be obtained when the camera is activated.

  FIG. 13A shows an optical zoom to the telephoto side only in the lens barrel 2A (first variable magnification optical system) in order to enlarge the specific subject P from the state shown in FIGS. 12A and 12B. The camera body 1 that has been subjected to (magnification operation) is shown as viewed from above. The angle of view of the lens barrel 2A is narrowed from θ1 to θ3 (<θ1) indicated by a dotted line. For this reason, if the angle of view of the lens barrels 2B and 2F adjacent to the lens barrel 2A is the original θ1 indicated by the dotted line, a part of these and the angle of view of the lens barrel 2A do not overlap with each other. Azimuth imaging is not possible.

  Therefore, in the present embodiment, the field angles of the lens barrels 2B and 2F that acquire partial images joined to the partial images acquired through the lens barrel 2A are widened to θ4 (> θ1) on the wide angle side from θ1. Control. That is, the lens barrels 2B and 2F are optically zoomed to the wide angle side. Thereby, a part of the angle of view of the lens barrels 2B and 2F can be superimposed on the narrowed angle of view of the lens barrel 2A, and continuous omnidirectional imaging can be performed. At this time, the angle of view θ4 of the lens barrels 2B and 2F is calculated and set so that the same omnidirectional imaging close distance L2 (that is, the distance R1) is maintained as before the angle of view of the lens barrel 2A is changed. It is desirable.

  Since the angle of view of the other lens barrels 2C to 2E in the horizontal plane is not changed, the overlapping area of the angle of view of the lens barrels 2B and 2C and the overlapping area of the angle of view of the lens barrels 2E and 2F are enlarged. The omnidirectional imaging close distance near those lens barrels is closer than L2. In this case, the user may be notified that the omnidirectional imaging close distance L2 is shortened by only a part of the area, or the feature point detection range in the joining process may be expanded.

  13B shows the lens barrels 2A, 2D, 2G, and 2H when the camera body 1 in the state shown in FIG. 13A is viewed from the horizontal direction (the arrow direction in FIG. 13A). The angle of view in the vertical plane is shown. Similar to FIG. 13A, the field angle of the lens barrel 2A is narrowed from θ1 to θ3 (<θ1) indicated by the dotted line. For this reason, if the angle of view of the lens barrels 2G and 2H adjacent to each other in the vertical direction of the lens barrel 2A remains the original θ2 indicated by the dotted line, these portions and the angle of view of the lens barrel 2A do not overlap, Continuous omnidirectional imaging becomes impossible.

  Therefore, in this embodiment, control is performed so that the angle of view of the lens barrels 2G and 2F is widened to θ5 (> θ2) on the wide angle side from θ2. Thereby, a part of the field angles of the lens barrels 2G and 2H can be overlapped with the narrowed field angle of the lens barrel 2A, and continuous omnidirectional imaging can be performed.

  At this time, as described with reference to FIG. 13A, the angle of view θ5 of the lens barrels 2G and 2H is the same as the omnidirectional imaging close distance L2 (that is, the distance R1) before the angle of view of the lens barrel 2A is changed. It is desirable to be calculated and set so as to be maintained. That is, it is desirable that the angle of view of θ5 is set so as to pass through Q1 and Q2 in FIG. However, when such a wide angle of view cannot be set even when the lens barrels 2G and 2H are in the wide-angle end state, as shown in the figure, omnidirectional imaging can be performed at the angle of view θ5 from the omnidirectional imaging closest distance L2. It is necessary to change to the omnidirectional imaging close distance L3 (that is, the distance R2). In this case, continuous omnidirectional imaging cannot be performed at the preset omnidirectional imaging close distance L2, and the user is notified that the omnidirectional imaging close distance changes from L2 to L3 (> L2). desirable.

  Thus, when the number of lens barrels 2 for performing omnidirectional imaging is different between the horizontal plane and the vertical plane, the imaging lens of the lens barrel in the horizontal plane and the imaging lens of the top-side lens barrel The focal length and magnification may be varied.

  In addition, when the lens barrel includes an imaging lens with an ultra-wide field angle exceeding 180 °, there is a possibility that light rays may be emitted by the camera body 1, so that the lens barrel can be used as a subject as in this embodiment. It is desirable to pay out to the side.

  As described above, for example, when performing optical zooming to the telephoto side of the lens barrel 2A among the eight lens barrels 2, the optical to the wide-angle side of the lens barrels 2B, 2F, 2G, 2H adjacent thereto is performed. By performing zooming, continuous omnidirectional imaging can be maintained. However, there is a condition between the arrangement angle of the lens barrel and the angle of view for enabling this.

In FIG. 13A, an equal arrangement angle between the lens barrels 2A to 2F adjacent in the horizontal plane (an angle formed by the optical axes of the lens barrels 2A to 2F) is defined as θ. If the angle of view θ2 of the lens barrel 2A and the angle of view θ1 of the lens barrel 2F do not overlap at all at an infinite distance, continuous omnidirectional imaging cannot be performed. The reason that the angle of view does not overlap at an infinite distance is that the line at the end of the half angle of view θ3 / 2 of the lens barrel 2A and the line of the end of the half angle of view θ1 / 2 of the lens barrel 2F are opened in parallel. This is because the angle X between these lines is 0 or more. For this reason, the condition that the angle X is smaller than 0 is a condition for a part of the angle of view of adjacent lens barrels to overlap each other. The angle X is obtained by subtracting the sum of the half angle of view θ3 / 2 of the lens barrel 2A and the half angle of view θ1 / 2 of the lens barrel 2F from the arrangement angle θ. That is,
X = θ− (θ3 / 2 + θ1 / 2) <0
θ <θ3 / 2 + θ1 / 2 (1)
The relationship is established.

  Thus, in this embodiment, the narrowing of the field angle of the lens barrel that has been optically zoomed to the telephoto side is compensated by the field angle that is wider than the optical zoom to the wide angle side of the adjacent lens barrel. At this time, it is necessary to satisfy the condition that the sum of the half angles of view of these adjacent lens barrels is at least larger than the arrangement angle between these lens barrels. Since the expression (1) is a condition for performing continuous omnidirectional imaging at an infinite distance, as described above, when the omnidirectional imaging close distance is set shorter than the infinite distance, it is further reduced by half. It is necessary to increase the overlap of the angle of view by increasing the sum of the angle of view. In any case, since it is necessary to join partial images obtained through separate lens barrels, it is desirable to perform imaging with a wider angle of view as a margin for joining.

  FIG. 14 shows the first and second and third group lenses LS1, 223 (LS2), LS3 constituting the imaging lens (variable magnification optical system) in the lens barrel 2 in the wide-angle end state as shown in FIG. And the image sensor 301 are shown. The upper diagram shows light rays incident on the imaging lens in a neutral state in which the center of the second group lens 233 constituting the image stabilizing shift optical system (anti-vibration lens) coincides with the optical axis O of the imaging lens. Yes. Further, the lower diagram shows light rays incident on the imaging lens in a state in which the second group lens 233 is shifted in the direction of the arrow with respect to the optical axis O by the anti-vibration mechanism. The light beam is incident on the image sensor 301 at a position shifted by Δ on the opposite side to the arrow direction with respect to the case where the second group lens 233 is not shifted.

  This shift of the second group lens 233 is intended to reduce image blur caused by camera shake due to camera shake or the like. However, by shifting the second group lens 233 regardless of image blur, an image in which the position of the optical axis O is shifted by Δ can be acquired. A method for tracking a subject in omnidirectional imaging by performing optical axis shifting using the shift of the second group lens group 233 with a plurality of lens barrels 2 will be described below.

  The omnidirectional imaging while tracking the subject is performed in a state where the camera body 1 is fixed by the tripod 5 as shown in FIG. Further, in the following, as an example, the subject captured within the angle of view of the lens barrel (first variable magnification optical system) 2A is directed toward the angle of view of the lens barrel (second variable magnification optical system) 2B. A case of moving will be described.

  FIG. 15A shows a state in which the specific subject P that is enlarged in FIG. 13A is tracked by shifting the optical axis in the lens barrel 2A when the specific subject P moves in the direction of the arrow, as viewed from above. ing. The angle of view of θ3 of the lens barrel 2A is directed to the specific subject P that is moving while being narrowed by the optical zoom to the telephoto side at the original position P ′ of the specific subject P as indicated by the dotted line. At this time, the lens barrel 2A shifts the optical axis by an angle δ1 corresponding to the amount of movement of the specific subject P.

  A part of the original angle of view θ3 of the lens barrel 2A before tracking the specific subject P and the angle of view θ4 of the lens barrels 2B and 2F overlap each other so as to satisfy the omnidirectional imaging close distance L2. The positions were points Q3 and Q4, respectively. In order to satisfy the omnidirectional imaging close distance L2 by shifting the optical axis of δ1 for tracking the specific subject P in the lens barrel 2A, the angle of view of θ3 of the lens barrel 2A and the images of the lens barrels 2B and 2F are satisfied. The positions where the corners overlap each other need to be points Q5 and Q6, respectively. However, if the angle of view of the lens barrel (third variable magnification optical system) 2F remains θ4, the angle of view reaches only the point Q4 and is continuous at a position satisfying the omnidirectional imaging close distance L2. Cannot perform omnidirectional imaging. Therefore, in FIG. 15A, the angle of view of the lens barrel 2F is set to θ5 (> θ4) by optical zoom to the wide angle side of the lens barrel 2F so as to compensate for the change of the angle of view of the lens barrel 2A. spread. As a result, the θ5 field angle of the lens barrel 2F and the θ3 field angle of the lens barrel 2A partially overlap at the point Q6, and as a result, continuous omnidirectional images can be acquired.

  At this time, the moving amount of the specific subject P increases the amount of overlap between the angle of view of the lens barrel 2A and the angle of view of the lens barrel 2B, but these are closer to the camera body 1 than the omnidirectional imaging close distance L2. Since the overlapping at the point Q7 does not cause an angle of view defect, it may be left as it is. Further, the lens barrel 2B is optically zoomed to the telephoto side so that the angle of view is narrowed to θ6 (<θ4) and overlaps a part of the angle of view of the lens barrel 2A at the point Q5. Good. Since margins for joining the partial images obtained through the lens barrels 2B, 2A, and 2F are necessary, the points Q3 to Q6 that are actually at the omnidirectional imaging close distance L2 are preferably closer to the camera body 1. The angle of view of the lens barrels 2C and 2E adjacent to the lens barrels 2B and 2F whose angle of view has been changed is originally partially overlapped at a position closer to the camera body 1 than the omnidirectional imaging close distance L2, so that The angle of view θ1 may be sufficient. However, the angle of view of the lens barrels (third variable magnification optical system) 2C and 2E may be appropriately changed within a range that satisfies the omnidirectional imaging close distance L2.

  15B, similarly to FIG. 15A, when the specific subject P enlarged in FIG. 13A moves in the direction of the arrow, the specific subject P is moved by shifting the optical axis in the lens barrel 2A. The tracking is seen from above. The angle of view of θ3 of the lens barrel 2A is directed to the specific subject P that is moving while being narrowed by the optical zoom to the telephoto side at the original position P ′ of the specific subject P as indicated by the dotted line. At this time, the lens barrel 2A shifts the optical axis by an angle δ1 corresponding to the amount of movement of the specific subject P.

  Similarly to FIG. 15A, a part of each of the original angle of view θ3 of the lens barrel 2A and the angle of view θ4 of the lens barrels 2B and 2F before tracking the specific subject P is omnidirectionally imaged. The overlapping positions so as to satisfy the close distance L2 were points Q3 and Q4, respectively. In order to satisfy the omnidirectional imaging close distance L2 by shifting the optical axis of δ1 for tracking the specific subject P in the lens barrel 2A, the angle of view of θ3 of the lens barrel 2A and the images of the lens barrels 2B and 2F are satisfied. The positions where the corners overlap each other need to be points Q5 and Q6, respectively. However, if the angle of view of the lens barrel 2F remains θ4, the angle of view reaches only the point Q4, and continuous omnidirectional imaging cannot be performed at a position satisfying the omnidirectional imaging close distance L2. For this reason, in FIG. 15B, optical zoom to the wide-angle side of the lens barrel 2F is not performed (with the angle of view kept at θ4), and the optical axis is shifted by an angle δ2 in the lens barrel 2F. By bringing the angle toward the lens barrel 2A, the change in the angle of view of the lens barrel 2A is compensated. As a result, a part of the angle of view θ4 of the lens barrel 2F and a part of the angle of view θ3 of the lens barrel 2A overlap at the point Q6. As a result, continuous omnidirectional images can be acquired. Control of the angle of view of the lens barrels 2B, 2F, 2C, and 2E is performed as described with reference to FIG.

  Similarly to FIGS. 15A and 15B, FIG. 16A also shows that when the specific subject P enlarged in FIG. 13A moves in the direction of the arrow, the optical axis in the lens barrel 2A shifts. The state of tracking the specific subject P is shown as viewed from above. The angle of view of θ3 of the lens barrel 2A is directed to the specific subject P that is moving while being narrowed by the optical zoom to the telephoto side at the original position P ′ of the specific subject P as indicated by the dotted line. At this time, the lens barrel 2A shifts the optical axis by an angle δ1 corresponding to the amount of movement of the specific subject P.

  Similarly to FIGS. 15A and 15B, a part of each of the original angle of view θ3 of the lens barrel 2A and the angle of view θ4 of the lens barrels 2B and 2F before tracking the specific subject P is obtained. Are overlapped so as to satisfy the omnidirectional imaging close distance L2 at points Q3 and Q4, respectively. In order to satisfy the omnidirectional imaging close distance L2 by shifting the optical axis of δ1 for tracking the specific subject P in the lens barrel 2A, the angle of view of θ3 of the lens barrel 2A and the images of the lens barrels 2B and 2F are satisfied. The positions where the corners overlap each other need to be points Q5 and Q6, respectively. However, if the angle of view of the lens barrel 2F remains θ4, the angle of view reaches only the point Q4, and continuous omnidirectional imaging cannot be performed at a position satisfying the omnidirectional imaging close distance L2. For this reason, in FIG. 16A, the optical axis is shifted by an angle δ3 (<δ2) in the lens barrel 2F to bring the angle of view toward the lens barrel 2A, and the optical angle of the lens barrel 2F toward the wide angle side is increased. The change in the angle of view of the lens barrel 2A is compensated by expanding the angle of view to θ7 (> θ4) by zooming. Thereby, a part of the angle of view θ7 of the lens barrel 2F and a portion of the angle of view θ3 of the lens barrel 2A overlap at the point Q6, and as a result, continuous omnidirectional images can be acquired. Control of the angle of view of the lens barrels 2B, 2F, 2C, and 2E is performed as described with reference to FIG.

  As shown in FIG. 16A, the angle of view of the lens barrel 2F is changed by both the optical zoom to the wide angle side and the optical axis shift, so that the angle of view of the lens barrel 2F before the change is close to the wide angle end. This is effective when the angle at which the optical axis can be shifted is small. Even when the optical zoom to the wide-angle side of the lens barrel 2F and the optical axis shift are used in combination, a part of the angle of view of the lens barrel 2F and the angle of view θ3 of the lens barrel 2A is a point Q6. There is also a possibility that they cannot be overlapped (omnidirectional imaging close distance L2 cannot be satisfied). In this case, the magnification of the lens barrel 2A is lowered (that is, the angle of view is made larger than θ3) or the omnidirectional imaging close distance L2 is lengthened.

  FIG. 16B shows a state in which the specific subject P has moved further in the direction of the arrow from the state shown in FIGS. 15A, 15B, and 16A. Subject tracking by shifting the optical axis of the lens barrel 2A is limited by the angle limit value for shifting the optical axis. Therefore, in order to continue tracking the specific subject P, it is necessary to change the lens barrel to be tracked, such as the lens barrel 2B, the lens barrel 2C,. At this time, the movement of the specific subject P is estimated using a motion vector described later, and the lens barrel to be tracked may be changed according to the result. FIG. 16B shows a lens barrel 2A in which the optical axis is shifted from θ3 to θ8 (> θ3) while shifting the optical axis from angle δ4 (> δ1), and the angle of view is set to θ8. This shows a state in which the lens barrel 2B that has shifted the optical axis to the lens barrel 2A side takes over tracking. The specific subject P is within the angle of view of both the lens barrels 2A and 2B.

  The lens barrels 2A and 2B capture the specific subject P at the same magnification (view angle θ8). For this reason, the positions where the angle of view of the lens barrels 2A and 2B and the angle of view of the lens barrels 2F and 2C adjacent to each other in the state where the omnidirectional imaging close distance L2 is set overlap each other. It is necessary to be Q8 and Q7, respectively. For this reason, in the lens barrel 2C, the optical axis shift of the angle δ5 is performed on the lens barrel 2B side and the optical zoom is performed on the wide angle side to widen the angle of view from θ1 to θ9 (> θ1). Further, in the lens barrel 2F, the optical axis is shifted to the lens barrel 2B side of δ5, which is larger than the angle δ3 shown in FIG. 16A, and the field angle is set by performing optical zooming to the telephoto side. The angle is narrowed from θ7 to θ9 (<θ7).

  Further, for the lens barrels 2D and 2E adjacent to the lens barrels 2C and 2F, in order to satisfy the omnidirectional imaging close distance L2, a part of their angles of view overlap at points Q9 and Q10 (that is, Performs the following control (to compensate for changes in the angle of view of the lens barrels 2C and 2F). That is, for the lens barrel 2D, the angle δ6 is shifted to the lens barrel 2C side and the optical zoom is performed to the wide angle side to widen the angle of view from θ1 to θ10 (> θ1). On the other hand, with respect to the lens barrel 2E, the optical axis shift of the angle δ6 is performed on the lens barrel 2F side and the optical zoom is performed on the wide angle side to widen the angle of view from θ1 to θ10.

  In this way, from the one lens barrel (2A) where the adjacent lens barrels (2A, 2B) have the same magnification (view angle) and the optical axis is shifted so as to track the specific subject P first. The optical axis of the other lens barrel (2B) toward the one lens barrel is shifted so as to take over the tracking. Thereby, it is possible to enlarge and track the specific subject P in a wide range. The lens barrels (2C to 2F) other than the tracking lens barrel also perform optical axis shift and optical zoom so as to compensate for the change in the angle of view of the tracking lens barrel. As a result, continuous omnidirectional imaging can be performed while enlarging and tracking the specific subject P.

  In FIG. 16B, the case where the lens barrels 2D and 2E are supplemented with the angle of view by both the axis shift and the optical zoom has been described, but only one of them may be performed. In FIG. 16B, the angle of view of the lens barrel 2F is reduced from θ7 to θ9 because the angle of the optical axis shift of the lens barrel 2F is increased to δ5 so that the angle of view of θ7 is the lens barrel. This is because it is deviated by 2F itself (that is, exceeding the limit field angle 2γ of the lens barrel 2F). The limit field angle is substantially equal to an angle obtained by adding an angle shift due to the optical axis shift to the left and right to the field angle at the wide-angle end of the lens barrel.

  Further, the angle of view of the lens barrel 2A was set to θ3 until the states of FIGS. 15A, 15B, and 16A, but the tracking from the lens barrel 2A to the lens barrel 2B was performed. When taking over, the angle of view was changed from θ3 to θ8. This is because the angle of view is distributed within the limit angle of view by the lens barrel 2A and the lens barrel 2F in order to maintain the omnidirectional imaging close distance L2. The same applies to the lens barrel 2B and the lens barrel 2C. However, it is desirable to maintain the field angles of the lens barrels 2A and 2B at θ3 if the omnidirectional imaging close distance L2 can be maintained without causing a problem with the limit field angle.

  Although the case where tracking is performed while the specific subject P is enlarged by shifting the optical axis of the lens barrel 2 has been described so far, the optical axis is shifted in order to enlarge a still subject that is not on the optical axis of the lens barrel 2. May be.

  Next, a process for changing the control for tracking the enlarged specific subject P according to the moving direction and moving speed of the specific subject P will be described. FIG. 17 shows a state in which the moving specific subject P is enlarged and captured by the lens barrel 2A as viewed from above. A thick arrow in the figure indicates a direction in which the specific subject P may move. As a moving direction of the specific subject P, a direction approaching the camera body 1 is D1, a direction away is D2, a direction toward the lens barrel 2B is D3, and a direction toward the lens barrel 2F is D4. These intermediate movements in the four directions are represented by a combination of two of the four directions. The imaging of the specific subject P is repeated at a predetermined cycle, and the direction and magnitude of the motion vector indicating the displacement direction and the displacement amount of the specific subject P between the temporally continuous previous image and subsequent image are detected. Thus, the moving direction and moving speed of the specific subject P can be acquired. The user can set a direction threshold and a speed threshold in advance for each of the movement direction and the movement speed. The direction threshold is a moving direction (angle) of the specific subject P with respect to the lens barrel 2A in which the lens barrel 2F adjacent to the lens barrel 2A can compensate for the change in the angle of view of the lens barrel 2A by shifting its optical axis. It is the maximum value.

  FIG. 18A shows a motion vector acquired when the specific subject P moves in the direction D1 smaller than the direction threshold at a speed slower than the speed threshold. Here, the contour of the face of the person who is the specific subject P and the motion vector of the body part are shown. In this case, since the specific subject P is approaching the lens barrel 2A, the direction of the motion vector is a direction in which the specific subject P is enlarged, and the size of the motion vector is small. At this time, the lens barrel 2A does not shift the optical axis, and the size of the specific subject P in the partial image acquired through the lens barrel 2A is maintained by expanding the angle of view by optical zoom to the wide angle side. Further, for the lens barrels 2B and 2F adjacent to the lens barrel 2A, the overlapping area increases by expanding the angle of view of the lens barrel 2A with respect to these angles of view, so that the angle of view is maintained as it is. To do. Thereby, the omnidirectional imaging close distance becomes shorter.

  FIG. 18B shows a motion vector acquired when the specific subject P is moving in the direction D1 at a speed faster than the speed threshold. In this case, since the specific subject P is approaching the lens barrel 2A at high speed, the direction of the motion vector is the direction in which the specific subject P is enlarged as in FIG. 18A, but compared with FIG. The motion vector is large. At this time as well, the lens barrel 2A is not shifted in the optical axis as in the case of FIG. 18A, but is acquired through the lens barrel 2A with a wide angle of view by high-speed optical zoom to the wide-angle side. The size of the specific subject P in the image is maintained. Further, the lens barrels 2B and 2F adjacent to the lens barrel 2A are the same as those in FIG. Thereby, the omnidirectional imaging close distance becomes shorter.

  On the other hand, when the specific subject P is moving in the direction D2 at a speed slower than the speed threshold and at a higher speed, the directions of the motion vectors are opposite to those in FIGS. The lens barrel 2A does not shift the optical axis, and narrows the angle of view by optical zooming to the telephoto side in order to maintain the size of the specific subject P in the partial image acquired through the lens barrel 2A. Further, as for the lens barrels 2B and 2F adjacent to the lens barrel 2A, since the field angle of the lens barrel 2A is narrowed with respect to these field angles, their overlapping area is reduced. The distance gets longer. In this situation, since the specific subject P moves away from the camera body 1, the omnidirectional imaging close distance may be allowed to be long, or the lens barrel 2B, The angle of view may be widened by optical zooming to the wide-angle side of 2F.

  FIG. 19A shows a motion vector acquired when the specific subject P moves in the direction D3 larger than the direction threshold at a speed slower than the speed threshold. In this case, since the specific subject P is moving from the angle of view of the lens barrel 2A toward the angle of view of the lens barrel 2B, the direction of the motion vector is the right direction in the figure, and the magnitude of the motion vector is small. At this time, for the lens barrel 2F adjacent on the opposite side of the moving direction of the lens barrel 2A and the specific subject P, the three types of images described in FIGS. 15A, 15B, and 16A are used. A method of compensating for the angular change is conceivable. When the moving speed of the specific subject P is slower than the speed threshold and the moving direction is larger than the direction threshold as in this case, the change in the angle of view of the lens barrel 2A for enlarging and tracking the specific subject P is detected by the lens barrel 2F. It is determined that it cannot be compensated only by shifting the optical axis. At this time, priority is given to widening the angle of view in the lens barrel 2F as shown in FIG. If this cannot compensate for the change in the angle of view, the optical axis is also shifted as shown in FIG. 16A, and the subject tracking to the lens barrel B is finally taken over.

  In FIG. 17, when the specific subject P is moving in a thin arrow direction between the D3 direction and the D2 direction at a speed slower than the speed threshold value, the specific subject P moves from the angle of view of the lens barrel 2A to the image of the lens barrel 2B. It moves slowly toward the corner and moves away from the camera body 1. In this case, when the moving speed is slower than the speed threshold and the moving direction is smaller than the direction threshold, the change in the angle of view of the lens barrel 2A for enlarging and tracking the specific subject P is shifted by the optical axis of the lens barrel 2F. It is determined that it can be supplemented with only. At this time, priority is given to shifting the optical axis in the lens barrel 2F as shown in FIG. If this cannot compensate for the change in the angle of view, the angle of view is widened as shown in FIG. 16A, and the object tracking to the lens barrel B is taken over as shown finally.

  FIG. 19B shows a motion vector acquired when the specific subject P is moving in the direction D3 larger than the direction threshold at a speed faster than the movement threshold. In this case, since the specific subject P is moving from the angle of view of the lens barrel 2A toward the angle of view of the lens barrel 2B, the direction of the motion vector is the right direction in the figure, and the size is shown in FIG. Bigger than). At this time, with respect to the lens barrel 2F adjacent on the side opposite to the moving direction of the lens barrel 2A and the specific subject P, the angle of view of 360 ° is set by the lens barrels 2A to 2F as described with reference to FIG. Cannot be compensated for each other, it is impossible to cope with a change in the angle of view of the lens barrel 2A due to subject tracking. When the moving speed is faster than the speed threshold and the moving direction is larger than the direction threshold as in this case, the change in the angle of view of the lens barrel 2A for enlarging and tracking the specific subject P is shifted by the optical axis of the lens barrel 2F. It is determined that it cannot be compensated only by At this time, as shown in FIG. 16B, the tracking of the subject to the lens barrel B is carried out while mutually compensating for an angle of view of 360 ° by the lens barrels 2A to 2F.

  In FIG. 17, when the specific subject P moves in the direction between the D3 direction and the D2 direction at a speed faster than the speed threshold, the specific subject P changes from the angle of view of the lens barrel 2A to the angle of view of the lens barrel 2B. Although it is moving fast toward the camera, it is moving away from the camera body 1. When the moving direction is smaller than the direction threshold as in this case, but the moving speed is faster than the speed threshold, the change in the angle of view of the lens barrel 2A for enlarging and tracking the specific subject P is shifted by the optical axis of the lens barrel 2F. It is determined that it cannot be compensated only by At this time, as shown in FIG. 16A, priority is given to optical axis shifting and wide-angle optical zooming in the lens barrel 2F. If this is not enough to compensate for the change in the angle of view, as shown in FIG. 16B, the 360 ° angle of view is mutually compensated by the lens barrels 2A to 2F, and the subject tracking to the lens barrel B is taken over. I do.

  The case where the moving direction of the specific subject P is the D1, D2, D3, and the directions between the D2 and D3 directions has been described above. However, the description about the D4 direction and other intermediate directions is the same as that of the lens barrel 2B. The same applies to the lens barrel 2F, for example.

  FIG. 20 shows the system configuration of the camera body 1. Each of the lens barrels 2 (2A to 2H) includes an optical unit 300 (1st to 3rd group barrels 21 to 22), an image sensor 301, and a motor drive unit 302 (drive motor 29). The optical unit 300 is moved in the optical axis direction by the motor drive unit 302. The amount of movement at this time is transmitted as a control signal from the CPU 306 serving as the control means to the motor drive unit 302 via the drive control unit 305.

  The subject image formed by the optical unit 300 is converted into an electrical signal by the image sensor 301 driven by the image sensor driver 303. The CPU 306 controls the driving of the image sensor 301 via the image sensor driver 303.

  The image processing unit 304 performs A / D conversion on the analog imaging signal output from the imaging element 301 of each lens barrel 2 and zooms the lens barrel 2 described above on the partial image as the obtained digital imaging signal. Luminance correction or the like according to the F value for each magnification is performed. The image processing unit 304 functions as a luminance correction unit. An image processing unit 304 as an image generating unit performs image processing such as gradation correction and white balance on the partial image obtained from each lens barrel 2. The lens barrel 2 (2A to 2H), the imaging element 301 and the image processing unit 304 provided in each of them constitute an imaging means.

  The image generation unit 309 performs a joining process on the partial images from the eight lens barrels 2 generated by the image processing unit 304 to generate an omnidirectional image. At this time, the image generation unit 309 functioning as a contrast detection unit detects the contrast of each partial image and determines a high contrast region and a low contrast region in the partial image. In response to the determination result, the CPU 306 causes the lens barrel 2 to perform an optical zoom necessary for including the stitch portion in the low contrast region of the partial image.

  The memory 307 includes a volatile memory or a non-volatile memory, and temporarily stores imaging data such as partial images and omnidirectional images, and stores a processing program executed by the CPU 306. The compression processing unit 308 compresses and encodes the captured image data using an encoding method such as JPEG. The communication control unit 310 transmits the compression-encoded imaging data to the outside.

  Note that the image generation unit 309 that performs partial image joining processing to generate an omnidirectional image is not necessarily provided in the omnidirectional camera. That is, the image data of the partial image may be sent to an image generation apparatus such as an external personal computer via the communication control unit 310, and the omnidirectional image may be generated by the image generation apparatus.

  The subject detection unit 311 and the subject determination unit 312 recognize a specific subject to be enlarged in the omnidirectional image, and determine the lens barrel 2 that is capturing the specific subject. Based on the determination result, the CPU 306 causes the lens barrel 2 capturing the specific subject to perform optical zoom toward the telephoto side, and causes the lens barrel 2 adjacent thereto to perform optical zoom toward the wide angle side. . Further, the CPU 306 detects the motion vector of the specific subject from the sequentially generated omnidirectional images, and compensates for the subject tracking described with reference to FIGS. 15A to 19B and the accompanying change in the angle of view. The lens barrel 2 is controlled. As described above, the CPU 306 is a control unit and also corresponds to a motion vector detection unit.

  In addition, the CPU 306 receives a command (instruction) for instructing optical zooming from the external instruction device 400 such as a personal computer or a smartphone to the telephoto side of the specific lens barrel 2 through wired or wireless communication via the communication control unit 310. Accept input. In accordance with the command, the CPU 306 causes the specific lens barrel 2 to perform optical zoom toward the telephoto side, and causes the lens barrel 2 adjacent thereto to perform optical zoom toward the wide angle side. Thereby, an omnidirectional imaging system including the omnidirectional camera and the external pointing device 400 is configured.

  FIG. 21 shows the lens when the lens barrel 2A zooms in on the specific subject P that moves to the angle of view of the lens barrel 2B as described with reference to FIGS. 15 (A) to 16 (B). The control of the lens barrels 2A to 2E is shown. A CPU 306 as a computer executes this processing in accordance with an imaging control program that is a computer program.

  In step S01, the CPU 306 obtains an omnidirectional image by performing omnidirectional imaging. Then, the CPU 306 determines whether or not the subject detection unit 311 has detected a specific subject P such as a person from the omnidirectional image. If the specific subject P is not detected, the CPU 306 returns to step S01. When the specific subject P is detected, the CPU 306 proceeds to step S02, and the lens barrel 2 including the specific subject P detected by the subject determination unit 312 among the eight lens barrels 2 (2A to 2H) within the angle of view. (Here, the lens barrel 2A) is determined. At this time, subject information such as the facial features of the specific subject P and the color of the clothes for tracking the specific subject P is stored.

  Next, in step S03, the CPU 306 detects a motion vector for the specific subject P. When the motion vector is detected, that is, when the specific subject P is moving, the CPU 306 proceeds to step S04. On the other hand, if the motion vector is not detected, that is, if the specific subject P is stationary, the CPU 306 repeats the determination in step S03.

  In step S04, the CPU 306 causes the lens barrel 2A to perform optical zooming toward the telephoto side to enlarge the specific subject P, and starts tracking the specific subject P by shifting the optical axis.

  Next, in step S05, the CPU 306 determines whether or not the moving speed of the specific subject P is lower than the speed threshold based on the magnitude of the motion vector detected in step S03. If the moving speed is lower than the speed threshold, the CPU 306 proceeds to step S06.

  In step 06, the CPU 306 determines whether or not the moving direction of the specific subject P is smaller than the direction threshold value from the direction of the motion vector. When the moving direction is smaller than the direction threshold value, the CPU 306 proceeds to step S07.

  In step S07, the CPU 306 selects a field angle correction A that compensates for a field angle change caused by subject tracking of the lens barrel 2A only by shifting the optical axis of the lens barrel 2F. In step S08, the CPU 03 causes the lens barrel 2F to shift the optical axis (shift operation of the second group lens 233) as described with reference to FIG. Thereafter, in order to continuously detect the movement of the specific subject P, the CPU 306 returns to step 03, and repeats the processing after step S04 if a motion vector is detected.

  On the other hand, if it is determined in step S06 that the moving direction of the specific subject P is larger than the direction threshold value, the CPU 306 proceeds to step S09. In step S09, the CPU 306 selects the angle of view correction B that compensates for the change in the angle of view of the lens barrel 2A due to subject tracking only by optical zooming to the wide angle side (enlargement of the angle of view). In step S10, the CPU 03 causes the lens barrel 2F to perform optical zoom to the wide angle side as described with reference to FIG. Thereafter, in order to continuously detect the movement of the specific subject P, the CPU 306 returns to step 03, and repeats the processing after step S04 if a motion vector is detected.

  If it is determined in step S05 that the moving speed of the specific subject P is higher than the speed threshold, the CPU 306 proceeds to step S11. In step S11, as in step S06, the CPU 306 determines whether or not the moving direction of the specific subject P is smaller than the direction threshold from the direction of the motion vector. If the moving direction is smaller than the direction threshold, the CPU 306 proceeds to step S12.

  In step S12, the CPU 306 shifts the angle of view of the lens barrel 2A tracking the specific subject P moving at high speed by shifting the optical axis of the lens barrel 2F and optical zooming to the wide angle side (enlarging the angle of view). ) To select the angle of view correction C to be compensated. In step S13, the CPU 306 causes the lens barrel 2F to perform the optical zoom to the wide angle side and the optical axis shift as described with reference to FIG. At this time, since the moving speed of the specific subject P is fast, priority is given to optical zoom to the wide angle side of the lens barrel 2F, that is, to widen the angle of view. Thereafter, in order to continuously detect the movement of the specific subject P, the CPU 306 returns to step 03, and repeats the processing after step S04 if a motion vector is detected.

  On the other hand, if it is determined in step S11 that the moving direction of the specific subject P is larger than the direction threshold value, the CPU 306 proceeds to step S14. In step S14, since the moving speed of the specific subject P is high and the moving direction cannot compensate the change in the angle of view of the lens barrel 2A with the lens barrel 2F, the CPU 306 performs tracking of the subject from the lens barrel 2A to the lens mirror. The angle-of-view correction D that is carried over to the cylinder 2B is performed. In step S15, as described in FIG. 16B, the CPU 306 causes the lens barrel 2A to take over the tracking of the specific subject P from the lens barrel 2A and also optically zooms the lens barrels 2A to 2F. The angle of view of 360 ° is complemented by shifting the optical axis. Thereafter, in order to continuously detect the movement of the specific subject P, the CPU 306 returns to step 03, and repeats the processing after step S04 if a motion vector is detected.

  As described above, in this embodiment, the angle of view change due to the enlarged tracking of the specific subject P in the lens barrel 2A is changed according to the result of comparing the moving direction and moving speed of the specific subject P with different threshold values. The method (processing) to be compensated by the lens barrels 2B to 2F is changed. Thereby, irrespective of the moving method and moving speed of the specific subject P, it is possible to realize omnidirectional imaging with a continuous field angle of 360 ° while accurately enlarging and tracking the specific subject P.

In this embodiment, the case where each lens barrel has a three-group variable magnification optical system has been described. However, this is an example, and other types of variable magnification optical systems may be used. Moreover, although the present Example demonstrated the case where a moving magnet type was used as an anti-vibration mechanism, a moving coil type may be sufficient.
(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

1 Camera body 2 (2A to 2H) Lens barrel 306 CPU

Claims (9)

Imaging means for acquiring a plurality of images that are continuously connected to each other by imaging through a plurality of variable magnification optical systems facing different directions;
Control means for controlling the operation of the plurality of variable magnification optical systems,
Each of the plurality of variable magnification optical systems includes a shift optical system capable of shifting with respect to the optical axis of the variable magnification optical system;
When the control means determines that an object to be moved is included in the angle of view of the first variable power optical system among the plurality of variable power optical systems, the first variable power optical system includes a telephoto lens. And the shift operation is performed so as to track the subject, and the at least one other variable power optical system different from the first variable power optical system is used. An image pickup apparatus that performs at least one of a magnification change operation and a shift operation so as to compensate for a change in angle of view due to the magnification change operation and the shift operation in a variable magnification optical system.   2. The control unit according to claim 1, wherein the control unit changes the operation of the at least one other variable power optical system for compensating for a change in the angle of view in accordance with a moving direction and a moving speed of the subject. The imaging device described in 1.   When the control unit determines that the subject is moving toward the field angle of the second variable power optical system among the at least one other variable power optical system, the second variable power optical The imaging apparatus according to claim 1, wherein the image pickup apparatus causes the system to perform a zooming operation toward the telephoto side and to perform the shift operation so as to take over the tracking of the subject. Imaging means for acquiring a plurality of images that are continuously connected to each other by imaging through a plurality of variable magnification optical systems facing different directions;
Control means for controlling the operation of the plurality of variable magnification optical systems,
Each of the plurality of variable magnification optical systems includes a shift optical system capable of shifting with respect to the optical axis of the variable magnification optical system;
The control means determines that a subject moving toward the angle of view of the second variable power optical system is included in the angle of view of the first variable power optical system among the plurality of variable power optical systems. In this case, the first variable power optical system is caused to perform a zooming operation toward the telephoto side, the shift operation is performed so as to track the subject, and the second variable power optical system is further moved toward the telephoto side. An image pickup apparatus characterized in that the zooming operation is performed and the shift operation is performed so as to take over the tracking of the subject.   The control means applies the first and second variable power optics to at least one third variable power optical system different from the first and second variable power optical systems among the plurality of variable power optical systems. The imaging apparatus according to claim 4, wherein at least one of a scaling operation and the shift operation is performed so as to compensate for a change in a field angle due to the scaling operation and the shift operation in the system. Motion vector detection means for acquiring a moving direction and a moving speed of the subject by detecting a motion vector from the image;
6. The imaging apparatus according to claim 1, wherein the control unit controls the shift operation of the first variable magnification optical system based on the moving direction and the moving speed. 7. . The imaging device according to any one of claims 1 to 6,
An image pickup system comprising: an instruction device that inputs an instruction for causing the control means to control a magnification operation of the plurality of magnification optical systems. An imaging apparatus that acquires a plurality of images that are continuously connected to each other by imaging through a plurality of variable magnification optical systems oriented in different directions, and each of the plurality of variable magnification optical systems includes the variable magnification optical system In a computer of an imaging apparatus including a shift optical system capable of shifting with respect to the optical axis of
In response to determining that the subject moving within the angle of view of the first variable power optical system among the plurality of variable power optical systems is included,
A process for causing the first variable magnification optical system to perform a zooming operation toward the telephoto side and performing the shift operation so as to track the subject;
Magnification operation so that at least one other magnifying optical system different from the first magnifying optical system compensates for a change in field angle due to the magnifying operation and the shift operation in the first magnifying optical system And an imaging control program for executing a process for performing at least one of the shift operations. An imaging apparatus that acquires a plurality of images that are continuously connected to each other by imaging through a plurality of variable magnification optical systems oriented in different directions, and each of the plurality of variable magnification optical systems includes the variable magnification optical system In a computer of an imaging apparatus including a shift optical system capable of shifting with respect to the optical axis of
In response to determining that the subject moving toward the field angle of the second variable power optical system is included in the field angle of the first variable power optical system among the plurality of variable power optical systems,
A process for causing the first variable magnification optical system to perform a zooming operation toward the telephoto side and performing the shift operation so as to track the subject;
An imaging control program that causes the second zoom optical system to perform a zooming operation toward the telephoto side and to perform the shift operation so as to take over the tracking of the subject.