JP7016722B2 - Imaging device and imaging control program - Google Patents

Description

The present invention relates to an imaging device suitable for wide-field imaging such as omnidirectional imaging and panoramic imaging.

As a method for performing wide-field imaging as described above, there is known a technique of joining a plurality of captured images obtained by a plurality of cameras so as to form a continuous image. When wide-field imaging is used for surveillance purposes such as crime prevention, it is often the case that a specific subject image is magnified and observed in detail from the captured wide landscape image. Conventionally, in such a case, a part of the captured image is cut out and electronically zoomed to enlarge the image. However, deterioration of image quality due to electronic zoom becomes a problem.

Patent Document 1 includes a wide-angle camera capable of capturing almost the entire monitoring area and a movable camera having a pan head mechanism and capable of changing the imaging direction and zoom magnification according to an external signal. The configured imaging system is disclosed. In this imaging system, the position of the subject to be tracked is specified from the wide-field image acquired by the wide-angle camera, and the imaging direction and zoom magnification of the movable camera are controlled so that the tracked image of the subject with high resolution can be obtained.

Japanese Unexamined Patent Publication No. 2015-194901

However, the imaging system disclosed in Patent Document 1 uses a wide-angle camera and a movable camera arranged at positions separated from each other, and one camera can be used for a wide-field image and a magnified image of a specific subject. Cannot be obtained.

Further, a camera has been proposed that can capture not only the omnidirectional image in the horizontal direction but also the omnidirectional (omnidirectional) imaging including the zenith direction and the ground direction at one time with one camera. However, in a camera that performs omnidirectional imaging, a member protruding from the camera, such as a tripod that supports the camera, is reflected in the image. In such a case, conventionally, an incomplete omnidirectional image excluding the region including the protruding member is first acquired, and then an image of the region captured so as not to include the protruding member is acquired first. A complete omnidirectional image was obtained by pasting it on the directional image.

The present invention provides an image pickup apparatus capable of performing wide-field imaging in which even if a projecting member is present, the image of the projecting member can be eliminated as much as possible by one imaging.

The image pickup apparatus as one aspect of the present invention includes an image pickup means for acquiring a plurality of images that have passed through a plurality of variable magnification optical systems facing different directions and are continuously connected to each other by image pickup, and the plurality of image pickup devices. The angle of view setting means for controlling the magnification operation of each of the variable magnification optical systems to set the angle of view for each variable magnification optical system, and the plurality of magnification optical systems even if the zoom magnifications of the plurality of variable magnification optical systems are different. A nodal point control means for controlling the nodal points of the plurality of variable magnification optical systems so as to make the parallax between partial images obtained through the variable magnification optical system substantially the same, and a main body holding the plurality of variable magnification optical systems. And a protrusion detecting means for detecting the protruding member, the angle of view setting means detects the angle of view of each of the plurality of variable magnification optical systems based on the detection result of the protrusion detecting means. It is characterized in that the angle of view setting control is performed so that the protruding member does not enter any of the angles of view of the plurality of variable magnification optical systems.

Further, an image pickup control program as a computer program for causing the computer of the image pickup apparatus to execute the angle of view setting control also constitutes another aspect of the present invention.

According to the present invention, even if a protruding member is present, it is possible to perform wide-field imaging in which the reflection of the protruding member is eliminated or is reduced as much as possible by one imaging.

External perspective view of an omnidirectional camera according to an embodiment of the present invention. The perspective view of the lens barrel in an Example. An exploded perspective view of the lens barrel in the embodiment. The exploded perspective view of the 2nd group lens barrel in an Example. The front view of the 2nd group lens barrel in an Example. Sectional drawing (collapse position) of the lens barrel in an Example. Sectional drawing (WIDE position) of the lens barrel in an Example. Sectional drawing (TELE position) of the lens barrel in an Example. The perspective view which shows the state which extended the tripod of the camera of an Example. The perspective view which shows the activation state of the camera of an Example. The conceptual diagram which shows the angle of view distribution in an Example. The figure which shows the lens angle of view (horizontal and vertical direction) in the activated state in an Example. The figure which shows the lens angle of view (horizontal and vertical direction) in a telephoto state in an Example. The conceptual diagram which shows the image pickup in the optical zoom in the camera of an Example. The figure which shows the omnidirectional image obtained by the camera (wide-angle state) of an Example. The figure which shows the omnidirectional image obtained by the camera (telephoto state) of an Example. Sectional drawing of the camera (telephoto state) of an Example in a vertical direction. The block diagram which shows the structure of the omnidirectional imaging system including the camera of an Example. The flowchart which shows the image pickup control processing of the camera of an Example. FIG. 3 is a perspective view showing a state in which the lighting unit is projected in the camera of the embodiment. The figure which shows the movable range of the lighting unit and the angle of view of a lens barrel in the camera of an Example. The figure which shows the range which the angle of view of the lens barrel in the camera of an Example can take. The figure which shows the angle of view of the leg of a tripod and the lens barrel in the camera of an Example. The figure which shows the grounding part of a tripod and the angle of view of a lens barrel in the camera of an Example. 21 is a diagram showing a state in which the movable lighting unit 4 described with reference to FIGS. 21 and 22 is set at the third position. A flowchart illustrating whether or not to allow the reflection of a movable lighting unit or a protruding member such as a tripod on a photographed image and the setting of a close range for imaging.

Hereinafter, examples of the present invention will be described with reference to the drawings.

FIG. 1 shows the appearance of an omnidirectional camera as an image pickup apparatus according to an embodiment of the present invention. This omnidirectional camera joins a plurality of images (partial images) obtained by imaging through a plurality of imaging optical systems facing different directions to form an image having an imaging range of 360 ° horizontally and 360 ° vertically. Generate a certain omnidirectional image (composite image).

1 is the camera body. The camera body 1 has a spherical exterior member 1a. Lens barrels 2 (2A to 2H) accommodating an image pickup lens face the outer directions in which the image pickup lenses are different from each other at six points in the horizontal plane and two points in the vertical direction (upper surface portion and lower surface portion) of the camera body 1, respectively. It is held (arranged) like this. Specifically, the lens barrels 2A to 2F face different directions in the horizontal plane, and the lens barrels 2G and 2H face the upper side and the lower side, which are different directions in the vertical direction, respectively.

The vertical straight line direction is a direction in which a straight line extends along a direction that receives gravity when the camera body 1 is fixed to a horizontal camera installation surface using a built-in tripod 5 described later, and is also referred to as a vertical direction. The horizontal plane is a plane orthogonal to the vertical direction, and the direction (direction) included in the horizontal plane is called the horizontal direction. All of the image pickup lenses in the total of eight lens barrels 2 are variable magnification optical systems capable of variable magnification operation (optical zoom) for changing the focal length. Although the variable magnification optical system actually corresponds to an image pickup lens, in the following description, the lens barrel including the image pickup lens will be described as corresponding to the variable magnification optical system.

A lens barrier 20 that opens and closes the front surface of the image pickup lens is provided at the front end of each lens barrel 2. The opening and closing of the lens barrier 20 is performed between the retracted state of the lens barrel 2 shown in FIG. 1 and the protruding state of the lens barrel 2 protruding from the camera body 1 according to the ON / OFF of the power button 7 arranged on the upper surface of the camera body 1. It is performed in conjunction with the operation of feeding and pulling in.

Further, between the lens barrel 2 (2G) on the upper surface of the camera body 1 and the lens barrels 2 (2A to 2F) at six locations in the horizontal plane, the subject around the camera is evenly illuminated. A ring-shaped illumination unit 3 is provided. Further, between the lens barrel 2 (2H) on the lower surface of the camera body 1 and the lens barrels 2 (2A to 2F) at six locations in the horizontal plane, the three movable lighting units 4 and the camera body 1 are stabilized. A built-in tripod (hereinafter, simply referred to as a tripod) 5 for fixing the tripod horizontally is provided. The position of the movable lighting unit 4 can be adjusted to an arbitrary lighting position protruding from the camera body 1 from the storage position shown in the figure. The user can arbitrarily change the length and angle of the legs of the tripod 5 from the stored state shown in the figure, whereby the height of the camera body 1 from the camera installation surface can be adjusted. The movable lighting unit 4 and the tripod 5 protruding from the storage position correspond to the protruding members.

A battery chamber 6 is provided on the lower surface of the camera body 1, and power is supplied from the battery stored in the battery chamber 6 to the camera body 1. Further, the upper surface of the camera body 1 is provided with a zoom button 8 for selecting a lens barrel 2 for performing optical zoom among the eight lens barrels 2 and instructing the optical zoom thereof. Further, a release button 9 for starting an image pickup preparation operation (focus adjustment operation and photometric operation) and an image pickup operation (exposure of an image sensor described later) is also provided.

The optical zoom, the image pickup preparation operation, and the image pickup operation can also be remotely controlled by communication from an external instruction device such as a personal computer or a smartphone.

FIG. 2 shows one of eight lens barrels 2 (2A to 2H) provided in the camera body 1. The lens barrel 2 is a retractable lens barrel, and the lens barrel 2 is housed in the camera body 1 when the power of the camera is off, and can be imaged by extending outward from the camera body 1 when the power is on. Further, as described above, since the lens barrier 20 covers the front surface of the image pickup lens in the power-off state, scratches on the front surface and intrusion of dust into the lens barrel are prevented.

FIG. 3 shows the lens barrel 2 in an exploded manner. The image pickup lens is composed of a 1st group lens, a 2nd group lens and a 3rd group lens (not shown). Reference numeral 21 is a group 1 lens barrel that holds the group 1 lens and is provided with the lens barrier 20 described above. Reference numeral 22 is a diaphragm unit for adjusting the amount of light. Reference numeral 23 is a 2nd group lens barrel that holds a 2nd group lens and is provided with an anti-vibration lens mechanism and a shutter.

Reference numeral 24 denotes a moving cam ring, and a cam groove portion for moving the group 1 lens barrel 21, the aperture unit 22, and the group 2 lens barrel 23 in the optical axis direction is provided on the inner peripheral portion thereof. A gear portion to which the driving force from the driving motor 29 is transmitted is provided on the outer periphery of the moving cam ring 24. The moving cam ring 24 is rotatably held around the optical axis with respect to the group 1 lens barrel 21, the aperture unit 22, the group 2 lens barrel 23, and the linear cylinder 25 and the fixed cylinder 27 described later, from the drive motor 29. It is rotationally driven by receiving the driving force of. The straight-ahead cylinder 25 guides the group 1 lens barrel 21, the aperture unit 22, and the group 2 lens barrel 23 in the optical axis direction, and prevents rotation around the optical axis. The 3rd group lens barrel 26 holds the 3rd group lens. The fixed cylinder 27 accommodates the group 1 lens barrel 21, the aperture unit 22, the group 2 lens barrel 23, the moving cam ring 24, and the group 3 lens barrel 26 so as to be movable in the optical axis direction. A cam groove portion for moving the moving cam ring 24 in the optical axis direction is provided on the inner peripheral portion of the fixed cylinder 27.

Reference numeral 28 is an image pickup element holding unit for holding an image pickup element (not shown), in which the fixed cylinder 27 is fixed and the drive motor 29 described above is held. Reference numeral 28a is a lens barrel flexible wiring board, which is attached to the image pickup element holding unit 28. The lens barrel flexible wiring board 28a is fixed to the outside of the aperture flexible wiring board 221 connected to the drive unit of the aperture unit 22 and the group 2 flexible wiring board 231 connected to the shutter drive unit of the group 2 lens barrel 23. Connected with.

The configuration of the lens barrel 2 described above is merely an example, and a lens barrel that performs optical zooming only by moving the lens inside may be used instead of the retractable type.
FIG. 4 shows the anti-vibration lens mechanism in the group 2 lens barrel 23 shown in FIG. 3 in an exploded manner. Further, FIG. 5 shows an anti-vibration lens mechanism seen from the subject side in the optical axis direction (however, excluding the second group flexible substrate 231 and the sensor holder 232 described later). 239 is a 2-group base, which is the base of the 2-group lens barrel 23. The second group base 239 is provided with three follower pins 239a that engage with the cam groove portion of the moving cam ring 24 described above. Further, a shutter yoke 239b is attached to a part of the region of the second group base 239 in the direction around the optical axis. The cover member 239c is covered on the shutter yoke 239b from the subject side and fixed to the second group base 239. The shutter actuator including the shutter yoke 239b and the rotor (not shown) is a two-point switching actuator in which the stop position of the arm provided on the rotor is switched by switching the energization direction.

Reference numeral 235 is a group 2 lens holder, and the group 2 lens 233 (LS2) is fixed to the group 2 lens holder 235 together with a mask member 234 for cutting harmful light by adhesion or the like. Further, magnets 235A and 235B are held in the second group lens holder 235. In the following description, the subscripts A and B of the reference numerals correspond to the A direction (pitch direction) and the B direction (yaw direction) shown in FIG. 5, respectively. The magnet 235A is held by the group 2 lens holder 235 so that the north pole and the south pole are arranged in the A direction, and the magnet 235B is held so that the north pole and the south pole are arranged in the B direction. Hooks 235a are provided at two positions of the second group lens holder 235, and one end of the tension spring 236 is hooked on each hook 235a.

237A and 237B are coil units including a coil and a bobbin. The coil units 237A and 237B are fixed to the concave portions provided at positions facing the magnets 235A and 235B in the optical axis direction at different positions from the shutter yoke 239b in the second group base 239 by a fixing means such as adhesive. There is. The power supply to the coil is performed by connecting the second group flexible substrate 231 to each terminal portion embedded in the bobbin and electrically connected to the coil.

The other end of the spring 236 hooked on the group 2 lens holder 235 is hooked on the hook 239d of the group 2 base 239. Three non-magnetic balls 238 are sandwiched between the second group base 239 and the second group lens holder 235. The spring 236 urges the group 2 lens holder 235 toward the group 2 base 239 in the optical axis direction. As a result, the group 2 lens holder 235 is pressed against the group 2 base 239 via the ball 238 in the optical axis direction. The second group lens holder 235 can smoothly move (shift) in a plane orthogonal to the optical axis while rolling the ball 238. By shifting the group 2 lens holder 235 in the plane according to camera shake such as camera shake, vibration isolation control for suppressing image shake on the image sensor can be performed.

The sensor holder 232 is arranged so as to cover the magnets 235A and 235B of the second group lens holder 235 and their peripheral portions from the subject side, and is fixed to the second group base 239. The sensor holder 232 positions and holds the Hall elements 231A and 231B, which will be described later.

Reference numeral 231 is a group 2 flexible substrate, to which the coil units 237A and 237B described above and the shutter actuator are connected. Further, Hall elements 231A and 231B are mounted on the group 2 flexible substrate 231. By detecting the change in the magnetic field from the magnets 235A and 235B that shift with the 2nd group lens holder 235 by the Hall elements 231A and 231B, the amount of movement of the 2nd group lens holder 235 from the neutral position on the optical axis, that is, the position can be determined. Can be detected. This makes it possible to perform feedback control of the position of the second group lens holder 235 in the vibration isolation control.

FIG. 6 shows a cross section of the lens barrel 2 in a retracted state (collapsed position). In the retracted state, the 1st, 2nd and 3rd group lenses LS1, LS2, LS3 are arranged so that the distance between them is minimized, and the lens barrier 20 is closed.

FIG. 7 shows a cross section of the lens barrel 2 in the wide-angle end state (Wide position). By driving the drive motor 29 from the retracted state, the 1st group lens barrel 21 and the moving cam ring 24 are extended so as to protrude from the fixed cylinder 27 (camera body 1), and the 1st, 2nd and 3rd group lenses LS1, LS2, LS3. Is placed at the position shown in the figure and is ready for imaging. At this time, the angle of view of the lens barrel 2 is maximized, and the image area obtained through one lens barrel 2 is the widest in the omnidirectional image described later.

FIG. 8 shows a cross section of the lens barrel 2 in the telephoto end state (Tele position). By driving the drive motor 29 from the wide-angle end state, the 1st group lens barrel 21 and the moving cam ring 24 are further extended with respect to the fixed cylinder 27 from the wide-angle end state, and the 1st, 2nd and 3rd group lenses LS1, LS2, LS3 It is placed at the position shown in the figure. At this time, the angle of view of the lens barrel 2 is minimized, and the image area obtained through one lens barrel 2 in the omnidirectional image is the narrowest. In this embodiment, a case where a lens barrel having a variable magnification optical system having three lens groups is used will be described, but the number of lens groups may be a number other than three.

When you want to enlarge the image area of a specific subject in an omnidirectional image, the optical image of that subject is displayed on the image sensor by performing an optical zoom to the telephoto side of the lens barrel 2 that is imaging that subject. It can be magnified and projected. Therefore, it is possible to obtain an enlarged subject image without deterioration of image quality as in the case of electronic zoom. Each of the eight lens barrels 2 provided in the camera can continuously change the angle of view within the range from the wide-angle end state to the telephoto end state.

FIG. 9 shows a user manually pulling out the tripod 5 and fixing it to the camera installation surface in preparation for the camera body 1 before imaging. The three leg portions 5a of the tripod 5 can be extended in the C direction from the retracted state shown in FIG. 1, and the camera body 1 is supported by the three ground contact portions 5b. The angle between the three legs 5a can be changed from α to β in the D direction, and each leg 5a can be expanded and contracted, so the height of the camera body 1 can be freely adjusted by combining them. Can be adjusted.

The camera body 1 has a built-in mechanism for matching the angles of the three legs 5a with respect to the vertical direction. As a result, if the camera installation surface is horizontal, the lens barrels 2A to 2F provided on the camera body 1 are accurately oriented horizontally, and the lens barrels 2G and 2H are accurately above the vertical direction (heaven). You can face the side) and the bottom (ground side).

FIG. 10 shows a state in which the power button 7 provided on the upper surface of the camera body 1 is turned on by the user to activate the camera. All the lens barrels 2 (2A to 2H) extend from the power off state, the lens barrier 20 opens, and an image can be taken. In this state, the angles of view of the eight lens barrels 2 are set so that omnidirectional imaging can be performed by adding the vertical direction to the vertical 360 °.

FIG. 11 shows the distribution of an image region (hereinafter referred to as a partial image) with respect to eight lens barrels 2A to 2H in an omnidirectional image which is an image captured by omnidirectional imaging. Partial images obtained through each of the lens barrels 2A to 2F in the horizontal 360 ° are shown by ImgA to ImgF. Further, partial images obtained through the lens barrel 2G on the top side and the lens barrel 2H on the ground side are shown by ImgG and ImgH.

The boundary line between the partial images is a stitch portion in which adjacent partial images are joined together in order to obtain one continuous omnidirectional image. In this embodiment, an image larger than the partial image is acquired through each of the lens barrels 2A to 2H. Then, a common feature point is detected from a peripheral area larger than the partial image in two of the eight acquired images, and the two acquired images are joined so that the feature points overlap, so that the subject shifts in the stitch portion. A partial image is obtained that is spliced together so that there is no such thing. An omnidirectional image is generated by performing such a joining process on all acquired images.

FIG. 12A shows the angle of view of the lens barrel 2 (2A to 2F) in the horizontal plane when the camera body 1 is viewed from above. The angles of view of the six lens barrels 2A to 2F arranged at equal intervals in the circumferential direction in the horizontal plane are θ1 respectively. Then, omnidirectional imaging within a continuous 360 ° range is possible in a region outside the circle having a radius R1 in which some of the angles of view of the lens barrels 2 adjacent to each other in the circumferential direction overlap each other. R1 is the distance from the center of the camera body 1 (the position of the optical axis of the lens barrel 2G) to the position where continuous omnidirectional imaging is possible, and L1 is the distance from the center of the camera body 1 to the lens barrels 2A to 2F, respectively. The distance to the tip of the lens. L2 indicates the omnidirectional imaging close range obtained by subtracting L1 from R1, that is, the shortest imaging distance at which discontinuity does not occur between the respective image angles of the lens barrels 2A to 2F.

In this embodiment, the angle of view of the lens barrel 2 will be described as a simple conical angle of view, not as an angle of view considering the short side, long side, and diagonal of the image pickup element. Further, the amount of extension of the lens barrel 2 changes depending on the zoom, but the amount of change is sufficiently small for the omnidirectional imaging close-up distance L2 and can be ignored. Therefore, the lens barrel can be seen from the center of the camera body 1 regardless of the zoom position. Let L1 be the distance to each tip of 2A to 2F.

FIG. 12B shows the angles of view of the lens barrels 2A, 2D, 2G, and 2H in the vertical plane when the camera body 1 in the state shown in FIG. 12A is viewed from the horizontal direction (the direction of the arrow in FIG. 12A). Shows. The angles of view of the lens barrels 2A and 2D are θ1, respectively. On the other hand, the angle of view of each of the lens barrels 2G and 2H is θ2 (> θ1). Continuous omnidirectional imaging is possible in the region outside the circle with radius R1 where some of the angles of view of the lens barrels 2A, 2D, 2G, and 2H overlap.

Similar to FIG. 12A, in order to obtain the omnidirectional imaging close-up distance L2, it is necessary to set the distance from the center of the camera body 1 to the position where continuous omnidirectional imaging is possible as R1. However, since the number of lens barrels 2G and 2H used for omnidirectional imaging is smaller in the vertical plane than in the horizontal plane, the angle of view of these lens barrels 2G and 2H is set to θ2, which is larger than θ1.

At the time of starting the camera, for example, the angle of view θ1 of the lens barrels 2A to 2F and the angle of view θ2 of the lens barrels 2G and 2H are set so that the shortest omnidirectional imaging close range L2 can be obtained. However, the narrower angles of view θ1 and θ2 may be set so that a longer omnidirectional image pickup close range L2 can be obtained when the camera is activated.

In FIG. 13A, only the lens barrel 2A (first variable magnification optical system) is subjected to optical zoom (magnification operation) toward the telephoto side in order to magnify the specific subject P from the state shown in FIG. 12A and 12B. The camera body 1 is shown when viewed from above. The angle of view of the lens barrel 2A is narrowed from θ1 shown by the dotted line to θ3 (<θ1). Therefore, if the angles of view of the lens barrels 2B and 2F on both sides of the lens barrel 2A are the original θ1 indicated by the dotted lines, some of them and the angles of view of the lens barrel 2A do not overlap, and all of them are continuous. Orientation imaging becomes impossible.

Therefore, in this embodiment, the angle of view of the lens barrels 2B and 2F for acquiring the partial image spliced to the partial image acquired through the lens barrel 2A is widened to θ4 (> θ1) on the wide-angle side of θ1. Control. That is, the lens barrels 2B and 2F (second variable magnification optical system) are made to perform optical zoom to the wide-angle side. As a result, a part of the angle of view of the lens barrels 2B and 2F can be overlapped with the angle of view of the narrowed lens barrel 2A, and continuous omnidirectional imaging can be performed. At this time, the angle of view θ4 of the lens barrels 2B and 2F is calculated and set so that the same omnidirectional imaging close range L2 (that is, the distance R1) as before the angle of view of the lens barrel 2A is maintained. Is desirable.

Since the angles of view of the other lens barrels 2C to 2E in the horizontal plane are not changed, the overlapping area of the angles of view of the lens barrels 2B and 2C and the overlapping area of the angles of view of the lens barrels 2E and 2F are expanded. The closest distance to omnidirectional imaging near those lens barrels is closer than L2. In this case, the user may be notified that the omnidirectional imaging close-up range L2 is shortened only in a part of the region, or the detection range of the feature points in the joining process may be expanded.

FIG. 13B shows the angles of view of the lens barrels 2A, 2D, 2G, and 2H in the vertical plane when the camera body 1 in the state shown in FIG. 13A is viewed from the horizontal direction (the direction of the arrow in FIG. 13A). Shows. Similar to FIG. 13A, the angle of view of the lens barrel 2A is narrowed from θ1 shown by the dotted line to θ3 (<θ1). Therefore, if the angles of view of the lens barrels 2G and 2H on both sides in the vertical direction of the lens barrel 2A remain the same as the original θ2 indicated by the dotted line, a part of them and the angle of view of the lens barrel 2A do not overlap. Continuous omnidirectional imaging becomes impossible.

Therefore, in this embodiment, the angle of view of the lens barrels 2G and 2H is controlled to be widened to θ5 (> θ2) on the wide-angle side of θ2. As a result, a part of the angle of view of the lens barrels 2G and 2H can be overlapped with the angle of view of the narrowed lens barrel 2A, and continuous omnidirectional imaging can be performed.

At this time, as described with reference to FIG. 13A, the angle of view θ5 of the lens barrels 2G and 2H maintains the same omnidirectional imaging close range L2 (that is, the distance R1) as before the angle of view of the lens barrel 2A was changed. It is desirable to calculate and set so as to. That is, it is desirable that the angle of view of θ5 is set so as to pass through Q1 and Q2 in FIG. 13B. However, if such a wide angle of view cannot be set even when the lens barrels 2G and 2H are in the wide-angle end state, omnidirectional imaging is possible at an angle of view θ5 from the omnidirectional imaging close-up distance L2 as shown in the figure. It is necessary to change to the omnidirectional imaging close range L3 (that is, the distance R2). In this case, continuous omnidirectional imaging cannot be performed at the preset omnidirectional imaging close range L2, and the user may be notified that the omnidirectional imaging close range changes from L2 to L3 (> L2). desirable.

In this way, when the number of lens barrels 2 for performing omnidirectional imaging differs between the horizontal plane and the vertical plane, the image pickup lens of the lens barrel in the horizontal plane and the image pickup lens of the lens barrel on the top and bottom side The focal length and magnification may be different.

Further, when the lens barrel is provided with an image pickup lens having an ultra-wide angle of view exceeding 180 °, light rays may be cast by the camera body 1, so the lens barrel is used as a subject as in the present embodiment. It is desirable to extend to the side.

As described above, of the eight lens barrels 2, for example, when the lens barrel 2A is optically zoomed to the telephoto side, the optics of the lens barrels 2B, 2F, 2G, and 2H adjacent to the lens barrel 2A to the wide-angle side. Continuous omnidirectional imaging can be maintained by zooming. However, there is a condition between the arrangement angle of the lens barrel and the angle of view to make this possible.

In FIG. 13A, θ is an even arrangement angle between the lens barrels 2A to 2F adjacent to each other in the horizontal plane (the angle formed by the optical axes of the lens barrels 2A to 2F). If the angle of view of θ3 of the lens barrel 2A and the angle of view of θ1 of the lens barrel 2F do not overlap at all even at an infinite distance, continuous omnidirectional imaging cannot be performed. The reason why the angles of view do not overlap at all even at infinity is that the line at the end of the half angle of view θ3 / 2 of the lens barrel 2A and the line at the end of the half angle of view θ1 / 2 of the lens barrel 2F open from parallel. This is because the angle X between these lines is 0 or more. Therefore, it is a condition that a part of the angle of view of the adjacent lens barrels overlaps with each other that the angle X becomes smaller than 0. The angle X is obtained by subtracting the sum of the half-angle of view θ3 / 2 of the lens barrel 2A and the half-angle of view θ1 / 2 of the lens barrel 2F from the arrangement angle θ. That is,
X = θ- (θ3 / 2 + θ1 / 2) <0
θ <θ3 / 2 + θ1 / 2 ... (1)
The relationship is established. Note that θ3 corresponds to the angle of view θ ₁ of the first variable magnification optical system, and θ1 corresponds to the angle of view θ ₂ of the second variable magnification optical system.

As described above, in this embodiment, the narrowing of the angle of view of the lens barrel that has been optically zoomed to the telephoto side is compensated for by a wider angle of view than the optical zoom to the wide-angle side of the lens barrel next to it. At this time, it is necessary to satisfy the condition that the sum of the half angles of view of these adjacent lens barrels is at least larger than the arrangement angle between these lens barrels. Since the equation (1) is a condition for performing continuous omnidirectional imaging at an infinity distance, if the omnidirectional imaging close range is set shorter than the infinity distance as described above, it is further half. It is necessary to increase the sum of the angles of view to increase the amount of overlap of the angles of view. Further, in any case, since it is necessary to perform a joining process of partial images obtained through separate lens barrels, it is desirable to perform imaging with a further widened angle of view as a margin for joining.

By using the optical zoom in this embodiment, an omnidirectional image and an enlarged image of a specific subject can be acquired at the same time. Since the omnidirectional image is acquired as a continuous image by the plurality of lens barrels 2 compensating for each angle of view, the appearance on the viewer does not change even if the position of the stitch portion changes. However, when an enlarged image of a specific subject is suddenly displayed in an omnidirectional image due to an optical zoom to the telephoto side of a certain lens barrel, it gives a sense of discomfort to the viewer. Therefore, in this embodiment, as shown in FIG. 14, partial images are continuously acquired during the scaling of the lens barrel 2 (during optical zoom). Here, as described in FIG. 13A and 13B, when the lens barrel 2A is optically zoomed to the telephoto side and the adjacent lens barrels 2B, 2F, 2G, and 2H are optically zoomed to the wide-angle side. Will be explained. In FIG. 14, W marked near each lens barrel 2 indicates the wide-angle side, and T indicates the telephoto side.

When the optical zoom is performed from the activated state of the camera to the telephoto side of the lens barrel 2A, at the same time, to the wide-angle side of the lens barrel 2B, 2F, 2G, 2H so as to satisfy the above-mentioned omnidirectional imaging close range. Optical zoom is performed. From the start of this optical zoom, imaging through each lens barrel during the optical zoom begins. As a result, a plurality of partial images ImgA (1 to n) passed through the lens barrel 2A during optical zooming from the wide-angle side to the telephoto side are sequentially acquired. Further, a plurality of partial images ImgB (1 to n), ImgF (1 to n), ImgG (1 to n) passed through the lens barrels 2B, 2F, 2G, and 2H during optical zooming from the telephoto side to the wide-angle side. , ImgH (1 to n) are sequentially acquired. Then, the first partial image acquired through each of the lens barrels 2A, 2B, 2F, 2G, and 2H and the partial image acquired through the lens barrels 2C to 2E without optical zoom are joined. As a result, the first omnidirectional image is generated.

The reason why the optical zoom of the lens barrels 2C to 2E is not performed is that the lens barrels 2B, 2F, 2G, and 2H are on the wide-angle side, that is, the optical zoom is in the direction in which the overlapping region with the angle of view of the lens barrels 2C to 2E increases. Because it is doing.

Similarly, n partial images are acquired through each of the lens barrels 2A, 2B, 2F, 2G, and 2H until the time when the optical zoom of the lens barrel 2A is completed, and n omnidirectional images are acquired.

By sequentially displaying the n omnidirectional images thus obtained on the viewer, it is possible to display the enlarged image of the specific subject so as to be magnified smoothly with high image quality (so as not to cause a sense of discomfort). Further, since the enlarged image of another subject existing in the vicinity of the specific subject gradually changes, it is possible to obtain an omnidirectional image with less discomfort.

Here, an example of performing optical zooming to the telephoto side of only one lens barrel has been described, but it is not always the case that the main subject is determined when acquiring an omnidirectional image. Therefore, a case where the optical zoom from the wide-angle side to the telephoto side is performed not only with one lens barrel but with all lens barrels will be described. Thereby, in this embodiment, eight magnified images can be acquired through the eight lens barrels 2A to 2H.

At this time, the eight lens barrels may be optically zoomed one by one from the wide-angle side to the telephoto side to sequentially acquire magnified images corresponding to the eight lens barrels. However, by simultaneously performing optical zooming to the telephoto side of the lens barrels 2A, 2C, 2E in the horizontal plane, and simultaneously performing optical zooming to the wide-angle side of the other lens barrels 2B, 2D, 2F, 2G, 2H. , Enlarged images that have passed through the lens barrels 2A, 2C, and 2E can be collectively acquired. Next, by simultaneously performing optical zooming to the telephoto side of the lens barrels 2B, 2D, 2F and optical zooming to the wide-angle side of the other lens barrels 2A, 2C, 2E, 2G, 2H, the lens Enlarged images that have passed through the lens barrels 2B, 2D, and 2F can be collectively acquired. Furthermore, by simultaneously performing optical zooming to the telephoto side of the lens barrels 2G and 2H while simultaneously performing optical zooming to the wide-angle side of the other lens barrels 2A to 2F, enlargement through the lens barrels 2G and 2H is performed. Images can be acquired together.

By simultaneously performing the optical zoom of a plurality of lens barrels in this way, the magnified image through all the lens barrels can be captured in a short time with a small number of imagings, as compared with the case where the optical zoom is performed for each lens barrel. It can be acquired, and the amount of image data can be further reduced.

FIG. 15 shows an example of an omnidirectional image acquired by omnidirectional imaging with the camera shown in FIG. 10 for a landscape of a skyscraper town. 30 is a group of buildings and 31 is a group of trees. 32 is the ground and 33 is the sky. The dotted line SG is a partial image (image inside the dotted line SG) obtained through the lens barrel 2G on the top side and a partial image (image outside the dotted line SG) obtained through the lens barrels 2A to 2F in the horizontal plane. It is a stitch part. The dotted line SH is a partial image (image outside the dotted line SH) obtained through the lens barrel 2H on the ground side and a partial image (image inside the dotted line SH) obtained through the lens barrels 2A to 2F in the horizontal plane. It is a stitch part with.

In such an omnidirectional image, the stitch portions SG and SH overlap the subject such as the building group 30 and the trees 31, and in particular, the window of the building group 30 is a feature point because the subject is a subject in which the same shape is continuously repeated. Is easy to falsely detect. As a result, the stitch portion may be distorted or discontinuous, and an unnatural omnidirectional image may be generated.

Therefore, in this embodiment, the camera body 1 is provided with a function as a contrast detecting means for detecting the contrast of the partial image. Then, in the partial image, the high contrast region (first region) including the building group 30 having high contrast and the low contrast region (second region) including the ground 32, the sky 33, etc. having low contrast are discriminated from each other. Further, the lens barrels 2G and 2H are optically zoomed so that the stitch portions SG and SH can generate a partial image included in the low contrast region. At this time, optical zooming of the other lens barrels 2A to 2F is also performed so as to compensate for the change in the angle of view of the lens barrels 2G and 2H.

Specifically, the optical zoom of the lens barrels 2G and 2H (first variable magnification optical system) to the telescopic side and the optical zoom of the lens barrels 2A to 2F (second variable magnification optical system) to the wide angle side. And do. As a result, the stitch portions (joining regions) SG and SH shown by the dotted line are moved to the stitch portions SG'and SH'shown by the solid line. By such optical zoom processing, it is possible to reduce the possibility that the stitch portion is distorted or discontinuous and an unnatural omnidirectional image is generated.

After determining the contrast, a partial image in which the stitch portion is included in the low contrast region may be newly acquired, or may be extracted from a plurality of partial images acquired during the above-mentioned optical zoom.

FIG. 16 is obtained when the lens barrel 2A is optically zoomed to the telephoto side and the lens barrels 2B, 2F, 2G, and 2H are optically zoomed to the wide-angle side as described with reference to FIG. 13A and 13B. The omnidirectional image to be taken is shown. Some variable magnification optical systems change their brightness (F value) according to the variable magnification. Specifically, it becomes darker (F value becomes larger) as the magnification on the telephoto side increases. Here, the brightness correction when a lens barrel whose F value changes according to the scaling factor is used will be described.

The partial image ImgA in the figure is an image obtained through the lens barrel 2A in the variable magnification state (zoom magnification) on the telephoto side of the lens barrels 2C to 2E. Partial images ImgB, ImgF, ImgG and ImgH are images obtained through each of the lens barrels 2B, 2F, 2G and 2H in a variable magnification state on the wide-angle side of the lens barrels 2C to 2E, respectively. The partial image ImgA is a darker image than the partial images ImgC to ImgE obtained through the lens barrels 2C to 2E, and the partial images ImgB, ImgF, ImgG and ImgH are brighter images than the partial images 2C to 2E.

The original angle of view of each lens barrel 2 is different due to the difference in the number of lens barrels 2 arranged in the horizontal plane and in the vertical plane, and the zoom magnification of each lens barrel 2 is different. The values are different, and the brightness of the obtained partial image is also different. This causes a difference in brightness in the stitch portion in the omnidirectional image.

Therefore, in this embodiment, the camera body 1 is provided with a function as a luminance correction means. This luminance correction function converts the difference in F value, which is determined in advance according to the zoom magnification, into the number of steps of brightness, and performs the luminance correction (luminance control) of the partial image using the number of steps.

This luminance correction may be performed so as to match the lens barrel 2 having the largest number of lens barrels having the same optical conditions including the F value among the eight lens barrels 2, or it may be determined in advance by the user. May be good. Since there is a high possibility that a high-brightness subject such as the sun or lighting is always present somewhere in the omnidirectional image, the brightness is rarely uniform and it is necessary to correct the brightness according to the imaging environment. By performing brightness correction in the camera body 1 that corrects the difference in F-number according to the difference in zoom magnification before the brightness correction according to the imaging environment, the difference in brightness at the stitch portion is small and natural. It is possible to acquire an omnidirectional image.

FIG. 17 shows a state in which the lens barrel 2A is optically zoomed to the telephoto side of the lens barrels 2C to 2E and the lens barrels 2B and 2F are optically zoomed to the wide-angle side as shown in FIG. 13A. The cross section when the camera body 1 is cut by the plane Z shown in the figure is shown. The spherical exterior member of the camera body 1 is not shown.

Generally, the entrance pupil position of the variable magnification optical system (hereinafter referred to as a nodal point) is located near the lens group (1st group lens LS1 in this embodiment) arranged closest to the subject side, and the image is magnified to the telephoto side. As the angle becomes narrower, the nodal point moves toward the imaging surface. When a panoramic image or an omnidirectional image is generated by joining multiple partial images obtained by imaging through a plurality of lens barrels, the nodal points of the plurality of lens barrels are brought as close as possible to the plurality of lenses. It is desirable to reduce the discrepancy between the partial images obtained through the lens barrel. This makes it possible to smoothly join the partial images. In this embodiment, the nodal points of the lens barrels 2A to 2F cannot be brought close to match, but as an alternative method, the nodal points P2A to P2F of these lens barrels 2A to 2F are on the same circle. The method of arranging in is explained.

In the camera body 1, a moving mechanism M for moving the entire lens barrel 2 in the optical axis direction by a drive source (not shown) different from the drive motor 29 is provided for each lens barrel 2 (2A to 2F). It is provided. A position sensor (not shown) detects the position of the lens barrel 2 in the optical axis direction and also detects the zoom magnification (zoom position) of the lens barrel 2, and the nodal point is predetermined even if the zoom magnification changes. The movement mechanism M is controlled so as to be located on the same circle C. Since the position of the nodal point is optically determined according to the zoom magnification, the position of the nodal point for each zoom magnification is not shown as the difference amount from a predetermined reference position (for example, the position at the wide-angle end state). You can store it in memory. Then, the lens barrel 2 may be moved by the moving mechanism M so as to cancel the difference amount corresponding to the detected zoom magnification.

In FIG. 17, the entire lens barrel 2A that has been optically zoomed to the telephoto side is moved to the subject side by the moving mechanism M with reference to the position P of the lens barrels 2C to 2E in the optical axis direction, and is on the wide-angle side. The entire lens barrels 2B and 2F that have been optically zoomed to are moved to the image pickup surface side. As a result, even if the zoom magnifications of the lens barrels 2A to 2F are different, the nodal points P2A to P2F of these lens barrels 2A to 2F can be positioned on the same circle C, and the lens barrels 2A to 2F can be positioned. The parallax between the partial images obtained through can be almost matched. Therefore, it is possible to generate an omnidirectional image in which the connection of a plurality of partial images is natural.

The same circle in which the nodal points P2A to P2F of the lens barrels 2A to 2F should be located means a circle having a width in the radial direction thereof. The width corresponds to the permissible range of nodal point deviation in which the parallax of the partial images acquired through the lens barrels 2A to 2F does not pose a problem in their connection.

FIG. 17 has described a case where the moving mechanism M is controlled so that the nodal points of the lens barrels 2A to 2F arranged in the horizontal plane are arranged on the same circle, but in the vertical plane shown in FIG. 13B. It can also be applied to the arranged lens barrels 2G, 2H, 2A, and 2D. In addition, a plurality of lens barrels are arranged so that their optical paths intersect each other, and the lens barrels are moved so as to cancel the movement of the nodal point due to the change in the zoom magnification of these multiple lens barrels. May be good.

FIG. 20 shows a state in which the movable lighting unit 4 is projected from the camera body 1. The movable lighting unit 4 is a lower surface portion between the lens barrel 2H and the lens barrels 2A to 2F in the camera body 1 and is between the leg portions 5a of the tripod 5 (lower side of the lens barrels 2B, 2D, 2F). ) Are provided, and the illumination position (protruding position) can be adjusted for each. The movable lighting unit 4 has a light emitting element (not shown) such as an LED or an Xe tube, a window portion 41 that transmits the illumination light from the light emitting element, and a holder 42 that holds them. Further, the movable lighting unit 4 includes an eleventh arm member 44 that rotatably holds the holder 42 around the shaft 43, and a second arm member 46 that rotatably supports the first arm portion 44 about the shaft 45. Has.

The second arm member 46 is rotatably supported around a shaft 47 provided on the camera body 1. Therefore, the holder 42 can be moved (protruded or changed its direction) with respect to the camera body 1 by rotating around the axes 43, 45, 47, and the required protrusion amount and illumination direction are requested by the user. Can be changed according to. Further, the light emitting element in the holder 42 and the circuit board (not shown) in the camera body 1 are connected via the flexible wiring board 48.

In this embodiment, the window portion 41 is exposed even in the stored state shown in FIG. However, the window portion may be completely stored in the camera body 1 in the stored state. Further, although the three movable lighting units 4 of this embodiment can be independently projected and stored with respect to the camera body 1, they are interlocked by a link mechanism and projected and stored with respect to the camera body 1. You may do so. At the time of omnidirectional imaging, there is a high possibility that there is a light source such as the sun or lighting directly above, and the lower side of the camera body 1 tends to be a shadow of the camera body 1. Therefore, by providing such a movable lighting unit 4, a lens mirror is provided. Appropriate exposure can be obtained by imaging through the cylinder 2H.

FIG. 21 shows the camera body 1 as viewed from the direction of the arrow in FIG. 20, and shows the movable area of the movable lighting unit 4 arranged between the lens barrels 2D and 2H among the three movable lighting units 4. There is. The tripod 5 is not shown. The second arm member 46 of the movable lighting unit 4 rotates about the axis 47 by an angle δ when the movable lighting unit 4 protrudes from the camera body 1 and is moved to the farthest position. The first arm member 44 extends from the tip of the second arm member 46, and the holder 42 is rotatably connected to the tip of the first arm member 44. Therefore, the movement locus of the tip of the movable lighting unit 4 when it moves to the position farthest from the camera body 1, that is, the outer edge of the movable region of the movable lighting unit 4 is approximately shown by the dotted line 48, and the movement locus thereof. You can move freely in the room.

The movable lighting unit 4 illuminates directly under the camera body 1 at the first position (indicated by the circle 1) in the figure, and illuminates diagonally below the camera body 1 at the second position (indicated by the circle 2). Illuminate the camera body 1 in the horizontal direction at the third position (indicated by the circle 3). When the movable lighting unit 4 is projected from the camera body 1, there is a risk that the movable lighting unit 4 will enter the angle of view of the lens barrels 2C, 2D, 2E, and 2H (it will be reflected in the partial image acquired by each). be. Here, the relationship between the angle of view of the lens barrels 2D and 2H and the position of the movable lighting unit 4 will be described.

The movable lighting unit 4 approaches the lens barrel 2H near the first position, but if the angle of view of the lens barrel 2H is made smaller than θ5, the movable lighting unit 4 does not enter the angle of view. Similarly, when the movable lighting unit 4 approaches the lens barrel 2D near the third position, if the angle of view of the lens barrel 2D is made smaller than θ6, the movable lighting unit 4 does not enter the angle of view. That is, by setting the angles of view of the lens barrels 2H and 2D to be smaller than θ5 and θ6, respectively, the movable lighting unit 4 can be placed in the movable area (dotted line 48) of the holder 42 regardless of the position. do not have. Therefore, through the lens barrels 2H and 2D, it is possible to acquire a continuous partial image at a distance farther than the distance to the position Q1 where some of these angles of view overlap each other.

The information indicating that the movable lighting unit 4 protrudes from the camera body 1 includes non-detection of a switch that detects that the holder 42 of the movable lighting unit 4 is stored in the camera body 1, and magnetic detection of a Hall element, a GMR sensor, or the like. It can be obtained by detecting a change in the magnetic field using an element. The switch and the magnetic detection element used at this time correspond to the protrusion detecting means. In this case, only the information indicating that the movable lighting unit 4 protrudes from the camera body 1 is limited to the angle of view of the lens barrels 2D and 2H regardless of where the movable lighting unit 4 is located in the movable area. Will be taken. That is, even if the lens barrels 2D and 2H are imaged at angles of view wider than the angles of view θ5 and θ6, respectively, the protrusion of the movable lighting unit 4 is detected so that the angles of view are narrower than those of θ5 and θ6. It is possible to control the lens barrels 2D and 2H to perform optical zoom to the telephoto side. Such control is called angle of view setting control.

By setting the angle of view of the lens barrels 2D and 2H after the change by the angle of view setting control to the telephoto side, it is possible to more reliably avoid the movable lighting unit 4 from entering the angle of view. However, normally, in order to shorten the omnidirectional imaging close range (L2), the angle of view of the lens barrels 2D and 2H is controlled so that the movable lighting unit 4 is located near the outer end of the angle of view. Is desirable. In this way, by controlling the angle of view setting so that the movable lighting unit 4 protruding from the camera body 1 does not enter the angle of view of the plurality of lens barrels 2, the movable lighting unit 4 is reflected in the omnidirectional image. It is possible to shorten the omnidirectional imaging close range while avoiding the above.

FIG. 22 shows the camera body 1 as viewed from the direction of the arrow in FIG. 20, similarly to FIG. 21, and shows the entire camera body 1 including the tripod 5. As described above, by setting the angle of view of the lens barrels 2D and 2H to the telephoto side, it is possible to more reliably avoid the protruding movable lighting unit 4 from entering the angle of view. However, when taking an image directly under the camera body 1, the distance to the camera installation surface (hereinafter referred to as the ground) such as the floor or the ground is short, so in order to generate an omnidirectional image, the telephoto side of the lens barrel 2H It is also necessary to consider the range in which the optical zoom to is possible. In the lens barrel D, the angle of view on the widest angle side where the movable lighting unit 4 does not enter is θ7, and the point where the lower end of the angle of view touches the ground is Q2. In this case, the angle of view on the farthest side of the lens barrel 2H is the angle of view θ8 at which the outer end of the angle of view on the lens barrel 2D side touches the ground at point Q2 (intersects the angle of view of the lens barrel 2D). It becomes.

Further, in the lens barrel 2H, the angle of view on the widest angle side where the movable lighting unit 4 does not enter is set to θ8, and the point where the outer end of the angle of view on the lens barrel 2D side comes into contact with the ground is set to Q3. In this case, the angle of view on the most telephoto side of the lens barrel 2D is the angle of view θ9 at which the lower end of the angle of view touches the ground at the point Q3 (intersects the angle of view of the lens barrel 2H).

When setting the omnidirectional imaging close range as short as possible, it is desirable to set θ7 and θ6, respectively, so that the angle of view of the lens barrel 2D and the angle of view of the lens barrel 2H overlap at the point Q1. However, when there is a specific subject to be magnified within the angle of view of the lens barrel 2D or 2H, the limit angle of view of the lens barrel 2D on the telephoto side is θ9, which is the limit of the lens barrel 2H on the telephoto side. The angle of view is θ8. That is, the range that can be taken by the angle of view θD of the lens barrel 2D when the movable lighting unit 4 is projected is
θ9 <θD <θ7
The possible range of the angle of view θH of the lens barrel 2H is
θ8 <θH <θ6
Will be. Therefore, as described with reference to FIG. 14, continuous (sequential) imaging can be performed in these angle-of-view ranges through the lens barrels 2D and 2H during optical zoom, and further, another lens barrel 2A can be performed. Continuous imaging through ~ 2C, 2E ~ 2G can also be performed.

By the way, in the angle of view setting control based only on the information indicating that the movable lighting unit 4 protrudes from the camera body 1 as described above, the movable lighting unit 4 is omnidirectional even when it is in the second position shown in FIG. It is not possible to set the close range for imaging to a distance shorter than the distance to Q1. Therefore, it is desirable that the angle of view of the lens barrels 2D and 2H can be controlled to be changed according to which of the first, second and third positions the movable lighting unit 4 is located.

Specifically, the camera body 1 stores the pattern of the image of the movable lighting unit 4 on the partial image obtained through the lens barrels 2D and 2H, and the subject or the movable lighting unit 4 is reflected by the pattern recognition. Let them know. Then, by this discrimination, the movable lighting unit 4 is moved to the telephoto side of these lens barrels 2D and 2H until it is not reflected in the partial image obtained through the lens barrels 2D and 2H (it does not fit in any angle of view). Allows the optical zoom of to be done automatically. As a result, it is possible to acquire an omnidirectional image without reflection of the movable lighting unit 4. By controlling the lens barrels 2D and 2H to an angle of view that makes the movable lighting unit 4 almost invisible (controlling the movable lighting unit 4 so that it does not enter the outer edge region of those angles of view), all directions are used. It is possible to bring the image close-up distance closer to the camera body 1 than to Q1.

In this embodiment, only the lens barrels 2D and 2H have been described, but the angle of view of the lens barrels 2C and 2E where the movable lighting unit 4 may enter the angle of view is also the image of the lens barrels 2D and 2H. It can be controlled in the same way as a corner. Also, when another movable lighting unit 4 arranged between the lens barrels 2B and 2H or between the lens barrels 2F and 2H protrudes from the camera body 1, the lens barrels 2B and 2H or the lens barrels 2F and 2H The angle of view can be controlled in the same manner as the angle of view of the lens barrels 2D and 2H.

FIG. 23 shows the angles of view of the lens barrels 2A to 2F and 2H in the plane Y1 passing through the leg portion 5a above the ground in a state where the leg portion 5a of the tripod 5 is extended and the ground contact portion 5b is fixed to the ground. Is shown. Since the lens barrel 2G is used for imaging in the heavenly direction, its angle of view is not described. The angle of view of the lens barrels 2A to 2H is automatically set by the angle of view setting control. The range of the angle of view corresponding to the lens barrels 2A to 2F and 2H is shown by curves 2A'to 2F' and 2H'. For example, the range outside the curve 2A'is the range of the angle of view of the lens barrel 2A. The angle of view of the lens barrel 2H is the range surrounded by the central circle 2H'.

The vicinity of the leg portion 5a of the tripod 5 is not included in the angle of view of any lens barrel 2. That is, the angle of view setting control is performed so that the leg portion 5a of the tripod 5 does not enter the angle of view of each lens barrel 2. At this time, similarly to the movable lighting unit 4, the angle of view of the tripod 5 is controlled so that the leg portion 5a of the tripod 5 is located near the outer edge of the angle of view of each lens barrel 2. Only a small range of is not included in the angle of view of any lens barrel 2. This makes it possible to acquire an omnidirectional image in which the tripod 5 is not reflected even when an image is taken in the direction directly below the camera body 1 provided with the tripod 5. Strictly speaking, the area near the leg portion 5a is not imaged on the plane Y1, but since it is rare that the subject exists in this small area, there is almost no problem.

FIG. 24 shows the angles of view of the lens barrels 2A to 2F and 2H in the plane Y2 passing through the ground contact portion 5b of the tripod 5 slightly above the ground in the same state as in FIG. 23. Also in FIG. 24, the angle of view of the lens barrel 2G is not described. The range of the angle of view corresponding to the lens barrels 2A to 2F and 2H is shown by the curves 2A ″ to 2F ″ and 2H ″. For example, the range outside the curve 2A ″ is the range of the angle of view of the lens barrel 2A. .. The angle of view of the lens barrel 2H is the range surrounded by the central circle 2H ".

The ground contact portion 5b is included in the outer edge region of the angle of view of the lens barrels 2A to 2H and does not enter inside the ground portion 5b (hereinafter, this is included only in the outer edge region of the angle of view or the outer edge of the partial image). It is said that it is reflected only in the area). As a result, it is possible to take an image of a subject close to the ground contact portion 5b, and it is possible to acquire an omnidirectional image without an angle of view defect. Image processing is performed in which the image 5c in the vicinity thereof is cut and pasted (interpolated) in the image area of the grounding portion 5b reflected in the omnidirectional image. Since the image 5c is a part of the omnidirectional image obtained by one imaging, the image processing is easier than the case where the image is separately imaged in order to obtain the image used for interpolation.

As with the movable lighting unit 4, it is desirable that the tripod 5 also be controlled to set the angle of view according to the amount of protrusion of the leg portion 5b from the camera body 1. Specifically, the pattern of the tripod 5 (leg portion 5a and the ground contact portion 5b) reflected on the partial image obtained through the lens barrels 2A to 2F and 2H is stored in the camera body 1 and reflected by pattern recognition. It is made to distinguish whether the subject or the tripod 5 is present. Then, by this discrimination, the tripod 5 is not reflected in the partial image obtained through the lens barrels 2A to 2F, 2H, or is reflected only in the outer edge region of the partial image of the lens barrels 2A to 2F, 2H. Allows the optical zoom to the telephoto side to be automatic. As a result, it is possible to acquire an omnidirectional image with no or almost no reflection of the tripod 5.

Further, not limited to the use of the above pattern recognition, the lens mirror corresponding to the opening angle of the tripod 5 (α to β shown in FIG. 9) and the extension amount (protrusion amount) of the leg portion 5a are detected. The angle of view of the cylinders 2A to 2F and 2H may be calculated, and the angle of view setting control may be performed based on the calculation result.

In this embodiment, the grounded portion 5b is included in the outer edge region of the angle of view of the plurality of lens barrels 2 for the reason of acquiring a large number of images for interpolating the image region of the grounded portion 5b of the tripod 5. Set the corner. For example, if the floor is not a uniform ground with low contrast, but a floor where paintings are drawn with tiles, it may be uncomfortable if the color of the floor on which the tripod is actually grounded is different, so interpolation is performed. It is better to acquire many images for interpolation in order to increase the choices of images to be performed. However, the ground contact portion 5b may be included in the outer edge region of the angle of view of at least one lens barrel 2.

As in FIG. 23, the ground contact portion 5b of the tripod 5 is not included in the angle of view of any of the lens barrels 2A to 2F and 2H, and the region outside the angle of view is interpolated with a nearby image. May be good.

In this embodiment, imaging is performed in a state in which each lens barrel 2 protrudes from the camera body 1 rather than in a retracted state (during non-imaging). This is because when there is a movable lighting unit 4 or a tripod 5 protruding from the camera body 1, the angle of view of the lens barrel 2 is narrower when the starting point is farther from the camera body 1 and the reflection is eliminated. It is preferable because it can be reduced or reduced.

Further, in this embodiment, the movable lighting unit 4 and the tripod 5 built in the camera body 1 have been described, but accessories such as an external strobe, an external microphone, an external tripod, a lens hood, and a lens filter mounted on the camera body 1 have been described. May be. In that case, since it is difficult to detect the protrusion amount of the external accessory and control the angle of view setting based on the calculation result, a pattern for storing the reflection pattern attached to the camera body 1 is stored. It is desirable to control the angle of view using recognition.

FIG. 18 shows the system configuration of the camera body 1. The lens barrels 2 (2A to 2H) each include an optical unit 300 (groups 1 to 3 lens barrels 21, 23, 26), an image sensor 301, and a motor drive unit 302 (drive motor 29). The optical unit 300 is moved in the optical axis direction by the motor drive unit 302. The amount of movement at this time is transmitted as a control signal from the CPU 306 as the control means to the motor drive unit 302 via the drive control unit 305. Further, the moving mechanism M shown in FIG. 17 is provided for each lens barrel 2.

The subject image formed by the optical unit 300 is converted into an electric signal by the image pickup device 301 driven by the image pickup device driver 303. The CPU 306 controls the drive of the image sensor 301 via the image sensor driver 303.

The image processing unit 304 A / D-converts the analog image pickup signal output from the image pickup element 301 of each lens barrel 2, and zooms the lens barrel 2 described above with respect to the partial image as the obtained digital image pickup signal. Brightness correction or the like is performed according to the F value for each magnification. The image processing unit 304 functions as a luminance correction means. Further, the image processing unit 304 as an image generation means performs image processing such as gradation correction and white balance on the partial image obtained from each lens barrel 2. The lens barrel 2 (2A to 2H), an image pickup element 301 provided in each, and an image processing unit 304 constitute an image pickup means.

The image generation unit 309 generates an omnidirectional image by performing a joining process on the partial images from the eight lens barrels 2 generated by the image processing unit 304. At this time, the image generation unit 309, which functions as a contrast detecting means, detects the contrast of each partial image and determines a high contrast region and a low contrast region in the partial image. In response to this determination result, the CPU 306 causes the lens barrel 2 to perform the optical zoom necessary for including the stitch portion in the low contrast region of the partial image.

The memory 307 is composed of a volatile memory or a non-volatile memory, and temporarily stores imaging data such as a partial image and an omnidirectional image, and stores a processing program executed by the CPU 306. The memory 307 also stores data showing the reflection pattern of the movable lighting unit 4 and the tripod 5 in the partial image.

The compression processing unit 308 compresses and encodes the image pickup data by a coding method such as JPEG. The communication control unit 310 transmits the compression-encoded image pickup data to the outside.

It should be noted that the image generation unit 309 that generates an omnidirectional image by performing the stitching process of the partial images does not necessarily have to be provided in the omnidirectional camera. That is, the captured data of the partial image may be sent to an image generation device such as an external personal computer via the communication control unit 310, and the image generation device may generate an omnidirectional image.

The subject detection unit 311 and the subject determination unit 312 recognize the magnifying specific subject in the omnidirectional image, and determine the lens barrel 2 that is capturing the specific subject. Based on the determination result, the CPU 306 causes the lens barrel 2 that is imaging a specific subject to perform optical zoom to the telephoto side, and causes the lens barrel 2 adjacent to the lens barrel 2 to perform optical zoom to the wide-angle side. ..

Further, the CPU 306 is a command (instruction) for instructing an optical zoom from an external instruction device 400 such as a personal computer or a smartphone to the telephoto side of a specific lens barrel 2 by wired or wireless communication via a communication control unit 310. Accept input. The CPU 306 causes a specific lens barrel 2 to perform optical zoom to the telephoto side according to the command, and causes the lens barrel 2 adjacent to the specific lens barrel 2 to perform optical zoom to the wide-angle side. This constitutes an omnidirectional imaging system including an omnidirectional camera and an external instruction device 400.

The flowchart of FIG. 19 shows the process for controlling the angle of view setting described above. The CPU 306 as a computer executes this process according to an image pickup control program which is a computer program.

In step S01, the CPU 306 as the protrusion detecting means detects whether or not at least one of the movable lighting unit 4 and the tripod 5 protrudes from the retracted state. Specifically, the CPU 306 reads data indicating the reflection pattern of the movable lighting unit 4 and the tripod 5 into the partial image from the memory 307. Further, the CPU 306 performs pre-imaging by setting the angle of view of the lens barrels 2A to 2F and 2H in the activated state of the camera. Then, the CPU 306 performs pattern recognition using the reflection pattern data for the partial images obtained through the lens barrels 2A to 2F and 2H. If the projecting member is reflected as a result of this pattern recognition, the CPU 306 determines that the projecting member is projecting from the retracted state, and proceeds to step S02. If not, it is determined that it does not protrude, and the process proceeds to step S04. At this time, if the information on the opening angle and the extension amount of the leg portion 5a of the tripod 5 can be acquired, the CPU 306 also acquires them.

In step S02, the CPU 306 is a lens mirror so that the protruding movable lighting unit 4 or the tripod 5 is not reflected in the partial image obtained through the lens barrels 2A to 2F, 2H, or is reflected only in the outer edge region of the partial image. The angle of view of the cylinders 2A to 2F and 2H is calculated. The angle of view calculated here is the same as the angle of view in the activated state or the angle of view on the telephoto side.

Next, in step S03, the CPU 306 causes the lens barrel of the lens barrels 2A to 2F, 2H whose angle of view calculated in step S02 is different from the angle of view in the activated state to perform optical zoom to the telephoto side. That is, the angles of view of the lens barrels 2A to 2F and 2H are reset. At this time, the CPU 306 uses the moving mechanism M provided for the lens barrel 2 subjected to the optical zoom, and the circle C in which the nodal point of the lens barrel 2 is located at the nodal point of another lens barrel 2. Control to be located on top.

In step S04, the CPU 306 performs omnidirectional imaging to acquire an omnidirectional image. Then, the CPU 306 determines whether or not the subject detection unit 311 has detected a specific subject P such as a person from this omnidirectional image. If the specific subject P is not detected, the CPU 306 repeats the determination in step S04. When the specific subject P is detected, the CPU 306 proceeds to step S05, and the lens barrel 2 (2) of the eight lens barrels 2 (2A to 2H) that captures the specific subject P detected by the subject determination unit 312 ( Hereinafter, the lens barrel 2A) is determined.

Next, in step S06, the CPU 306 causes the lens barrel 2A to perform optical zooming (zooming in) to the telephoto side via the drive control unit 305. At this time, the CPU 306 zooms in so that the specific subject P is captured in the largest angle of view of the lens barrel 2A. Further, in step S07, the CPU 306 optically zooms (zooms out) the lens barrel (peripheral lens barrel) 2B, 2F, 2G, 2H adjacent to the lens barrel 2A via the drive control unit 305 to the wide-angle side. To do. As a result, the angle of view narrowed by the optical zoom to the telephoto end of the lens barrel 2A can be compensated. At this time, the CPU 306 calculates the zoom amount of the lens barrels 2B, 2F, 2G, and 2H to the wide-angle side so as to satisfy the above-mentioned omnidirectional image pickup close range L2. Further, in steps S06 to S07, the CPU 306 has a moving mechanism M so that the positions of these nodal points are located on the same circle with the optical zoom of the lens barrels 2A, 2B, 2F, 2G, and 2H. To control.

Next, in step S08, the CPU 306 determines whether or not an omnidirectional image without a loss in the angle of view could be acquired. When there is an angle of view defect, the lens barrels 2B, 2F, 2G, 2H cannot perform optical zoom so as to satisfy the omnidirectional imaging close range L2, or the lens barrels 2B, 2F, 2G, 2H. This is a case where the angle of view does not overlap a part of the angle of view of the lens barrel 2A even at an infinite distance.

If there is an angle of view defect, the CPU 306 returns to step S07 and causes the lens barrels 2B, 2F, 2G, and 2H to perform optical zooming to the wider angle side. This is repeated until there is no angle of view defect. On the other hand, when there is no angle of view defect, the CPU 306 proceeds to step S09 and causes the image generation unit 309 to perform a joining process of eight partial images generated by the image processing unit 304 to generate an omnidirectional image. Then, in step S10, the omnidirectional image is stored in the memory 307, and this process is terminated.

In the first embodiment, an example in which the angle of view is controlled so that the protruding movable lighting unit 4 or the tripod 5 is not reflected in the partial image of the lens barrels 2A to 2F and 2H or is reflected only in the outer edge region of the partial image. Stated. In the second embodiment, an example is described in which the angle of view is controlled so as to allow the image while reducing the image as much as possible when there is a scene for which the image is to be acquired with priority over the image.

FIG. 25 shows a state in which the movable lighting unit 4 described in FIGS. 21 and 22 is set at the third position. As described above, the lens barrel 2D is controlled to have an angle of view θ6 and the lens barrel 2H is controlled to have an angle of view θ5 so that the movable lighting unit 4 is not reflected. The omnidirectional imaging close-up distance L3 capable of forming an omnidirectional image at this time is the distance from the tip of the lens barrel 2D or 2H (FIG. 25) to the arc including Q1 where the angles of view of the lens barrel 2D and the lens barrel 2H intersect. .. Here, even if there is a subject S whose image is desired to be acquired as an omnidirectional image near the camera body 1, since it is inside the omnidirectional imaging close range L3, the subject S does not enter the angle of view of the lens barrels 2D and 2H. A part of S1 will be missing. At this time, if the user wants to allow some reflection in the captured image of the movable lighting unit 4 and capture the entire subject S, the subject S can be photographed without loss by changing the omnidirectional imaging close range setting to L4. It becomes possible to do. The lens barrel 2D widens the angle of view from θ6 to θ10, and the lens barrel 2H has the maximum wide-angle angle of view. The distance L4 is the closest distance to the omnidirectional image pickup, and the subject S is all photographed. However, since the movable lighting unit 4 enters the angle of view as shown by the dotted line at the enlarged angle of view θ10 of the lens barrel 2D, the movable lighting unit 4 is reflected in the captured image. If the positional relationship between the lens barrel 2D and the movable lighting unit 4 is as shown in FIG. 25, it is possible to prevent the image from being reflected in the captured image by changing the position of the movable lighting unit 4. Further, in the image captured by the movable lighting unit 4 taken by the lens barrel 2D, if it is desired to give priority to the one in which the movable lighting unit 4 is not reflected over the partial defect of the subject S, the image processing is performed at the time of stitching. It is also possible to return to the original angle of view.

Next, the flowchart of FIG. 26 will be described. In step S101, the CPU 306 as the protrusion detecting means detects whether or not at least one of the movable lighting unit 4 and the tripod 5 protrudes from the retracted state. The specific detection method is as described above, and if there is a protrusion, the process proceeds to step S102, and if there is no protrusion, the process proceeds to step S109.

In step S102, the CPU 306 describes the lens barrels 2A to 2F and 2H when the protruding movable lighting unit 4 or the tripod 5 is not reflected in the partial image obtained through the lens barrels 2A to 2F and 2H and when the protruding movable lighting unit 4 or the tripod 5 is reflected in the partial image. Calculate the angle of view.

In step S103, the CPU 306 causes the lens barrel of the lens barrels 2A to 2F, 2H whose angle of view calculated in step S102 is different from the angle of view in the activated state to perform optical zoom to the telephoto side.

In step S104, the user's setting for the reflection on the lens barrels 2A to 2F and 2H is confirmed. If the user has previously set the setting to allow the reflection, the process proceeds to step S109, and if the setting does not allow the reflection, the process proceeds to the step S105. At this time, the specification may be such that a warning is issued to the user when the reflection is confirmed and the setting is made on the spot.

In step S105, the setting of the image pickup close range determined by the angle of view of the lens barrels 2A to 2H is confirmed. Step S106 is a case where it is determined that the movable lighting unit 4 or the tripod 5 that protrudes at the imaging close range set in advance by the user is reflected in the partial image, a warning is issued to the user, and the image is changed to the imaging close range where the image is no longer reflected. Proceed to. If the image is not allowed and the close range of the image is not desired to be changed, the process returns to step S104 and asks whether to allow the image again. By repeating this, the user is allowed to select whether to allow the projection member to be reflected or to change the image pickup close range to the infinite side. At this time, the specifications may be such that it is determined in advance whether to allow the reflection of the projecting member or to change the close range of the image. Since steps S106 and subsequent steps are the same as in the first embodiment, they will be omitted.

In this way, in the example of selecting whether or not to allow the reflection, there is no problem even if the user reflects the entire subject S near the camera body 1 so that the projecting member covers the entire subject S, or the close range of the image is not reflected. Make sure to confirm whether to change up to. As a result, various options can be provided depending on the scene that the user wants to shoot, and the degree of freedom in shooting can be improved. Further, even if the projecting member is inevitably reflected, the reflection can be reduced as much as possible by controlling the angles of view of the respective lens barrels 2A to 2F and 2H.

Here, if the projection member is allowed to be reflected, the projection member is also reflected in the finally joined omnidirectional image, but the images taken by the individual lens barrels 2A to 2F and 2H are included. Even if the projecting member is reflected, the projecting member may be joined without including the projecting member at the time of joining, or the projecting member may be erased by the above-mentioned image processing.

In this embodiment, the case where each lens barrel has a three-group type variable magnification optical system has been described, but this is an example, and other types of variable magnification optical systems may be used.
(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Each of the above-described embodiments is only a representative example, and various modifications and changes can be made to each embodiment when the present invention is implemented.

1 Camera body 2 (2A-2H) Lens lens barrel 306 CPU

Claims (9)

An imaging means for acquiring a plurality of images that have passed through a plurality of variable magnification optical systems facing in different directions and are continuously connected to each other by imaging.
An angle of view setting means for controlling the scaling operation of each of the plurality of scaling optical systems to set the angle of view for each scaling optical system.
The nodal points of the plurality of variable magnification optical systems are controlled so that the parallax between the partial images obtained through the plurality of variable magnification optical systems is substantially the same even if the zoom magnifications of the plurality of variable magnification optical systems are different. Nodal point control means and
The main body that holds the plurality of variable magnification optical systems and
It has a protrusion detecting means for detecting a protrusion member, and has
Based on the detection result of the protrusion detecting means, the angle of view setting means detects the angle of view of each of the plurality of variable magnification optical systems, and the detected protrusion member is any image of the plurality of variable magnification optical systems. An image pickup device characterized in that it controls the angle of view setting so that it does not enter the corner. The imaging device according to claim 1, wherein the protrusion detecting means detects the reflection of the protruding member in the plurality of images. The imaging device according to claim 1 or 2, wherein the protruding member emits illumination light. The imaging device according to any one of claims 1 to 3, wherein the protruding member is a tripod for fixing the main body. The image pickup apparatus according to claim 4, wherein the angle of view setting means controls the angle of view setting according to at least one of an opening angle formed by a plurality of legs of the tripod and an extension amount of the legs. The imaging according to any one of claims 1 to 5, wherein the plurality of variable magnification optical systems stored in the main body during non-imaging project with respect to the main body during imaging. Device. An imaging means for acquiring a plurality of images that have passed through a plurality of variable magnification optical systems facing in different directions and are continuously connected to each other by imaging.
An angle of view setting means for controlling the scaling operation of each of the plurality of scaling optical systems to set the angle of view for each scaling optical system.
The nodal points of the plurality of variable magnification optical systems are controlled so that the parallax between the partial images obtained through the plurality of variable magnification optical systems is substantially the same even if the zoom magnifications of the plurality of variable magnification optical systems are different. Nodal point control means and
The main body that holds the plurality of variable magnification optical systems and
It has a protrusion detecting means for detecting a protruding member protruding from the main body.
The angle of view setting means sets the angle of view of each of the plurality of variable magnification optical systems based on the detection result of the protrusion detecting means so that the amount of reflection of the detected protrusion member is reduced. An image pickup device characterized by controlling the angle setting. It is a control method of an image pickup device that acquires a plurality of images that have passed through a plurality of variable magnification optical systems facing in different directions and are continuously connected to each other by imaging.
A step of controlling the scaling operation of each of the plurality of scaling optical systems to set the angle of view for each scaling optical system, and
The nodal points of the plurality of variable magnification optical systems are controlled so that the parallax between the partial images obtained through the plurality of variable magnification optical systems is substantially the same even if the zoom magnifications of the plurality of variable magnification optical systems are different. Steps and
It has a step to detect a protruding member, and
In the step to be set, the angle of view of each of the plurality of variable magnification optical systems is determined based on the detection result of the step to be detected, and the angle of view of any of the plurality of variable magnification optical systems in which the detected protruding member is detected. A control method for an image pickup device, which is characterized in that an angle of view setting control is performed so as not to enter the image. To the computer of an image pickup device that acquires multiple images that have passed through multiple variable magnification optical systems facing in different directions and are connected to each other so as to be continuous by image pickup.
The angle of view setting control for setting the angle of view for each of the variable magnification optical systems by controlling the variable magnification operation of each of the plurality of variable magnification optical systems is performed.
The nodal points of the plurality of variable magnification optical systems are controlled so that the parallax between the partial images obtained through the plurality of variable magnification optical systems is substantially the same even if the zoom magnifications of the plurality of variable magnification optical systems are different. ,
Detect the protruding member,
In the angle of view setting control, based on the detection result of the projecting member, the angle of view of each of the plurality of variable magnification optical systems is detected, and the detected angle of view of the projecting member is any of the plurality of variable magnification optical systems. An image control program characterized by being set so that it does not enter.

Images