JP2021061505A - Imaging apparatus - Google Patents

Abstract

To prevent reflection of a high-luminance subject.SOLUTION: An imaging apparatus comprises: a plurality of imaging units capable of imaging different imaging directions; a detection unit which detects a high-luminance region in a range to be imaged by the imaging units; and a control unit which controls the imaging units and excludes the high-luminance region detected by the detection unit from the range to be imaged by the imaging units.SELECTED DRAWING: Figure 4

Description

The present invention relates to an imaging device.

A compound eye camera that acquires parallax images for the right eye and the left eye, respectively, is known (see Patent Document 1). However, with a conventional camera, the image quality deteriorates due to the reflection of a high-brightness subject.

Japanese Unexamined Patent Publication No. 2009-188931

The imaging device according to one aspect of the present invention controls a plurality of imaging units capable of imaging different imaging directions, a detection unit that detects a high-luminance region in a range to be imaged by the plurality of imaging units, and the plurality of imaging units. A control unit is provided which excludes the high-luminance region detected by the detection unit from the range of imaging by the plurality of imaging units.

FIG. 1A is a schematic view of the image pickup apparatus according to the first embodiment as viewed from the front, and FIG. 1B is a schematic view of the image pickup apparatus as viewed from the side. 2 (a) to 2 (c) are schematic views of the two imaging units of FIG. 1 as viewed from above, and are views showing different open states of the two imaging units. 3 (a) to 3 (d) are schematic views showing the orientations of the two imaging units in different open states. It is a figure explaining the structure of an image display device. It is a schematic diagram explaining a shooting range and a non-shooting range. It is a schematic diagram explaining a shooting range. FIG. 7A is a schematic diagram illustrating the shooting range of the imaging device before avoiding the high-brightness subject, and FIG. 7B is the shooting range of the imaging device after avoiding the high-brightness subject by the first method. FIG. 7 (c) is a schematic diagram illustrating the shooting range of the imaging device after avoiding a high-luminance subject by the second method, and FIG. 7 (d) is a schematic diagram illustrating the high image by the third method. It is a schematic diagram which illustrates the shooting range of the image pickup apparatus after avoiding a bright subject. It is a flowchart explaining the process flow of a program. FIG. 9A is a schematic diagram illustrating the shooting range of the imaging device before avoiding the high-brightness subject, and FIG. 9B is the shooting range of the imaging device after avoiding the high-brightness subject by the second method. 9 (c) is a schematic diagram focusing on the non-photographing range of FIG. 9 (b), and FIG. 9 (d) is an image pickup apparatus whose roll state is being changed from FIG. 9 (c). It is a schematic diagram which illustrates the shooting range. FIG. 10A is an enlarged view of the non-shooting range of FIG. 9C, and FIG. 10B is an enlarged view of the non-shooting range after changing the roll state. In the third embodiment, it is a schematic diagram showing the photographing range in each frame when the image pickup apparatus images 6 frames. FIG. 12 (a) is a schematic view of the image pickup apparatus according to the fourth embodiment as viewed from above, and FIG. 12 (b) is a schematic view of the image pickup apparatus as viewed from the side.

(First Embodiment)
The image pickup apparatus according to the first embodiment of the present invention has two image pickup units. Each image pickup unit has a fisheye lens and an image pickup element provided on the image plane of the fisheye lens. In each image pickup unit, a subject image formed by a fisheye lens is imaged by an image pickup device. This imaging device has a function of avoiding the reflection of a high-brightness subject. In the following description, the configuration of the image pickup apparatus will be described first, and then the function of avoiding the reflection of a high-luminance subject will be described. It should be noted that avoiding the reflection of a high-brightness subject is simply referred to as avoiding a high-brightness subject.

<Appearance of imaging device>
FIG. 1A is a schematic view of the image pickup apparatus 100 according to the present embodiment as viewed from the front, and FIG. 1B is a schematic view of the image pickup apparatus 100 as viewed from the side. 2 (a) to 2 (c) are schematic views of the two imaging units of the imaging device 100 of FIG. 1 as viewed from above. In the image pickup apparatus 100, the first image pickup unit 200 and the second image pickup unit 300 are supported by the connection unit 400. The connecting portion 400 has a function similar to that of a so-called hinge, and the first imaging unit 200 and the second imaging unit 300 can rotate around the axis Ax. With this configuration, the open state (opening angle θ1 around the axis Ax) of the imaging unit (first imaging unit 200 and second imaging unit 300) can be changed. In the present embodiment, the open state can be changed in the range of 0 degrees from the state shown in FIG. 1 (opening angle θ1 = 360 degrees).

Further, the connecting portion 400 is rotatably supported by the support member 410 and the shaft member 420 around the shaft Bx that intersects the shaft Ax. With this configuration, the roll state (rotation angle θ2 around the axis Bx) of the imaging unit can be changed while maintaining the open state (opening angle θ1) of the imaging unit. In the present embodiment, the roll state can be changed in the range of ± 90 degrees from the state shown in FIG. 1 (θ2 = 0 degrees). In FIG. 1B, the direction indicated by the counterclockwise arrow with respect to the axis Bx is defined as the + direction.

Furthermore, the support member 410 is rotatably supported by the grip member 450 and the shaft member 430 around a shaft Gx coaxial with the grip member 450. With this configuration, the pan state (rotation around the axis Gx) of the image pickup apparatus 100 is maintained while maintaining the open state (opening angle θ1) of the image pickup unit and the roll state (rotation angle θ2) of the image pickup unit. The angle θ3) can be changed. In the present embodiment, the pan state can be changed within a range of ± 180 degrees from the state shown in FIG. 1 (θ3 = 0 degrees). In FIGS. 1A and 1B, the direction indicated by the clockwise arrow when the axis Gx is viewed from above is defined as the + direction.

In both FIGS. 1A and 1B, the opening angle θ1 indicating the open state is 360 degrees, the rotation angle θ2 indicating the roll state is 0 degrees, and the rotation angle θ3 indicating the pan state is 0 degrees. Illustrate the case.
Further, in FIGS. 2 (a), 2 (b), and 2 (c), the rotation angle θ2 indicating the roll state is 0 degrees, and the rotation angle θ3 indicating the pan state is 0 degrees, and the open state is obtained. The case where the opening angle θ1 indicating the above is 360 degrees, 270 degrees, and 180 degrees, respectively, is illustrated. The straight line arrows in FIGS. 1 (a) and 2 (a) to 2 (c) indicate the photographing direction D as the image pickup apparatus 100. Since the photographing direction D of the imaging device 100 corresponds to the overlapping range DUP (FIG. 6 described later) in which the imaging range of the first imaging unit 200 and the imaging range of the second imaging unit 300, which will be described in detail later, overlap. , The direction of imaging by the first imaging unit 200 and the direction of imaging by the second imaging unit 300 do not always match.

The rod-shaped gripping member 450 is provided coaxially with the axis Gx parallel to the axis Ax of the connecting portion 400. The reason why the shaft Gx of the gripping member 450 is not provided coaxially with the shaft Ax of the connecting portion 400 is that the user's hand holding the gripping member 450 is reflected in the first imaging unit 200 and the second imaging unit 300. This is to avoid.

When the image pickup device 100 is used, for example, the user holds the grip member 450 of the image pickup device 100 in a state where the rotation angle θ2 indicating the roll state is 0 degrees and the rotation angle θ3 indicating the pan state is 0 degrees. Hold it in front of the body and turn the connection portion 400 side of the imaging unit toward the photographing direction D as the imaging device 100. In other words, the side of the support member 410 that pivotally supports the connecting portion 400 via the shaft member 420 is directed toward the main subject. In FIG. 1A, assuming that the main subject side (in the direction of the straight arrow) is the front when viewed from the connecting portion 400, the gripping member 450 is located behind the connecting portion 400, and therefore the hand of the user who grips the gripping member 450. Is less likely to be captured by the first imaging unit 200 and the second imaging unit 300. At this time, the user keeps the gripping member 450 in the direction perpendicular to the ground (floor surface).
If the user's hand holding the gripping member 450 is allowed to be reflected in the first imaging unit 200 and the second imaging unit 300, the axis Gx of the gripping member 450 is changed to the axis Ax of the connecting unit 400. It may be provided coaxially with.

The release button 810 is provided on the surface of the gripping member 450 that faces the connection portion 400, that is, the surface that faces the main subject (direction of the straight arrow) during shooting. The release button 810 can be operated with the index finger. Further, in the grip member 450, an open / close switch 820, a rotary switch 821, and a rotary switch 822 are provided on a surface opposite (rearward) to the connection portion 400. The open / close switch 820 and the rotary switches 821 and 822 can all be operated with the thumb.

The open / close switch 820 is an operation unit for opening and closing the image pickup unit (first image pickup unit 200 and second image pickup unit 300) in order to change the open state of the image pickup unit (first image pickup unit 200 and second image pickup unit 300). is there. The open / close switch 820 is composed of, for example, a rocker type operating member, and when one end is pressed, a signal for changing the opening angle θ1 is output, and when the other end is pressed, the open / close switch 820 is described. A signal for changing the opening angle θ1 in a decreasing direction is output.

The rotation switch 821 rotates the connection unit 400 that supports the image pickup unit around the axis Bx in order to change the roll state of the image pickup unit (first image pickup unit 200 and the second image pickup unit 300). It is an operation unit for. The rotation switch 821 is composed of, for example, a rocker type operating member, and when one end is pressed, a signal for changing the rotation angle θ2 in the + direction is output, and when the other end is pressed, the rotation switch 821 is rotated. A signal for changing the angle θ2 in the − direction is output.

In order to change the pan state of the imaging unit (first imaging unit 200 and the second imaging unit 300), the rotation switch 822 rotates the support member 410 supporting the connecting portion 400 around the axis Gx. It is an operation unit for making it. The rotation switch 822 is composed of, for example, a rocker type operating member, and when one end is pressed, a signal for changing the rotation angle θ3 in the + direction is output, and when the other end is pressed, the rotation switch 822 is rotated. A signal for changing the angle θ3 in the − direction is output.

The first imaging unit 200 is provided with a fisheye lens 210, and the second imaging unit 300 is provided with a fisheye lens 310. Both the fisheye lens 210 and the fisheye lens 310 have an angle of view of more than 180 degrees. Therefore, each of the first imaging unit 200 and the second imaging unit 300 can image a wider range than the hemisphere. With this configuration, the image pickup apparatus 100 can perform the following imaging by changing the opening angle θ1.

For example, the opening angle θ1 = 360 degrees (the first open state) illustrated in FIG. 2A is set, and the first imaging unit 200 and the second imaging unit 300 each have a wider range than the hemisphere. When the image is taken, a 360-degree spherical image can be taken in the horizontal and vertical directions. At that time, it is necessary to secure a margin for connecting the first image acquired by the first imaging unit 200 and the second image acquired by the second imaging unit 300 as an image of the whole celestial sphere. Can be done.
Further, the opening angle θ1 = 180 degrees (the third open state) illustrated in FIG. 2C is set, and the first imaging unit 200 and the second imaging unit 300 each have a wider range than the hemisphere. When the image is taken, the same direction in a wider range than the hemisphere can be imaged from different positions.

<Various open states>
3 (a) to 3 (d) are schematic views showing directions of imaging by the first imaging unit 200 and the second imaging unit 300 in various open states. The first image pickup unit 200 includes a fisheye lens 210 and an image pickup element 220 arranged on the image plane of the fisheye lens 210. The second image pickup unit 300 includes a fisheye lens 310 and an image pickup device 320 arranged on the image plane of the fisheye lens 310.

FIG. 3A is a diagram showing a first open state, and corresponds to the cases of FIGS. 1 (a), 1 (b), and 2 (a). The direction of imaging by the first imaging unit 200 and the direction of imaging by the second imaging unit 300 are opposite to each other and differ by 180 degrees. Therefore, the optical axis X200 of the fisheye lens 210 that constitutes the optical system of the first imaging unit 200 and the optical axis X300 of the fisheye lens 310 that constitutes the optical system of the second imaging unit 300 are coaxial. As described above, in the first open state, an image of the whole celestial sphere can be obtained by joining the image acquired by the first imaging unit 200 and the image acquired by the second imaging unit 300. ..

FIG. 3B is a diagram showing a second opening state (180 degrees <opening angle θ1 <360 degrees), and corresponds to the case of FIG. 2B. The direction of imaging by the first imaging unit 200 and the direction of imaging by the second imaging unit 300 are different by θ1-180 degrees. For example, when the opening angle θ1 = 270 degrees, the direction of imaging by the first imaging unit 200 and the direction of imaging by the second imaging unit 300 are different from each other by 270-180 = 90 degrees.

FIG. 3C is a diagram showing a third open state, and corresponds to the case of FIG. 2C. The direction of imaging by the first imaging unit 200 and the direction of imaging by the second imaging unit 300 are parallel. That is, the optical axis X200 of the fisheye lens 210 that constitutes the optical system of the first imaging unit 200 and the optical axis X300 of the fisheye lens 310 that constitutes the optical system of the second imaging unit 300 are parallel and do not intersect.

FIG. 3D is a diagram showing a fourth open state (0 degree <open angle θ1 <180 degrees), and is a direction in which the first image pickup unit 200 takes an image and a direction in which the second image pickup unit 300 takes an image. Is different from θ1-180 degrees. For example, when the opening angle θ1 = 90 degrees, the direction of imaging by the first imaging unit 200 and the direction of imaging by the second imaging unit 300 are different by 90-180 = −90 degrees. When the imaging directions of the first imaging unit 200 and the second imaging unit 300 differ by a negative angle, the optical axes X200 and X300 of the imaging units intersect each other.

<Configuration of imaging device>
The configuration of the image pickup apparatus 100 will be described with reference to the block diagram illustrated in FIG. The image pickup apparatus 100 includes a first image pickup unit 200, a second image pickup unit 300, a connection unit 400, a control unit 500, a display unit 700, and an operation member 800, and the storage medium 900 can be attached and detached. It is composed. The operating member 800 includes the release button 810, the open / close switch 820, the rotary switch 821, and the rotary switch 822.

(First imaging unit 200, second imaging unit 300)
When the first imaging unit 200 and the second imaging unit 300 are set to an opening angle θ1 = 180 degrees as shown in FIGS. 2 (c) and 3 (c), the optical axes X200 and X300 are aligned. A stereo camera is configured which is arranged at a predetermined distance L (referred to as a baseline length). The fisheye lenses 210 and 310 are composed of, for example, equidistant projection type fisheye lenses. With this configuration, in the images obtained by the image sensors 220 and 320, the distance from each of the optical axes X200 and X300b represents the direction (angle) of the object (subject). Further, the difference in the position on the image of the same object reflected in the images obtained by the image sensors 220 and 320 represents the parallax. Therefore, the distance to the object can be obtained by using the principle of triangulation based on the image signal by the image sensor 220 and the image signal by the image sensor 320. Obtaining distance information from images obtained by the image sensors 220 and 320 of a stereo camera is also referred to as stereo matching.
The light from the object passes through the fisheye lens 210 and is incident on the image sensor 220, and is transmitted through the fisheye lens 310 and is incident on the image sensor 320. The image signals output from the image sensor 220 of the first image sensor 200 and the image sensor 320 of the second image sensor 300 are sent to the control unit 500, respectively.

(Connection part 400)
The connection unit 400 is provided with a first angle sensor 401 that detects the open state (opening angle θ1) of the first image pickup unit 200 and the second image pickup unit 300, and a drive unit 402 that changes the open state. .. The detection signal output from the first angle sensor 401 is sent to the control unit 500.
The drive unit 402 includes a motor and a deceleration mechanism that transmits the drive force of the motor to the first and second image pickup units 200 and 300. The drive unit 402 and the first image pickup unit 200 around the shaft Ax (FIG. 1) via the deceleration mechanism by driving the motor based on the drive signal transmitted from the control unit 500 (drive control unit 540). The second imaging unit 300 and the second imaging unit 300 are rotated at equal angles in opposite directions. As a result, the opening angle θ1 is controlled.

(Support member 410)
The support member 410 includes a second angle sensor 421 that detects the roll state (rotation angle θ2) of the image pickup units (first image pickup unit 200 and the second image pickup unit 300), and a drive unit 422 that changes the roll state. It is provided. The detection signal output from the second angle sensor 421 is sent to the control unit 500.
The drive unit 422 has a motor and a reduction mechanism for transmitting the driving force of the motor to the shaft member 420. The drive unit 422 rotates the connection unit 400 around the shaft Bx (FIG. 1) via the reduction mechanism by driving the motor based on the drive signal sent from the control unit 500 (drive control unit 540). As a result, the rotation angle θ2 is controlled.

(Gripping member 450)
The gripping member 450 is provided with a third angle sensor 431 that detects the pan state (rotation angle θ3) of the image pickup apparatus 100, and a drive unit 432 that changes the pan state. The detection signal output from the third angle sensor 431 is sent to the control unit 500.
The drive unit 432 has a motor and a reduction mechanism for transmitting the driving force of the motor to the shaft member 430. The drive unit 432 rotates the support member 410 around the shaft Gx (FIG. 1) via the reduction mechanism by driving the motor based on the drive signal sent from the control unit 500 (drive control unit 540). As a result, the rotation angle θ3 is controlled.

(Control unit 500)
The control unit 500 is composed of a CPU, a ROM, a RAM, and the like, and controls the operation of each unit of the image pickup apparatus 100 based on a control program. The control unit 500 includes an image pickup control unit 510, a signal processing unit 520, a recording / reading control unit 530, a drive control unit 540, a shooting (non-shooting) range calculation unit 550, a high-luminance region detection unit 560, and a rotation amount calculation unit 570. And, the image generation unit 580 is included.

The image pickup control unit 510 causes the first image pickup unit 200 and the second image pickup unit 300 to perform an image pickup operation, respectively, based on the operation signal from the release button 810 of the operation member 800. The signal processing unit 520 performs predetermined signal processing on the image signals output from the image sensor 220 of the first image sensor 200 and the image sensor 320 of the second image sensor 300, respectively. In addition to interpolation processing and gradation correction processing, the signal processing unit 520 can also perform conversion processing for converting a celestial sphere image into a two-dimensional image, for example, an equirectangular projection image.

The recording / reading control unit 530 is used for recording processing for recording image data captured by the first imaging unit 200 and the second imaging unit 300 and signal-processed by the signal processing unit 520 on the storage medium 900, respectively. A read-out process for reading out image data recorded on the storage medium 900 is performed.

The drive control unit 540 controls the drive unit 402 that changes the open state, the drive unit 422 that changes the roll state, and the drive unit 432 that changes the pan state.
(Change of open state)
Change the open state as follows. The drive control unit 540 sends a drive signal to the drive unit 402 based on the operation signal from the open / close switch 820 of the operation member 800. By driving the motor of the drive unit 402, the first and second imaging units 200 and 300 rotate in opposite directions around the axis Ax of the connection unit 400, and the opening angle θ1 changes. As described above, the drive control unit 540 sends a drive signal for driving the drive unit 402 in a direction of increasing the opening angle θ1 or reduces the opening angle θ1 depending on the position where the open / close switch 820 is pressed. A drive signal for driving the drive unit 402 is transmitted in the direction.

For example, when the user operates the open / close switch 820, the motor starts driving and the first and second imaging units 200 and 300 rotate. In the image pickup apparatus 100 of the embodiment, while the user continuously operates the open / close switch 820, the operation of changing the opening angle θ1 is continued, and when the user stops the operation of the open / close switch 820, the opening / closing switch 820 is opened. The operation of changing the angle θ1 is stopped.
When the user operates the open / close switch 820 once, the motor may be automatically continued to be driven until the first angle sensor 401 detects the preset opening angle θ1.
Further, as will be described later, in order to avoid a high-intensity subject, the drive control unit 540 can automatically change the opening angle θ1 based on the change amount of the opening angle θ1 calculated by the rotation amount calculation unit 570. ing.

(Change of roll status)
The roll state is changed as follows. The drive control unit 540 sends a drive signal to the drive unit 422 based on the operation signal from the rotation switch 821 of the operation member 800. By driving the motor of the drive unit 422, the image pickup unit (first image pickup unit 200 and the second image pickup unit 300) rotates around the axis Bx of the shaft member 420 while maintaining the opening angle θ1, and the rotation angle θ2 is changed. Change. As described above, the drive control unit 540 sends a drive signal for driving the drive unit 422 in a direction that changes the rotation angle θ2 in the + direction depending on the position where the rotation switch 821 is pressed, or the rotation angle θ2. A drive signal for driving the drive unit 422 is sent in a direction that changes the-direction.

For example, when the user operates the rotary switch 821, the motor starts driving and the image pickup unit rotates. In the image pickup apparatus 100 of the embodiment, while the user continuously operates the rotation switch 821, the operation of changing the rotation angle θ2 is continued, and when the user stops the operation of the rotation switch 821, the rotation switch 821 is rotated. The operation of changing the angle θ2 is stopped.
As will be described later, in order to avoid a high-intensity subject, the drive control unit 540 can automatically change the rotation angle θ2 based on the amount of change in the rotation angle θ2 calculated by the rotation amount calculation unit 570. ing.

(Change of bread state)
Furthermore, the pan state is changed as follows. The drive control unit 540 sends a drive signal to the drive unit 432 based on the operation signal from the rotary switch 822 of the operation member 800. By driving the motor of the drive unit 432, the image pickup unit (first image pickup unit 200 and the second image pickup unit 300) maintains the opening angle θ1 (open state) and the rotation angle θ2 (roll state), and the shaft member 430. The rotation angle θ3 changes by rotating around the axis Gx of. As described above, the drive control unit 540 sends a drive signal for driving the drive unit 432 in a direction that changes the rotation angle θ3 in the + direction depending on the position where the rotation switch 822 is pressed, or the rotation angle θ3. A drive signal for driving the drive unit 432 is sent in a direction that changes the-direction.

For example, when the user operates the rotary switch 822, the motor starts driving and the image pickup unit rotates. In the image pickup apparatus 100 of the embodiment, while the user continuously operates the rotation switch 822, the operation of changing the rotation angle θ3 is continued, and when the user stops the operation of the rotation switch 822, the rotation switch 822 is rotated. The operation of changing the angle θ3 is stopped.
As will be described later, in order to avoid a high-intensity subject, the drive control unit 540 can automatically change the rotation angle θ3 based on the amount of change in the rotation angle θ3 calculated by the rotation amount calculation unit 570. ing.

The description will be continued with reference to FIG. The shooting (non-shooting) range calculation unit 550 is based on the opening angle θ1, the rotation angle θ2, the rotation angle θ3, and the design information of the optical systems of the first imaging unit 200 and the second imaging unit 300. The shooting range captured by the imaging device 100, and when turned inside out, the non-shooting range not captured by the imaging device 100 is calculated. FIG. 5 is a schematic diagram illustrating a shooting range and a non-shooting range. FIG. 5 corresponds to the second open state of the image pickup apparatus 100 (FIG. 3 (b): 180 degrees <θ1 <360 degrees). In FIG. 5, when the side of the support member 410 that pivotally supports the connection portion 400 is directed to the shooting direction D as the image pickup device 100, the region on the shooting direction D side is the shooting range shot by the image pickup device 100. Is. On the contrary, the region OB on the gripping member 450 side of the photographing device 100 is a non-photographing range that is not photographed by the imaging device 100. The imaging range captured by the image pickup apparatus 100 becomes wider as the opening angle θ1 indicating the opening state of the imaging device 100 approaches 360 degrees, and narrower as the opening angle θ1 indicating the opening state approaches 180 degrees. On the contrary, the non-imaging range that is not photographed by the image pickup apparatus 100 becomes narrower as the opening angle θ1 approaches 360 degrees, and becomes wider as the opening angle θ1 approaches 180 degrees.

The high-luminance region detection unit 560 is based on the data of the first image captured by the first image pickup unit 200 and the data of the second image captured by the second image pickup unit 300, and is based on the data of the image pickup device 100. Detects high-brightness subjects such as the sun in the shooting range. In the example of FIG. 5, a region corresponding to the sun S included in the background of the subject P such as a group of buildings is detected.

An example of a detection method in which the high-luminance region detection unit 560 detects a high-luminance subject will be described below. The high-luminance region detection unit 560 compares the average brightness of the first image based on the data of the first image with the average brightness of the second image based on the data of the second image, and is bright. It is determined that a high-intensity subject exists in the other image. The high-brightness area detection unit 560 further searches for a saturated area in the brighter image of the first or second image, in other words, a area in which the value indicating brightness exceeds a predetermined threshold value. .. When a saturated region is detected, it is determined that the region corresponds to the sun S. On the other hand, when the saturated region is not detected, it is determined that a high-intensity subject such as the sun S does not exist in the shooting range of the image pickup apparatus 100. When the high-brightness region detection unit 560 detects a saturated region in the first image, it determines that a high-brightness subject exists in the shooting range B200 and is saturated in the second image. When the region is detected, it is determined that a high-luminance subject exists in the shooting range B300.

The image pickup apparatus 100 according to the embodiment can automatically change at least one of the above-mentioned open state, roll state, and pan state in order to avoid a high-luminance subject. The details of the control for avoiding the high-intensity subject will be described later, but the rotation amount calculation unit 570 is necessary to change the opening angle θ1 when changing the above-mentioned opening state in order to avoid the high-intensity subject. The amount of rotation of the motor of the drive unit 402 is calculated.
Further, the rotation amount calculation unit 570 calculates the rotation amount of the motor of the drive unit 422 required to change the rotation angle θ2 when changing the roll state described above in order to avoid a high-brightness subject.
Furthermore, when changing the pan state described above in order to avoid a high-brightness subject, the rotation amount calculation unit 570 calculates the rotation amount of the motor of the drive unit 432 required to change the rotation angle θ3.

The image generation unit 580 is a first image captured and acquired by the first image pickup unit 200 and the second image pickup unit 300 based on an instruction transmitted to the control unit 500 by the user operating the operation member 800. And, based on at least one of the data of the second image or at least one of the data of the first and second images recorded on the storage medium 900, a display image to be displayed on the display unit 700 is generated. ..
Further, the setting menu screen and the like are generated based on the instruction transmitted to the control unit 500 by the user operating the operation member 800.

(Display unit 700)
The display unit 700 is composed of, for example, a liquid crystal display panel, an organic EL display panel, or the like, and is provided on a surface of the first imaging unit 200 or the second imaging unit 300 where a fisheye lens is not provided. The display unit 700 displays the display image generated by the image generation unit 580, the setting menu screen, and the like based on the instruction transmitted from the control unit 500.

The display unit 700 is provided with an operation detection unit 710 such as a touch panel switch on the display surface thereof. The operation detection unit 710 detects a contact operation (touch operation, swipe operation, etc.) on the display surface by the user, and sends an operation signal indicating the contacted position to the control unit 500.

(Operating member 800)
The operation member 800 includes buttons and switches such as the release button 810, the open / close switch 820, the rotation switch 821, and the rotation switch 822 of FIG. 1, and sends an operation signal according to the operation of the user to the control unit 500.

(Storage medium 900)
The storage medium 900 is composed of, for example, a memory card that can be attached to and detached from the image pickup apparatus 100. The recording and reading of data on the storage medium 900 is controlled by the recording / reading control unit 530 described above.

The storage medium 900 contains the data of the first image captured by the first imaging unit 200, the data of the second image captured by the second imaging unit 300, and the first angle sensor 401 at the time of imaging. The information indicating the detected opening angle θ1, the information indicating the rotation angle θ2 detected by the second angle sensor 421 at the time of imaging, and the information indicating the rotation angle θ3 detected by the third angle sensor 431 at the time of imaging are Recorded in association.

By recording the information indicating the opening angle θ1 at the time of imaging in the storage medium 900, the first is based on the opening angle θ1, the design information of the optical systems of the first imaging unit 200 and the second imaging unit 300, and the like. It is possible to obtain an overlapping range DUP (FIG. 6 to be described later) in which the imaging range B200 by the imaging unit 200 and the imaging range B300 by the second imaging unit 300 overlap.

Further, by recording the information indicating the rotation angle θ2 at the time of imaging in the storage medium 900, it becomes possible to obtain the roll state of the imaging device 100. Further, by recording the information indicating the rotation angle θ3 at the time of imaging in the storage medium 900, it is possible to obtain the pan state of the imaging device 100 at the time of imaging.

The image pickup apparatus 100 described above is used for fixed-point shooting for shooting a still image at a predetermined time, surveillance shooting for continuing to shoot a moving image at a predetermined frame rate, and recording shooting for an event or the like. .. In particular, the closer the opening angle θ1 is to 360 degrees, the wider the field of view image that is closer to the whole celestial sphere can be obtained, so that one unit can obtain a wide range of images.

<Function to avoid high-brightness subjects>
The imaging device 100 of the embodiment detects a high-intensity subject existing in the imaging range imaged by the first imaging unit 200 and the second imaging unit 300, and avoids the detected high-intensity subject for photographing. In addition, it has a function of changing at least one of the above-mentioned open state, roll state, and pan state. The high-intensity subject is the sun when it is outdoors during the day, and the moon, fireworks, an illumination light source, etc. when it is outdoors at night. If it is indoors, it may be an illumination light source. Regarding the control for exerting the function of avoiding such a high-intensity subject, an example in which the sun S exists as a high-intensity subject in a scene of outdoor shooting in the daytime will be described below.

FIG. 6 is a schematic diagram illustrating a shooting range imaged by the first imaging unit 200 and the second imaging unit 300, where reference numeral B200 is the imaging range taken by the first imaging unit 200 and reference numeral B300 is the second. The shooting range by the imaging unit 300 is shown. Further, the reference numeral DUP indicates an overlapping range in which the shooting range B200 and the shooting range B300 overlap. The region included in at least one of the shooting range B200 and the shooting range B300 corresponds to the shooting range of the imaging device 100 described with reference to FIG. Further, the region not included in either the shooting range B200 or the shooting range B300 corresponds to the non-shooting range OB described with reference to FIG.
When the control unit 500 detects a high-intensity subject within the shooting range of the imaging device 100, in other words, the shooting range of at least one of the shooting ranges B200 and B300, the first method to the third method described below will be described. Using any of the methods, control is performed to avoid the high-luminance subject from the shooting range of the image pickup apparatus 100 so that the high-luminance subject is not captured. The user selects and sets any one of the first to third methods in advance. Two methods or three methods may be selected.

(First method)
In the first method, a high-intensity subject is avoided by changing the photographing range of the imaging device 100 to a small size while maintaining the photographing direction D as the imaging device 100. That is, the shooting range of the imaging device 100 is narrowed so that the high-brightness subject is out of the shooting range of the imaging device 100.
Since the image pickup apparatus 100 is configured so that the open state of the first image pickup unit 200 and the second image pickup unit 300 can be changed as described above, the shooting range B200 by the first image pickup unit 200 and the second image pickup unit 300 are used. The width of the shooting range of the image pickup apparatus 100 can be changed by widening or narrowing the overlapping range DUP that overlaps with the shooting range B300.

FIG. 7 is a diagram illustrating a first method of setting a shooting range of the image pickup apparatus 100 while avoiding a high-luminance subject. FIG. 7A is a schematic view illustrating the shooting range of the image pickup apparatus 100 before avoiding the high-intensity subject, and FIG. 7B is an image pickup apparatus after avoiding the high-intensity subject by the first method. It is a schematic diagram which illustrates the photographing range of 100. Similar to FIG. 6, the shooting range B200 by the first imaging unit 200 and the shooting range B300 by the second imaging unit 300 overlap in the overlapping range DUP.

In the case of FIG. 7A, the image pickup apparatus 100 sets the opening angle θ1 indicating the open state to 300 degrees, the rotation angle θ2 indicating the roll state to 0 degrees, and the rotation angle θ3 indicating the pan state to 0 degrees as initial settings. It is assumed that the photographing direction D as the image pickup apparatus 100 is directed to the east direction and is fixed to a tripod or the like.

The scale in the left-right direction indicates the direction in which the image is taken. It is assumed that the center in the left-right direction is east (0 degree), the right direction is +, and the left direction is-. FIG. 7A shows a shooting range B200 including the east-southeast (+30 degrees) to the west-northwest (-150 degrees) and a shooting range B300 including the west-southwest (+150 degrees) to the east-northeast (-30 degrees). Is done. The overlap range DUP is from east-northeast (-30 degrees) to east-southeast (+30 degrees).
Then, it is assumed that the sun S exists in the northwest (around +135 degrees) and the sun S is included in the photographing range B300.

When the image pickup device 100 is set to avoid a high-intensity subject from the shooting range of the image pickup device 100 by the first method, the control unit 500 determines that the saturated region in the shooting range B200 or the shooting range B300 is set. When detected by the high-luminance region detection unit 560, the overlapping range DUP where the photographing range B200 and the photographing range B300 overlap is widened, and the photographing range of the imaging device 100 is narrowed. The rotation amount calculation unit 570 calculates the amount of change in the opening angle θ1 indicating the open state of the image pickup device 100, which is necessary for narrowly changing the shooting range of the image pickup device 100.

In the example of FIG. 7B, in order to remove the sun S existing in the northwest (+135 degrees) from the shooting range B300, the rotation amount calculation unit 570 shoots from the south-southwest (+120 degrees) to the north-northeast (-60 degrees). The opening angle θ1 = 240 degrees is calculated so as to be in the range B300.
In the first method, the shooting direction D (east in this example) as the imaging device 100 is maintained, so that even after the opening angle θ1 is changed from 300 degrees to 240 degrees, the center of the overlapping range DUP in the left-right direction remains. It remains east (0 degrees). Therefore, the shooting range B200 calculated by the shooting (non-shooting) range calculation unit 550 is from south-southeast (+60 degrees) to north-northwest (-120 degrees).
After changing the opening angle θ1 from 300 degrees to 240 degrees, the overlapping range DUP is from south-southeast (+60 degrees) to north-northeast (-60 degrees).

(Second method)
In the second method, the high-intensity subject is avoided from the shooting range of the image pickup apparatus 100 by changing the pan state while maintaining the open state of the image pickup apparatus 100. That is, the image pickup device 100 is panned so that the high-intensity subject is out of the shooting range of the image pickup device 100.
Since the image pickup apparatus 100 is configured so that the pan state can be changed as described above, the overlapping range DUP in which the shooting range B200 by the first imaging unit 200 and the shooting range B300 by the second imaging unit 300 overlap is maintained. , The shooting direction D as the image pickup apparatus 100 can be changed.

FIG. 7C is a schematic view illustrating the shooting range of the image pickup apparatus 100 after avoiding a high-luminance subject by the second method. The shooting range of the image pickup apparatus 100 before avoiding the high-luminance subject is as illustrated in FIG. 7 (a).

When the image pickup device 100 is set to avoid a high-intensity subject from the shooting range of the image pickup device 100 by the second method, the control unit 500 determines that the saturated region in the shooting range B200 or the shooting range B300 is set. When it is detected by the high-luminance region detection unit 560, the shooting direction D is changed while maintaining the shooting range of the imaging device 100. The rotation amount calculation unit 570 calculates the amount of change in the rotation angle θ3 indicating the pan state of the image pickup apparatus 100, which is necessary for removing the sun S from the photographing range B300.

In the example of FIG. 7C, in order to remove the sun S existing in the northwest (+135 degrees) from the shooting range B300, the rotation amount calculation unit 570 sets the shooting direction D as the image pickup device 100 to 30 degrees from the east to the north. The rotation angle θ3 = -30 degrees is calculated to be changed.
Since the second method maintains the open state of the image pickup apparatus 100, the center of the overlapping range DUP in the left-right direction (that is, the shooting direction D) is east after changing the rotation angle θ3 from 0 degrees to -30 degrees. It changes from (0 degrees) to east-northeast (-30 degrees). Therefore, the shooting range B300 calculated by the shooting (non-shooting) range calculation unit 550 is from south-southwest (+120 degrees) to north-northeast (-60 degrees), and the shooting range B200 is from east (0 degrees) to west (0 degrees). Up to -180 degrees).
After changing the rotation angle θ3 from 0 degrees to -30 degrees, the overlapping range DUP is from the east (0 degrees) to the north-northeast (-60 degrees).

(Third method)
In the third method, the high-intensity subject is avoided from the shooting range of the image pickup apparatus 100 by changing the roll state while maintaining the open state and the pan state of the image pickup apparatus 100. That is, the roll state of the image pickup apparatus 100 is changed so that the high-intensity subject is out of the shooting range of the image pickup apparatus 100.
Since the image pickup apparatus 100 is configured so that the roll state can be changed as described above, the overlapping range DUP in which the shooting range B200 by the first imaging unit 200 and the shooting range B300 by the second imaging unit 300 overlap is maintained. Moreover, the roll state can be changed without changing the shooting direction D of the image pickup apparatus 100.

FIG. 7D is a schematic diagram illustrating the shooting range of the image pickup apparatus 100 after avoiding a high-luminance subject by the third method. The shooting range of the image pickup apparatus 100 before avoiding the high-luminance subject is as illustrated in FIG. 7 (a).

When the image pickup device 100 is set to avoid a high-intensity subject from the shooting range of the image pickup device 100 by the third method, the control unit 500 determines that the saturated region in the shooting range B200 or the shooting range B300 is set. When detected by the high-luminance region detection unit 560, the roll state is changed while maintaining the shooting range and the shooting direction D of the image pickup apparatus 100. The rotation amount calculation unit 570 calculates the amount of change in the rotation angle θ2 indicating the roll state of the image pickup apparatus 100, which is necessary to avoid the sun S from the photographing range B300.

In the example of FIG. 7D, in order to remove the sun S existing in the northwest (+135 degrees) from the photographing range B300, the rotation amount calculation unit 570 calculates the rotation angle θ2 = 15 degrees in order to change the roll state.
Since the third method maintains the open state and the pan state of the image pickup apparatus 100, even after the rotation angle θ2 is changed from 0 degrees to 15 degrees, the center of the overlapping range DUP in the left-right direction is east (0 degrees). ) Remains.

<Explanation of flowchart>
The flow of processing of the program executed by the control unit 500 in order to avoid a high-luminance subject will be described with reference to the flowchart of FIG. As initial settings, the image pickup apparatus 100 has an opening angle θ1 indicating an open state of 300 degrees, a rotation angle θ2 indicating a roll state of 0 degrees, and a rotation angle θ3 of a pan state of 0 degrees. It is assumed that the shooting direction D as is directed to the east direction and is fixed to a tripod or the like. Here, the initial value of 300 degrees of the opening angle θ1 is an example, and it may be set to any of 180 degrees to 360 degrees. Further, the shooting direction D is directed to the east as an example, and the shooting direction D may be directed to north, south, west, or any other direction. Further, the initial values of the rotation angles θ2 and θ3 do not necessarily have to be 0 degrees.
The image pickup apparatus 100 can perform fixed point shooting at a predetermined time in the state set as described above, or perform surveillance shooting at a predetermined frame rate. When the control unit 500 is set to perform control for avoiding a high-luminance subject, the control unit 500 executes the program for performing the process of FIG. 8 between fixed-point shooting or between surveillance shooting.

In step S10 of FIG. 8, the control unit 500 causes the image pickup apparatus 100 to acquire two images. Specifically, the image pickup control unit 510 sends an instruction to the first image pickup unit 200 and the second image pickup unit 300 to perform an image pickup operation, so that the first image pickup unit 200 acquires the first image. , The second imaging unit 300 acquires the second image.

In step S20, the control unit 500 supplies the high-luminance region detection unit 560 with the data of the first image and the data of the second image in order to check which of the first image and the second image is brighter. Let the brightness values be compared between.

In step S30, the control unit 500 causes the high-luminance region detection unit 560 to determine whether or not the difference between the brightness values of the first and second images is equal to or greater than a predetermined value. Such a determination is based on the idea that a luminance difference of a predetermined value or more occurs between an image including a high-luminance subject and an image not including a high-luminance subject. When the high-luminance region detection unit 560 detects a difference of a predetermined value or more, the control unit 500 positively determines step S30 and proceeds to step S40. When the high-luminance region detection unit 560 does not detect a difference of a predetermined value or more. Step S30 is negatively determined, and the process according to FIG. 8 is terminated. If the difference between the brightness values of the first and second images is not equal to or greater than a predetermined value, it is considered that a high-luminance subject is not captured in either of the first and second images. Therefore, since it is not necessary to avoid a high-luminance subject, the processing after S30 is not performed.

In step S40, the control unit 500 causes the high-luminance region detection unit 560 to detect the high-luminance region included in the brighter image of the two images whose brightness values are compared. The high-brightness region detection unit 560 is based on the idea that when a high-brightness region such as the sun S is captured in the image, the data in the region corresponding to the sun S or the like is saturated, and the brighter one of the above two images. In the image of, the saturation region where the value indicating the brightness exceeds a predetermined threshold value is searched. When the high-luminance region detection unit 560 detects a saturated region, the control unit 500 determines that the sun S or the like is reflected in the saturated region.

In step S50, the control unit 500 sets to avoid a high-brightness subject such as the sun S so as not to capture the high-brightness subject, and ends the process according to FIG. When the control unit 500 is set to avoid a high-brightness subject from the shooting range of the image pickup apparatus 100 by the first method, the control unit 500 changes the above-mentioned opening state to avoid a high-brightness subject such as the sun S. When the control unit 500 is set to avoid a high-brightness subject from the shooting range of the image pickup apparatus 100 by the second method, the control unit 500 changes the pan state described above to avoid a high-brightness subject such as the sun S. When the control unit 500 is set to avoid a high-brightness subject from the shooting range of the image pickup apparatus 100 by the third method, the control unit 500 changes the roll state described above to avoid a high-brightness subject such as the sun S.

According to the first embodiment described above, the following effects can be obtained.
(1) The imaging device 100 has a high range of imaging by the first imaging unit 200 and the second imaging unit 300, and the first imaging unit 200 and the second imaging unit 300, which can image different imaging directions. The high-luminance region detection unit 560 that detects the brightness region, the first imaging unit 200, and the second imaging unit 300 are driven and controlled, and the high-luminance region detected by the high-luminance region detection unit 560 is first imaged. It has a unit 200 and a drive control unit 540 that is removed from the imaging ranges B200 and B300 of the second imaging unit 300. With this configuration, it is possible to prevent a high-intensity subject such as the sun S from being reflected in any one of the plurality of imaging units (first imaging unit 200, second imaging unit 300).

(2) The drive control unit 540 changes the imaging direction of the first imaging unit 200 and the second imaging unit 300 to set the high-luminance region into the imaging range of the first imaging unit 200 and the second imaging unit 300. Since it is removed from B200 and B300, it is possible to appropriately avoid the reflection of a high-intensity subject such as the sun S.

(3) The image pickup apparatus 100 has a first image pickup unit 200 and a second image pickup unit 300 as a plurality of image pickup units whose imaging direction can be changed, and is a first image pickup direction of the first image pickup unit 200. The first imaging unit 200 and the second imaging unit 300 are placed around the first axis (axis Ax) so that the imaging direction and the second imaging direction, which is the imaging direction of the second imaging unit 300, can be changed. A connecting portion 400 is provided as a first supporting portion that supports the rotation. The imaging range B200 imaged by the first imaging unit 200 and the imaging range B300 imaged by the second imaging unit 300 have overlapping regions (corresponding to overlapping range DUP) and non-overlapping regions that do not overlap with each other. .. The connection unit 400 is controlled by the drive control unit 540, and changes the size of the overlapping region by changing the angle between the first imaging direction and the second imaging direction around the first axis (axis Ax). Includes a drive unit 402 that excludes the high-brightness region from the shooting range B200 and the shooting range B300. The angle between the first imaging direction and the second imaging direction corresponds to the open state of the first imaging unit 200 and the second imaging unit 300. With this configuration, for example, by bringing the open state of the first imaging unit 200 and the second imaging unit 300 closer to the third open state (opening angle θ1 = 180 degrees), the shooting ranges B200 and B300 overlap. Increasing the area (overlapping amount) increases the non-photographing range OB of the image pickup apparatus 100. As a result, a high-brightness region such as the sun S is likely to be included in the non-shooting range OB, so that the reflection of a high-brightness subject on either the first imaging unit 200 or the second imaging unit 300 is appropriately avoided. be able to.

(4) The image pickup apparatus 100 supports the first image pickup unit 200 and the second image pickup unit 300 so as to rotate around a second axis (axis Gx) parallel to the first axis (axis Ax). A gripping member 450 is provided as a second support portion. The gripping member 450 is controlled by the drive control unit 540, and by rotating the first imaging unit 200 and the second imaging unit 300 around the second axis (axis Gx), the overlapping region (overlapping amount) Includes a drive unit 432 that removes the high-brightness region from the shooting range B200 and the shooting range B300 without changing the size of. With this configuration, a high-brightness subject such as the sun S can be excluded from the shooting ranges B200 and B300 of the image pickup device 100 by changing the shooting direction D (pan state of the image pickup device 100) as the image pickup device 100. That is, even if the overlapping regions (overlapping amounts) of the shooting ranges B200 and B300 imaged by the first imaging unit 200 and the second imaging unit 300 are not changed, the first imaging unit 200 and the second imaging unit 200 The reflection of a high-intensity subject on any of the 300 can be appropriately avoided.

(5) The image pickup apparatus 100 supports the first image pickup unit 200 and the second image pickup unit 300 so as to rotate around a third axis (axis Bx) intersecting the first axis (axis Ax). A support member 410 is provided as a third support portion. The support member 410 is controlled by the drive control unit 540, and by rotating the first imaging unit 200 and the second imaging unit 300 around the third axis (axis Bx), the overlapping region (overlapping amount) Includes a drive unit 422 that removes the high-brightness region from the shooting range B200 and the shooting range B300 without changing the size of. With this configuration, by changing the roll state of the image pickup device 100, a high-intensity subject such as the sun S can be excluded from the shooting ranges B200 and B300 of the image pickup device 100. That is, even if the shooting direction D (pan state of the image pickup device 100) of the image pickup device 100 is not changed, the image of a high-brightness subject can be reflected on either the first image pickup unit 200 or the second image pickup unit 300. Can be avoided appropriately.

The following modifications are also within the scope of the present invention, and one or more of the modifications can be combined with the above-described embodiment.
(Modification example 1)
In the image pickup apparatus 100 according to the first embodiment described above, the high-brightness region detection unit 560 automatically detects a region corresponding to the sun S appearing as a high-brightness subject in the first image or the second image. An example of doing so was explained. Instead, the control unit 500 exists in the shooting range of the imaging device 100 by using the season and time of shooting, the position where the imaging device 100 is installed (latitude / longitude information), and the directional information indicating the imaging direction D. It may be configured to estimate the position of the sun S. By estimating the position of the sun S, it is possible to omit the process of detecting the region corresponding to the sun S based on the first image and the second image.

Information necessary for estimating the position of the sun S existing in the shooting range of the image pickup apparatus 100 (information indicating the relationship between the shooting time and the azimuth and altitude of the sun at the time of shooting, etc.) is not shown in the control unit 500. It may be stored in the memory of the above, or may be configured to be acquired from an external device via a communication unit (not shown). The control unit 500 is provided with a function of calculating positioning information (latitude, longitude, altitude) indicating the position of the imaging device 100 based on the information transmitted from the GPS satellite.

The control unit 500 includes information acquired from an external device or information read from a memory in the control unit 500, positioning information calculated based on information from a GPS satellite, and information on the imaging device 100 (opening angle θ1, rotation angle). The imaging range of the imaging device 100 is combined with θ2, a rotation angle θ3, design information such as the angle of view of the optical system of the first imaging unit 200 and the second imaging unit 300, and the photographing direction D) as the imaging device 100. Estimate the position of the sun S existing in.

The control unit 500 controls the exclusion of the sun S from the shooting range of the image pickup apparatus 100 so that the sun S at the estimated position is not captured, as in the first embodiment, from the first method to the third method. This is performed using a preset method among the methods.

According to the first modification, since the control unit 500 estimates the position of the sun S existing in the photographing range of the imaging device 100, the first image and the first image for detecting the region corresponding to the sun S are obtained. Even if the acquisition of the second image and the detection of the region corresponding to the sun S appearing as a high-brightness subject in the first image or the second image are omitted, the reflection of the high-brightness subject can be avoided. Can be done.

(Modification 2)
In the first embodiment, a method used for control for avoiding a high-intensity subject such as the sun S from the shooting range of the image pickup apparatus 100 is preset in the first to third methods. An example was explained. As the method used for control to avoid a high-luminance subject, a different method may be set for each shooting scene. The control unit 500 determines the shooting scene based on at least one of the first image and the second image. When the control unit 500 determines the shooting scene, the control unit 500 performs control for avoiding a high-brightness subject by using a method associated with the determined shooting scene.

For example, the flash of fireworks, which is a high-intensity subject in a fireworks scene, the illumination, which is a high-intensity subject in a night scene, the illumination light, which is a high-intensity subject in a night sports scene, and the spotlight in a theater or stage scene. Depending on the type of light source, the amount of light of the light source, the position of the main subject, whether the main subject is stationary, the main subject is moving, the imaging device 100 is fixed, or the imaging device 100 is moving. The control for avoiding the high-intensity subject in each shooting scene is performed by the first method, the second method, the third method, or the control for avoiding the high-intensity subject. Whether or not it is decided in advance. For example, if there is an unnecessary lighting light source for fireworks photography and night view photography, the first method is adopted, for night sports photography, the second method is adopted, and for fine weather photography, the third method is adopted. Decide to adopt. Then, the control unit 500 controls to avoid a high-brightness subject by using a method associated with the determined shooting scene.

According to the second modification, the method used for control to avoid a high-brightness subject can be set for each shooting scene such as fireworks shooting, night view shooting, night sports shooting, theater or concert stage shooting, etc. Control to avoid the subject can be appropriately performed.
In addition, whether the main subject is stationary, the main subject is moving, the image pickup device 100 is fixed, or the image pickup device 100 is moving is also included in the determination of the shooting scene, so that the subject has high brightness. Control to avoid the above can be performed more appropriately.

(Second Embodiment)
<Control to avoid high-brightness subjects>
In the second embodiment, control is performed by combining a plurality of methods among the above-mentioned first to third methods used in the control for avoiding a high-luminance subject. If control is performed using only one of the first to third methods in order to avoid a high-brightness subject from the shooting range of the image pickup apparatus 100, the subject to be shot is not shot together with the sun S. It is assumed that the image is included in the range OB and the desired shooting target is not captured.

9 (a) to 9 (d) are schematic views illustrating the shooting range of the image pickup apparatus 100 when the control for avoiding a high-luminance subject is performed according to the second embodiment. In FIG. 9A, the opening angle θ1 of the image pickup apparatus 100 is set to 330 degrees, the rotation angle θ2 indicating the roll state is set to 0 degrees, and the rotation angle θ3 indicating the pan state is set to 0 degrees. It is assumed that the shooting direction D is directed to the east and is fixed to a tripod or the like. The scale in the left-right direction indicates the direction in which the image is taken, the center of the left-right direction is east (0 degree), the right direction is +, and the left direction is-.

In FIG. 9A, the sun S is included in the right end of the shooting range B300 by the second imaging unit 300 in the shooting range of the imaging device 100 whose shooting direction D is set to the east (0 degree). .. The shooting range B300 is from west-southwest (+165 degrees) to east-northeast (-15 degrees), and the shooting range B200 is from east-southeast (+15 degrees) to west-northwest (-165 degrees). The control unit 500 shall perform control using the second method in order to avoid a high-luminance subject. Specifically, the pan state of the image pickup device 100 is changed based on the rotation angle θ3 calculated by the rotation amount calculation unit 570, and the shooting direction D as the image pickup device 100 is changed by 30 degrees from the east to the north (rotation). Angle θ3 = -30 degrees).

FIG. 9B is a schematic view showing an imaging range of the image pickup apparatus 100 after the pan state is changed. In the second method, since the open state of the image pickup apparatus 100 is maintained, the shooting range B300 after changing the rotation angle θ3 indicating the pan state from 0 degrees to -30 degrees is from the southwest (+135 degrees) to the northeast (-45). The shooting range B200 is from east-northeast (-15 degrees) to west-southwest (-195 (that is, +165) degrees).

According to FIG. 9B, the photographing direction D as the image pickup apparatus 100 is 180 degrees opposite to the sun S, that is, east-northeast (-30 degrees). FIG. 9 (c) is a schematic view focusing on the non-photographing range OB of FIG. 9 (b). The scale in the left-right direction indicates the direction in which the image is taken, and the center in the left-right direction is west-southwest (+150 degrees), which is 180 degrees opposite to the image direction D. The west corresponds to ± 180 degrees.

FIG. 10A is an enlarged view of the non-photographing range OB in FIG. 9C. Although not shown in FIG. 9 (c), it is assumed that 24 people P1 to P24 are lined up in a circle surrounding the image pickup device 100 at 360 degrees around the image pickup device 100. The user wants to photograph the person P1 to P24 with the image pickup apparatus 100. However, according to FIG. 10A, the persons P3 and P4 located in the same direction as the sun S among the persons P1 to P24 due to the inclusion of the sun S in the non-photographing range OB by the control using the second method. Is included in the non-shooting range OB. That is, in the example of FIG. 10A, since the persons P3 and P4 are not included in either the shooting range B200 or the shooting range B300, they are not captured even if they are shot.

In the second embodiment, the control unit 500 detects the main subject based on at least one of the first image and the second image, and the persons P1 to P24 as the subjects are included in the non-shooting range OB. Control is performed by combining the third method so as not to prevent it. FIG. 9D is a schematic view illustrating the imaging range of the image pickup apparatus 100 while changing the roll state (that is, the rotation angle θ2) while maintaining the pan state (that is, the rotation angle θ3) of FIG. 9C. .. The rotation amount calculation unit 570 sets the rotation angle θ3 to −90 degrees so that the band-shaped non-shooting range OB is set along the left-right direction in which the persons P1 to P24 as the subject are lined up. FIG. 10B is an enlarged view of the non-photographing range OB after changing the roll state. According to FIG. 10B, the sun S is included in the non-shooting range OB so as not to be photographed, while the person P3 and the person P4 not captured in FIG. 10A are included in the photographing range B200. Can be.

According to the second embodiment described above, not only the first method or the second method, but also the subject to be photographed by combining the first method and the second method with the third method. Since the rotation angle θ2 is set so that the band-shaped non-shooting range OB is set along the direction in which the objects are lined up, it is possible to appropriately perform control for avoiding a high-luminance subject.

(Third Embodiment)
When the opening angle θ1 is not 360 degrees by performing the control using the first method described above, the first image acquired by the first imaging unit 200 and the first image acquired by the second imaging unit 300 Even if the two images are joined together, a 360-degree spherical image cannot be obtained. Specifically, the area corresponding to the non-photographing range OB shown in a strip shape in the schematic views of FIGS. 7, 9 and 10 is insufficient. In the third embodiment, when a still image is taken by the image pickup apparatus 100, the first and second images of a plurality of frames are taken with different rotation angles θ2 indicating the roll state, and the first of the plurality of frames is taken. By synthesizing the second image and the second image, the area lacking from the spherical image is suppressed to be smaller than when only one frame is captured.

FIG. 11 is a schematic view showing the shooting range in each frame when the imaging device 100 shoots 6 frames in the third embodiment, focusing on the non-shooting range OB. The shooting range indicated by reference numeral F1 is the shooting range of the imaging device 100 set to the same (rotation angle θ2 = 0 degrees) as in FIG. 9C, and the shooting range B300 and the shooting range with the non-shooting range OB in between. It has a B200.

The photographing range indicated by the reference numeral F2 is the photographing range of the imaging device 100 set to the rotation angle θ2 = −30 degrees indicating the roll state, and has the photographing range B300 and the photographing range B200 with the non-imaging range OB interposed therebetween.

The photographing range indicated by reference numeral F3 is the photographing range of the imaging device 100 set to the rotation angle θ2 = −60 degrees indicating the roll state, and has the photographing range B300 and the photographing range B200 with the non-imaging range OB interposed therebetween.

The photographing range indicated by reference numeral F4 is the photographing range of the imaging device 100 set to the rotation angle θ2 = −90 degrees indicating the roll state, and has the photographing range B300 and the photographing range B200 with the non-imaging range OB interposed therebetween.

The photographing range indicated by reference numeral F5 is the photographing range of the imaging device 100 set to the rotation angle θ2 = −120 degrees indicating the roll state, and has the photographing range B300 and the photographing range B200 with the non-imaging range OB interposed therebetween.

The photographing range indicated by reference numeral F6 is the photographing range of the imaging device 100 set to the rotation angle θ2 = −150 degrees indicating the roll state, and has the photographing range B300 and the photographing range B200 with the non-imaging range OB interposed therebetween.

The control unit 500 takes images of the six frames illustrated in FIG. 11 in order, and the first and second imaging units 200 take images of the six frames, respectively, and the six sets of the first and second images are acquired. Based on the data in the above, a compositing process for narrowing the non-photographing range OB is performed. By performing this compositing process, the non-shooting range OB existing in a band shape in FIG. 9C is interpolated by the data of the first and second images in other frames and surrounds the sun S as a high-brightness subject. It is suppressed to a dodecagonal area. That is, the dodecagonal region that is lacking from the spherical image after the compositing process can be made smaller than the strip-shaped region that is lacking when only one frame is imaged.

Instead of imaging 6 frames, for example, 2 frames are imaged in the photographing range indicated by the reference numeral F1 and the imaging range indicated by the reference numeral F4, and based on the data of the two sets of the first and second images, the imaging is performed. A compositing process that narrows the non-photographing range OB may be performed. By performing this compositing process, the non-shooting range OB existing in a band shape in the case of the shooting range indicated by the reference numeral F1 is interpolated with the data of the first and second images obtained in the case of the shooting range indicated by the reference numeral F4. , It is suppressed to a square area surrounding the sun S as a high-brightness subject. That is, the quadrangular region that is lacking from the spherical image after the compositing process can be made smaller than the strip-shaped region that is lacking when only one frame is imaged.

According to the third embodiment described above, the following effects can be obtained. That is, the image pickup apparatus 100 was performed by the first image pickup unit 200 and the second image pickup unit 300, respectively, before and after the rotation about the third axis (axis Bx). The second imaging was performed, and the first image (first and second images in the first frame) obtained in the first imaging and the second image (first and second images in the second frame) obtained in the second imaging were performed. A control unit 500 that functions as a generation unit that generates a third image that interpolates the non-shooting range by using the second image) is provided. With this configuration, for example, when the opening angle θ1 is not 360 degrees, the first image acquired by the first imaging unit 200 and the second image acquired by the second imaging unit 300 In the case where a 360-degree spherical image cannot be obtained even if the images are joined together, the area missing from the spherical image after the joining process is smaller than in the case where only one frame is used for the first and second images. It can be suppressed.

(Modification example 3)
When the opening angle θ1 of the image pickup apparatus 100 described above is not 360 degrees, the dodecagonal to quadrangular region described above is insufficient from the spherical image even when the synthesis process according to the third embodiment is performed. In the third modification, the control unit 500 fills the missing region from the spherical image with the created image.

Specifically, the control unit 500 uses the data of the image located around the deficient area to duplicate the data corresponding to substantially the same area (area) as the deficient area, and the deficient area. The data is complemented with the duplicated data.
Further, the control unit 500 generates image data having similar characteristics to the data of the image located in the surrounding area based on the data of the image located around the insufficient area, and the control unit 500 generates the data of the image having similar characteristics to the data of the image located in the surrounding area. The data may be supplemented with the above-generated data.

According to the third modification, the control unit 500 complements the image data created by duplication or generation for the missing area from the spherical image, so that, for example, the missing area of the spherical image is created. It is possible to prevent the user's usability from being impaired by being complemented with data unrelated to the spherical image, such as blackout. In particular, when the deficient region of the spherical image is the region surrounding the sun S, the surrounding of the deficient region is an image of the sky, so it is necessary to create data of an image of the sky whose characteristics are similar to those of the surrounding sky. Can be done. Therefore, the complemented portion becomes less noticeable, and the complemented portion can be appropriately complemented without impairing the usability of the user.

(Modification example 4)
Instead of supplementing the missing area from the spherical image with image data having similar characteristics to the surrounding sky, a logo mark, a symbol mark, an icon, or the like may be used to fill the area. By filling with marks or icons that are familiar to the user, the missing area can be filled appropriately without impairing the user's usability, unlike the case where the missing area of the spherical image is blacked out. ..

(Modification 5)
The above-described image pickup device 100 has been described as an example in which the user holds it by hand or fixes it on a tripod. However, by mounting the grip member 450 of the image pickup device 100 on a self-propelled trolley and running the trolley. The image pickup apparatus 100 may be configured to be movable.

Further, the image pickup device 100 may be mounted on the drone, and the image pickup device 100 may be configured to be movable by flying the drone. When the image pickup device 100 is mounted on the drone, instead of changing the pan state (rotation angle θ3 around the axis Gx) of the image pickup device 100, the direction of the drone is changed in the hovering state in which the drone is stationary in the air (ladder). The shooting direction D of the image pickup apparatus 100 may be changed accordingly.

The dolly or drone equipped with the image pickup apparatus 100 may be moved and controlled so as to move between the main subject and the high-intensity light source. At this time, the control unit 500 of the image pickup device 100 controls the shooting direction D so that the shooting direction D as the image pickup device 100 faces the main subject. When the shooting direction D of the image pickup apparatus 100 faces the main subject, the high-intensity light source is located in the direction 180 degrees opposite to that of the main subject. It can be appropriately excluded from the shooting range of.

(Fourth Embodiment)
The imaging apparatus according to the fourth embodiment has five imaging units. Each image pickup unit has a fisheye lens and an image pickup element provided on the image plane of the fisheye lens, similarly to the first image pickup unit 200 and the second image pickup unit 300 in the above-described embodiment. The five image pickup units are configured to be able to individually switch between a state in which the subject image imaged by the fisheye lens is captured by the image pickup device and a state in which imaging is paused.

FIG. 12 (a) is a schematic view of the image pickup apparatus 100X according to the fourth embodiment as viewed from above, and FIG. 12 (b) is a schematic view of the image pickup apparatus 100X as viewed from the side. The cube-shaped imaging device 100X is provided with a fisheye lens 210U of a first imaging unit 200U on the upper surface. On each of the four sides of the cube shape, the fisheye lens 210S of the second imaging unit 200S, the fisheye lens 210E of the third imaging unit 200E, the fisheye lens 210N of the fourth imaging unit 200N, and the fisheye lens 210W of the fifth imaging unit 200W, respectively. Is provided.

As shown in FIG. 12B, in the image pickup apparatus 100X, the lower surface of the cube shape is supported by the support member 460. The support member 460 is provided along an axis Gx perpendicular to the ground (floor surface), one end of which supports the image pickup apparatus 100X, and the other end of which is fixed on the carriage 600. The bogie 600 is provided with wheels 610 that can be steered and driven by a motor (not shown). Each of the four wheels 610 is a steering wheel and a driving wheel. The image pickup apparatus 100X is configured to be movable in any direction by driving four wheels 610. For example, by driving the four wheels 610 in the same direction to translate the carriage 600, the image pickup apparatus 100X moves in an arbitrary direction. Further, the image pickup apparatus 100X is rotated by driving the four wheels 610 in different directions by 90 degrees to rotate the carriage 600 around the axis Gx.

With this configuration, the first imaging unit 200U, the second imaging unit 200S, the third imaging unit 200E, the fourth imaging unit 200N, and the fifth imaging unit 200W are each wider than the hemisphere. The range can be imaged. The control unit of the image pickup apparatus 100X can obtain an image as illustrated below by having two of the five image pickup units perform imaging and suspending the remaining three image pickup units, for example.

(Example 1)
The second imaging unit 200S and the fourth imaging unit 200N each image a wider range than the hemisphere, and the first imaging unit 200U, the third imaging unit 200E, and the fifth imaging unit 200W are paused. And, as in the case of FIG. 3A, two images having different imaging directions by 180 degrees are acquired. In other words, it is possible to photograph a 360-degree spherical image in the horizontal and vertical directions. At that time, it is necessary to secure a margin for connecting the second image acquired by the second imaging unit 200S and the fourth image acquired by the fourth imaging unit 200N as an image of the whole celestial sphere. Can be done.

(Example 2)
Further, the third imaging unit 200E and the fifth imaging unit 200W are used to image a wider range than the hemisphere, and the first imaging unit 200U, the second imaging unit 200S, and the fourth imaging unit 200N are combined. When paused, two images with different imaging directions of 180 degrees are acquired, as in the case of FIG. 3A. In other words, it is possible to photograph a 360-degree spherical image in the horizontal and vertical directions. At that time, it is necessary to secure a margin for connecting the third image acquired by the third imaging unit 200E and the fifth image acquired by the fifth imaging unit 200W as an image of the whole celestial sphere. Can be done.

(Example 3)
For example, the second imaging unit 200S and the third imaging unit 200E can image a wider range than the hemisphere, and the first imaging unit 200U, the fourth imaging unit 200N, and the fifth imaging unit 200W can be imaged. When paused, two images with different imaging directions of 90 degrees are acquired, as in the case of FIG. 3B. In the case of (Example 3), it is not included in either the second image acquired by the second imaging unit 200S and the third image acquired by the third imaging unit 200E, and is not included in the image pickup device 100X. A non-shooting range OB that is not shot occurs.

(Example 4)
For example, the fourth imaging unit 200N and the first imaging unit 200U image a wider range than the hemisphere, and the second imaging unit 200S, the third imaging unit 200E, and the fifth imaging unit 200W are suspended. Then, as in the case of FIG. 3B, two images having different imaging directions by 90 degrees are acquired. In the case of (Example 4), it is not included in either the fourth image acquired by the fourth imaging unit 200N and the first image acquired by the first imaging unit 200U, and is not included in the imaging device 100X. A non-shooting range OB that is not shot occurs.
When two imaging units provided on adjacent surfaces in a cubic shape are used for imaging, two imaging units may be selected in different combinations other than the above (Example 3) and (Example 4). In either case, a non-shooting range OB that is not captured by the imaging device 100X occurs.

When the control unit of the image pickup apparatus 100X detects a high-intensity subject in at least one of the shooting ranges of the two image pickup units that perform imaging, the control unit is either the fourth method or the fifth method described below. Or, by using both the fourth method and the fifth method, control is performed to avoid the high-luminance subject from the shooting range of the image pickup apparatus 100X so that the high-luminance subject is not captured.

In the fourth embodiment, the fourth method or the fifth method may be set in advance as a method used for controlling to avoid a high-intensity subject such as the sun S from the shooting range of the image pickup apparatus 100X. Alternatively, as in the modification 2 described above, a different method may be set for each shooting scene determined based on the image of at least one of the two imaging units for imaging.

(Fourth method)
In the fourth method, a high-brightness subject is avoided by switching between imaging and pause for each imaging unit. Specifically, by switching from the setting for acquiring the spherical image of the above (Example 1) or (Example 2) to the setting for acquiring the image as described above (Example 3) or (Example 4), The image pickup device 100X avoids a high-intensity subject. The control unit of the image pickup device 100X performs imaging with two of the five image pickup units so that the high-brightness subject is out of the shooting range of the image pickup device 100X (that is, included in the non-shooting range OB), and three The imaging unit is paused.
In the present embodiment, an example in which imaging is performed by two imaging units that do not receive light from a high-brightness subject will be described, but if there is only one imaging unit that receives light from a high-brightness subject, Only one imaging unit may be paused and controlled so that four imaging units perform imaging.

(Fifth method)
In the fifth method, a high-intensity subject is avoided by moving and rotating the image pickup apparatus 100X. Specifically, the control unit of the image pickup device 100X moves the image pickup device 100X so that the high-intensity subject is out of the shooting range of the image pickup device 100X (that is, included in the non-shooting range OB).
Further, the control unit of the image pickup device 100X rotates the image pickup device 100X around the axis Gx so that the high-intensity subject is out of the shooting range of the image pickup device 100X (that is, included in the non-shooting range OB).

According to the fourth embodiment described above, the following effects can be obtained.
(1) The imaging device 100X includes a first imaging unit 200U, a second imaging unit 200S, a third imaging unit 200E, a fourth imaging unit 200N, and a fifth imaging unit 200W capable of imaging different imaging directions. , The first imaging unit 200U, the second imaging unit 200S, the third imaging unit 200E, the fourth imaging unit 200N, and the detection unit that detects the high-brightness region in the range imaged by the fifth imaging unit 200W. , The first imaging unit 200U, the second imaging unit 200S, the third imaging unit 200E, the fourth imaging unit 200N, and the fifth imaging unit 200W are driven and controlled to detect a high-luminance region detected by the detection unit. Has a control unit that excludes the first imaging unit 200U, the second imaging unit 200S, the third imaging unit 200E, the fourth imaging unit 200N, and the fifth imaging unit 200W from the imaging range. With this configuration, the control unit of the imaging device 100X includes a plurality of imaging units (first imaging unit 200U, second imaging unit 200S, third imaging unit 200E, fourth imaging unit 200N, fifth imaging unit). The imaging unit includes a high-brightness subject such as the sun S in the shooting range (in other words, the non-shooting range OB includes the sun S or the like), and the high-brightness subject such as the sun S is suspended. However, it is possible to avoid being reflected in the operating imaging unit.

(2) Further, the trolley 600 (wheel 610) as a drive control unit of the image pickup apparatus 100X includes a first image pickup unit 200U, a second image pickup unit 200S, a third image pickup unit 200E, and a fourth image pickup unit 200N. A high-brightness subject is included in the shooting range of the resting imaging unit of the fifth imaging unit 200W (in other words, the high-brightness subject is included in the non-shooting range OB of the imaging unit that performs shooting). By moving and rotating the image pickup device 100X, it is possible to appropriately avoid the reflection of a high-intensity subject such as the sun S.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Aspects in which the configurations shown in the embodiments and modifications are used in combination are also included in the scope of the present invention. Other aspects conceivable within the scope of the technical idea of the present invention are also included within the scope of the present invention.

100 ... Imaging device, 200, 200U, 200E, 200S, 200N, 200W, 300 ... Imaging unit, 210, 210U, 210E, 210S, 210N, 210W, 310 ... Fisheye lens, 220, 320 ... Image sensor, 400 ... Connection unit, 410 ... Support member, 420 ... 430 ... Shaft member, 450 ... Gripping member, 500 ... Control unit, 540 ... Drive control unit, 550 ... Shooting (non-shooting) range calculation unit, 560 ... High brightness region detection unit, 570 ... Rotation Quantitative calculation unit, 580 ... image generation unit, 700 ... display unit, 800 ... operation member, Ax, Bx, Gx ... axis, P ... subject, S ... sun

Claims (6)

Multiple imaging units capable of capturing different imaging directions,
A detection unit that detects a high-luminance region in the range to be imaged by the plurality of imaging units, and a detection unit.
A control unit that controls the plurality of imaging units and excludes the high-luminance region detected by the detection unit from the range of imaging by the plurality of imaging units.
An imaging device comprising. In the imaging device according to claim 1,
The control unit is an imaging device that changes the imaging direction of the plurality of imaging units and excludes the high-luminance region from the range of imaging by the plurality of imaging units. In the imaging device according to claim 2,
The plurality of imaging units include a first imaging unit and a second imaging unit whose imaging directions can be changed, a first imaging direction which is an imaging direction of the first imaging unit, and a second imaging direction which is an imaging direction of the second imaging unit. It is provided with a first support unit that supports the first image pickup unit and the second image pickup unit so as to rotate around the first support shaft so that the image pickup direction can be changed.
The first imaging range captured by the first imaging unit and the second imaging range imaged by the second imaging unit have overlapping regions that overlap each other and non-overlapping regions that do not overlap with each other.
The control unit controls the first support unit to change the size of the overlapping region by changing the first imaging direction and the second imaging direction around the first support axis. An imaging device that excludes the high-luminance region from the first imaging range and the second imaging range. In the imaging device according to claim 3,
The image pickup unit includes a second support unit that supports the first image pickup unit and the second image pickup unit so as to rotate around a second support shaft parallel to the first support shaft.
The control unit controls the second support unit and rotates the first imaging unit and the second imaging unit around the second support shaft without changing the size of the overlapping region. An imaging device that excludes the high-luminance region from the first imaging range and the second imaging range. In the imaging device according to claim 3 or 4,
The image pickup unit includes a third support unit that supports the first image pickup unit and the second image pickup unit so as to rotate around a third support shaft that intersects the first support shaft.
The control unit controls the third support unit and rotates the first imaging unit and the second imaging unit around the third support shaft without changing the size of the overlapping region. An imaging device that excludes the high-luminance region from the first imaging range and the second imaging range. In the imaging device according to claim 5,
The first image obtained in the first photographing performed before the first imaging unit and the second imaging unit rotate around the third support shaft, and the first imaging unit and the second imaging unit A third image that interpolates the first imaging range and the non-imaging range outside the second imaging range using the second image obtained in the second imaging performed after rotating around the third support axis. An imaging device including a generation unit that generates.

Images