JP2017188871A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus that uses a rolling shutter type first image sensor and second image sensor to capture images in which a subject is partly duplicated, with a reduced time shift between boundary regions of the images.SOLUTION: An imaging apparatus 100 includes a rolling shutter type first image sensor 110 and second image sensor 111, a timing generator 114 that controls operation timings of the two image sensors, and a controller 112 that subjects generated image data to image processing and controls the timing generator 114. The two image sensors capture images of a subject to be partly duplicated and generate image data. The controller 112 controls the timing generator 114 in such a way that the timings of the exposure periods of the respective lines of the duplicated region generated by the two image sensors match.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an imaging apparatus using two rolling shutter type image sensors.

  Patent Document 1 discloses an imaging apparatus that performs clipping processing in an arbitrary range from a wide-angle video and obtains a video having a composition desired by a photographer.

  Patent Document 2 discloses a multi-view imaging apparatus that corrects rolling shutter distortion, which is one of the features of a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

JP 2010-147925 A JP 2013-120435 A

  The present disclosure relates to an imaging apparatus that reduces temporal deviation in a boundary portion of images captured so that a part of a subject overlaps by using a first image sensor and a second image sensor of a rolling shutter system. provide.

  An imaging apparatus according to the present disclosure includes a rolling shutter first image sensor and a second image sensor, a timing generator, and a controller. The first image sensor and the second image sensor generate image data by imaging so that a part of the subject overlaps each other. The timing generator controls the operation timing of the first image sensor and the second image sensor. The controller performs image processing on the generated image data. Further, the controller controls the timing generator so that the exposure periods of the respective lines included in the overlapping area generated by the first image sensor and the second image sensor are matched.

  The imaging apparatus according to the present disclosure reduces temporal shift in a boundary portion of images captured so that a part of a subject overlaps using a first image sensor and a second image sensor of a rolling shutter system. It is effective for.

1 is a block diagram illustrating a configuration of an imaging device in Embodiment 1. Schematic diagram of image data captured by the first image sensor and the second image sensor in the first embodiment Timing chart of signals related to operations of first image sensor and second image sensor in the first embodiment Timing chart of first image sensor and second image sensor showing rolling shutter distortion Timing chart of first image sensor and second image sensor in Embodiment 1 Timing chart of first image sensor and second image sensor in Embodiment 2 FIG. 7 is a block diagram illustrating a configuration of an imaging device according to Embodiment 3. Timing chart of signals related to operation of first image sensor according to Embodiment 3

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

  The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

(Embodiment 1)
Hereinafter, Embodiment 1 will be described with reference to FIGS.

[1-1. Constitution]
FIG. 1 is a block diagram illustrating a configuration of the imaging apparatus 100 according to the first embodiment. The imaging apparatus 100 includes a first image sensor 110, a second image sensor 111, a timing generator (TG) 114, a controller 112, and a memory 115.

  The first image sensor 110 and the second image sensor 111 capture a subject image and generate image data. For example, the number of horizontal pixels is 3840 pixels and the number of vertical pixels is 2160 pixels, and 60 frames of 4K resolution image data is obtained per second. The subject image is formed by an optical system (not shown). These image sensors are often composed of CMOS.

  The TG 114 generates a signal CLK1 necessary for the operation of the first image sensor 110. Further, a signal CLK <b> 2 necessary for the operation of the second image sensor 111 is generated according to the setting of the controller 112. For example, it is a signal related to an electronic shutter operation, a still image or moving image imaging operation, a data reading operation, and the like. Details of this operation will be described later.

  The controller 112 performs various image processing on the output image data of the first image sensor 110 and the second image sensor 111. For example, the controller 112 performs white balance adjustment processing, gamma processing, YC conversion processing, rolling shutter distortion correction, image composition processing, compression processing, and the like on the output image data. The controller 112 also controls the TG 114 to control the operation timing of the first image sensor 110 and the second image sensor 111. Furthermore, the controller 112 controls the entire imaging apparatus 100 according to a computer program described in software or firmware.

  The memory 115 temporarily stores image data that is the output of the first image sensor 110 and the second image sensor 111. The memory 115 temporarily stores image data being processed by the controller 112 and image data after the processing is completed. The memory 115 is also used as a program memory for the controller 112, and temporarily stores program instructions, data, a program diagram related to exposure control, and the like. In short, the memory 115 temporarily stores data in order to realize image processing and program execution by the first image sensor 110 and the second image sensor 111.

  FIG. 2 is a schematic diagram illustrating an example in which the first image sensor 110 and the second image sensor 111 have captured a part of the stationary subject 200 (overlapping region in FIG. 2) so as to overlap. Specifically, FIG. 2A shows the image data 120 of the upper portion (200A) of the subject 200 imaged by the first image sensor 110, and FIG. 2C shows the image data taken by the second image sensor 111. FIG. 2B shows image data 122 obtained by synthesizing image data of the subject 200 captured by the first image sensor 110 and the second image sensor 111. FIG. Indicates.

  The first image sensor 110 reads image data in the reading direction of the image sensor (in the direction of the arrow in FIG. 2) for each line from the start line 0Ha to the end line (La) Ha to be imaged. The second image sensor 111 reads image data in the reading direction of the image sensor (in the direction of the arrow in FIG. 2) for each line from the start line 0Hb to the end line (Lb) Hb for imaging. The optical system of the first image sensor 110 and the optical system of the second image sensor 111 are arranged in a direction perpendicular to the readout direction so that a part of the subject to be imaged overlaps. In FIG. 2, the overlapping area is from 0Hb to (La) Ha.

  The number of lines (ΔL) included in the overlapping area is determined by comparing each pixel with respect to image data obtained by imaging a test chart or the like with the first image sensor 110 and the second image sensor 111, for example. Calculated.

[1-2. Operation]
The operation of the imaging apparatus 100 configured as described above will be described below.

  FIG. 3 is a timing chart showing an example of a signal generated by the TG 114 in FIG. The horizontal synchronization signal H1 is a signal indicating the horizontal synchronization timing of the first image sensor 110. The vertical synchronization signal V1 is a signal indicating the vertical synchronization timing of the first image sensor 110. The vertical synchronization signal V1 operates at a cycle T1. The horizontal synchronization signal H2 is a signal indicating the horizontal synchronization timing of the second image sensor 111. The vertical synchronization signal V2 is a signal indicating the vertical synchronization timing of the second image sensor 111. The vertical synchronization signal V2 operates at a period T2. The horizontal synchronization signal H1 and the horizontal synchronization signal H2 are generated, for example, by processing the clock of the same oscillator at least one of multiplication and division. Therefore, the horizontal synchronization signal H1 and the horizontal synchronization signal H2 are synchronized. The vertical synchronization signal V1 is generated, for example, by dividing the operation clock C1 (not shown). The vertical synchronization signal V2 is generated, for example, by dividing the operation clock C2 (not shown). Note that the operation clocks C1 and C2 are synchronized. The vertical synchronization signal V2 is a clock delayed by Δt from the vertical synchronization signal V1 in accordance with the setting from the controller 112.

  4A and 4B are timing charts showing exposure timings of the first image sensor 110 and the second image sensor 111. FIG. Here, an example in which the first image sensor 110 and the second image sensor 111 are operated in the external trigger synchronization mode will be described.

  4A and 4B, the start line 0Ha represents the first line imaged by the first image sensor 110. That is, it is a line from the imaging start point (PS1) of the first line to the imaging end point (PE1). An end line (La) Ha represents an end line taken by the first image sensor 110. That is, it is a line from the imaging start point (QS1) of the end line to the imaging end point (QE1).

  With the falling edge of the external trigger (vertical synchronization signal V1) of the first image sensor 110 as a trigger, the exposure of the start line 0Ha is completed in PE1, and reading is started. After the completion of line reading, the charge accumulated in the pixel is reset and the next exposure is started. The exposure time of the first image sensor 110 is the interval (T1) between the falling edge and the next falling edge of the external trigger (vertical synchronization signal V1). As shown by the one-dot chain line starting with PE1 on the start line 0Ha and ending with the tip QE1 of the arrow on the end line (La) Ha, the image data of each line (from the start line 0Ha to the end line (La) Ha) sequentially ( The output (reading) of D1), the resetting of the charge of the line after completion of reading, and the exposure are repeated.

  The period E1 is the time from the start of exposure (PS1) of the start line 0Ha imaged by the first image sensor 110 to the start of exposure (QS1) of the end line (La) Ha. The period R1 is the time required to read out one frame of image data captured by the first image sensor 110. In this embodiment, the line charge reset time after completion of line reading is included in the period R1, and the period R1 is equal to the period E1.

  The start line 0Hb represents the first line imaged by the second image sensor 111. That is, it is a line from the imaging start point (MS1) of the first line to the imaging end point (ME1). An end line (Lb) Hb represents an end line taken by the second image sensor 111. That is, it is a line from the imaging start point (NS1) of the end line to the imaging end point (NE1). With the falling edge of the external trigger (vertical synchronization signal V2) of the second image sensor 111 as a trigger, the exposure of the start line 0Hb is completed in ME1 and reading is started. After the completion of line reading, the charge accumulated in the pixel is reset and the next exposure is started. The interval (T2) from the falling edge to the next falling edge of the external trigger (vertical synchronization signal V2) is the exposure time of the second image sensor 111. Image data of each line from the start line 0Hb to the end line (Lb) Hb sequentially, as shown by the two-dot chain line starting from ME1 on the start line 0Hb and ending at the tip NE1 of the arrow on the end line (Lb) Hb The output (reading) of (D2), the resetting of the charge of the line after the reading is completed, and exposure are repeated.

  The period E2 is the time from the exposure start (MS1) of the start line 0Hb imaged by the second image sensor 111 to the exposure start (NS1) of the end line (Lb) Hb. The period R2 is the time required to read out one frame of image data captured by the second image sensor 111. In this embodiment, the line charge reset time after completion of line reading is included in the period R2, and the period R2 and the period E2 are equal.

  As described above, in the external trigger synchronization mode, the exposure period (reading timing) of each line corresponding to the horizontal direction of the image sensor is controlled using the external signal (vertical synchronization signal) as a trigger.

  FIG. 4A shows an exposure timing when the subject 201 is imaged in a state where the external trigger (vertical synchronization signal V1) of the first image sensor 110 and the external trigger (vertical synchronization signal V2) of the second image sensor 111 are synchronized. It is a timing chart which shows. Since it is a rolling shutter system, the timing of the exposure periods of the start line 0Ha (PS1-PE1) and the end line (La) Ha (QS1-QE1) are different. Therefore, for example, when the subject 201 is moving in the right direction in the drawing at a high speed relative to the imaging device 100, the image data read from the first image sensor 110 is an image such as the image data 202A. It becomes. Since the vertical synchronization signal V1 and the vertical synchronization signal V2 are synchronized, the exposure period of the start line 0Hb (MS1-ME1) of the second image sensor 111 is the start line 0Ha (PS1-PE1) of the first image sensor 110. ) Is the same as the exposure period timing. Therefore, the image data read from the second image sensor 111 is an image such as the image data 203A.

  When the number of lines in the overlapping area is ΔL, the leading line of the overlapping area is the (La−ΔL) Ha line of the first image sensor 110 and the 0Hb line (MS1-ME1) of the second image sensor 111. Equivalent to. The end line of the overlapping region corresponds to the (La) Ha line (QS1-QE1) of the first image sensor 110 and the (ΔL) Hb line of the second image sensor 111. However, the timings of the exposure periods are different.

  As described above, in FIG. 4A, when the subject is relatively moving, the timing of the exposure period of the overlapping area of the first image sensor 110 and the second image sensor 111 is different, and thus the first image sensor 110. There is a temporal shift in the overlapping area in the image data generated by the second image sensor 111.

  FIG. 4B is a timing chart of the first image sensor 110 and the second image sensor 111 in the present embodiment. Unlike FIG. 4A, the subject 201 is moved at a timing when only the external trigger (vertical synchronization signal V2) of the second image sensor 111 is delayed by Δt with respect to the external trigger (vertical synchronization signal V1) of the first image sensor 110. It is a timing chart which shows the exposure timing at the time of imaging.

  In FIG. 4B, the first image sensor 110 reads each line using a vertical synchronization signal V1 having a period T1 as a trigger. After completion of line reading, the next exposure is started. A period T1 is an exposure period of the first image sensor 110. The second image sensor 111 reads each line using the vertical synchronization signal V2 having a period T2 delayed by Δt from the vertical synchronization signal V1 as a trigger. Specifically, after the completion of line reading, the charge accumulated in the pixel is reset, and the next exposure is started. For example, when the subject 201 is moving at a high speed in the right direction in the figure relative to the imaging device 100, the subject 201 starts with PE1 on the start line 0Ha read from the first image sensor 110, and ends with the end line (La ) The image data in the period of the alternate long and short dash line that ends at the point QE1 of the arrow on Ha is an image like the image data 202B, similar to the image data 202A in FIG. On the other hand, unlike FIG. 4A, the start line 0Hb of the second image sensor 111 is the exposure timing delayed by Δt from the start line 0Ha of the first image sensor 110. Therefore, the image data in the period of the two-dot chain line starting from ME1 on the start line 0Hb read from the second image sensor 111 and ending at the tip NE1 of the arrow on the end line (Lb) Hb is like the image data 203B. It becomes an image.

  When the number of lines in the overlapping area is ΔL, the leading line of the overlapping area is the (La−ΔL) Ha line of the first image sensor 110 and the 0Hb line (MS1-ME1) of the second image sensor 111. Equivalent to. The end line of the overlapping region corresponds to the (La) Ha line (QS1-QE1) of the first image sensor 110 and the (ΔL) Hb line of the second image sensor 111. And the timing of the exposure period of each line of the overlapping region matches.

  Since the image data of the overlapping area is the same, for example, the controller 112 uses the data up to the overlapping area end line (QS1-QE1) (hatched portion of D1) as the image data of the first image sensor 110, and As the image data of the second image sensor 111, data from the line after the end of the overlapping region (the hatched portion of D2) is used. At this time, reading from the two image sensors is congested (overlapped) in some sections.

[1-3. Effect]
As described above, in the present embodiment, the imaging apparatus 100 includes the first image sensor 110, the second image sensor 111, the TG 114, and the controller 112. The first image sensor 110 and the second image sensor 111 capture an image so that part of the subject image overlaps to generate image data. The controller 112 controls the TG 114 so that the exposure periods of the respective lines included in the overlapping areas of the generated image data are matched.

  Thereby, the timing difference of the exposure period in each line of the overlapping area imaged by the first image sensor 110 and the second image sensor 111 is reduced. For this reason, temporal deviation of the image near the boundary captured by the first image sensor 110 and the second image sensor 111 is reduced.

  Further, in the present embodiment, the controller 112 determines the frame synchronization timing of the second image sensor 111 with respect to the frame synchronization timing (V1) of the first image sensor 110 according to the number of lines ΔL included in the overlapping region. The timing generator (TG) 114 is controlled so as to delay (V2).

  Thereby, the timing of the exposure period of each line of the first image sensor 110 and the second image sensor 111 can be controlled by the frame period. Therefore, the load on the controller 112 is reduced.

  In this embodiment, the controller 112 delays the exposure timing of the second image sensor 111 by Δt [seconds] that satisfies the following condition (1) with respect to the exposure timing of the first image sensor 110. It is desirable to control the TG 114 to make it happen.

Δt / E1 = 1−ΔL / L1 (1)
here,
E1: Time [seconds] from the start of exposure of the start line to the start of exposure of the end line taken by the first image sensor 110;
L1: Number of horizontal lines taken by the first image sensor 110,
ΔL: the number of lines that the overlapping area in the first image sensor 110 has,
It is.

  Thereby, the timing of the exposure period in each line of the overlapping area captured by the first image sensor 110 and the second image sensor 111 can be appropriately controlled, and the first image sensor 110 and the second image sensor 111 can be controlled. The time shift of the image near the boundary to be imaged is reduced. Therefore, the generated image data can be corrected for rolling shutter distortion by performing image processing in the same manner as an image obtained from one rolling shutter type image sensor.

  In the present embodiment, the controller 112 synthesizes an image in a specified range among the image data generated by the first image sensor 110 and the second image sensor 111.

  As a result, it is possible to capture a wider-angle video while maintaining the resolution obtained from one image sensor.

  In the present embodiment, the controller 112 cuts out part of the generated image data.

  Thereby, for example, an image including a subject of interest can be cut out from a wide-angle image of the imaging device 100 installed so as to look over the entire stadium. Therefore, it becomes possible to acquire an image with an arbitrary angle of view, and the options for the acquired image can be expanded.

(Embodiment 2)
The second embodiment will be described below with reference to FIG. The second embodiment is a modification of the first embodiment.

[2-1. Configuration etc.]
The configuration and the overlapping area are the same as those in the first embodiment.

[2-2. Operation]
In FIG. 5, the subject 201 is imaged at a timing when the external trigger (vertical synchronization signal V2) of the second image sensor 111 is delayed by Δt with respect to the external trigger (vertical synchronization signal V1) of the first image sensor 110. It is a timing chart which shows the exposure timing in the case.

  The first image sensor 110 starts exposure of each line with a vertical synchronization signal V1 having a cycle T1 as a trigger. The exposure time is the time set for the first image sensor 110 (the time from the start of 0 Ha exposure to the start of reading of the first image sensor 110). The period T1 is the time from the start of exposure of the first line (PS1-PE1) of the first image sensor 110 to the completion of reading of the end line (NS1-NE1) of the second image sensor 111. Yes, one frame period.

  The period E1 is the time from the exposure start point (PS1) of the start line 0Ha imaged by the first image sensor 110 to the exposure start point (QS1) of the end line (La) Ha. The period R1 is a time for reading out one frame of image data captured by the first image sensor 110. In the present embodiment, the period R1 and the period E1 are equal.

  The second image sensor 111 starts exposure of each line with a falling edge of the vertical synchronization signal V2 having a period T2 delayed by Δt from the vertical synchronization signal V1 as a trigger. The exposure time is the time set for the second image sensor 111 (the time from the start of 0 Hb exposure to the start of reading of the second image sensor 111), and is equal to the exposure time of the first image sensor 110. For example, when the subject 201 is moving at a high speed in the right direction in the figure relative to the imaging device 100, the image data read from the first image sensor 110 is the same as the image data 202B in FIG. An image such as the image data 202C is obtained. The exposure start of the start line 0Hb (MS1-ME1) of the second image sensor 111 is an exposure timing delayed by Δt from the start of exposure of the start line 0Ha (PS1-PE1) of the first image sensor 110. Accordingly, the image data read from the second image sensor 111 is an image such as the image data 203C.

  When the number of lines in the overlapping area is ΔL, the leading line of the overlapping area is the (La−ΔL) Ha line of the first image sensor 110 and the 0Hb line (MS1-ME1) of the second image sensor 111. Equivalent to. The end line of the overlapping region corresponds to the (La) Ha line (QS1-QE1) of the first image sensor 110 and the (ΔL) Hb line of the second image sensor 111. Therefore, the timings of the exposure periods of the respective lines in the overlapping area coincide.

  Since the image data of the overlapping area is the same, for example, for each frame, the controller 112 outputs data up to the overlapping area end line (QS1-QE1) (shaded portion of D1) as the image data of the first image sensor 110. Are sequentially read out from the line after the end of the overlapping area as the image data of the second image sensor 111 (shaded portion of D2).

[2-3. Effect]
As shown in FIG. 5, reading from the first image sensor 110 and the second image sensor 111 is not congested (does not overlap). Thereby, the bandwidth of the memory 115 can be reduced. As a result, the degree of freedom in design is improved.

(Embodiment 3)
Hereinafter, the third embodiment will be described with reference to FIGS.

[3-1. Constitution]
An outline of the configuration of the imaging apparatus 600 according to the third embodiment will be described.

  FIG. 6 is a block diagram illustrating a configuration of the imaging apparatus 600.

  The imaging device 600 includes a first imaging unit 300, a second imaging unit 400, and an external device 500.

  The first imaging unit 300 includes a first image sensor 310, a timing generator (TG) 314, a controller 312, and a memory 315. The controller 312 outputs the image data generated by the first image sensor 310 to the external device 500 connected externally.

  The second imaging unit 400 has the same configuration as the first imaging unit 300, and includes a second image sensor 410, a timing generator (TG) 414, a controller 412, and a memory 415. The controller 412 outputs the image data generated by the second image sensor 410 to the external device 500 connected externally.

  The external device 500 includes a controller 512, a memory 515, and a display unit 518.

  The first image sensor 310 and the second image sensor 410 capture a subject image and generate image data. For example, the number of horizontal pixels is 3840 pixels and the number of vertical pixels is 2160 pixels, and 60 frames of 4K resolution image data is obtained per second. The subject image is formed by an optical system (not shown). These image sensors are often composed of CMOS.

  The TG 314 generates a signal CLK3 necessary for the operation of the first image sensor 310. When the external signal CLK5A is input from the controller 512, the CLK3 signal is switched to the external signal CLK5A via, for example, a PLL (Phase Locked Loop). The TG 414 generates a signal CLK4 necessary for the operation of the second image sensor 410. Further, when the external signal CLK5B is input from the controller 512, the CLK4 signal is switched to the external signal CLK5B through, for example, a PLL. Signal switching will be described later.

  The controller 512 performs various image processing on the output image data of the first image sensor 310 and the second image sensor 410. For example, the controller 512 performs white balance adjustment processing, gamma processing, YC conversion processing, rolling shutter distortion correction, image composition processing, image cutout processing, compression processing, and the like on the output image data. The controller 512 also controls the TG 314 and the TG 414 to control the operation timing of the first image sensor 310 and the second image sensor 410. The controller 512 controls the entire imaging apparatus 600 according to a computer program described in software or firmware.

  The memory 315 temporarily stores image data that is the output of the first image sensor 310. The memory 315 temporarily stores image data being processed by the controller 312 and image data after the processing is completed. The memory 315 is also used as a program memory for the controller 312 and temporarily stores program instructions, data, a program diagram related to exposure control, and the like. In short, the memory 315 temporarily stores data for realizing image processing and program execution in the first image sensor 310.

  The memory 415 temporarily stores image data that is the output of the second image sensor 410. The memory 415 temporarily stores image data being processed by the controller 412 and image data after the processing is completed. Further, the memory 415 is also used as a program memory of the controller 412, and temporarily stores program instructions, data, a program diagram related to exposure control, and the like. In short, the memory 415 temporarily stores data for realizing the image processing in the second image sensor 410 and the execution of the program.

  The memory 515 temporarily stores image data that is the output of the first image sensor 310 and the second image sensor 410. The memory 515 temporarily stores image data being processed by the controller 512 and image data after the processing is completed. Further, the memory 515 is also used as a program memory of the controller 512, and temporarily stores program instructions, data, a program diagram related to exposure control, and the like. In short, the memory 515 temporarily stores data in order to realize image processing and program execution by the first image sensor 310 and the second image sensor 410.

  The display unit 518 is a display that displays the image data image-processed by the controller 512, and is a liquid crystal display or the like. The composite data of the first image sensor 310 and the second image sensor 410, data obtained by cutting out a part of the first image sensor 310 and the second image sensor 410, an operation panel, and the like are displayed.

  As in the first embodiment, the image data generated by the first imaging unit 300 and the image data captured by the second imaging unit 400 are overlapped (number of lines ΔL) as part of the subject to be imaged. It is arranged in a direction perpendicular to the reading direction.

[3-2. Operation]
FIG. 7 is a timing chart showing an example of signals related to the operation of the first image sensor 310 in the third embodiment. The horizontal synchronization signal H3 is a signal indicating the horizontal synchronization timing of the first image sensor 310. The vertical synchronization signal V3 is a signal indicating the vertical synchronization timing of the first image sensor 310. Until the external signal CLK5A is input, the first imaging unit 300 operates with the signal CLK3 generated by the TG 314. When, for example, a signal of horizontal synchronization timing is input as the external signal CLK5A, the phase difference of the horizontal synchronization signal H3 from the CLK5A is fed back via the PLL, and is gradually synchronized with the CLK5A (Tlock). Similarly, the second imaging unit 400 is synchronized when the external signal CLK5B is input.

  The controller 512 delays the vertical synchronization timing of the second image sensor 410 by Δt with respect to the vertical synchronization signal V3 (frame synchronization timing) of the first image sensor 310 according to the number of overlapping lines ΔL. This delay amount is the same as in the first and second embodiments.

[3-3. Effect]
Even when a plurality of imaging devices having only one image sensor are combined to produce a wide-angle image from these images, the effect of reducing the temporal shift of the overlapping region as in the first and second embodiments can be obtained.

(Other embodiments)
As described above, Embodiments 1 to 3 have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can also be applied to embodiments that have been changed, replaced, added, omitted, and the like.

  Therefore, other embodiments will be exemplified below.

  In the first to third embodiments, the CMOS image sensor (110, 111, 310, 410) has been described as an example of the image sensor. In the present disclosure, the image sensor may be an image sensor having a rolling shutter. Therefore, the image sensor is not limited to a CMOS image sensor. Moreover, although the example which records as image data using all the pixels of an image sensor was shown, you may use a part of all the pixels as an effective pixel. In this case, it is possible to calculate the number of lines ΔL that the overlapping area has using invalid pixels.

  In the first to third embodiments, the controller (112, 312, 412, 512) has been described as an example of controlling the imaging device (100, 600) and the imaging unit (300, 400). In the present disclosure, the controllers (112, 312, 412, 512) may be configured by hard logic, or may be configured by a microcomputer using a program. If the controllers (112, 312, 412, 512) are realized by hard logic, the processing speed can be improved. If it is realized by a microcomputer, the processing contents can be changed by changing the program, so that the degree of freedom in designing the controller can be improved. Various image processing performed by the controllers (112, 312, 412, 512) includes white balance adjustment processing, gamma correction processing, scratch correction processing, aberration correction processing, YC conversion processing, rolling shutter distortion correction, and the like. The present invention is not limited to these, and all image processing may be performed, or only a part thereof may be performed. Such image processing may be performed by an image processor outside the controller (112, 312, 412, 512), and the image processor may be modularized in the image sensor.

  In the first to third embodiments, the trigger for starting reading of the image sensors (110, 111, 310, 410) is represented by a falling signal, but the present invention is not limited to this, and a rising signal may be used. Further, the case where the number of readout lines of the first image sensor 110 and the second image sensor 111 is equal and the number of readout lines of the first image sensor 310 and the second image sensor 410 is equal is shown. It is not limited to. The number of readout lines of the first image sensor (110 or 310) may be large, or the number of readout lines of the second image sensor (111 or 410) may be large.

  In the first to third embodiments, the example in which the exposure period of the image sensor is controlled at the edge-to-edge interval of the external trigger is not limited to this, and the exposure period (reading timing) is set in the image sensor. You may do it. What is necessary is just to control the exposure period (reading timing) of an image sensor according to the specification of the image sensor to be used.

  In the first to third embodiments, the external trigger synchronization mode using the vertical synchronization signal as the trigger for controlling the exposure start timing of the image sensor has been described as an example. You may control the exposure period of each line using a horizontal synchronizing signal as a trigger which controls the exposure period of each line which an image sensor images. What is necessary is just to control the exposure period (reading timing) of each line which an image sensor images according to the specification of the image sensor to be used.

  In the first to third embodiments, the read timing of the first image sensor (110, 310) and the second image sensor (111, 410) has been described with reference to FIGS. 4B and 5. 4B and 5, the reading of the second image sensor 111 is delayed by Δt with respect to the first image sensor 110, but if the reading direction of the image sensor is the opposite direction, the second image sensor The readout timing of the first image sensor 110 may be delayed by Δt with respect to 111.

  In the first to third embodiments, the TG is provided outside the controller. However, the TG may be provided inside the controller or the image sensor, and the exposure period (reading timing) of each line captured by the image sensor may be controlled. It can be anywhere if possible.

  In the first and second embodiments, the example in which the video of the image data of the first image sensor 110 is used as the image data of the overlapping region is shown. However, the image data of the overlapping area may be the image data of the second image sensor 111, or may be a video that has been subjected to image processing based on both image data.

  In the first to third embodiments, the synchronization related to the control of the image sensor has been described with reference to FIGS. 3 and 7. 3 and 7 have been described with reference to the horizontal synchronization signal as an example, the signal to be synchronized may be an operation clock of the image sensor or a vertical synchronization signal. Any clock can be used as a reference for the operation of the image sensor, and the present invention is not limited to this.

  In Embodiment 3, a liquid crystal display is used for the display portion 518. The present disclosure is not limited to this, and may be, for example, an organic EL (Electro-Luminescence) display or an externally connected monitor.

  The above-described embodiments are for illustrating the technique in the present disclosure, and various modifications, replacements, additions, omissions, and the like can be made within the scope of the claims and their equivalents.

  The present disclosure is applicable to an imaging apparatus that creates a wide-angle image by combining a plurality of images and cuts out an arbitrary range. Specifically, the present disclosure can be applied to a digital camera, a digital video camera, a surveillance camera, a mobile phone with a camera function, a smartphone, and the like.

DESCRIPTION OF SYMBOLS 100 Image pick-up device 110 1st image sensor 111 2nd image sensor 112 Controller 114 Timing generator (TG)
115 Memory CLK1 Operation signal of the first image sensor 110 CLK2 Operation signal of the second image sensor 111 V1 Vertical synchronization signal of the first image sensor 110 H1 Horizontal synchronization signal of the first image sensor 110 V2 Second image sensor 111 Vertical synchronization signal H2 Horizontal synchronization signal of the second image sensor 111 120 Image data of the upper part of the subject 121 Image data of the lower part of the subject 122 Combined image data 200 Subject 200A Upper part of the subject 200B Lower part of the subject 201 Subject 202A First 202B Image data generated by the first image sensor 110 202C Image data generated by the first image sensor 110 202C Image data generated by the first image sensor 110 203A Image data generated by the second image sensor 111 3B the second image data 300 a first image sensor 111 is image data 203C second image sensor 111 that generated the production of the imaging unit 310 first image sensor 312 controller 314 a timing generator (TG)
315 Memory CLK3 Operation signal of the first image sensor 310 400 Second imaging unit 410 Second image sensor 412 Controller 414 Timing generator (TG)
415 Memory CLK4 Operation signal of second image sensor 410 500 External device 512 Controller 515 Memory 518 Display unit CLK5A External signal sent from external device 500 to first imaging unit 300 CLK5B From external device 500 to second imaging unit 400 External signal sent 600 Imaging device

Claims (5)

A first image sensor and a second image sensor, which are rolling shutter systems that capture and capture image data so that part of the subject overlaps;
A timing generator for controlling operation timing of the first image sensor and the second image sensor;
A controller for performing image processing on the generated image data and controlling the timing generator;
With
The controller controls the timing generator so that the timing of the exposure period of each line included in the overlapping area generated by the first image sensor and the second image sensor matches.
Imaging device. The controller controls the timing generator to delay the frame synchronization timing of the second image sensor with respect to the frame synchronization timing of the first image sensor according to the number of lines included in the overlapping region.
The imaging device according to claim 1. The controller controls the timing generator so that the frame synchronization timing of the second image sensor is delayed by Δt seconds that satisfies the following condition (1) with respect to the frame synchronization timing of the first image sensor. To
The imaging device according to claim 1:
Δt / E1 = 1−ΔL / L1 (1)
here,
E1: Time [seconds] from the start of exposure of the start line to the start of exposure of the end line taken by the first image sensor;
L1: Number of horizontal lines taken by the first image sensor,
ΔL: the number of lines of the overlapping region that the first image sensor captures,
It is. The controller synthesizes an image in a specified range among image data generated by the first image sensor and the second image sensor.
The imaging device according to claim 1. The controller cuts out an image in a specified range from image data generated by the first image sensor and the second image sensor.
The imaging device according to claim 1.

Images