JP2008278430A - Imaging apparatus, imaging method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of improving a time resolution and a luminance resolution. <P>SOLUTION: An imaging apparatus of the present invention comprises: an image pickup unit for picking up a plurality of pickup images of a subject by receiving light from approximately the same subject during each of a plurality of light receiving terms partially overlapping the other light receiving term; and a partial exposure image generation unit for generating a partial exposure image to be obtained in a case where light from the subject is received within a term of a time length, included in the plurality of light receiving terms, shorter than the plurality of light receiving terms based on image contents of the plurality of picked-up images and the overlapping term of the plurality of light receiving terms. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program.

When a monitoring area is photographed using a plurality of cameras with different exposure amounts, and an object that enters the monitoring area is detected, an image portion with an appropriate exposure amount is extracted from a plurality of images with different exposure amounts, and then dynamically An apparatus for synthesizing an image with a wide range is known (for example, see Patent Document 1).
JP 2004-96488 A

  In the apparatus of Patent Document 1, it is necessary to set the frame rate low in order to take a long exposure time in order to ensure an appropriate exposure in a dark place, and the resolution with respect to time is lowered. For this reason, there is a possibility that the behavior of a suspicious person or the like cannot be recorded appropriately.

  Accordingly, an object of the present invention is to provide an imaging apparatus, an imaging method, and a program that can solve the above-described problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous specific examples of the present invention.

  In order to solve the above-described problem, the imaging apparatus according to the first aspect of the present invention receives light from substantially the same subject in each of a plurality of light receiving periods in which some of the periods overlap with other light receiving periods. And a time shorter than the plurality of light receiving periods included in the plurality of light receiving periods based on the imaging unit that captures the plurality of captured images of the subject, and the image content of the plurality of captured images and the period in which the plurality of light receiving periods overlap. A partial exposure image generation unit configured to generate a partial exposure image to be obtained when light from the subject is received during the length period.

  A difference image generating unit that generates a difference image between the first captured image and the second captured image captured in the first light receiving period and the second light receiving period in which the imaging unit partially overlaps the first light receiving period; A partial exposure image acquisition unit that acquires a partial exposure image to be obtained when light is received in a period other than a period that overlaps the second light reception period, and the partial exposure image generation unit is generated by the difference image generation unit Based on the difference image and the partial exposure image acquired by the partial exposure image acquisition unit, it should be obtained when light from the subject is received during the second light receiving period other than the period overlapping the first light receiving period. A partially exposed image may be generated.

  The imaging unit includes a mirror unit that divides the light from the subject into a plurality of lights, a plurality of light receiving units that respectively receive the plurality of lights divided by the mirror unit, and another light receiving unit A light reception control unit that receives light by overlapping a part of the light reception period and a captured image generation unit that generates a plurality of captured images based on the amount of light received by the plurality of light reception units. Good.

  The imaging unit includes a light receiving unit having a plurality of light receiving elements, an optical low-pass filter that disperses light from a subject and receives the light by the plurality of light receiving elements, and light dispersed by the optical low-pass filter to each of the plurality of light receiving elements. Based on the received light amount of a plurality of light receiving elements that receive light from the subject during the same period, You may have a captured image production | generation part which produces | generates a captured image for every said period.

  In the imaging method according to the second aspect of the present invention, a plurality of captured images of a subject are received by receiving light from substantially the same subject in each of a plurality of light receiving periods in which some periods overlap with other light receiving periods. Light from the subject during a period of time shorter than the plurality of light receiving periods included in the plurality of light receiving periods based on the image capturing unit that captures the image content and the period in which the image contents of the plurality of captured images and the plurality of light receiving periods overlap And an image generation unit that generates a partial exposure image to be obtained when light is received.

  According to a third aspect of the present invention, there is provided a program for an imaging apparatus, wherein the imaging apparatus emits light from substantially the same subject in each of a plurality of light receiving periods in which some periods overlap with other light receiving periods. Based on an image pickup unit that picks up a plurality of picked-up images of the subject, the image content of the picked-up images, and a period in which the plurality of light receiving periods overlap, shorter than the plurality of light receiving periods included in the plurality of light receiving periods It functions as an image generation unit that generates a partial exposure image to be obtained when light from a subject is received during a period of time.

  The above summary of the invention does not enumerate all the necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can record a motion of a suspicious person etc. in a dark place appropriately can be provided.

  Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the scope of claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

  FIG. 1 shows an example of a usage environment of an imaging system 100 according to an embodiment. The imaging system 100 includes a plurality of imaging units 110a and 110b (hereinafter collectively referred to as the imaging unit 110), an image processing unit 120, an imaging control unit 130, a display unit 140, and a communication line 150.

  The imaging unit 110 a provides a captured image obtained by imaging the subject 180 to the image processing unit 120 through the communication line 150. The imaging unit 110 captures a plurality of captured images with light from the subject 180 and supplies the captured images to the image processing unit 120. Note that the imaging unit 110 receives light from the subject 180 in each of a plurality of light receiving periods in which some of the periods overlap, and generates a captured image in each of the plurality of light receiving periods. Then, the image processing unit 120 generates a plurality of partial exposure images obtained when imaging is performed in a light receiving period shorter than the light receiving period from the plurality of captured images captured by the imaging unit 110a, and supplies the partial exposure images to the display unit 140. . The display unit 140 simultaneously displays a plurality of partial exposure images generated by the image processing apparatus 120 in a form that can be viewed at a time. Note that the imaging unit 110a and the image processing unit 120 may be mounted on one device as an imaging device, for example.

  Note that the operation of the imaging unit 110b is substantially the same as the operation of the imaging unit 110a, and a description thereof will be omitted. The display unit 140 may display a plurality of partial exposure images generated by the imaging unit 110b in the same manner as displaying the plurality of partial images generated by the imaging unit 110a.

  Note that the image processing unit 120 detects the movement in the imaging region by the imaging units 110a and 110b based on the image contents of the plurality of captured images captured by the imaging units 110a and 110b. When no movement is detected in the image processing unit 120, the imaging control unit 130 continuously captures images without overlapping light receiving periods as described above. Then, when a motion is detected in the image processing unit 120, the imaging control unit 130 may perform imaging with overlapping light reception periods. Note that the imaging system 100 may be a monitoring system that monitors a monitoring area using a captured image obtained by imaging the monitoring area or a partial exposure image generated from the captured image. The communication line 150 includes a communication network such as a computer network and a communication path such as a bus.

  FIG. 2 shows an example of a block configuration of the image processing unit 120. The image processing unit 120 includes a motion calculating unit 265, a motion region specifying unit 260, a difference image generating unit 220, a moving object image generating unit 270, a moving object size specifying unit 275, a partial exposure image generating unit 230, and a partial exposure image storage unit. 240, a partial exposure image acquisition unit 250, and an object image composition unit 280.

  The imaging unit 110 receives light from substantially the same subject 180 and captures a plurality of captured images of the subject 180 in each of a plurality of light receiving periods whose partial periods overlap with other light receiving periods. Then, the partial exposure image generation unit 230 includes a period of time shorter than the plurality of light receiving periods included in the plurality of light receiving periods based on the image content of the plurality of captured images and the period in which the plurality of light receiving periods overlap. A partial exposure image to be obtained when light from the subject 180 is received is generated. The partial exposure image storage unit 240 stores the partial exposure image generated by the partial exposure image generation unit 230.

  For example, the difference image generation unit 220 generates a difference image between a first captured image and a second captured image captured by the imaging unit 110 in a first light receiving period and a second light receiving period in which some periods overlap. And the partial exposure image acquisition part 250 acquires the partial exposure image which should be obtained when light-receiving in periods other than the period which overlaps with a 2nd light reception period among 1st light reception periods. Specifically, the partial exposure image acquisition unit 250 is a part of the first light reception period other than the period overlapping with the second light reception period from the partial exposure images stored in the partial exposure image storage unit 240. You may acquire the partial exposure image which should be obtained when it receives light.

  The partial exposure image generation unit 230 overlaps the first light reception period in the second light reception period based on the difference image generated by the difference image generation unit 220 and the partial exposure image acquired by the partial exposure image acquisition unit 250. A partial exposure image to be obtained when light from the subject 180 is received in a period other than the period to be generated is generated. Specifically, the partial exposure image generation unit 230 calculates the sum of the difference image generated by the difference image generation unit 220 and the partial exposure image acquired by the partial exposure image acquisition unit 250 in the first light receiving period. It is generated as a partial exposure image to be obtained when light from the subject 180 is received in a period other than the period overlapping with the light receiving period.

  The partial exposure image generation unit 230 includes image contents of the first captured image and the second captured image captured by the imaging unit 110 in the first light receiving period and the second light receiving period in which some periods overlap, and the first light receiving period. And a second light receiving period based on the overlapping period, a partial exposure image in at least one of the overlapping period, the period other than the overlapping period in the first light receiving period, and the period other than the overlapping period in the second light receiving period Is generated. Then, the motion calculation unit 265 calculates the moving speed of the object based on the image contents of the first captured image and the second captured image. Then, the moving object image generation unit 270 generates an image in which the object moves at the speed calculated by the movement calculation unit 265. Then, the object image synthesis unit 280 synthesizes the image generated by the moving object image generation unit 270 with the partial exposure image generated by the partial exposure image generation unit 230. Then, the partial exposure image obtained by combining the images of the moving objects by the moving object image generation unit 270 is supplied to the display unit 140 and displayed.

  The motion area specifying unit 260 specifies an area where a moving object exists in each of the first captured image and the second captured image based on the image contents of the first captured image and the second captured image. And the difference image generation part 220 produces | generates the difference image of the area | region which overlaps among the area | regions which the motion area specific | specification part 260 specified in the 1st captured image and the 2nd captured image. Then, the moving object size specifying unit 275 specifies the size of the moving object based on the image content of the difference image generated by the difference image generating unit 220.

  Then, the moving object image generation unit 270 generates an image of the moving object having the size specified by the moving object size specifying unit 275. Then, the object image synthesis unit 280 synthesizes the image generated by the moving object image generation unit 270 with the partial exposure image generated by the partial exposure image generation unit 230. The moving object image generation unit 270 may generate an image of an object in which the object having the size specified by the moving object size specifying unit 275 moves at the speed calculated by the movement calculating unit 265.

  As described above, the image processing unit 120 can generate a plurality of images from captured images obtained by overlapping a part of the light receiving periods, so that an image that finely represents how the object moves can be generated. Can be provided. In addition, since the image can be captured in a long light receiving period, the S / N ratio of the signal can be improved. For this reason, according to the imaging system 100, a higher quality image can be provided compared with the case where it imaged in a short light reception period.

  FIG. 3 shows an example of the light receiving period. As will be described with reference to FIGS. 8 and 9, the imaging unit 110 receives light from the subject 180 by overlapping a part of each period with each of the plurality of sets of light receiving elements. For example, the imaging unit 110 generates a captured image P301 by causing a set of light receiving elements to receive light from the subject 180 during a period from time t1 to time t4. Then, the imaging unit 110 sequentially receives the light from the subject 180 by shifting the plurality of sets of other light receiving elements by a time width ΔT shorter than the exposure time (t4−t1), thereby capturing the captured image P302 and the captured image. An image P303 and a captured image P304 are generated.

  Specifically, the imaging unit 110 generates a captured image P302 by causing another set of light receiving elements to receive light during a period from time t2 after ΔT from time t1 to time t5 after ΔT from time t4. To do. Further, the imaging unit 110 generates a captured image P303 by receiving light from another set of light receiving elements during a period from time t3 after ΔT from time t2 to time t6 after ΔT from time t5. In addition, the imaging unit 110 generates a captured image P304 by receiving light from another set of light receiving elements during a period from time t4 after ΔT from time t3 to time t7 after ΔT from time t6.

  Here, the images that are expected to be captured when the period from time t1 to time t2, the period from time t2 to time t3, and the period from time t3 to time t4 are set as the light receiving period are p1, p2, and If p3, it can be said that the captured image P301 is an image obtained by adding p1, p2, and p3. In other words, it can be expressed as P301 = p1 + p2 + p3. Similarly, images that are expected to be captured when the period from time t4 to time t5, the period from time t5 to time t6, and the period from time t6 to time t7 are set as the light receiving period are p4, Assuming p5 and p6, the captured images P302, P303, and P304 can be expressed as P302 = p2 + p3 + p4, P303 = p3 + p4 + p5, and P304 = p4 + p5 + p6.

  Here, p1, p2, p3, p4, p5, and p6 are examples of partial exposure images in the present embodiment. The partial exposure image generation unit 230 generates partial exposure images p1, p2, p3, from the image contents of the captured images P301, P302, P303, and P304 and the light receiving periods t1 to t4, t2 to t5, t3 to t6, and t4 to t7. Generate p4, p5, and p6.

  As an example, the partial exposure image acquisition unit 250 acquires partial exposure images p1 and p6. When the partial exposure images p1 and p6 are known, there are four unknown partial exposure images, p2, p3, p4, and p5. On the other hand, since there are four expressions relating to the captured images P301, P302, P303, and P304 as described above, the partial exposure image generation unit 230 generates partial exposure images p1, p2, p3, p4, p5, and p6. can do. For example, the partial exposure image generation unit 230 can generate the partial exposure image p4 by p4 = P302− (p2 + p3) = P302−P301 + p1.

  As another example of generating the partial exposure image, the partial exposure image generation unit 230 calculates a spatial frequency component for each of the captured images captured by the imaging unit 110, and a frequency component higher than a predetermined spatial frequency component. A picked-up image having a position is specified from among a plurality of picked-up images, and a partial exposure image is generated from the specified picked-up image.

  For example, the partial exposure image generation unit 230 specifies a captured image having a higher spatial frequency component among the captured images P301, P302, P303, and P304 with higher priority as a captured image for generating a partial exposure image. Here, if P303 is specified as the captured image having the highest spatial frequency component by the partial exposure image generation unit 230, the partial exposure image generation unit 230 regards p3 = p4 = p5 = P303 / 3. Thereby, the partial exposure image generation unit 230 generates the partial exposure images p1, p2, p3, p4, p5, and p6 as p2 = P302-p3-p4, p1 = P301-p2-p3, and the like. be able to.

  The captured image having a higher spatial frequency component may be a captured image having a higher average value of the spatial frequency components, and the area of the image region having a frequency component having a frequency lower than a predetermined frequency is larger. A small captured image may be used. If there is a moving object in the captured image, the captured image has a lower spatial frequency component. For this reason, since a captured image having a higher frequency component indicates that the subject change during the imaging period is smaller, by assuming that such a captured image is the sum of the partial exposure images, A partial exposure image can be generated more accurately.

  In the example of this figure, the length of the shift time ΔT is set to 1/3 of the light reception time, but the length of the shift time ΔT only needs to be shorter than the light reception time, and is not limited to the example of this figure. . Further, in the example of this figure, the time at which the light receiving period starts is sequentially shifted by ΔT for each set of light receiving elements, but the shift time ΔT may be variable. In the example of this figure, the length of the light receiving period is the same for all sets of light receiving elements, but the length of the light receiving period of at least one set is different from the length of the light receiving periods of the other sets. May be.

  FIG. 4 shows an example of a moving object in the captured image. In the captured image P301, the object moves over a range from the position captured as the object image 400 to the position captured as the object image 405. In this case, the moving object is imaged over the range from the x coordinate x0 to x1 in the captured image P301.

  On the other hand, while the captured image P302 is captured, the object moves over a range from the position captured as the object image 402 to the position captured as the object image 407 in the captured image P302. In this case, the moving object is imaged over the range from the x coordinate x3 to x2 in the captured image P302.

  In this case, the motion region specifying unit 260 extracts the outline of the object from the captured image P301 and the captured image P302 by edge extraction or the like. Then, the motion region specifying unit 260 specifies, as a motion object, an object whose position changes between the captured image P301 and the captured image P302 among objects having colors or shapes that match with a predetermined degree of matching. . Then, the motion area specifying unit 260 specifies the area included in the object as the motion area. As described above, the motion region specifying unit 260 specifies a region where a moving object exists in each of the plurality of captured images based on the image contents of the plurality of captured images.

  Then, the motion calculation unit 265 calculates a value obtained by dividing the difference between the x coordinate x2 of the tip of the object image in the captured image P302 and the x coordinate x1 of the tip of the object image in the captured image P301 by ΔT in the x direction of the object. Calculate as

  Next, a method for calculating the size of the moving object by the moving object size specifying unit 275 will be described. When the difference between the captured image P302 and the captured image P301 generated by the difference image generation unit 220 is taken, the moving object moves from the position where it is captured as the object image 402 to the position where it is captured as the object image 405. During this time, the image information of the object imaged is canceled. Therefore, in the difference image, the residual of the image due to the object appears between the position of the object image 400 and the position of the object image 402 and between the position of the object image 405 and the position of the object image 406.

  In this case, the moving object size specifying unit 275 specifies the x coordinate x4 of the tip where the residual of the image due to the object exists in the direction in which the object moves in the difference image generated by the difference image generating unit 220. In addition, the moving object size specifying unit 275 specifies the x coordinate x3 of the tip of the moving object in the direction opposite to the moving direction in the captured image P302. Then, the moving object size specifying unit 275 specifies the length L from the x coordinate x3 to x4 as the length of the object in the x direction.

  In the example of this figure, an object that moves in the x direction is taken as an example, and the method of specifying the size component of the object in the x direction and the velocity component in the x direction has been described. Similarly, in the y direction, A magnitude component and a velocity component can be specified. Therefore, in the following description, the operation of each unit with respect to an object moving in the x direction will be described so as not to complicate the description.

  FIG. 5 shows an example of an area where the object image composition unit 280 composes an object image. The moving object image generation unit 270 generates an object image having the length L of the object specified by the moving object size specifying unit 275. For example, the moving object image generation unit 270 may cut out an area having a length L from the object area in the captured image P301 or P302 and generate the cut out area as an object image. Then, the object image composition unit 280 arranges the object image generated by the moving object image generation unit 270 in the area 460 having a length L in the partial exposure image p1, and outputs the object image to the display unit 140.

  The object image composition unit 280 may set a region 460 having a length L at the center of gravity of the region where the object is expected to be located in the partial exposure image p1. Specifically, the object image composition unit 280 may set an area 460 of length L having a center of gravity at the midpoint between x0 and x4.

  FIG. 6 shows another example of a region where the object image composition unit 280 composes an object image. The object image composition unit 280 may set an area 660 where an object is expected to exist in the partial exposure image p1. Specifically, the object image composition unit 280 arranges the object image in an area from x0 to x4 in the x coordinate of the partial exposure image p1. Note that the moving object image generation unit 270 may extract an area having a length from x0 to x4 from the captured image P301 and generate the extracted area as an object image.

  FIG. 7 shows another example of the light receiving period. In the light receiving period of this figure, the imaging unit 110 captures the captured image P300 in the light receiving period from time t1 to time t2, and the imaging unit 110 captures the captured image P304 in the light receiving period from time t5 to time t6. Except for this, it is the same as the light receiving period described in FIG.

  According to the light receiving period as shown in the figure, p1 = P300 and p5 = P304. Therefore, the partial exposure image generation unit 230 can calculate the partial exposure images p1, p2, p3, p4, p5, and p5 by solving the equations relating to P302, P302, and P303 described in FIG.

  FIG. 8 shows an example of a block configuration of the imaging unit 110. The imaging unit 110 includes a plurality of mirror units 850a-c (hereinafter collectively referred to as mirror units 850), a plurality of light receiving units 810a-d (hereinafter collectively referred to as light receiving units 810), a light reception control unit 830, and a captured image. A generation unit 840 and a communication path 820 are included. Each of the plurality of light receiving units 810 includes a plurality of light receiving elements. Each of the plurality of light receiving elements may be a CCD type element or a CMOS type element.

  The mirror unit 850 divides the light from the subject 180 into a plurality of lights. The plurality of light receiving units 810 receive the plurality of lights divided by the mirror unit 850, respectively. Specifically, the mirror unit 850a divides the light from the subject 180 into two. The light receiving unit 810a receives one of the lights divided by the mirror unit 850a. Then, the mirror unit 850b divides the other light divided by the mirror unit 850a into two. The light receiving unit 810b receives one of the lights divided by the mirror unit 850b. The mirror unit 850c divides the other light divided by the mirror unit 850b into two. The light receiving unit 810c receives one of the lights divided by the mirror unit 850c. The light receiving unit 810d receives the other light divided by the mirror unit 850c.

  The mirror unit 850 may be a beam splitter. The mirror unit 850 may be a half mirror. Further, the mirror unit 850 may have a division ratio that can provide each of the light receiving units 810 with substantially the same amount of light per unit time. Further, the division ratio of the mirror unit 850 and the light receiving sensitivity of the light receiving unit 810 may be adjusted so that the overall sensitivity including the division ratio of the mirror unit 850 and the light receiving sensitivity of the light receiving unit 810 is substantially the same.

  The light reception control unit 830 causes each of the plurality of light reception units 810 to receive light by overlapping the light reception periods of the other light reception units 810 with a part of the period. For example, as described with reference to FIGS. 2 and 7, the light reception control unit 830 shifts the light reception start time and causes each of the plurality of light reception units 810 to receive light. Then, the captured image generation unit 840 generates a plurality of captured images based on the amount of light received by each of the plurality of light receiving units 810. Note that the light reception control unit 830 and the captured image generation unit 840 transmit and receive signals to and from the light reception unit 810 through a communication path 820 such as a bus.

  FIG. 9 shows another example of the block configuration of the imaging unit 110. The imaging unit 110 includes a plurality of light receiving units 910a-d (hereinafter collectively referred to as a light receiving unit 910), a light reception control unit 930, a captured image generation unit 940, an optical low-pass filter 950, and a communication path 920. Each of the plurality of light receiving units 910 includes a plurality of light receiving elements. Each of the light receiving units 910 may be a light receiving element array in which a plurality of light receiving elements are arranged in a line, for example.

  The optical low-pass filter 950 disperses the light from the subject 180 and receives the light over the plurality of light receiving units 910 in a direction substantially orthogonal to the direction in which the light receiving elements are arranged in the light receiving unit 910. Then, the light reception control unit 930 receives the light dispersed by the optical low-pass filter so that each of the plurality of light reception units 910 overlaps the light reception period of the other light reception units 910 and a part of the period. For example, as described with reference to FIGS. 2 and 7, the light reception control unit 930 shifts the light reception start time and causes each of the light reception units 910 to receive light. Then, the captured image generation unit 940 generates captured images for each period based on the received light amounts of a plurality of light receiving elements included in the light receiving unit 910 that has received light from the subject 180 during the same period. Generate. Note that the light reception control unit 930 and the captured image generation unit 940 transmit and receive signals to and from the light reception unit 810 through a communication path 820 such as a bus.

  Each of the plurality of light receiving elements may be a CCD type element or a CMOS type element. When the light receiving element is a CCD type element, the captured image generation unit 940 may read the amount of light received by the plurality of light receiving elements included for each light receiving unit 910, that is, for each row of light receiving elements.

  Thus, the optical low-pass filter 950 disperses the light from the subject 180 and causes the plurality of light receiving elements to receive the light. Then, the light reception control unit 930 receives the light dispersed by the optical low-pass filter 950 so that each of the plurality of light receiving elements overlaps the light receiving period of the other light receiving elements and a part of the period. Then, the captured image generation unit 940 generates a captured image for each period based on the amounts of light received by the plurality of light receiving elements that have received light from the subject 180 during the same period.

  In the above description, the case where the captured image is generated by receiving light over the continuous light reception time has been described as an example. In addition, the imaging unit 110 may generate a captured image based on the amount of light received by intermittently receiving light from the subject 180. In addition, the imaging unit 110 may generate a single captured image by adding a plurality of images obtained from each of a plurality of different light receiving periods.

  FIG. 10 shows an example of the hardware configuration of the imaging system 100. The imaging system 100 includes a CPU peripheral unit having a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 connected to each other by a host controller 1582, and a communication interface connected to the host controller 1582 by an input / output controller 1584. 1530, an input / output unit including a hard disk drive 1540 and a CD-ROM drive 1560, and a legacy input / output unit including a ROM 1510 connected to an input / output controller 1584, a flexible disk drive 1550, and an input / output chip 1570.

  The host controller 1582 connects the RAM 1520, the CPU 1505 that accesses the RAM 1520 at a high transfer rate, and the graphic controller 1575. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 and controls each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 and the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the hard disk drive 1540, the communication interface 1530, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The hard disk drive 1540 stores programs and data used by the CPU 1505. The communication interface 1530 is connected to the network communication device 1598 to transmit / receive programs or data. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program executed when the imaging system 100 is started, a program depending on the hardware of the imaging system 100, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520. The input / output chip 1570 connects various input / output devices via a flexible disk drive 1550 and, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

  A program executed by the CPU 1505 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The program stored in the recording medium may be compressed or uncompressed. The program is installed in the hard disk drive 1540 from the recording medium, read into the RAM 1520, and executed by the CPU 1505.

  The program executed by the CPU 1505 causes the imaging system 100 to function as the imaging unit 110, the image processing unit 120, the imaging control unit 130, and the display unit 140 described with reference to FIGS.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and the program may be provided to the imaging unit 110 via the network.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

1 is a diagram illustrating an example of a configuration of an imaging system 100. FIG. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 120. FIG. It is a figure which shows an example of a light reception period. It is a figure which shows an example of the moving object in a captured image. It is a figure which shows an example of the area | region which synthesize | combines an object image. It is a figure which shows another example of the area | region which synthesize | combines an object image. It is a figure which shows another example of a light reception period. 2 is a diagram illustrating an example of a block configuration of an imaging unit 110. FIG. It is a figure which shows another example of the block configuration of the imaging part. 2 is a diagram illustrating an example of a hardware configuration of an imaging system 100. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging system 110 Imaging device 120 Image processing part 130 Imaging control part 140 Display part 150 Communication line 180 Subject 220 Difference image generation part 230 Partial exposure image generation part 240 Partial exposure image storage part 250 Partial exposure image acquisition part 260 Motion area specification part 265 Motion calculating unit 270 Moving object image generating unit 275 Moving object size specifying unit 280 Object image synthesizing unit 810 Light receiving unit 820 Communication path 830 Light receiving control unit 840 Captured image generating unit 850 Mirror unit 920 Communication path 910 Light receiving unit 930 Light receiving control unit 940 Captured image generation unit 950 Optical low-pass filter

Claims (10)

In each of a plurality of light receiving periods in which some periods overlap with other light receiving periods, an imaging unit that receives light from substantially the same subject and captures a plurality of captured images of the subject;
Based on the image content of the plurality of captured images and a period in which the plurality of light receiving periods overlap, light from the subject is received during a period of time shorter than the plurality of light receiving periods included in the plurality of light receiving periods. An image pickup apparatus comprising: a partial exposure image generation unit configured to generate a partial exposure image to be obtained in the case of. A difference image generation unit that generates a difference image between the first captured image and the second captured image captured in the first light receiving period and the second light receiving period in which the imaging unit overlaps a part of the period;
A partial exposure image acquisition unit that acquires a partial exposure image to be obtained when light is received in a period other than the period that overlaps the second light reception period in the first light reception period;
The partial exposure image generation unit overlaps the first light reception period in the second light reception period based on the difference image generated by the difference image generation unit and the partial exposure image acquired by the partial exposure image acquisition unit. The imaging apparatus according to claim 1, further comprising: generating a partial exposure image to be obtained when light from the subject is received in a period other than the period to be performed. The partial exposure image generation unit includes image contents of a first captured image and a second captured image captured by the imaging unit in the first light receiving period and the second light receiving period in which some periods overlap, and the first Based on a period in which one light-receiving period and the second light-receiving period overlap, other than the overlapping period, a period other than the overlapping period in the first light-receiving period, and a period other than the overlapping period in the second light-receiving period Generating a partially exposed image in at least one of the periods;
The imaging device
A motion calculation unit that calculates the speed at which the object moves based on the respective image contents of the first captured image and the second captured image;
A motion object image generation unit that generates an image of the object moving at a speed calculated by the motion calculation unit;
The imaging apparatus according to claim 1, further comprising an object image synthesis unit that synthesizes an image generated by the moving object image generation unit with a partial exposure image generated by the partial exposure image generation unit. A motion region specifying unit that specifies a region where a moving object exists in each of the first captured image and the second captured image, based on the image contents of the first captured image and the second captured image;
A difference image generating unit that generates a difference image of an overlapping region among the regions specified by the motion region specifying unit in the first captured image and the second captured image;
A moving object size specifying unit for specifying the size of the moving object based on the image content of the difference image generated by the difference image generating unit;
The imaging apparatus according to claim 3, wherein the moving object image generation unit generates an object image in which an object having a size specified by the moving object size specifying unit moves at a speed calculated by the movement calculating unit. The partial exposure image generation unit includes image contents of a first captured image and a second captured image captured by the imaging unit in the first light receiving period and the second light receiving period in which some periods overlap, and the first Based on a period in which one light-receiving period and the second light-receiving period overlap, other than the overlapping period, a period other than the overlapping period in the first light-receiving period, and a period other than the overlapping period in the second light-receiving period Generating a partially exposed image in at least one of the periods;
The imaging device
A motion region specifying unit for specifying a region where a moving object exists in each of the first captured image and the second captured image based on the image contents of the first captured image and the second captured image;
A difference image generating unit that generates a difference image of an overlapping region among the regions specified by the motion region specifying unit in the first captured image and the second captured image;
A moving object size specifying unit for specifying the size of the moving object based on the image content of the difference image generated by the difference image generating unit;
A moving object image generating unit that generates an image of the moving object having a size specified by the moving object size specifying unit;
The imaging apparatus according to claim 1, further comprising: an object image synthesis unit that synthesizes an image generated by the moving object image generation unit with a partial exposure image generated by the partial exposure image generation unit. The imaging unit
A mirror section that divides the light from the subject into a plurality of lights;
A plurality of light receiving portions that respectively receive a plurality of lights divided by the mirror portion;
A light receiving control unit that receives light by overlapping a part of the light receiving period of the other light receiving unit with each of the plurality of light receiving units, and
The imaging apparatus according to claim 1, further comprising: a captured image generation unit configured to generate the plurality of captured images based on the amount of light received by the plurality of light receiving units. The imaging unit
A light receiving section having a plurality of light receiving elements;
An optical low-pass filter that disperses light from a subject and receives the light by the plurality of light receiving elements;
A light receiving control unit configured to receive the light dispersed by the optical low-pass filter by overlapping each of the plurality of light receiving elements with a light receiving period of another light receiving element and a part of the period;
2. The imaging according to claim 1, further comprising: a captured image generation unit configured to generate a captured image for each period based on light reception amounts of a plurality of light receiving elements that receive light from the subject during the same period. apparatus. The plurality of light receiving elements form a plurality of light receiving element arrays in which a plurality of light receiving elements are respectively arranged,
The optical low-pass filter disperses light from a subject and receives light over the plurality of light receiving element rows in a direction substantially orthogonal to a direction in which the light receiving elements are arranged in the light receiving element row,
The light receiving control unit receives light dispersed by the optical low-pass filter by overlapping each of the plurality of light receiving element arrays with a light receiving period of a part of the other light receiving element arrays,
The captured image generation unit generates a captured image for each period based on the amount of light received by a plurality of light receiving elements included in a light receiving element array in which the light reception control unit receives light from a subject during the same period. Item 8. The imaging device according to Item 7. In each of a plurality of light receiving periods in which some periods overlap with other light receiving periods, an imaging unit that receives light from substantially the same subject and captures a plurality of captured images of the subject;
Based on the image content of the plurality of captured images and the period in which the plurality of light receiving periods overlap, light from the subject is received during a period of time shorter than the plurality of light receiving periods included in the plurality of light receiving periods. An imaging method comprising: an image generation unit that generates a partial exposure image to be obtained. A program for an imaging apparatus, wherein the imaging apparatus is
An imaging unit that receives light from substantially the same subject and captures a plurality of captured images of the subject in each of a plurality of light receiving periods in which some periods overlap with other light receiving periods;
Based on the image content of the plurality of captured images and the period in which the plurality of light receiving periods overlap, light from the subject is received during a period of time shorter than the plurality of light receiving periods included in the plurality of light receiving periods. A program that functions as an image generation unit that generates a partial exposure image to be obtained.

Images