JP6284604B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program capable of generating a bulb-captured image by comparatively bright synthesis and having an arbitrarily adjustable exposure time in an image processing apparatus such as a digital still camera. About.

  Conventionally, in a single-lens reflex imaging apparatus, observation of a subject image has been performed using an optical viewfinder. However, recently, an imaging device that observes a subject image by a live view display in which an optical viewfinder is eliminated and an image read from an image sensor is displayed on a rear liquid crystal monitor or the like is commercially available. Also, instead of the optical viewfinder, the live view display image is displayed using the electronic viewfinder separately from the rear LCD monitor, etc., and the display using the rear LCD monitor and the electronic viewfinder can be switched. Imaging devices are also commercially available.

  Conventionally, regardless of whether it is an optical viewfinder or an electronic viewfinder, the image data cannot be read from the image sensor during exposure for long periods such as bulb photography. I couldn't do it, and I checked the image after shooting. For this reason, for the photographer, the exposure setting such as the aperture and ISO sensitivity and the timing for starting and ending the exposure must be determined by the photographer himself / herself based on the brightness of the subject, which is due to underexposure or excessive exposure. It was not easy to obtain a desired shot image without shooting failure.

  Therefore, in order to solve such a problem, the imaging apparatus disclosed in Patent Document 1 reads image data from the image sensor at predetermined time intervals, and simply reads the image data each time the image data is read from the image sensor. Cumulative addition is performed, and an image obtained thereby is displayed on a liquid crystal monitor or the like, so that the progress of exposure can be displayed.

  Further, the imaging device disclosed in Patent Document 2 continuously reads out image data from an image sensor and generates bulb photographed image data by comparatively bright combination. This comparatively bright combination is an image combination in which the luminance level of each pixel of the image data is compared, the larger one is selected, and the combined luminance level is obtained.

JP 2005-117395 A Japanese Patent No. 4148586

  In the imaging apparatus disclosed in Patent Document 1, the current exposure information can be notified to the photographer as progress information during bulb photography, and failures such as underexposure and overexposure can be reduced. However, as a subject of bulb photography, when photographing light trails of fireworks launched in the night sky, which are common, and star light trails, the longer the exposure time, the longer the light trail length and the background ( In these cases, the brightness of the night sky) may change. In such a case, it is necessary to finish the shooting with an exposure time that does not cause the background brightness to be overexposed or underexposed, and the length of the light trail is not related to the photographer's intention. It will be decided by that.

  On the other hand, in the imaging device disclosed in Patent Literature 2, image data continuously read from the imaging device is compared with the luminance level for each pixel of the image data, and the brighter pixel is selected. Captured image data is generated by comparatively bright combination with a luminance level. For this reason, the brightness of the background is maintained at the brightness of the single image before composition regardless of the exposure time, and the exposure time is set to match the intention of the photographer to capture the light trace. Can do.

  However, in any of the imaging devices disclosed in Patent Documents 1 and 2, in order for the photographer to capture the brightness of the background and the length and brightness of the light trail as intended, according to the brightness of the subject, It is necessary to adjust the aperture, ISO sensitivity, and exposure time for capturing a single image before synthesis. This adjustment requires years of shooting experience and brightness confirmation by preview, and requires advanced shooting techniques.

  Also, since stars and fireworks are shot in a dark environment, if you check the image in a bright environment such as daytime or a room after shooting, the shot image will appear to have a different brightness from the time of shooting. I often shoot with inadequate brightness.

  The present invention has been made in view of such circumstances, an image processing apparatus and an image processing device that prevent exposure failure of bulb photography, which is difficult, and enable photography of a photograph according to the photographer's intention. It is an object to provide a method and a program.

In order to achieve the above object, an image processing apparatus according to a first aspect of the present invention includes a storage unit that stores a plurality of image data that are sequentially input, and each of the two frames of image data that are read from the storage unit. A comparatively bright combination processing unit that compares the pixel data corresponding to each of a plurality of pixels and uses the larger pixel data as new pixel data to perform comparatively bright combination processing that forms comparatively bright combined image data, and the storage Comparing the corresponding pixel data of each of the plurality of pixels constituting the two-frame image data read from the image unit, and constructing comparative dark composite image data using the smaller pixel data as new pixel data A comparative dark composition processing unit for performing comparative dark composition processing, and each of the plurality of pixels constituting the comparative bright composite image data and the comparative dark composite image data A difference image synthesizing unit for generating a new shelf difference image data by calculating the difference in pixel data response, and a dark image data multiplied by multiplying the arbitrary gain value to the relatively dark composite image data, to the difference image data An image synthesis unit that synthesizes multiplication difference image data multiplied by an arbitrary gain value and generates a synthesized image .

In the image processing apparatus according to a second aspect based on the first aspect , the comparatively bright combination processing unit accumulates first image data generated first based on the image data read from the storage unit. A plurality of pixel data constituting the second image data generated based on the image data read out from the storage unit and stored as the comparatively bright composite image data and the cumulative comparative bright composite image data The pixel data corresponding to each of the plurality of pixel data is compared, and the larger pixel data is used as new pixel data, and the comparatively bright composite processing for forming the cumulative comparative bright composite image data is performed. synthesis processing unit stores in the storage unit the first image data that has been initially generated based on image data read out from the storage unit as a cumulative relatively dark composite image data, or the storage unit A plurality of pixel data composing the second image data generated based on the read image data is compared with corresponding pixel data of the plurality of pixel data composing the cumulative comparative dark composite image data. The smaller pixel data is used as new pixel data, and a comparative dark composition process is performed to form the cumulative comparative dark composite image data. The difference image composition unit is configured to perform the cumulative comparative light composite image data and the cumulative comparative dark composition. The difference between the pixel data corresponding to each of the image data is calculated to generate new difference image data, and the image composition unit includes cumulative multiplication dark image data obtained by multiplying the cumulative comparison dark composite image data by an arbitrary gain value, Then, the difference image data obtained by multiplying the difference image data by an arbitrary gain value is synthesized to generate a synthesized image.

According to a third aspect of the present invention, there is provided an image processing apparatus for storing a plurality of sequentially input image data and a plurality of pixels constituting the two frames of image data read from the storage unit. The difference with the corresponding pixel data is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the pixel data corresponding to each of the plurality of pixels constituting the image data of the two frames is compared. The larger pixel data is set as new pixel data. On the other hand, when the difference is smaller than the predetermined value, the average value of the plurality of pixel data constituting the image data of the two frames is set as new pixel data. The comparatively bright composite processing unit that performs comparatively bright composite processing and generates comparatively bright composite image data, and each of the plurality of pixels constituting the two frames of image data read from the storage unit The difference between the pixel data corresponding to each is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the pixel data corresponding to each of the plurality of pixels constituting the image data of the two frames is calculated. If the difference is smaller than the predetermined value, the smaller one of the pixel data is set as new pixel data. On the other hand, when the difference is smaller than the predetermined value, the average value of the plurality of pixel data respectively constituting the two frames of image data As the data, a comparative dark composition processing unit that performs image composition processing and generates comparative dark composite image data, and pixel data corresponding to each of a plurality of pixels constituting the comparative bright composite image data and the comparative dark composite image data a difference image combining unit that calculates the difference to generate a new difference image data, and the dark image data multiplied by multiplying the arbitrary gain value to the relatively dark composite image data, the difference image de It synthesizes the multiplication difference image data obtained by multiplying the arbitrary gain value data, comprising an image combining unit for generating a composite image.
The image processing apparatus according to the fourth invention, the cumulative in the third aspect of the invention, the lighten compositing processing unit, the first image data that has been initially generated based on image data read out from the storage unit A plurality of pixel data constituting the second image data generated based on the image data read out from the storage unit and stored as the comparatively bright composite image data and the cumulative comparative bright composite image data The difference between the pixel data corresponding to each of the plurality of pixel data is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the pixel corresponding to each of the plurality of pixel data constituting each of the two frames of image data When the data is compared and the larger pixel data is set as new pixel data, and the difference is smaller than the predetermined value, a plurality of images constituting each of the two frames of image data are obtained. A comparatively bright combination process for generating the cumulative comparatively bright combined image data is performed using the average value of the data as new pixel data, and the comparative dark combining process unit is first configured based on the image data read from the storage unit A plurality of pixel data constituting the second image data generated based on the image data read out from the storage unit. And the difference between the pixel data corresponding to each of the plurality of pixel data constituting the cumulative comparison dark composite image data is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the image data of the two frames is formed. When the pixel data corresponding to each of the plurality of pixels is compared and the smaller pixel data is set as new pixel data, and the difference is smaller than the predetermined value, Using the average value of a plurality of pieces of pixel data constituting each of the two frames of image data as new pixel data, a comparative dark composition process for generating the cumulative comparative dark composite image data is performed, and the difference image composition unit is configured to perform the cumulative comparison. A difference between pixel data corresponding to each of the bright composite image data and the cumulative comparative dark composite image data is calculated to generate new difference image data, and the image composition unit adds an arbitrary gain to the cumulative comparative dark composite image data. The cumulative multiplied dark image data multiplied by the value and the multiplied difference image data obtained by multiplying the difference image data by an arbitrary gain value are combined to generate a combined image.

An image processing method according to a fifth aspect of the present invention is an image processing method in an image processing apparatus having a storage unit for storing a plurality of sequentially input image data, wherein the two frames of image data read from the storage unit are configured. The corresponding pixel data of each of the plurality of pixels is compared, and the larger pixel data is used as new pixel data to perform comparatively bright combination processing that forms the comparatively bright combined image data. Comparing the corresponding pixel data of each of the plurality of pixels constituting the two-frame image data thus output, and using the smaller pixel data as new pixel data, the comparative dark composite image data constituting the comparative dark composite image data Pixels corresponding to each of the plurality of pixels constituting the comparatively bright composite image data and the comparative dark composite image data by performing the synthesis process By calculating the difference over data to generate a new shelf difference image data, and the dark image data multiplied by multiplying the arbitrary gain value to the relatively dark composite image data, multiplied by the arbitrary gain value to the difference image data The multiplication difference image data is synthesized to generate a synthesized image .
An image processing method according to a sixth aspect of the present invention is an image processing method in an image processing apparatus having a storage unit for storing a plurality of sequentially input image data, wherein two frames of image data read from the storage unit are configured. The difference between the pixel data corresponding to each of the plurality of pixels to be compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, each of the plurality of pixels constituting the two-frame image data If the difference is smaller than the predetermined value, the plurality of pixels constituting the image data of the two frames are compared. Performs comparatively bright composite processing using the average value of the data as new pixel data, generates comparatively bright composite image data, and constructs two frames of image data read from the storage unit The difference between the pixel data corresponding to each of the plurality of pixels is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the difference between each of the plurality of pixels constituting the image data of the two frames is determined. Comparing the corresponding pixel data, the smaller pixel data is set as new pixel data. On the other hand, when the difference is smaller than the predetermined value, the average of the plurality of pixel data respectively constituting the two frames of image data Using the value as new pixel data, image synthesis processing is performed to generate comparative dark composite image data, and the difference between pixel data corresponding to each of the plurality of pixels constituting the comparative bright composite image data and the comparative dark composite image data the calculates and generates a new difference image data, and the dark image data multiplied by multiplying the arbitrary gain value to the relatively dark composite image data, optionally in the difference image data Synthesizes the multiplication difference image data obtained by multiplying the in-value, to generate a composite image.

According to a seventh aspect of the invention, there is provided a program for executing a computer of an image processing apparatus having a storage unit for storing a plurality of sequentially input image data, wherein the two frames of image data read from the storage unit The corresponding pixel data of each of the plurality of pixels constituting the image data are compared, and the larger pixel data is used as new pixel data, and the comparatively bright synthesized image data constituting the relatively bright synthesized image data is performed. The pixel data corresponding to each of the plurality of pixels constituting the two-frame image data read from the image data are compared, and the smaller pixel data is used as new pixel data to form comparative dark composite image data. A comparative dark composition process is performed, and each of the plurality of pixels constituting the comparative bright composite image data and the comparative dark composite image data is processed. Each generates by calculating the difference between the corresponding pixel data new shelf differential image data, and the dark image data multiplied by multiplying the arbitrary gain value to the relatively dark composite image data, any gain in the difference image data The multiplication difference image data multiplied by the value is synthesized to generate a synthesized image .
According to an eighth aspect of the invention, there is provided a program for executing a computer of an image processing apparatus having a storage unit for storing a plurality of sequentially input image data, wherein two frames of image data read from the storage unit The difference between the pixel data corresponding to each of the plurality of pixels constituting the image is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the plurality of pixels constituting the image data of the two frames If the difference is smaller than the predetermined value, the plurality of pixels constituting the image data of the two frames are compared. The comparatively bright composite processing is performed by using the average value of the pixel data as new pixel data to generate comparatively bright composite image data, and the two frames read from the storage unit The difference between the pixel data corresponding to each of the plurality of pixels constituting the image data is compared with a predetermined value. When the difference is equal to or greater than the predetermined value, the plurality of the plurality of pixels constituting the image data of the two frames are compared. Pixel data corresponding to each of the pixels is compared and the smaller pixel data is set as new pixel data. On the other hand, when the difference is smaller than the predetermined value, a plurality of pieces of image data of two frames are formed. Using the average value of the pixel data as new pixel data, image synthesis processing is performed, comparative dark composite image data is generated, and each of the plurality of pixels constituting the comparative bright composite image data and the comparative dark composite image data is handled. by calculating the difference between the pixel data to generate new difference image data, and the dark image data multiplied by multiplying the arbitrary gain value to the relatively dark composite image data, the difference It synthesizes the multiplication difference image data obtained by multiplying the arbitrary gain value to the image data to generate a composite image.

  According to the present invention, it is possible to provide an image processing apparatus, an image processing method, and a program that prevent exposure of bulb photography with a high degree of difficulty and that can take a photograph according to the photographer's intention.

It is a block diagram which shows mainly an electrical structure of the camera which concerns on 1st Embodiment of this invention. 5 is a flowchart showing an operation when a comparatively bright combination mode is selected in a valve mode in the camera according to the first embodiment of the present invention. 5 is a flowchart showing an operation when a comparatively bright combination mode is selected in a valve mode in the camera according to the first embodiment of the present invention. It is a figure which shows the operation screen at the time of exposure adjustment in the camera which concerns on 1st Embodiment of this invention. It is a figure which shows comparative bright and dark composition in the camera which concerns on 1st Embodiment of this invention. It is a figure explaining exposure adjustment in the camera concerning a 1st embodiment of the present invention. It is a flowchart which shows operation | movement when the valve mode is selected in the camera which concerns on 2nd Embodiment of this invention.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. A camera according to a preferred embodiment of the present invention is a digital camera, and roughly displays image data based on image data read from an image sensor on a display unit and performs image processing for recording according to an operation of a release button. The recorded image data is recorded in the external memory. Also, when the comparatively bright combination mode is set as the bulb shooting mode, the comparatively bright combined image data is generated by the comparatively bright combination using the brighter pixel for each pixel of the image data during the bulb shooting, and this image data is used. Thus, the progress of the exposure state can be displayed during bulb photography.

  Further, the camera generates comparative dark composite image data by comparative dark combining using a darker pixel for each pixel of the image data, and outputs difference output image data from the comparative bright composite image data and the comparative dark composite image data. Is generated. The generated comparative dark composite image data is used as background output image data, and the difference output image data is used as light trace output image data. For the bright composite image data, the brightness of the background and the brightness of the light trail can be adjusted independently.

  FIG. 1 is a block diagram mainly showing an electrical configuration of a camera according to the first embodiment as a preferred embodiment of the present invention. The camera in the present embodiment includes an imaging unit 1, an image processing unit 10, a system control unit 20, and a bus 31 and each unit connected thereto.

  The lens 2 forms an optical image of the subject on the image sensor 4. This lens 2 is provided with an aperture for determining an aperture value for adjusting the exposure amount. The mechanical shutter 3 controls the shutter speed by performing exposure and shading to the image sensor 4 by opening and closing operations. The image sensor 4 includes an image sensor such as a CMOS image sensor or a CCD image sensor, converts an optical image of a subject formed by the lens 2 into an electrical signal for each pixel, and converts the image data into the image processing unit 10 and the bus. To 31. The bus 31 is a signal line for transmitting and receiving signals between the blocks.

  The image processing unit 10 performs image processing on the image data output from the image sensor 4, and performs an image comparison / synthesis unit 11, an image average synthesis unit 12, a difference image synthesis unit 13, an image addition synthesis unit 14, an image synthesis unit 15, And a development processing unit 16.

  The image comparison / synthesis unit 11 compares the image data continuously read from the image sensor 4 with the output of each pixel of the image data stored in the internal memory 33, and performs comparative bright processing and comparative dark synthesis for each pixel. Processing is performed, and composite image data is generated using the comparatively bright composite image data and the comparative dark composite image data generated by these processes.

  That is, in the comparatively bright process in the image comparison / synthesis unit 11, the pixel data constituting the image data generated based on the image data first read from the image sensor 4 is stored in the internal memory 33 as the comparatively bright composite image data. To do. Next, the pixel data constituting the image data generated based on the image data read from the image sensor 4 and the plurality of pixel data constituting the cumulative comparative bright composite image data stored in the internal memory 33 Compare the corresponding pixel data. When the difference between the corresponding pixel data is equal to or greater than a predetermined value, the larger pixel data, that is, the brighter pixel data is detected, and this brighter pixel data is used for cumulative comparison. Reconstruct bright composite image data. On the other hand, the image comparison / synthesis unit 11 replaces the corresponding pixel data with the average value calculated by the image average synthesis unit 12 when the difference is smaller than a predetermined value as a result of comparing the corresponding pixel data. This process is repeated every time image data is read from the image sensor 4. By performing the comparative light processing, an image of a light trail of a star in the night sky can be obtained (see FIG. 5B described later).

  Further, in the comparative dark process in the image comparison / synthesis unit 11, the pixel data constituting the image data generated based on the image data first read from the image sensor 4 is stored in the internal memory 33 as the comparative dark composite image data. To do. Next, the pixel data constituting the image data generated based on the image data read from the image sensor 4 and the plurality of pixel data constituting the cumulative comparative dark composite image data stored in the internal memory 33 are as follows. Each corresponding pixel data is compared. If the difference between the corresponding pixel data is greater than or equal to a predetermined value, the smaller one, that is, the darker pixel data is detected, and the darker pixel data is used to perform cumulative comparison. Reconstruct dark composite image data. On the other hand, the image comparison / synthesis unit 11 replaces the corresponding pixel data with the average value calculated by the image average synthesis unit 12 when the difference is smaller than a predetermined value as a result of comparing the corresponding pixel data. This process is repeated every time image data is read from the image sensor 4. By performing comparative dark processing, it is possible to erase the light trails of the stars in the night sky and obtain a background image (see FIG. 5C described later).

  The image average composition unit 12 compares the corresponding pixel data with respect to a plurality of pixel data constituting the cumulative comparative bright composite image data or the cumulative comparative dark composite image data, and when the difference is smaller than a predetermined value, The average value of the corresponding pixel data is calculated.

  The difference image composition unit 13 calculates a difference output for each pixel of the comparatively bright composite image data and the comparative dark composite image data, and generates difference image data. That is, the difference image data is image data obtained by calculating a difference for each pixel data from the comparative dark composite image data from the comparative bright composite image data, and the background image data is lost and the image data is only the light trace. (See FIG. 5D described later).

  The image addition / synthesis unit 14 adds the outputs for each pixel to generate addition synthesis data. That is, every time image data is read from the image sensor 4 at a predetermined cycle, the image addition / synthesis unit 14 adds the image data to generate addition / synthesis data. By displaying on the basis of the added synthesized data, it is possible to display the shooting progress during bulb shooting. In addition, in the flowchart (refer FIG.2, 3) mentioned later, the imaging | photography progress display based on this addition synthetic data is abbreviate | omitted.

  The image composition unit 15 multiplies the comparative dark composite image data and the difference image data by a gain, and performs an addition process. The comparative dark composite image data generated by the image comparison / combination unit 11 is a bright portion where only the background moves (see FIG. 5C). This is image data consisting of light traces (see FIG. 5D). By multiplying each of these image data by a gain and performing addition processing, the brightness of the light trail portion and the background portion can be adjusted independently. This adjustment will be described later with reference to FIGS. 4 to 6 and S43 to S55 of FIG.

  The development processing unit 16 performs development processing such as demosaicing, white balance adjustment, gamma correction, and image compression on the RAW image data generated by the image composition unit 15.

  The internal memory 33 temporarily stores various setting information necessary for camera operation and image data in the middle of image processing. The internal memory 33 is configured by a nonvolatile memory such as a flash memory or a DRAM.

  The external memory 36 is a non-volatile storage medium that can be loaded into the camera body or fixed inside, and is, for example, an SD card or a CF card. The external memory 36 records the image data developed by the development processing unit 16, and at the time of reproduction, the recorded image data can be read out and output to the outside of the camera.

  The display unit 37 includes a rear display unit such as a TFT (Thin Film Transistor) liquid crystal or an organic EL, or an EVF (Electronic Viewfinder), and displays an image developed by the development processing unit 16.

  The input IF 38 includes an operation member such as a release button, a touch panel for inputting a touch operation on the rear display unit, and the like, and instructs various shooting operations such as mode setting and release based on a user operation.

  The exposure determination unit 34 determines whether the exposure is proper from the output of the image data read from the image sensor 4 or the image data generated by each combining unit. The comparison unit 35 compares output differences for each pixel of a plurality of image data to be compared and synthesized. It is also possible to compare the output difference for each pixel with a predetermined value (threshold level).

  The system control unit 20 includes a CPU (Central Processing Unit), and performs overall control of the camera according to a program stored in the internal memory 33. The system control unit 20 includes an exposure time calculation unit 21 and an exposure time control unit 22. The exposure time calculation unit 21 calculates an optimal exposure time (shutter speed) from the exposure determination result in the exposure determination unit 34. The exposure time control unit 22 controls the opening / closing timing processing of the mechanical shutter 3 according to the calculation result in the exposure time calculation unit 21. The exposure time calculation unit 21 and the exposure time control unit 22 may be processed by software or may be processed by hardware.

  Next, an outline of the operation of the camera shown in FIG. 1 will be described. Based on the control by the system control unit 20, the focus lens constituting the lens 2 is adjusted, the diaphragm is set to a predetermined value, the mechanical shutter 3 is opened, and the optical image is converted into an electrical signal by the image sensor 4. . Image data read from the image sensor 4 is subjected to predetermined image processing in the image processing unit 10 and recorded in the external memory 36. The image data that has undergone predetermined image processing by the image processing unit 10 is resized and then displayed on the display unit 37.

  The system control unit 20 receives an instruction from the user via the input IF 38, and performs timing control such as exposure start of the image sensor 4, signal readout, etc., opening / closing timing control of the mechanical shutter 3, lens aperture control and autofocus control. . In addition, the system control unit 20 receives image data from the image processing unit 10 and performs control such as image display by the display unit 37 and storage of image data in the external memory 36.

  Next, the processing flow of the camera according to the present embodiment will be described using the flowcharts shown in FIGS. This flowchart is executed by the system control unit 20 controlling each unit in accordance with a program stored in the internal memory 33.

  In this flowchart, a sequence in the case where the user selects the bulb photographing mode by the input IF and selects the comparatively bright combination mode in the bulb photographing mode is illustrated. Even in the bulb photographing mode, the normal bulb photographing in which the progress display during exposure is not performed, and the addition composite display mode in which the progress display during exposure is performed with an added image are omitted. In addition, when the addition composite display mode which is a kind of the progress display mode is selected, the image data is read from the image sensor 4 at a predetermined cycle during bulb photographing, and the read image data is sequentially added. It can be displayed on the display unit 37 that the screen gradually becomes brighter as the exposure time elapses. In the comparative bright combination mode, which is a kind of progress display mode, the brightness of the background is constant regardless of the exposure time, and the length of the light trace can be adjusted according to the exposure time.

  The user adjusts the direction of the camera and the focal length (zoom) of the lens so that the subject to be photographed can be photographed by checking the display unit 37 and the optical viewfinder (not shown) in advance when performing bulb photographing. If necessary, photographing information such as a focus lens position, an aperture value, and ISO sensitivity may be set via an input IF 38 such as a button or a touch panel.

  In the flow shown in FIG. 2, the system control unit 20 determines whether or not the 1st release switch is turned on in response to the first pressing operation of the release switch (S1). If the result of this determination is not on, a standby state is entered.

  If the result of determination in step S1 is that the 1st release switch is on, AF and AE are next carried out (S3). Here, the system hand control unit 20 performs AF (autofocus) to focus on the subject. In AF, the system control unit 20 drives the focus lens in the lens 2 so that the contrast value of the image data repeatedly read from the image sensor 4 becomes the maximum value. In addition, AE (automatic exposure control) is performed to calculate an exposure time, an aperture, and the like for proper exposure.

  Once AF and AE have been carried out, it is next determined whether or not the 2nd release switch is on (S5). Here, the system control unit 20 determines whether a relatively stronger pressing operation than the first pressing operation is performed and the 2nd release switch is turned on. If the result of this determination is that the 2nd release switch is not on, processing returns to step S3 and AF / AE operations are repeated.

  If the result of determination in step S5 is that the 2nd release switch is on, comparatively bright combination in the comparatively bright combination mode is performed. First, the image reading cycle T is reflected (S7). This period T is a period for reading image data from the image sensor 4 and may be a value set manually by the user or a value corresponding to the appropriate exposure time calculated in AE in step S3.

  If the image reading cycle T is reflected, then exposure is started (S9). Here, the system control unit 20 causes the image sensor 4 to start exposure. Once exposure is started, it is next determined whether or not the exposure time T has elapsed (S11). Here, it is determined whether time T has elapsed after the start of exposure in step S9. It will be in a standby state until time T passes.

  If the exposure time T elapses as a result of the determination in step S11, next, image data is read and exposure of the next frame is started (S13). Subsequently, the image data read in step S13 is stored in the internal memory 33 (S15), and at the same time, exposure for the next photographing is started.

  After the image data is stored in the internal memory 33, comparative bright / dark combination processing is performed (S17). However, when image data is first read from the image sensor 4 in the comparative bright combination mode, the comparative bright / dark combination processing is skipped and the comparative bright / dark combination processing is performed in the second and subsequent frames. The comparative light / dark combination processing for the second and subsequent frames is stored in the internal memory 33 and the image data read in step S13 after the time T has elapsed when the result of determination in step S25 described later is No. Image data (cumulative comparative bright composite image data and cumulative comparative dark composite data).

  The comparative light / dark combination process in the second frame is performed as follows. The internal memory 33 temporarily stores image data for the first frame and the second frame. The system control unit 20 transfers the read image data of the first frame and the second frame temporarily stored in the internal memory 33 to the comparison unit 35 and compares the pixel outputs of the same address of the two images with each other. The other pixel data is selected as the pixel data of the combined image, and set as comparatively bright composite image data (cumulative comparative bright composite image data). Also, the pixel outputs at the same address of the two images are compared with each other, and the smaller pixel data is selected as the pixel data of the combined image, and is set as comparative dark composite image data (cumulative comparative dark composite image data).

  The comparatively bright composite image data and comparative dark composite image data generated in step S17 are stored in the internal memory 33 (S19).

  Once stored in the internal memory, next, development processing is performed (S21). Here, the system control unit 20 performs, for example, demosaicing processing, γ correction processing, noise reduction processing, YC signal generation processing, and resizing processing on the image data temporarily stored in the internal memory 33 by the development processing unit 16. Let the development process. The resizing process is performed in order to match the number of display pixels of the display element because the number of pixels of the read image data is larger than the display pixels of the liquid crystal display or the electronic viewfinder.

  Once the development process is performed, a developed image is displayed (S23). Here, the system control unit 20 causes the display unit 37 to display the comparatively bright composite image developed by the development processing unit 16 as an exposure progress image. The user can check the progress image at the time of bulb photography.

  Once the developed image is displayed, it is next determined whether or not the 2nd release switch is detected to be off (S25). In the present embodiment, bulb photographing is started until the user presses and releases the release button, that is, when the 2nd release switch is turned on, and the bulb photography is terminated when the 2nd release switch is turned off. It should be noted that, after the 2nd release switch is first turned on, the bulb photography may be continued even if it is turned off, and the bulb photography may be terminated when the 2nd release switch is turned on again.

  If the result of determination in step S25 is that the 2nd release switch is not OFF, processing returns to step S11. The image data for the third and subsequent frames are also read from the image sensor after the exposure time T has elapsed (S13) and stored in the internal memory 33 (S15). The comparison unit 35 reads the image data read from the image sensor 4 and the accumulated comparatively bright composite image data stored in the internal memory 33 (if the third frame is composite image data, the first and second frames). Comparatively bright composite image data) is subjected to comparatively bright composite processing (S17), and is stored in the internal memory 33 as cumulative comparative bright composite image data (S19). The stored cumulative comparative bright composite image data is subjected to development processing (S21), and is displayed on the display unit 37 as an exposure progress image (S23). Similarly, the image data read from the image sensor 4 and the cumulative comparative dark composite image data (if the third frame composite image data, the comparative dark composite image data of the first and second frames) are comparatively dark combined. Processed (S17) and stored in the internal memory 33 as accumulated comparative dark composite image data (S19).

  As described above, during bulb photographing, every time T elapses, image data is read from the image sensor 4, the comparatively bright combining process and the comparative dark combining process are performed, and the comparatively bright combined process image is displayed. It is displayed on the part 37 as an exposure progress image.

  As a result of the determination in step S25, when it is detected that the 2nd release switch is turned off, bulb photographing end processing is performed.

  Next, in step S43 and subsequent steps, exposure adjustment processing is performed. In this exposure adjustment process, the brightness of the background and the light trace portion of the final photographed image can be adjusted independently. First, it is determined whether or not to adjust the brightness of the background / light trail (S43). In this step, since an adjustment necessity screen (for example, an exposure adjustment necessity screen as shown in FIG. 4A) is displayed on the display unit 37, the user looks at the exposure adjustment necessity screen to determine the background. Confirm that the brightness of the light trail is as intended. If it is as intended, the exposure adjustment “NO” is selected by the input IF 38 (S43 → No). In this case, the accumulated comparatively bright composite image data and its development processed image data are stored in the external memory, and the photographing is completed.

  On the other hand, as a result of the determination in step S43, when the exposure adjustment “Yes” is selected (S43 → Yes), the background / light trace brightness adjustment screen (for example, as shown in FIG. 4B) is displayed on the display unit. An exposure adjustment operation screen) is displayed on the display unit 37.

  When an exposure adjustment gain setting operation is performed in this state, the set operation amount is acquired (S45). In the example shown in FIG. 4B, an adjustment indicator is displayed for each of the background and the light trace. When the user instructs to increase the luminance by sliding the indicator to the right or to decrease the luminance by sliding the indicator to the left, the system control unit 20 instructs the user via the touch panel in the input IF 38. Enter.

  The system control unit 20 changes the brightness adjustment amount of the background and light trace input by the user's operation to how much the brightness is changed with respect to the displayed image to an adjustment gain. Assuming that the background adjustment gain is α and the light trace adjustment gain is β, for example, when there is an input for adjusting the brightness of the background by +1 stage and the light trace by −1 stage, α = 2, β = Convert to 0.5.

Next, when an exposure adjustment gain setting operation is performed, next, difference output image data is generated (S47). Here, the system control unit 20 inputs the comparatively bright composite image data and the comparative dark composite image data to the differential image composition unit 13, generates differential output image data by the following equation (1), and stores it in the internal memory 33. .
Difference output image data = Comparative light composite image data−Comparative dark composite image data (1)

  Here, as shown in FIG. 5, the image data before synthesis read out at the exposure time T is comparatively dark composite image data (FIG. 5C) unless the background brightness changes. This corresponds to image data reflecting the brightness of the background excluding the trace. On the other hand, the difference output image data (FIG. 5D) is image data obtained by subtracting the background (comparative dark composite image data) (FIG. 5C) from the comparatively bright composite image data (FIG. 5B). Therefore, it becomes image data in which only the light trace is captured.

If difference output image data is generated in step S47, next, composite image data is generated (S49). Here, the system control unit 20 causes the image composition unit 15 to output the comparative dark composite image data and the difference output image data stored in the internal memory 33, and generates composite image data according to the following equation (2). That is, two image data of the image data obtained by multiplying the comparative dark composite image data by the adjustment gain α and the image data obtained by multiplying the difference output image data by the adjustment gain β are added and combined to generate composite image data. The generated composite image data is stored in the internal memory 33.
Composite image data = (Comparative dark composite image data × α) + (Differential output image data × β) (2)
Here, A + B indicates the combined composite image data of the image data A and the image data B.

  Through the above-described synthesis, image data is generated by synthesizing the background brightness adjusted image data (comparative dark synthesized image data × α) and the light trace brightness adjusted image data (difference output image data × β), and the user's operation Image data whose brightness is adjusted in accordance with is generated.

  When the composite image data is generated in step S49, next, development processing is performed (S51), and an image is displayed on the display unit 37 (S53). Once image display has been carried out, it is next determined whether or not the brightness is OK (S55). Here, as shown in FIG. 4, since the adjustment screen is displayed again, the user confirms the image whose brightness has been adjusted, and touches the exposure adjustment “Yes” if readjustment is necessary. . If the brightness of the composite image is insufficient or excessive, the process returns to step S45, and after the brightness adjustment value is input, the brightness of the background / light trace is adjusted again.

  FIG. 6 shows an example of background / light trace adjustment. FIG. 6A shows an example of light trace brightness adjustment. Here, the number of trajectories is adjusted. The image P2 is a reference image, the image P1 is subjected to minus correction, and the image P3 is subjected to plus correction. As can be seen from FIG. 6A, only the light trace can be adjusted without changing the brightness of the background.

  FIG. 6B is an example of background brightness adjustment. The image P5 is a reference image (the same image as P2), the image P4 is subjected to minus correction, and the image P6 is subjected to plus correction. As can be seen from FIG. 6B, only the brightness of the background can be adjusted without changing the brightness of the light trace.

  On the other hand, if it is determined in step S55 that the user has adjusted the brightness as intended, the user performs a touch operation on exposure adjustment “NO” via the input IF 38. In this case, the result of determination in step S55 is brightness OK, the composite image and its developed image are saved in the external memory 36 (S57), and the shooting is completed.

  As described above, in the first embodiment of the present invention, as shown in FIG. 6, while confirming the comparatively bright composite image after shooting, the user can brighten the background and the light trace in accordance with the intention. Can be adjusted. For this reason, it is possible to prevent a shooting failure due to the exposure failure of bulb shooting, which is difficult, and to take a photo according to the photographer's intention.

Next, a modification of the first embodiment of the present invention will be described. In the first embodiment, when the comparative light / dark combination is performed in the image comparison / synthesis unit 11, the comparison unit 35 compares the pixel data for each pixel to determine whether it is large or small. On the other hand, in this modification, difference pixel data between pixel data outputs to be compared is calculated, and the difference pixel data is compared with a threshold level (predetermined value) TH in the comparison unit 35, and is larger than the threshold level. The pixel data is a value obtained by performing comparatively bright / dark combination in the image comparison / synthesis unit 11, and an average value is calculated by the image average combination unit 12 for pixels smaller than the threshold level, and this average value is used as an image output after synthesis. That is, the following processing is performed.
Difference pixel data ≧ TH → Comparative light / dark combination Difference pixel data <TH → Average combination

  The image data read from the image sensor 4 includes random noise. For backgrounds that do not change in brightness, the maximum value of random noise is reflected in the cumulative bright composite image data in the comparative bright synthesis of multiple image data, and the minimum random noise value is reflected in the cumulative comparative dark composite image in the comparative dark synthesis. Reflected in the data. For this reason, in this modification, by setting the threshold level TH to a value of about random noise, it is possible to suppress image quality deterioration due to random noise and improve the image quality of the final composite image.

  Next, a second embodiment of the present invention will be described with reference to FIG. In the first embodiment, the exposure time (reading period T) for one frame from the image sensor 4 is constant during bulb photography. On the other hand, in the second embodiment, the exposure time for one frame from the image sensor 4 is changed according to the ambient brightness.

  That is, in the first embodiment, when the brightness of the background changes during shooting from when the 2nd release switch is turned on to when it is turned off, for example, astronomical shooting at dusk or dawn, When fireworks shine up and the brightness of the background changes, the background when the background appears brightest is reflected in the final cumulative comparative light composition. On the other hand, the background when the background appears darkest is reflected in the cumulative comparative dark composite image data. Since the difference composite image data is a difference between these image data, a difference in the change in brightness of the background is included together with the output of the light trace.

  When the photographer adjusts the brightness of the background / light trace as described in the first embodiment using the difference composite image data, not only the brightness of the light trace but also the brightness of the background changes. Thus, composite image data not intended by the photographer is generated. Therefore, in the second embodiment, in order to solve this problem, by changing the exposure time of one frame in accordance with the change in ambient brightness, the change in the brightness of the image at the time of comparatively bright / dark combining is performed. I try to make it as small as possible.

  The configuration in this embodiment is the same as the block diagram shown in FIG. 1 according to the first embodiment, and only the flowchart shown in FIG. 2 is replaced with FIG. In addition, the flowchart of FIG. 7 is only described with respect to this difference since steps S16A to S16E are only added to the flowchart shown in FIG.

  In the flow shown in FIG. 7, the image reading cycle T is reflected in step S7, and exposure is started (S9). When the exposure time T of the first frame has passed (S11), the image data is read and the next shooting is performed. Exposure is started (S13), and the read image data is stored in the internal memory 33 (S15).

  When the image data is stored in the internal memory in step S15, it is determined whether or not the exposure time for the second and subsequent frames has passed T / N (S16A). Here, N is an integer, and T / N is shorter than time T. The image sensor 4 performs exposure at time T for the first frame, and performs exposure at time T / N for the second and subsequent frames.

  As a result of the determination in step S16A, when the exposure time T / N elapses, image data is read and exposure for the next photographing is started (S16B). The read image data is stored in the internal memory 33 (S16C).

  Once stored in the internal memory, addition image processing is then performed (S16D). Here, the image addition / synthesis unit 14 adds the read image data for the second and subsequent frames. That is, in step S16B, image data is added each time image data is read. The added image data is stored in the internal memory 33 and used in the next added image processing.

Once the addition image processing has been carried out, it is next determined whether or not the exposure difference between the addition image data and the first frame image data stored in the internal memory 33 in step S15 is smaller than 1 / N ( S16E). Here, the comparison unit 35 compares the exposure amount of the image data acquired by the exposure of the first frame with the exposure amount of the image data acquired by the addition image processing. The comparison is performed in the comparison unit 35 by calculating the exposure difference in the following equation (3).
Exposure difference = {(exposure amount of first frame image data) − (exposure amount of added composite image data)} / (first frame image data exposure amount) (3)

  In order to calculate the exposure amount, an image pixel output average value may be calculated using an AE (photometry) method mounted on a digital camera in recent years.

  As a result of the determination in step S16E, if the exposure difference is 1 / N or more, the process returns to step S16A and the above-described addition / synthesis processing is repeated. Thereby, the exposure amount of the addition composite image data gradually increases.

  As a result of the determination in step S16E, when the exposure difference becomes 1 / N or less, the added synthesized image data is stored in the internal memory 33, and the added synthesized image data and the read image data of the first frame are used for comparison. Dark synthesis is performed (S17), and the result is stored in the internal memory 33 as cumulative comparative bright composite image data and cumulative comparative dark composite image data, respectively (S19).

  Thereafter, until the 2nd release switch is turned off, the exposure difference from the first frame becomes 1 / N or less in the above-described sequence for the image data read every time T / N (N is an integer). Additive synthesized image data is generated, and cumulative comparative bright synthesis, cumulative comparative dark synthesis, and comparative synthesis processing are performed. If the 2nd release switch is turned off as a result of the determination in step S25, step S31 and subsequent steps shown in FIG. 3 are executed. Since this flow is the same as that of the first embodiment, description thereof is omitted, but the brightness of the background / light trace is adjusted according to the photographer's intention and the final image data is stored in the external memory 36. To do.

  As described above, in the second embodiment, the background brightness is close to the brightness of the captured image of the first frame (exposure difference of 1 / N or less). Even when the brightness of the background changes, it is possible to prevent generation of composite image data not intended by the photographer.

  As described above, in each embodiment or modification of the present invention, the darker pixel is used for each pixel of the image data using the image data continuously read from the image sensor at predetermined time intervals. The comparative dark composite image data is generated by the comparative dark composite, and the comparative bright composite image data is generated by the comparative bright composite using the brighter pixel for each pixel of the image data. Difference output image data is generated from the image data (see S35 and S47 in FIG. 3, and FIGS. 5B, 5C, and 5D). Then, the generated comparative dark composite image data is used as background image data, and the difference output image data is used as light trace output image data. For the comparatively bright composite image data, the brightness of the background and the brightness of the light trail can be adjusted independently of each other. For this reason, the exposure failure of bulb photography with high difficulty is prevented, and photography according to the photographer's intention is made possible.

  Further, when generating the composite of the comparative dark composite image data and the comparative bright image data, the pixel data of the same address are compared with each other, and the larger or smaller pixel data is used. Therefore, the configuration is simple and processing can be performed quickly. In addition, when compositing the comparative dark composite image data and the comparative bright image data, if the difference between the pixel data exceeds a predetermined value, the comparative dark composite image data or the comparative bright composite is used by using large or small pixel data. When image data is generated and the difference between the pixel data is equal to or smaller than a predetermined value, comparative dark composite image data or comparatively bright composite image data is generated using an average value of the pixel data. By determining using the difference value, the influence of pixel noise can be reduced.

  In each embodiment and modification of the present invention, when the comparative light / dark combination processing is performed, only the latest cumulative comparative bright composite image data / cumulative comparative dark composite image data is stored (see FIG. 2). Step S19, S37 in FIG. 3, S19 in FIG. 7), a series of past accumulated bright composite image data / cumulative dark composite image data is stored, and after bulb photography is completed, based on these image data, respectively. Difference output image data may be generated and image data intended by the user may be selected.

  In each embodiment and modification of the present invention, a comparatively bright composite image is displayed as an image display. However, the present invention is not limited to this, and a comparative dark composite image and a differential output image are displayed if selected by the user. Anyway.

  In each embodiment and modification of the present invention, a digital camera has been described as an apparatus for photographing. However, a digital single-lens reflex camera, a mirrorless camera, or a compact digital camera may be used as the camera. It may be a camera for moving images such as a camera or a movie camera, and may be a camera built in a mobile phone, a smart phone, a personal digital assistant (PDA), a game device, or the like. In any case, the present invention can be applied to any device for bulb photography that can perform bulb photography.

  Of the techniques described in this specification, the control mainly described in the flowchart is often settable by a program and may be stored in a recording medium or a recording unit. The recording method for the recording medium and the recording unit may be recorded at the time of product shipment, may be a distributed recording medium, or may be downloaded via the Internet.

  In addition, regarding the operation flow in the claims, the specification, and the drawings, even if it is described using words expressing the order such as “first”, “next”, etc. It does not mean that it is essential to implement in this order.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 1 ... Imaging part, 2 ... Lens (aperture), 3 ... Mechanical shutter, 4 ... Image sensor, 10 ... Image processing part, 11 ... Image comparison combining part, 12 ... Image average synthesizing unit, 13 ... Difference image synthesizing unit, 14 ... Image addition synthesizing unit, 15 ... Image synthesizing unit, 16 ... Development processing unit, 20 ... System control unit, 21 ... Exposure time calculation unit, 22 ... exposure time control unit, 31 ... bus, 33 ... internal memory, 34 ... exposure determination unit, 35 ... comparison unit, 36 ... external memory, 37 ... Display unit, 38 ... Input IF

Claims (8)

A storage unit for storing a plurality of image data sequentially input ;
Compared pixel data corresponding to each of the plurality of pixels constituting the two-frame image data read out from the storage unit, and using the larger pixel data as new pixel data, comparatively bright composite image data A comparatively bright combination processing unit for performing comparatively bright combination processing that constitutes
Comparison dark composite image data by comparing pixel data corresponding to each of a plurality of pixels constituting the two-frame image data read from the storage unit, and using the smaller pixel data as new pixel data A comparative dark synthesis processing unit for performing comparative dark synthesis processing that constitutes
A difference image synthesizing unit for generating a corresponding calculates the difference between the pixel data new shelf differential image data for each of a plurality of pixels constituting the relatively bright composite image data and the relatively dark composite image data,
An image composition unit that synthesizes the multiplication dark image data obtained by multiplying the comparative dark composite image data by an arbitrary gain value and the multiplication difference image data obtained by multiplying the difference image data by an arbitrary gain value, and generates a composite image;
An image processing apparatus comprising: The comparatively bright combination processing unit stores the first image data first generated based on the image data read from the storage unit in the storage unit as cumulative comparative bright combined image data, and reads from the storage unit. Comparing the plurality of pixel data constituting the second image data generated based on the output image data and the corresponding pixel data of the plurality of pixel data constituting the cumulative comparative bright composite image data, The larger pixel data is used as new pixel data, and the comparatively bright combination processing that constitutes the cumulative comparative bright combined image data is performed.
The relatively dark combination processing unit, the first image data that has been initially generated based on image data read out from the storage unit as a cumulative relatively dark composite image data stored in the storage unit, read from the storage unit Comparing a plurality of pixel data constituting the second image data generated based on the outputted image data and corresponding pixel data of the plurality of pixel data constituting the cumulative comparative dark composite image data, Using the smaller pixel data as new pixel data, a comparative dark composition process is performed that constitutes the cumulative comparative dark composite image data.
The difference image composition unit calculates a difference between corresponding pixel data of the cumulative comparative bright composite image data and the cumulative comparative dark composite image data to generate new difference image data,
The image composition unit synthesizes the cumulative multiplication dark image data obtained by multiplying the cumulative comparative dark composite image data by an arbitrary gain value and the multiplication difference image data obtained by multiplying the difference image data by an arbitrary gain value. Generate
The image processing apparatus according to claim 1 . A storage unit for storing a plurality of image data sequentially input ;
The difference between the pixel data corresponding to each of the plurality of pixels constituting the two-frame image data read from the storage unit is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, When pixel data corresponding to each of a plurality of pixels constituting the image data of two frames is compared, and the larger pixel data is set as new pixel data. On the other hand, when the difference is smaller than the predetermined value, A comparatively bright composite processing unit that performs comparatively bright composite processing using the average value of each of the plurality of pixel data constituting the two frames of image data as new pixel data, and generates comparatively bright composite image data;
The difference between the pixel data corresponding to each of the plurality of pixels constituting the two frames of image data read from the storage unit is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the 2 Comparing the pixel data corresponding to each of the plurality of pixels constituting the image data of the frame, the smaller pixel data is set as new pixel data, and when the difference is smaller than the predetermined value, A comparative dark composition processing unit that performs image composition processing by using an average value of a plurality of pixel data constituting each of the two frames of image data as new pixel data, and generates comparative dark composite image data;
A difference image composition unit that calculates a difference between pixel data corresponding to each of a plurality of pixels constituting the comparative bright composite image data and the comparative dark composite image data to generate new difference image data;
An image composition unit that synthesizes the multiplication dark image data obtained by multiplying the comparative dark composite image data by an arbitrary gain value and the multiplication difference image data obtained by multiplying the difference image data by an arbitrary gain value, and generates a composite image;
An image processing apparatus comprising: The comparatively bright combination processing unit stores the first image data first generated based on the image data read from the storage unit in the storage unit as cumulative comparative bright combined image data, and reads from the storage unit. The difference between the plurality of pixel data constituting the second image data generated based on the output image data and the plurality of pixel data constituting the cumulative comparative bright composite image data is compared with a predetermined value. When the difference is equal to or greater than the predetermined value, the pixel data corresponding to each of the plurality of pixel data constituting the image data of the two frames is compared, and the larger pixel data is set as new pixel data. On the other hand, when the difference is smaller than the predetermined value, the average value of a plurality of pixel data constituting each of the two frames of image data is used as new pixel data, and the cumulative comparison Performed lighten compositing process of generating formed image data,
The comparative dark synthesis processing unit stores the first image data generated first based on the image data read from the storage unit in the storage unit as cumulative comparative dark synthesized image data, and reads the first image data from the storage unit. The difference between the plurality of pixel data constituting the second image data generated based on the output image data and the corresponding pixel data of the plurality of pixel data constituting the cumulative comparative dark composite image data is defined as a predetermined value. If the difference is equal to or greater than the predetermined value, the pixel data corresponding to each of the plurality of pixels constituting the image data of the two frames is compared, and the smaller pixel data is set as a new pixel. On the other hand, when the difference is smaller than the predetermined value, the average value of a plurality of pixel data constituting each of the two frames of image data is used as new pixel data, and the cumulative comparative darkness is obtained. It compares dark combination process of generating formed image data,
The difference image composition unit calculates a difference between corresponding pixel data of the cumulative comparative bright composite image data and the cumulative comparative dark composite image data to generate new difference image data,
The image composition unit synthesizes the cumulative multiplication dark image data obtained by multiplying the cumulative comparative dark composite image data by an arbitrary gain value and the multiplication difference image data obtained by multiplying the difference image data by an arbitrary gain value. Generate
The image processing apparatus according to claim 3 . In an image processing method in an image processing apparatus having a storage unit for storing a plurality of image data sequentially input ,
Compared pixel data corresponding to each of the plurality of pixels constituting the two-frame image data read out from the storage unit, and using the larger pixel data as new pixel data, comparatively bright composite image data The comparative light composition processing that composes
Comparison dark composite image data by comparing pixel data corresponding to each of a plurality of pixels constituting the two-frame image data read from the storage unit, and using the smaller pixel data as new pixel data The comparative dark synthesis process that composes
By calculating the respective differences of the corresponding pixel data of each of a plurality of pixels constituting the relatively bright composite image data and the relatively dark composite image data to generate a new shelf difference image data,
The composite dark image data obtained by multiplying the comparative dark composite image data by an arbitrary gain value and the multiply differential image data obtained by multiplying the difference image data by an arbitrary gain value are combined to generate a composite image .
An image processing method. In an image processing method in an image processing apparatus having a storage unit for storing a plurality of image data sequentially input ,
The difference between the pixel data corresponding to each of the plurality of pixels constituting the two-frame image data read from the storage unit is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, When pixel data corresponding to each of a plurality of pixels constituting the image data of two frames is compared, and the larger pixel data is set as new pixel data. On the other hand, when the difference is smaller than the predetermined value, , Performing a comparatively bright composite process using the average value of each of the plurality of pixel data constituting the two frames of image data as new pixel data, and generating comparatively bright composite image data,
The difference between the pixel data corresponding to each of the plurality of pixels constituting the two frames of image data read from the storage unit is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the 2 Comparing the pixel data corresponding to each of the plurality of pixels constituting the image data of the frame, the smaller pixel data is set as new pixel data, and when the difference is smaller than the predetermined value, An image synthesizing process is performed by using an average value of a plurality of pixel data constituting each of the two frames of image data as new pixel data, and comparative dark synthesized image data is generated.
Calculating a difference between pixel data corresponding to each of a plurality of pixels constituting the comparative light composite image data and the comparative dark composite image data, and generating new difference image data ;
The composite dark image data obtained by multiplying the comparative dark composite image data by an arbitrary gain value and the multiply differential image data obtained by multiplying the difference image data by an arbitrary gain value are combined to generate a composite image .
An image processing method. In a program for executing a computer of an image processing apparatus having a storage unit for storing a plurality of image data sequentially input ,
Compared pixel data corresponding to each of the plurality of pixels constituting the two-frame image data read out from the storage unit, and using the larger pixel data as new pixel data, comparatively bright composite image data The comparative light composition processing that composes
Comparison dark composite image data by comparing pixel data corresponding to each of a plurality of pixels constituting the two-frame image data read from the storage unit, and using the smaller pixel data as new pixel data The comparative dark synthesis process that composes
By calculating the respective differences of the corresponding pixel data of each of a plurality of pixels constituting the relatively bright composite image data and the relatively dark composite image data to generate a new shelf difference image data,
The composite dark image data obtained by multiplying the comparative dark composite image data by an arbitrary gain value and the multiply differential image data obtained by multiplying the difference image data by an arbitrary gain value are combined to generate a composite image .
A program for causing the computer to execute the above. In a program for executing a computer of an image processing apparatus having a storage unit for storing a plurality of image data sequentially input ,
The difference between the pixel data corresponding to each of the plurality of pixels constituting the two-frame image data read from the storage unit is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, When pixel data corresponding to each of a plurality of pixels constituting the image data of two frames is compared, and the larger pixel data is set as new pixel data. On the other hand, when the difference is smaller than the predetermined value, , Performing a comparatively bright composite process using the average value of each of the plurality of pixel data constituting the two frames of image data as new pixel data, and generating comparatively bright composite image data,
The difference between the pixel data corresponding to each of the plurality of pixels constituting the two frames of image data read from the storage unit is compared with a predetermined value, and when the difference is equal to or greater than the predetermined value, the 2 Comparing the pixel data corresponding to each of the plurality of pixels constituting the image data of the frame, the smaller pixel data is set as new pixel data, and when the difference is smaller than the predetermined value, An image synthesizing process is performed by using an average value of a plurality of pixel data constituting each of the two frames of image data as new pixel data, and comparative dark synthesized image data is generated.
Calculating a difference between pixel data corresponding to each of a plurality of pixels constituting the comparative light composite image data and the comparative dark composite image data, and generating new difference image data ;
The composite dark image data obtained by multiplying the comparative dark composite image data by an arbitrary gain value and the multiply differential image data obtained by multiplying the difference image data by an arbitrary gain value are combined to generate a composite image .
A program for causing the computer to execute the above.