JP6138211B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program capable of applying a special effect similar to a cross filter.

  Conventionally, a technique for applying a special effect by attaching a cross filter, which is an optical filter, is known when taking a still image. In recent years, it has been proposed to apply a special effect similar to that of a cross filter by image processing instead of shooting with a cross filter when shooting a still image in an imaging apparatus such as a digital camera.

  For example, in Patent Document 1, a portion with high luminance is extracted from an image taken with a still image, and a light streak (hereinafter referred to as “light stripe”) is applied to the center of high luminance by image processing, which is similar to a cross filter. An electronic still camera capable of achieving the special effects is disclosed. According to this electronic still camera, the optical streak effect can be easily obtained only by digital processing using an electrical switch without mounting an optical cross filter.

JP-A-6-30373

  In recent digital cameras, a live view display in which a subject image is displayed as a moving image is displayed on a display unit such as a liquid crystal monitor, and a photographer generally shoots a still image or a movie while checking the live view display. ing. Also in this live view display, it is convenient if image processing that gives a special effect similar to the above-described cross filter can be performed. However, when performing live view display as a moving image, it has been difficult to perform image processing that provides a cross filter effect.

  That is, in order to give the cross filter effect, the entire image is scanned to find a bright spot, and the cross image must be drawn according to the brightness and color of the found bright spot, and a series of these processes is performed. It takes a very long time. In particular, (1) the larger the target image, the longer the time for searching for bright spots, and (2) the longer the length of the cross filter, the longer the processing time. . Furthermore, (3) the processing time increases and the processing time fluctuates as the number of bright spots on the subject increases.

  If a still image is photographed and a recorded still image is edited as in the past, image processing for providing a cross filter effect is possible. However, when a moving image is processed at a predetermined frame rate, for example, about 30 fps (frame-per-second), such as live view display or moving image shooting, image processing for providing a cross filter effect is not performed. Because of the above (1) to (3), the processing load is large, and in reality it was difficult to give a cross filter effect. In addition, because of the above (3), if the shooting direction is changed in normal live view display or moving image shooting, the frame rate will change with the change in processing time, resulting in an unnatural live view display or moving image. It became an expression.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an imaging apparatus, an imaging method, and a program capable of performing a cross-filter process suitable for live view display and moving images.

In order to achieve the above object, an imaging apparatus according to a first invention includes an imaging unit that converts a subject image into an electrical signal and outputs image data in a live view, and once for each predetermined frame for the image data . The entire image data is divided into a plurality of areas, at least one of which is set as a bright spot detection area, and a bright spot within the bright spot detection area set by the bright spot area setting section. A bright spot detection section for detecting a point, a bright spot processing section for performing bright spot processing at the bright spot position of the bright spot detected by the bright spot detection section, and an image that has been bright spot processed by the bright spot processing section The image output unit for outputting for display or recording, and for all the regions divided into a plurality of regions by the bright spot region setting unit, at the position where the bright spot is detected by the bright spot detection unit, By the bright spot processing unit When subjected to the bright spot treatment, and a control unit repeating the bright point processing.

In the imaging apparatus according to a second aspect of the present invention, in the first aspect, the bright spot area setting unit sets the bright spot detection area at least in the vertical and horizontal directions.
In the imaging device according to a third aspect of the present invention, in the first aspect, the bright spot area setting unit sequentially sets the bright spot detection areas in a random order.

In the imaging device according to a fourth invention, in the first invention, the bright spot processing in the bright spot processing section is a cross filter process.
Imaging device according to a fifth invention, in the first aspect, the bright point processing unit, when the bright point processing successive frames, at least a portion of the brightness, the length of the striation, striation Change the number, the angle of light, or the color .

  In the imaging apparatus according to a sixth aspect of the present invention, in the first aspect, the bright spot processing unit limits the number of bright spots to be processed within the set bright spot detection area.

According to a seventh aspect of the present invention, there is provided an imaging method according to a first output step of converting a subject image into an electrical signal and outputting image data in a live view, and once for each predetermined frame with respect to the image data, the entire image data. was divided into a plurality of regions, of which a setting step of setting at least one area as a bright spot detection region, a detection step of detecting a bright point of the set the bright spot detection area, the detected bright A processing step for performing bright spot processing at the bright spot position of the point, and for all the areas divided into a plurality of areas in the setting step, at the position where the bright spot is detected in the detection step, when subjected to the bright point processing, a control step of repeating the bright point processing, the second output for outputting the image processed the bright spots or for recording display Has a step, a.

According to an eighth aspect of the present invention, there is provided a program for converting a subject image into an electrical signal and outputting image data in a live view, and for the image data, the entire image data is once per predetermined frame. A setting step of dividing the plurality of regions into at least one of them as a bright spot detection region, a detection step of detecting a bright spot in the set bright spot detection region, and the detected bright spot A processing step for performing bright spot processing at the bright spot position, and for all areas divided into a plurality of areas in the setting step, at the position where the bright spot is detected in the detection step, when subjected to bright point processing, a control step of repeating the bright spot treatment, leaving the second outputs the image processed the bright spots or for recording display A step, the causes the computer to execute.

  According to the present invention, it is possible to provide an imaging apparatus, an imaging method, and a program that can perform cross-filter processing suitable for live view display and moving images.

1 is a block diagram mainly showing an electrical configuration of a camera according to an embodiment of the present invention. It is the external appearance perspective view seen from the back side of the camera concerning one Embodiment of this invention. It is a figure explaining the area | region which gives the cross filter effect in the camera concerning one Embodiment of this invention. 4 is a flowchart illustrating a main operation of the camera according to the embodiment of the present invention. It is a flowchart which shows the live view operation | movement of the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the cross filter process of the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the saturation point detection process of the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the cross filter drawing process of the camera concerning one Embodiment of this invention. It is a flowchart which shows the modification of the live view operation | movement of the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the cross filter moving image photography of the camera concerning one Embodiment of this invention. It is a flowchart which shows operation | movement of the 1st modification of the cross filter video recording of the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of the production effect setting in the 1st modification of the cross filter video recording of the camera concerning one Embodiment of this invention. It is a flowchart which shows operation | movement of the 2nd modification of the cross filter video recording of the camera concerning one Embodiment of this invention. It is a figure explaining the image in the cross filter of the camera concerning one Embodiment of this invention. It is a figure explaining the image in the 2nd modification of cross filter animation photography of the camera concerning one embodiment of the present invention.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. A camera according to a preferred embodiment of the present invention is a digital camera, includes an imaging unit, converts a subject image into image data by the imaging unit, and converts the subject image into a main body based on the converted image data. Live view is displayed on the display unit arranged on the back of the camera. The photographer determines the composition and the photo opportunity by observing the live view display. During the release operation, image data is recorded on the recording medium. The image data recorded on the recording medium can be reproduced and displayed on the display unit when the reproduction mode is selected. When the cross filter mode is set, live view display subjected to the cross filter processing is performed, and image data subjected to the cross filter processing is recorded at the time of motion shooting.

  FIG. 1 is a block diagram mainly showing an electrical configuration of a camera 100 according to an embodiment of the present invention. An aperture mechanism 103, a shutter 105, and an image sensor 107 are disposed on the optical axis of the photographing lens 101. The output of the image sensor 107 is connected to the A / D converter 109, and the output of the A / D converter 109 is connected to the memory 110. The memory 110 is connected to the image processing unit 111 and the system control unit 116. The system control unit 116 includes an imaging control unit 108, a shutter control unit 106, an aperture control unit 104, a lens control unit 102, an exposure control unit 112, an AF processing unit 113, a flash control unit 121, a nonvolatile memory 118, and an external memory 114. The display unit 115, the operation unit 117, and the power control unit 120 are connected to each other. The above-described imaging control unit 108 is connected to the imaging element 107, the shutter control unit 106 is connected to the shutter 105, the aperture control unit 104 is connected to the aperture 103, and the lens control unit 102 is the imaging lens 101. It is connected to the. The power control unit 120 is connected to the power supply unit 119, and the flash control unit 121 is connected to the flash charging unit 122 and the flash light emitting unit 123, respectively.

  The photographing lens 101 is an optical system for condensing a subject light beam on the image sensor 107 and forming a subject image. The photographing lens 101 is moved in the optical axis direction by the lens control unit 102 that operates in response to an instruction from the system control unit 116, and the focus state changes. The diaphragm mechanism 103 adjusts the incident amount of the subject light beam incident on the image sensor 107 via the photographing lens 101. The aperture amount of the aperture mechanism 103 is controlled by the aperture controller 104 that operates in accordance with an instruction from the system controller 116.

  The shutter 105 opens and closes the light flux of the subject image formed by the photographing lens 101, and is configured by a known lens shutter, focal plane shutter, or the like. The shutter opening time (shutter speed value) of the shutter 105 is controlled by the shutter control unit 106 that operates in accordance with an instruction from the system control unit 116.

  The image sensor 107 functioning as an image capturing unit is a two-dimensional image sensor such as a CMOS image sensor or a CCD image sensor. A Bayer array color filter arranged on the front surface and a photo array corresponding to the color filter are provided. It is composed of photoelectric conversion elements such as diodes. An imaging region is configured by a pixel group including each color filter and a corresponding photoelectric conversion element. The image sensor 107 receives the light collected by the photographing lens 101 by each pixel and converts it into a photocurrent, accumulates this photocurrent in a capacitor, and sends it to the A / D converter 109 as an analog voltage signal (image signal). Output. The imaging control unit 108 controls the operation of the imaging element 107 in accordance with an instruction from the system control unit 116.

  The A / D conversion unit 109 converts an analog voltage signal (image signal) output from the image sensor 107 into a digital image signal (image data). The memory 110 is a storage unit that temporarily stores various data such as image data obtained by the A / D conversion unit 109 and image data processed by the image processing unit 111. In this specification, as long as the signal is based on the image signal output from the image sensor 107, the image includes not only the signal A / D converted by the A / D converter 109 but also the image processed signal. Sometimes referred to as data.

  The image processing unit 111 reads out image data temporarily stored in the memory 110 and performs image processing such as white balance correction processing, synchronization processing, and color conversion processing on the image data. The image processing unit 111 performs image compression when recording in an external memory 114 described later, and decompresses compressed image data read from the external memory 114.

  The exposure control unit 112 calculates subject brightness (brightness of the scene including the subject) using the image data temporarily stored in the memory 110. Of course, the subject brightness may be calculated using a dedicated photometric sensor.

  An AF (Auto Focus) processing unit 113 extracts a high-frequency component signal from image data temporarily stored in the memory 110, and acquires a focus evaluation value by AF integration processing. Based on the focus evaluation value, the system control unit 116 performs drive control through the lens control unit 102 so that the photographing lens 101 is in the in-focus position. Of course, the AF processing unit 113 may be provided with a dedicated sensor such as a TTL phase difference AF sensor, and the focus shift amount of the photographing lens 101 may be obtained based on the output of the dedicated sensor. The exposure control unit 112 and the AF processing unit 113 perform processing suitable for each shooting mode during still image shooting and during moving image shooting, and the processing of both is different.

  The system control unit 116 includes an ASIC (Application Specific Circuit) including a CPU (Central Processing Unit) and the like, and various types of the camera 100 such as the imaging control unit 108 and the flash control unit 121. Centrally control the sequence. As will be described later, the system control unit 116 functions as a bright spot area setting section that sets a bright spot area for image data from the image sensor 107. In addition, as a bright spot detection unit for detecting a bright spot within the set bright spot region, a bright spot processing unit for performing bright spot processing at the bright spot position of the detected bright spot, and bright spot processing. It also functions as an image output unit that outputs an image for display or recording.

  The operation unit 117 includes operation members such as a power button 117a, a release button 117b, a shooting mode dial 117c, a moving image button 117d, a function button 117e, a cross button 117f, an OK button 117g, a menu button, and various input keys as shown in FIG. including. When the user operates one of the operation members of the operation unit 117, the system control unit 116 executes various sequences according to the user's operation.

  The power button 117a in the operation unit 117 is an operation member for instructing the power on / off of the camera 100. The system control unit 116 is turned on when the power button 117a is pressed, and the power is turned off when the power button 117a is pressed again. . The release button 117b has a two-stage switch including a 1st release switch and a 2nd release switch. When the release button 117b is half-pressed, the 1st release switch is turned on. When the release button 117b is further pressed halfway and fully pressed, the 2nd release switch is turned on. When the 1st release switch is turned on, the system control unit 116 executes a shooting preparation sequence such as AE processing and AF processing. When the 2nd release switch is turned on, the system control unit 116 executes a still image shooting sequence to perform shooting.

  The moving image button 117d in the operation unit 117 is a button for instructing start and stop of moving image shooting. When the moving image button 117d is first depressed, moving image shooting starts. When the moving image button 117d is depressed again, moving image shooting is stopped. The shooting mode dial 117c is a dial for setting an auto mode, an aperture priority mode, a shutter speed priority mode, a movie shooting mode (moving image shooting mode), a scene mode, a bulb shooting mode, and the like.

  The external memory 114 is, for example, a storage medium that is detachable from the camera body, and stores the image data compressed by the image processing unit 111 and its associated data. The stored image data is read out and reproduced and displayed on the display unit 115. The recording medium for recording image data or the like is not limited to an external memory that can be attached to and detached from the camera body, but may be a recording medium such as a hard disk built in the camera body.

  The display unit 115 includes a liquid crystal monitor 115a (see FIG. 2) disposed on the back surface of the camera body and the like, and performs live view display based on image data. In addition, the display unit 115 reproduces and displays the captured image recorded in the external memory 114, and further displays a menu screen for setting an exposure control value and the like and setting a shooting mode and the like. On this menu screen, a cross filter mode to be described later is set. In addition to the menu screen, it may be set by an operation member such as the function button 117e. In addition, as long as it can display an image etc., not only a liquid crystal monitor but a display, such as organic EL, may be used.

  The nonvolatile memory 118 is an electrically rewritable nonvolatile memory, and stores various parameters necessary for the operation of the camera 100. The nonvolatile memory 118 also stores a program executed by the system control unit 116. The system control unit 116 reads parameters stored in the non-volatile memory 118 according to a program stored in the non-volatile memory 118, and executes various sequences.

  The power supply unit 119 supplies power necessary for the operation of each unit of the camera 100, and is configured by a power supply battery such as a secondary battery, for example. The power supply control unit 120 controls the power supply unit 119 such as detection of a power supply voltage and a remaining amount of a battery constituting the power supply unit 119.

  The flash control unit 121 controls the charging operation in the flash charging unit 122 and the light emitting operation in the flash light emitting unit 123 according to an instruction from the system control unit 116. The flash charging unit 122 includes a booster circuit that boosts the power supply voltage of the power supply unit 119 and a capacitor that stores energy using the boosted voltage. The flash charging unit 122 stores energy necessary for the flash light emitting unit 123 to emit light. . The flash light-emitting unit 123 includes, for example, a light-emitting tube such as a xenon (Xe) tube and a reflector, and the energy stored in the capacitor of the flash charging unit 122 when the light-emission instruction is received from the flash control unit 121. Use to emit light.

  Next, the appearance of the camera 100 according to the present embodiment will be described with reference to FIG. FIG. 2 is an external view of the camera 100 as seen from the back side. The interchangeable lens 20 is attached to the camera body 10. On the upper surface of the camera body 10, a power button 117a, a release button 117b, and a shooting mode dial 117c are arranged.

  In addition, a liquid crystal monitor 115a is disposed on the back of the camera body 10, thereby performing various displays such as live view display of a subject image, menu screen display, and playback display of a recorded image. A moving image button 117d and a function button 117e are arranged on the upper right side of the rear surface of the camera body 10, and a cross button 117g and an OK button 117f are arranged below these buttons. The cross button 117g moves the cursor on the menu screen or the like displayed on the liquid crystal monitor 115a, and can confirm the item selected by the cursor by pressing the OK button 117f.

  Next, the cross filter processing in the present embodiment will be described with reference to FIG. FIG. 3A shows a state where the pixel group of the image sensor 107 is divided into 24 areas of 4 × 6. In the present embodiment, a random number is generated for each frame by a randomizing function, an area divided into 24 is selected based on the random number, and this area and all four areas adjacent to this area are added to the effect imparting area ( (Bright spot detection area to which bright spot processing is performed), and bright spot processing is performed on the effect imparting area.

  For example, FIG. 3B is the first frame, and here, in each area of (1, 1), (1, 2), (2, 1), (2, 2), a brightness such as cross filter processing is obtained. Point processing is performed. In the subsequent second frame, as shown in FIG. 3 (c), bright spots such as cross filter processing are provided in the areas (4, 1), (4, 2), (5, 1), and (5, 2). Processing is performed. In the subsequent third frame, as shown in FIG. 3 (d), bright areas such as cross filter processing are provided in the areas (3, 2), (3, 3), (4, 2), and (4, 3). Processing is performed.

  In the fourth frame, as shown in FIG. 3E, bright spot processing such as cross filter processing is performed in each of the areas (2, 3), (2, 4), (3, 3), and (3,4). Is given. In the subsequent fifth frame, as shown in FIG. 3 (f), bright spots such as cross filter processing are provided in the areas (5, 2), (5, 3), (6, 2), and (6, 3). Processing is performed. In the subsequent sixth frame, as shown in FIG. 3G, bright spots such as cross filter processing are provided in the areas (1, 3), (1, 4), (2, 3), and (2, 4). Processing is performed.

  In the seventh frame, as shown in FIG. 3 (h), bright spot processing such as cross filter processing is performed in each area of (2, 1), (2, 2), (3, 1), (3, 2). Is given. In the subsequent 8th frame, as shown in FIG. 3 (i), bright areas such as cross filter processing are applied to the areas (4, 3), (4, 4), (5, 3), and (5, 4). Processing is performed. In the subsequent 9th frame, as shown in FIG. 3 (j), bright areas such as cross filter processing are provided in the areas (5, 1), (5, 2), (6, 1), and (6, 2). Processing is performed.

  In the 10th frame, as shown in FIG. 3 (k), bright spot processing such as cross filter processing is performed in each of the areas (5, 3), (5, 4), (6, 3), and (6, 4). Is given. In this embodiment, when the effect is applied to the 10th frame, the operation returns to the 1st frame again. As can be seen from FIG. 3, when the area selection is completed from the first frame to the 10th frame, the cross filter process is executed in all areas. The setting of these areas and the cross filter process as the bright spot process are executed by the system control unit 116.

  Next, the operation of the camera 100 in the present embodiment will be described using the flowcharts shown in FIGS. These flows are executed by the system control unit 116 according to a program stored in the nonvolatile memory 118. The flow shown in FIG. 4 is a main routine operation. This main routine starts executing when the power button 117a of the operation unit 117 is turned on.

  When the operation of the main routine is started, a live view operation is first performed (S1). In the live view operation, the image signal output from the image sensor 107 is subjected to image processing for live view display by the image processing unit 111, and the processed image data is displayed on the liquid crystal monitor 115 a of the display unit 115. The photographer looks at this live view display, determines the composition of the still image or moving image, and determines the shutter timing. Further, when live view display is performed when the cross filter mode is set, the image data is subjected to cross filter processing for the effect imparting region (bright spot detection region) described with reference to FIG. 3 and displayed. . Details of the live view operation will be described later with reference to FIG.

  When the live view operation is performed, it is next determined whether or not the release button 117b of the operation unit 117 is half-pressed, that is, whether or not the 1st release switch is on (S3). If the result of this determination is that the first release switch is on, AF processing is executed (S5). In this step, the system control unit 116 makes the subject image formed on the image sensor 107 the clearest based on the focus evaluation value obtained by the AF processing unit 113, in other words, the focus state. Thus, the focus state of the taking lens 101 is controlled via the lens control unit 102.

  Once the AF process is performed, photometry is performed (S7) and exposure calculation is performed (S9). In these steps, the exposure control unit 112 calculates subject brightness, and the system control unit 116 obtains exposure control values such as an aperture value and a shutter speed by using the calculated subject brightness. The aperture value and shutter speed are calculated by apex calculation or table reference.

  If exposure calculation is performed in step S9 or if the result of determination in step S3 is that the first release is not on, then the movie switch that turns on and off in conjunction with the movie button 117d changes from off to on. It is determined whether or not (S11). Here, the determination is made based on the state of the moving image button 117d of the operation unit 117.

  If the result of determination in step S11 is that the moving image switch has changed from off to on, it is next determined whether or not the moving image shooting flag is 0 and the still image shooting flag is x (S13). Here, 0 of the moving image flag indicates that moving image shooting is stopped, and 1 of the moving image flag indicates that moving image shooting is continuing. Still image shooting flag 1 indicates that still image shooting is being performed, and still image shooting flag 0 indicates that still image shooting has ended. The flag x for moving image shooting or still image shooting may be in an arbitrary state, that is, any state, and a flag indicating the previous state is held. When the camera 100 is reset, both the moving image shooting flag and the still image shooting flag are reset to zero.

  If the result of determination in step S13 is that the movie shooting flag is 0, that is, movie shooting has been stopped, the movie shooting flag is set to 1, and the still image shooting flag is set to x (S15). Here, x means that the flag is held without being changed. When this flag is set, shooting of a moving image is started (S17). Here, the image data acquired by the image sensor 107 is processed by the image processing unit 111 and then recorded in the external memory 114. At the same time, live view display is performed on the display unit 115. When moving image shooting is performed when the cross filter mode is set, the cross filter processing as described with reference to FIG. 3 is performed on the recording image data and recorded. Details of moving image shooting when the cross filter mode is set will be described later with reference to FIG.

  On the other hand, if the result of determination in step S13 is that the moving image shooting flag 1, that is, moving image shooting is in progress, the moving image shooting flag is set to 0 and the still image shooting flag is set to x (S19). Since it is detected in step S11 that the moving image button 117d has been pressed during moving image shooting, flag processing for stopping moving image shooting is performed in this step. When this flag is set, the movie shooting is stopped (S21). Here, processing for ending the recording operation of moving image data is performed.

  When shooting of a moving image is started in step S17 or when shooting of a moving image is stopped in step S21, it is next determined whether or not shooting of a still image has ended (S23). When the 2nd release is turned on in step S27 to be described later, still image shooting starts. When still image data is recorded in the external memory 114, still image shooting ends. Therefore, in this step, determination is made based on whether or not a series of these operations are being performed.

  If the result of determination in step S23 is that still image shooting has been completed, then the moving image shooting flag is set to x and the still image shooting flag is set to 0 (S25). Once this flag is set, it is next determined whether or not the 2nd release is on (S27). Here, it is determined whether or not the release button 117b of the operation unit 117 is fully pressed.

  If the result of determination in step S27 is that the 2nd release is on, then the moving image shooting flag is set to x and the still image shooting flag is set to 1 (S29). Then, still image shooting is executed (S31). Here, the image data acquired by the image sensor 107 is subjected to still image processing by the image processing unit 111 and then recorded in the external memory 114.

  If still image shooting has been carried out, it is next determined whether or not the moving image shooting flag is 0 and the still image shooting flag is 0 (S33). Here, it is determined whether it is not during moving image shooting or still image shooting. If the result of this determination is Yes, that is, if moving image shooting has ended and still image shooting has also ended, the process returns to step S1.

  If the result of determination in step S33 is No, that is, if moving image shooting or still image shooting is in progress, then the moving image shooting flag is 1 and the still image shooting flag is 0. It is determined whether or not (S35). Here, it is determined whether a moving image is being shot but a still image is not being shot.

  If the result of determination in step S35 is Yes, that is, if still image shooting has been completed and moving image shooting has been performed, the flow returns to step S3 to continue moving image shooting. On the other hand, if the result of determination in step S35 is No, that is, if moving image shooting has ended, but still image shooting has not ended, the process returns to step S11 to continue still image shooting.

  Thus, in the main flow in the present embodiment, live view display is performed, and moving image shooting is started and stopped alternately each time the moving image button 117d is operated, and in response to the release button 117b being fully pressed. To execute still image shooting. When the cross filter mode is set, the cross filter process is performed on the effect imparting area during live view display or moving image shooting.

  Next, details of the live view operation in step S1 will be described with reference to FIG. When the live view operation is started, it is determined whether or not the cross filter mode is set (S41).

  If the result of determination in step S41 is that the cross filter mode has been set, then a random number is generated (S43), and an effect imparting region is determined (S45). Here, as described with reference to FIG. 3, the pixels of the image sensor 107 are divided into 4 vertical areas and 6 horizontal areas, for a total of 24 areas, and an area is randomly selected from these areas, and based on the selected area. To determine the effect application region (bright spot detection region).

  First, a random number is generated, and an area determined by the random number and a total of four areas on the right side, lower side, and lower right side of the area are set as an effect imparting area. In the present embodiment, four areas are used as the effect imparting areas (bright spot detection areas), but the number of areas depends on the number of bright spot detection areas, the number of pixels in the bright spot detection areas, and the like. The number may be more or less than four.

In step S43, a random number is generated. In the present embodiment, the random number (RND) generates a value from 0.000000 to 0.9999999. In the example shown in FIG. 3, since it is divided into 4 vertical and 6 horizontal,
x = INT (RND × 6 + 1), y = INT (RND × 4 + 1)
Is defined as the position of the area, and the area next to the right side, the lower side, and the lower right side of the area may be defined as the bright spot detection region. Here, INT means an integer part in parentheses.

  Once the random number generation and the effect imparting area are determined, a cross filter process is performed (S47). Here, a cross-filter drawing process for detecting a bright bright spot portion in the effect applying region determined in step S45 and giving the same effect as the optical cross filter to the detected bright spot portion. Apply. Details of the cross filter processing will be described later with reference to FIGS.

  If the cross filter process is performed in step S47 or if the result of determination in step S41 is that the cross filter mode has not been set, then display is performed (S51). Here, live view display is performed on the display unit 115 based on image data from the image sensor 107. When the cross filter mode is set, live view display with cross filter processing is performed.

  Once the display has been carried out, it is next determined whether or not the acquisition of the next frame has been completed (S53). It is determined whether the image sensor 107 has acquired image data of the next frame. When the frame rate is 30 fps, frames are acquired once every 1/30 seconds. If the result of this determination is that the acquisition of the next frame has not been completed, the process returns to step S51. On the other hand, if acquisition has been completed, the process returns to step S41.

  Next, the cross filter processing in step S47 will be described with reference to FIG. When the cross filter process is started, a saturation point detection process is first performed (S61). Here, it is determined whether or not the RGB image data of all the pixels in the effect applying region determined in step S45 is equal to or higher than the luminance value corresponding to the saturation value, and a bright spot equal to or higher than the saturation value is detected. Details of the saturation point detection processing will be described later with reference to FIG.

  Once the saturation point detection process is performed, a cross filter drawing process is then performed (S63). Here, the bright spot detected in step S61 is subjected to image processing that produces the same effect as when the optical cross filter is mounted. Details of the cross filter drawing processing will be described later with reference to FIG.

  Next, the saturation point detection process in step S61 will be described with reference to FIG. If the flow of the saturation point detection process is entered, first, the flag f indicating the saturation point is reset to 0 (S71). Subsequently, coordinates (x, y) indicating the position of the pixel are initialized to x = 1 and y = 1 (S73). Note that the coordinates of the pixel have the position of (1, 1) at the upper left corner or the like in the set effect applying region.

  Subsequently, it is determined whether or not the R data at the position (x, y) is larger than 255 (S75). In the present embodiment, each RGB data constituting the image data is composed of 8 bits, and the maximum value is represented by 255. Therefore, the numerical value 255 may be determined as appropriate based on the number of bits of the image data, and may be a value smaller than the maximum value determined by the number of bits as long as it can be regarded as a saturated state.

  If the result of determination in step S75 is R (x, y) ≧ 255, then (R, G, B, f) (x, y) = (R, G, B, 1) (S77). Here, among the image data at the position (x, y), the R data, the G data, and the B data are kept at the acquired values, and the flag f indicating the saturation point is set to 1. By setting the flag f = 1, the image data of the pixel at the position (x, y) is saturated and indicates a bright spot.

  If (R, G, B, f) (x, y) = (R, G, B, 1) in step S77, or if R (x, y) <255 as a result of determination in step S75 Next, it is determined whether or not G (x, y) ≧ 255 (S79). Here, it is determined whether or not the G data at the position (x, y) is larger than 255.

  As a result of the determination in step S79, if G (x, y) ≧ 255, as in step S77, (R, G, B, f) (x, y) = (R, G, B, 1) (S81). If this processing is performed or if the result of determination in step S79 is G (x, y) <255, it is next determined whether or not B (x, y) ≧ 255 (S83). ). Here, it is determined whether or not the B data at the position (x, y) is larger than 255.

  If the result of determination in step S83 is B (x, y) ≧ 255, as in step S77, (R, G, B, f) (x, y) = (R, G, B, 1) (S85). If this process is performed or if the result of determination in step S83 is B (x, y) <255, then x indicating the pixel position is incremented, that is, 1 is added to x ( S87).

  If x is incremented in step S87, it is next determined whether or not x> maximum coordinates (S89). As described above, the effect imparting region is determined in step S45, and it is determined whether or not the x coordinate in the determined effect imparting region has reached the maximum coordinate. As a result of this determination, if x has not reached the maximum coordinate, the process returns to step S75, and it is detected whether any of the RGB data is saturated for the pixel at the position of the adjacent x coordinate.

  If the result of determination in step S89 is that x has reached the maximum coordinate, then y representing the pixel position is incremented, that is, 1 is added to y (S91). Subsequently, 1 representing x representing the pixel position is set (S93).

  Next, it is determined whether or not y> maximum coordinates (S95). It is determined whether or not the y coordinate within the effect applying region determined in step S45 has reached the maximum coordinate. If the result of this determination is that y has not reached the maximum coordinate, the process returns to step S75, and for any pixel at the position of the adjacent y coordinate, is any of the RGB data sequentially saturated from x = 1? To detect. On the other hand, when y reaches the maximum coordinate as a result of the determination, the flow returns to the original flow.

  Next, the cross filter drawing process in step S63 will be described with reference to FIG. When the flow of the cross filter drawing process is entered, first, similarly to step S73, the pixel position (x, y) of the image data is initialized to x = 1 and y = 1 (S101).

  Once the pixel position is initialized, it is next determined whether or not (R, G, B, f) f = 1 (S103). In step S77, S81, or S85, if at least one of the RGB image data is saturated, 1 is set to the saturation flag f. In this step, it is determined whether or not 1 is set in the saturation flag f for the image data at the pixel position (x, y).

  If the result of determination in step S103 is that the saturation flag 1 has been set to 1, next, a cross filter pattern of the color (R, G, B) is drawn (S105). Here, image processing is performed in which a light stripe is applied around the pixel position (x, y) where the saturation flag 1 has been set.

  If the cross filter pattern is drawn or if the result of determination in step S103 is that f of (R, G, B, f) is not 1, then the pixel position is changed as in step S87. Indicate x is incremented (S107). Once x has been incremented, it is next determined whether or not x> maximum coordinates, as in step S89 (S109). As a result of this determination, if x> the maximum coordinate is not satisfied, the process returns to step S103, and the above-described processing is repeated for the image data of the pixel at the coordinate position of the incremented x.

  On the other hand, if the result of determination in step S109 is that x> maximum coordinates, y representing the pixel position is incremented, as in step S91 (S111). Once y is incremented, 1 is set to x representing the position of the pixel, similarly to step S93 (S113).

  Subsequently, similarly to step S95, it is determined whether or not y> maximum coordinates (S115). As a result of this determination, if y> maximum coordinates are not satisfied, the process returns to step S103, and the above-described processing is repeated sequentially from x = 1 for the incremented y coordinates. On the other hand, if y> maximum coordinates as a result of the determination in step S115, the process returns to the original flow.

  As described above, in the flow of the live view operation in the present embodiment, when the cross filter mode is set, the effect applying region is determined every time the frame image for performing the live view display is acquired, and the determined effect applying region is determined. Bright spot detection (saturation point detection) is performed, and image processing (see S105) is performed that gives the same effect as the cross filter is attached to the detected bright spot (saturation point). The image subjected to the cross filter processing is displayed on the display unit 115 as a live view display. For this reason, the photographer can easily confirm the image on which the cross filter process has been performed.

  Next, a modified example of the live view operation in the present embodiment will be described with reference to FIG. In the live view operation according to the embodiment of the present invention described with reference to FIGS. 5 to 8, a saturation point is detected every time an image of one frame is acquired, and the image data of the detected saturation point is detected. A cross filter process for expressing the light stripes was performed. If the area for performing the cross filter processing is widened, the time required for the processing becomes long, and it becomes difficult to process every frame. Therefore, in this modification, cross filter processing is performed every several frames to shorten the processing time.

  The flow of the live view operation according to this modification example is only described with reference to steps S48 to S50, S55, and S57, as compared with the live view operation of FIG.

  When the live view operation flow of this modification is entered and the cross filter mode is set (S41 Yes), a random number is generated (S43), and an effect imparting region is determined (S43). Subsequently, cross filter image generation processing is performed (S48). Now, assuming that the acquired frame image 203 for live view display (see FIG. 14A), in step S48, an image composition cross-filter image 201 is generated as a separate image from the frame image 203. The image composition cross-filter image 201 is an image in which the light stripe 201a is generated in the effect applying area determined in step S45 based on the live view display frame image, and only the light stripe 201a is extracted. .

  If a cross filter image is generated in step S48, next, N is set as an effect applying flag (S49). The effect imparting flag is a natural number N indicating how many frames the effect of the cross filter is imparted. Cross filter processing is executed on the Nth acquired image.

  If the effect imparting flag is set, next, a cross filter image synthesis process is performed (S50). Here, as shown in FIG. 14B, the live view display frame image 203 is combined with the image combining cross-filter image 201 generated in step S48 to generate a display combined image 205.

  Once the cross filter image composition processing has been performed, display is then performed (S51), and it is determined whether acquisition of the next frame image has been completed (S53). If the result of this determination is that acquisition of the next frame image has not been completed, processing returns to step S51.

  On the other hand, if the result of determination in step S53 is that the acquisition of the next frame image has been completed, 1 is subtracted from the effect flag, and this subtraction value is used as the effect imparting flag (S55). Subsequently, it is determined whether or not this effect imparting flag is 0 (S57). If the result of this determination is that the effect flag is not 0, processing returns to step S50. Therefore, until the effect imparting flag becomes 0, the same cross filter image is synthesized with the newly acquired frame image for live view display without generating a new cross filter image in step S48, thereby combining the images for display. Generate an image and display it in live view.

  As a result of the determination in step S57, when the effect imparting flag becomes 0, the process returns to step S41. Therefore, if the cross filter mode is set, a new cross filter image for image synthesis is generated, and the above-described processing is repeated.

  As described above, in this modification, an image composition cross-filter image is generated for each predetermined frame (here, N frames), and even if the cross-filter processing takes as long as N frames, the live view There is no need to reduce the frame rate.

  Next, details of cross-filter moving image shooting when the cross-filter mode is set in moving image shooting starting in step S17 and stopping in step S21 will be described with reference to FIG. If the flow for cross-filter moving image shooting is entered, it is first determined whether or not the cross-filter mode is set (S121) as in step S41.

  If the result of determination in step S121 is that the cross filter mode has been set, processing similar to that in steps S43 to S57 in FIG. 9 is performed. That is, a random number is generated as in step S43 (S123), and an effect imparting region is determined based on the random number as in step S45 (S125).

  Once the effect imparting region is determined, next, a cross filter image generation process is performed as in step S48 (S127). Here, as shown in FIG. 14A, the image synthesis cross-filter image 201 is generated as a separate image from the frame image 203. However, during live view operation, image data having the number of pixels for live view display is used. However, when shooting a moving image, the frame image 203 having the number of pixels to be recorded in the external memory 114 and the image synthesis cross filter are used. The image 201 is processed.

  Once the cross filter image for image synthesis is generated, next, N is set in the effect imparting flag as in step S49 (S129), and cross filter image synthesis processing is performed as in step S50 (S131).

  If the result of determination in step S121 is that the cross filter mode has not been set, or if cross filter image composition processing has been performed in step S131, then display is performed (S141). Here, the image data for recording during moving image shooting subjected to the cross filter process is converted into display image data and displayed on the display unit 115. In parallel with this display, the cross-filtered image data is compressed and then recorded in the external memory 114. If the cross filter mode is not set, normal captured image display and recording are performed.

  Once the display has been carried out, next, as in step S53, it is determined whether or not the acquisition of the next frame has been completed (S143). If the result of this determination is that acquisition has not been completed, processing returns to step S141. On the other hand, when acquisition of the next frame is completed, next, as in step S55, 1 is subtracted from the effect flag, and this subtraction value is used as the effect imparting flag (S145). It is determined whether or not (S147). If this effect flag is not 0, the process returns to step S131. Therefore, until the effect imparting flag becomes 0, the same cross filter image is synthesized with the frame image of the newly acquired recording image without generating a new cross filter image in step S127, and for recording. A composite image is generated and recorded in the external memory 114. In addition, the captured image subjected to the cross filter process is converted for display and displayed on the display unit 115.

  On the other hand, if the result of the determination in step S147 is 0, the process returns to step S121. For this reason, if the cross filter mode is set, a new cross filter image for image synthesis is generated, and shooting of a moving image is continued.

  As described above, in the cross-filter moving image shooting in the present embodiment, the moving image subjected to the cross-filter process can be displayed and recorded by performing substantially the same processing as the live view operation described with reference to FIG. . In particular, in the case of moving image shooting, the amount of image data to be recorded becomes large and the processing time also takes. Therefore, the processing time can be shortened by performing the cross filter processing for each predetermined frame.

  Next, a first modification of cross-filter moving image shooting in the present embodiment will be described with reference to FIGS. In the present embodiment, the cross filter processing at the time of cross filter moving image shooting is image processing in which uniform light stripes are added. In this modification, a random number is generated in the cross filter process, and a random change is given by this random number.

  The cross-filter moving image shooting flow in the present modification shown in FIG. 11 is obtained by adding step S126 to the cross-filter moving image shooting flow in the embodiment of the present invention shown in FIG. Is the same as in the embodiment. In step S126, the effect of the cross filter process for the effect applying region determined in step S125 is set. Details of the effect setting will be described with reference to FIG.

  If the production effect setting flow is entered, first, a random number is generated (S151), and then the number of light stripes is set according to the generated random number (S153). As the number of light stripes, for example, a number close to the generated random number, such as 4, 6, 8, etc., is set.

  When the number of light stripes is set, a random number is generated again (S155), and then the length of the light stripe is set (S157). As the length of the light stripe, for example, a length close to the generated random number is set among short (= 4), middle (= 6), and long (= 8). The numbers in parentheses are random numbers.

  When the length of the light stripe is set, a random number is generated again (S159), and then the light stripe angle is set (S161). As the light stripe angle, for example, an angle close to the generated random number is set among 0 degrees (= 4), 30 degrees (= 6), and 45 degrees (= 8). The numbers in parentheses are random numbers.

  When the light stripe angle is set, a random number is generated again and the light stripe color is set (S165). As the light stripe color, for example, a color close to the generated random number is set among red (= 4), green (= 6), and blue (= 8). The numbers in parentheses are random numbers. When the light color is set, the original flow is restored.

  As described above, in this modification, the number of light stripes, the length, the angle, the color, and the like are changed every time the cross filter process is performed. For this reason, a variety of cross filter processes are performed, and the image becomes intriguing. In this modification, a random number is generated and an effect is set according to the random number. However, the present invention is not limited to this, and it is of course possible to give a predetermined programmed change over several frames. Absent. Further, the change of the light stripe is not limited to that illustrated, and may be a part thereof.

  Next, a second modification of cross-filter moving image shooting in the present embodiment will be described with reference to FIGS. In the embodiment and the first modification, the image synthesis cross-filter image is generated at the effect imparting flag N frame interval. This process is a cross filter process most suitable for motion JPEG. However, when MPEG is employed, the effect of the cross filter process may be applied only to key frames. Since the key frame usually appears every 2 seconds, a sufficient processing time can be taken. Therefore, the effect imparting region can be made wider than in the above-described embodiments and modifications.

  In this modification, in the flowchart of the first modification shown in FIG. 11, the setting of the effect applying flag in step S129 and the step of subtracting 1 from the effect applying flag in step S145 are omitted, and the determination in step S147 is a key. This is the same except that the determination is made as to whether or not the frame. Steps that perform the same processing as in the flow of FIG. 11 are assigned the same step numbers, and differences will be mainly described.

  If the cross filter mode is set as a result of the determination in step S121, a random number is generated (S123), an effect imparting area is determined according to the random number (125), and an effect is set (S126). ), A cross filter image generation process is performed in accordance with the set effect (S127). Here, as shown in FIG. 15A, an image composition cross-filter image 201 used for cross-filter processing is generated separately from the frame image 203.

  When the cross filter image generation process is performed in step S127, the effect imparting flag is set next in the first modification, but this setting is omitted because the effect imparting flag is not used in the present modification. Next, cross filter image synthesis processing is performed (S131). Here, as shown in FIG. 15B, the image synthesis cross-filter image 201 is synthesized with the frame image 203 to generate a synthesized image 205.

  Once the cross filter image composition processing has been performed, display and recording are then performed (S141), and it is determined whether or not acquisition of the next frame has been completed (S143). If the result of this determination is that acquisition of the next frame has not been completed, processing returns to step S141 and display continues.

  On the other hand, when the acquisition of the next frame is completed as a result of the determination in step S143, it is determined whether or not it is a key frame (S148). As described above, in MPEG, key frames appear at predetermined time intervals. In this step, it is determined whether or not the acquired frame is a key frame.

  If the result of determination in step S148 is not a key frame, processing returns to step S141, and the same cross-filter image is generated in step S127 without newly generating a cross-filter image. A composite image for recording is generated by combining with the image and recorded in the external memory 114. In addition, the captured image subjected to the cross filter process is converted for display and displayed on the display unit 115.

  On the other hand, if the result of determination in step S148 is a key frame, processing returns to step S121. Returning to step S121, if the cross filter mode is set, a new cross filter image for image synthesis is generated, and shooting of a moving image is continued.

  In this modified example, as shown in FIG. 15C, key frames appear at predetermined time intervals, and the composite image 205 is displayed / recorded only on the key frames, and only the frame image 203 is displayed / recorded during that time. To be recorded.

  Thus, in the second modified example of the present invention, the key frame is skillfully used, and the cross filter process is performed on the key frame. For this reason, management of the effect imparting flag becomes unnecessary.

  As described above, in one embodiment of the present invention and its modification, a bright spot detection area is set for the image data from the image sensor 107 (S45), and the bright spots in the bright spot detection area are set. (S61), the bright spot processing is performed at the bright spot position of the detected bright spot (S63), and the bright spot processed image is displayed or recorded (S51, S141). In this way, the bright spot processing is not performed on the entire imaging region of the image sensor 107, but the bright spot processing is performed on the bright spots in the bright spot detection area. For this reason, it is possible to perform cross-filter processing suitable for live view display and moving images.

  In the embodiment and the modification of the present invention, when setting the effect imparting region (bright spot detection region), the imaging region is divided into 24 areas, and 4 areas are set out of them. The number of divisions of the imaging region and the number of areas of the effect applying region may be changed as appropriate. What is necessary is just to set an effect provision area | region more than the minimum size vertically and horizontally. Further, the size of the area is constant, but it may be changed for each frame. In one embodiment or modification of the present invention, the effect imparting area (bright spot detection area) is set randomly by generating a random number. However, the bright spot area may be selected in order. Absent.

  In one embodiment or modification of the present invention, the number of bright spots to be processed in the set effect application area (bright spot detection area) is not limited, but the number of bright spots to be processed is limited. It may be. By limiting the number of bright spots, it is possible to prevent a long processing time.

  In one embodiment of the present invention, a cross filter process for adding a light streak to the center of high luminance is performed as the bright spot process. However, when a bright spot is detected, in addition to the cross filter processing, other bright spot processing such as saturation (white) around the bright spot or saturation in a star shape may be performed. Absent.

  In the embodiment and the modification of the present invention, the digital camera is used as the photographing device. However, the camera may be a digital single-lens reflex camera or a compact digital camera, such as a video camera or movie camera. It may be a camera for moving images such as a mobile phone, a personal digital assistant (PDA), a game machine, or the like.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 10 ... Camera body, 20 ... Interchangeable lens, 100 ... Digital camera, 101 ... Shooting lens, 102 ... Lens control part, 103 ... Aperture mechanism, 104 ... Aperture control part , 105 ... shutter, 106 ... shutter control unit, 107 ... imaging device, 108 ... imaging control unit, 109 ... A / D conversion unit, 110 ... memory, 111 ... Image processing unit 112... Exposure control unit 113... AF processing unit 114... External memory 115... Display unit 115 a. ... Operation unit, 117a ... Power button, 117b ... Release button, 117c ... Shooting mode dial, 117d ... Movie button, 117e ... Function button, 117f ... Cross button, 117g ..OK button, 118 ... nonvolatile memory, 119 ... power supply unit, 120 ... power supply control unit, 121 ... flash control unit, 122 ... flash charging unit, 123 ... flash emission , 201... Cross-filter image for image synthesis, 201a... Light strip, 203... Frame image, 205.

Claims (8)

An imaging unit that converts a subject image into an electrical signal and outputs image data in a live view;
A bright spot area setting unit that divides the entire image data into a plurality of areas once per predetermined frame for the image data, and sets at least one area as a bright spot detection area;
A bright spot detection unit for detecting a bright spot in the bright spot detection area set by the bright spot area setting unit;
A bright spot processing unit that performs bright spot processing at the bright spot position of the bright spot detected by the bright spot detection unit;
An image output unit for outputting the image subjected to the bright spot processing by the bright spot processing unit for display or recording;
When the bright spot processing unit performs the bright spot processing at the position where the bright spot detection unit detects the bright spot for all areas divided into a plurality of areas by the bright spot area setting unit Includes a control unit that repeats the bright spot processing, and
An imaging device comprising:   The imaging apparatus according to claim 1, wherein the bright spot area setting unit sets the bright spot detection area at least in a vertical and horizontal size.   The imaging apparatus according to claim 1, wherein the bright spot area setting unit sequentially sets the bright spot detection areas in a random order.   The imaging apparatus according to claim 1, wherein the bright spot processing in the bright spot processing unit is a cross filter process. The bright point processing unit, characterized by varying the time of the bright point processing successive frames, at least a portion of the brightness, the length of the striation, the number of striation, the angle of the striation, one color The imaging apparatus according to claim 1.   The imaging apparatus according to claim 1, wherein the bright spot processing unit limits the number of bright spots to be processed within a set bright spot detection area. A first output step of converting a subject image into an electrical signal and outputting image data in a live view;
A setting step for dividing the entire image data into a plurality of areas once per predetermined frame for the image data, and setting at least one area as a bright spot detection area;
A detection step for detecting a bright spot in the set bright spot detection area;
A processing step for performing bright spot processing at the bright spot position of the detected bright spot;
When the bright spot processing is performed in the processing step at the position where the bright spot is detected in the detection step for all areas divided into a plurality of areas in the setting step, the bright spot processing is performed. A control step that repeats
A second output step for outputting the bright spot processed image for display or recording;
An imaging method characterized by comprising: A first output step of converting a subject image into an electrical signal and outputting image data in a live view;
A setting step for dividing the entire image data into a plurality of areas once per predetermined frame for the image data, and setting at least one area as a bright spot detection area;
A detection step for detecting a bright spot in the set bright spot detection area;
A processing step for performing bright spot processing at the bright spot position of the detected bright spot;
When the bright spot processing is performed in the processing step at the position where the bright spot is detected in the detection step for all areas divided into a plurality of areas in the setting step, the bright spot processing is performed. A control step that repeats
A second output step for outputting the bright spot processed image for display or recording;
A program that causes a computer to execute.