JP2016220023A - Image processing apparatus, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device which allows for easy adjustment of dynamic range using a histogram expressed in a logarithmic axis.SOLUTION: In an image processing apparatus, an image captured by an image pickup device is read (S301), histogram of the image thus read is calculated (S302), and the histogram calculated, with the input axis as a logarithmic axis, is displayed in a display device. When the fact that the adjustment of dynamic range for an image is started by a user is detected (YES at S303), a dynamic range adjustment value specified by a user is acquired (S304), and a histogram where a specific range including the dynamic range adjustment value is resized, based on the dynamic range adjustment value thus acquired, is displayed (S307, S308).SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for performing image processing on image data of an image captured by an imaging apparatus such as a digital camera or a digital video camera.

  An EV value (Exposure Value) is generally used as a numerical value representing exposure (brightness) when imaging is performed with an imaging apparatus such as a digital camera. The EV value is a value at which the exposure is doubled at +1 and the exposure is halved at -1, and is expressed by the following formula 1 using the aperture value F and the shutter speed T.

  On the other hand, as an image processing apparatus realized by a CPU (Central Processing Unit) such as a computer executing image editing software, it has a function of displaying an exposure distribution of an image in a histogram, and the input axis of the histogram has a radix of 2 Some use the logarithmic axis. This is because the exposure changes by a power of 2 due to the change in EV value. Therefore, if the logarithmic axis having a radix of 2 is taken as the input axis, the exposure distribution moves in parallel with the change in the EV value. This is because the change is easily visually understood by the user.

  However, the histogram representing the exposure distribution often has a function of adjusting the dynamic range of the image in addition to the display of the exposure distribution. In that case, in the histogram represented by the logarithmic axis, a large difference is generated in the numerical value represented by one scale. Therefore, the adjustment accuracy at the time of adjusting the dynamic range differs for each pixel value, and adjustment is difficult.

  On the other hand, when the user is operating a scaled axis, the technology makes it easier to operate by enlarging the scale of the axis that the user is operating to increase the display accuracy of the scale. Has been proposed (see Patent Document 1).

JP 2009-205462 A

  However, since the technique disclosed in Patent Document 1 only enlarges the scale of the axis, it is not suitable for an operation performed while viewing a graph such as dynamic range adjustment using a histogram. In addition, since only the scale of the axis is enlarged, it is not suitable for an operation of reducing the accuracy so as to obtain an appropriate accuracy for a portion where the accuracy is too high.

  An object of the present invention is to provide an image processing apparatus that can easily adjust a dynamic range using a histogram represented by a logarithmic axis.

  An image processing apparatus according to the present invention is an image processing apparatus that performs image processing on an image picked up by an image sensor, wherein the calculation means calculates a histogram of the image, and the calculation means uses an input axis as a logarithmic axis. Based on the dynamic range adjustment value acquired by the display means for displaying the calculated histogram, the adjustment value acquisition means for acquiring the dynamic range adjustment value for adjusting the dynamic range of the image, and the adjustment value acquisition means, Display control means for displaying a histogram obtained by resizing the specific range including the dynamic range adjustment value on the display means.

  According to the present invention, the dynamic range can be easily adjusted using the histogram represented by the logarithmic axis by resizing and displaying the axis and the graph according to the size of the display area.

1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to an embodiment of the present invention. It is a block diagram which shows the functional block which concerns on 1st Embodiment of the image processing apparatus of FIG. It is a flowchart of the image processing by an image processing apparatus provided with the functional block shown in FIG. It is a figure which shows the example of a histogram display in step S307 of FIG. It is a figure which shows the example of a histogram display in step S308 of FIG. It is a block diagram which shows the functional block which concerns on 2nd Embodiment of the image processing apparatus of FIG. 7 is a flowchart of image processing performed by an image processing apparatus including the functional block illustrated in FIG. 6. It is a figure which shows the example of a histogram display in step S705 of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<Configuration of image processing apparatus>
FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus 100 according to an embodiment of the present invention. The image processing apparatus 100 includes a CPU 101, a display device 102, a memory 103, a storage device 104, and an input device 105. A specific example of the image processing apparatus 100 is a personal computer, but is not limited thereto, and may be an imaging apparatus such as a digital camera having an imaging unit using an imaging element and a display unit such as a liquid crystal panel. .

  The CPU 101 develops an operating system (OS) and an image processing program stored in the storage device 104 in the work area of the memory 103, thereby enabling the processing to be executed in each device and the entire image processing device 100. The general behavior. The display device 102 is a liquid crystal display, an organic EL display, or the like, and displays an image, a user interface, and a histogram.

  For the storage device 104, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like is used. The storage device 104 stores RAW image data and image processing parameters. In the present embodiment, it is assumed that the image data stored in the storage device 104 is image data of a captured image captured using an imaging device such as a digital camera that captures an image using a solid-state imaging device.

  The memory 103 is composed of, for example, a ROM or a RAM. Note that the OS and the image processing program may be stored in a nonvolatile memory such as a ROM constituting the memory 103 instead of the storage device 104. An operation by the user of the image processing apparatus 100 is performed through the input device 105. The input device 105 includes, for example, a keyboard, a mouse, a touch panel, and the like.

<First Embodiment>
FIG. 2 is a block diagram illustrating functional blocks according to the first embodiment of the image processing apparatus 100. In the first embodiment, the image processing apparatus 100 includes an image reading unit 200, a histogram calculation unit 201, a dynamic range adjustment detection unit 202, a dynamic range adjustment value acquisition unit 203, a display area size acquisition unit 204, and a histogram display unit 205. . The functions of these units are realized when the CPU 101 executes a predetermined program stored in the storage device 104 and each device included in the image processing apparatus 100 performs a predetermined operation based on a command from the CPU 101.

  The image reading unit 200 reads image data from the storage device 104 and writes it to the memory 103. In the present embodiment, it is assumed that image data of an image to be adjusted for dynamic range is read from the storage device 104. Further, it is assumed that the read image data is data with 8-bit accuracy (pixel value 0 to 255).

  The histogram calculation unit 201 calculates an image histogram based on the image data read to the memory 103. The dynamic range adjustment detection unit 202 detects that the user has started adjusting the dynamic range through the input device 105. The dynamic range adjustment detection unit 202 detects that the user has instructed the end of adjustment of the dynamic range through the input device 105. The dynamic range adjustment value acquisition unit 203 acquires a dynamic range value (dynamic range adjustment value) adjusted by the user through the input device 105.

  The display area size acquisition unit 204 acquires a display area size that is a size of an area in which the display device 102 can display a histogram. For example, when software for image editing is started up on a computer, the display device 102 displays a UI such as a menu or icon, and a window indicating a work area in which an image or the like can be displayed. It is well known that the size of the window can be matched to the entire display area (maximum size) of the display device 102 or limited to a part. The display area size of the histogram in the present embodiment refers to the size of the work area in the window displayed on the whole or a part of the display area of the display device 102. Therefore, when the size of the window is changed, the display area size of the histogram also changes with the change of the window size.

  In the following description, for the sake of convenience, it is assumed that the window size when the image editing software is started up is matched with the maximum size of the display area of the display device 102. Therefore, the display area size of the histogram is also the maximum size that can be displayed on the display device 102.

  The histogram display unit 205 is display control means for controlling the display form of the histogram on the display device 102 according to the situation. Specifically, the histogram display unit 205 displays a histogram (graph) on the display device 102 with a logarithmic axis having a radix of 2 as an input axis of pixel values. At that time, the histogram display unit 205 can perform a multiple graph display in which the entire histogram and a specific range of the histogram are displayed on the display device 102 as separate graphs. In addition, the histogram display unit 205 can perform single graph display that displays a histogram in which the scale ratio of a specific range is changed from the scale ratio of another range in the entire histogram, instead of the multiple graph display.

  In the image processing apparatus 100, the user can select through the input device 105 whether to display a plurality of graphs or a single graph. As will be described later, during dynamic range adjustment by the user, the CPU 101 automatically selects one of a plurality of graph displays and a single graph display according to the display area size.

  FIG. 3 is a flowchart of image processing for dynamic range adjustment by the image processing apparatus 100 including the functional blocks shown in FIG. The storage device 104 or the like stores a predetermined program describing the procedure of the image processing method shown in the flowchart of FIG. Each process shown in FIG. 3 is realized by the CPU 101 developing the predetermined program in the memory 103, and the CPU 101 controlling the operation of each apparatus constituting the image processing apparatus 100 and the process of each functional block.

  First, it is assumed that the user operates the image processing apparatus 100 and starts an image editing program (software). It is assumed that a screen (UI) capable of selecting an image such as a list of reduced images (thumbnail) of image data stored in the storage device 104 is displayed on the operation screen of the image editing program. However, the present invention is not limited to this, and it is sufficient that an environment capable of specifying image data and starting an image editing program is prepared. For example, there is a case where a window of a folder storing image data is displayed on the display device 102, and a program associated with an image is automatically started when an image in the window is selected. .

  When the user operates the input device 105 to select an image to be adjusted for dynamic range, the image reading unit 200 reads the image data of the selected image from the storage device 104 and stores it in the memory 103 in step S301. Further, the image of the selected image data is displayed on the display device 102 by the process of step S301.

  In subsequent step S <b> 302, the histogram calculation unit 201 calculates a histogram of the image data stored in the memory 103 and stores it in the memory 103. In step S <b> 302, the calculated histogram is displayed on the display device 102. Note that the display of the histogram on the display device 102 may be performed when the determination of the process in the next step S303 is YES. In step S <b> 302, for example, four histograms, that is, an RGB histogram and R, G, and B histograms are calculated and displayed on the display device 102.

  Thereafter, in step S303, the dynamic range adjustment detection unit 202 determines whether or not the start of dynamic range adjustment has been instructed. The user can issue a dynamic range adjustment start instruction from, for example, the UI menu of the activated image editing program. When the dynamic range adjustment detection unit 202 determines that the start of dynamic range adjustment is not instructed (NO in S303), the determination in step S303 is repeated at regular intervals. If the dynamic range adjustment detection unit 202 determines that an instruction to start dynamic range adjustment is given (YES in S303), the process proceeds to step S304.

  In step S <b> 304, the dynamic range adjustment value acquisition unit 203 acquires the dynamic range adjustment value indicated by the user through the input device 105 and stores it in the memory 103. In subsequent step S 305, the display area size acquisition unit 204 acquires the display area size and stores it in the memory 103. In step S306, the display area size acquisition unit 204 determines whether the display area size acquired in step S305 is equal to or smaller than a predetermined size. For example, 256 × 256 can be defined as a reference value (threshold value) in terms of resolution with a fixed size.

  If the display area size acquisition unit 204 determines that the display area size is larger than the certain size (NO in S306), the process proceeds to step S307. In step S <b> 307, the histogram display unit 205 performs a multiple graph display in which the display device 102 displays the histogram calculated in step S <b> 302 and another graph obtained by resizing the specific range before and after the dynamic range adjustment value acquired in step S <b> 304.

  FIG. 4 is a diagram showing a histogram display example in step S307. The left diagram in FIG. 4 shows a window 401 before starting dynamic range adjustment, displaying the histogram calculated in step S302. The window 401 is displayed in a part of the entire display area (not shown) of the display device 102.

  The right figure of FIG. 4 shows the window 402 at the time of dynamic range adjustment after the display conversion of the display of the window 401 is performed by the process of step S307. Here, a configuration in which another graph in which a range of ± 16 pixel values is expanded as a specific range before and after the dynamic range adjustment value is illustrated, and a pixel value 32 is specified as the dynamic range adjustment value. An example is shown.

  If the display area size acquisition unit 204 determines in step S306 that the display area size is equal to or smaller than a predetermined size (YES in S306), the process proceeds to step S308. In step S308, the histogram display unit 205 performs a single graph display in which the specific range before and after the dynamic range adjustment value acquired in step S304 is displayed as one histogram by changing the occupation ratio in the histogram.

  FIG. 5 is a diagram illustrating a histogram display example in step S308. The window 401 in the left diagram of FIG. 5 is the same as the window 401 in the left diagram of FIG. The right figure of FIG. 5 shows the window 501 after the display of the window 401 is converted by the process of step S308. Here, for comparison with step S307, a configuration in which a range of ± 16 pixel values is enlarged and displayed as a specific range before and after the dynamic range adjustment value is illustrated, and a pixel value 32 is designated as the dynamic range adjustment value. An example is shown.

  Here, the window 501 is displayed in the same size as the window 401. However, the present invention is not limited to this, and the image may be displayed in a size larger than the window 401 as much as possible. The size of the window 501 is inevitably smaller than the window 402 based on the determination result in step S306.

  This processing is terminated by the processing in steps S307 and S308. The user operates the enlarged portion of the histogram displayed in steps S307 and S308 through the input device 105 to adjust the dynamic range for the image read in step S301.

  A well-known technique can be used for the dynamic range adjustment method. For example, the user selects one histogram from among the RGB histogram and the R, G, B histograms. Then, the mouse pointer is aligned with the adjustment value in the enlarged portion, and the pointer is moved up and down with the pixel value designated by clicking the mouse. At this time, the CPU 101 executing the image editing program detects the movement of the pointer, performs display for changing the peak position and height of the entire histogram, and adjusts the dynamic range of the color of the image (not shown). Change in sync with. The user can adjust the dynamic range so that a desired image can be obtained while visually recognizing the entire histogram and image changes.

  As described above, in the present embodiment, when the user starts adjusting the dynamic range for a predetermined image, the display method is changed according to the display area size when the display size near the dynamic range adjustment value is changed. Switch. As a result, even when the dynamic range is adjusted using a histogram with the logarithmic axis as the input axis, the display area of the display device 102 is effectively used, and the influence of the accuracy difference due to the scale width is reduced and dynamic The range can be adjusted.

Second Embodiment
FIG. 6 is a block diagram illustrating functional blocks according to the second embodiment of the image processing apparatus 100. In the second embodiment, the image processing apparatus 100 includes an adjustment sensitivity changing unit 601 in addition to the units illustrated in FIG. The image reading unit 200, the histogram calculation unit 201, the dynamic range adjustment detection unit 202, the dynamic range adjustment value acquisition unit 203, the display area size acquisition unit 204, and the histogram display unit 205 illustrated in FIG. 6 are the same as those illustrated in FIG. It is. Therefore, these are denoted by the same reference numerals and description thereof is omitted here.

  The adjustment sensitivity changing unit 601 changes the operation sensitivity when the user designates the dynamic range adjustment value according to the dynamic range adjustment value designated by the user from the input device 105. The function of the adjustment sensitivity changing unit 601 is realized by the CPU 101 executing a predetermined program stored in the storage device 104.

  FIG. 7 is a flowchart of image processing performed by the image processing apparatus 100 including the functional blocks shown in FIG. The processing in steps S701 to S703 is the same as the processing in steps S301 to S303 described with reference to FIG. 3, and thus description thereof is omitted here.

  In step S <b> 704, the dynamic range adjustment value acquisition unit 203 acquires the dynamic range adjustment value at the start of the dynamic range adjustment specified by the user through the input device 105 and stores it in the memory 103. In subsequent step S705, the histogram display unit 205 displays a plurality of graphs. In the multiple graph display here, the histogram calculated in step S702 and another graph obtained by resizing the specific range before and after the dynamic range adjustment value acquired in step S704 are displayed on the display device 102.

  FIG. 8 is a diagram showing a histogram display example in step S705. The window 800 is displayed in a part of a display area (not shown) of the display device 102. A window similar to the window 401 of FIG. 4 is displayed on the display device 102 after the process of step S703 and before the process of step S704. For example, when a predetermined pixel value (here, pixel value 32) in the window 401 is designated by operating the mouse in step S704, the window 401 is switched to the window 800 by the process in step S705.

  The upper graph in the window 800 indicates that the pixel value 32 is designated as an adjustment value at the start of dynamic range adjustment in step S704. When the display is switched from the window 401 to the window 800, the pointer P of the mouse 810 moves to a position indicating the pixel value 32 in the lower graph.

  As in the first embodiment, the lower graph in the window 800 displays a graph in which a range of ± 16 pixel values is expanded as a specific range before and after the dynamic range adjustment value. Further, when the user moves the pointer of the mouse 810 to the upper graph of the window 800 and changes the designated pixel value, the pixel value range of the lower graph is changed and displayed so as to correspond to the designated pixel value. The In the window 800, pointers P of the mouse 810 are shown at a plurality of locations, but the pointers actually displayed after step S705 are only those pointing to the pixel value 32 in the lower graph (solid line display). is there.

  In step S <b> 706, the dynamic range adjustment value acquisition unit 203 acquires the current dynamic range adjustment value and stores it in the memory 103. The current dynamic range adjustment value means that the user operates the mouse 810 on the lower graph of the window 800 to move the pointer P from a position indicating the pixel value 32 to a position indicating the vicinity of the pixel value 24, for example. This indicates the value indicated by the subsequent pointer P.

  The dynamic range adjustment value at the start of dynamic range adjustment acquired in step S704 is “A”, and the current dynamic range adjustment value acquired in step S706 is “B”. Using these dynamic range adjustment values, in the next step S707, the adjustment sensitivity changing unit 601 changes the adjustment sensitivity according to the following equation 2.

  The lower graph of the window 800 shows a state in which the pointer P of the mouse 810 points to the vicinity of each of the pixel values 16, 24, 32, 36, and 42. These schematically show that the user has designated the dynamic range adjustment value by operating the mouse 810 and further moving the pointer P. It can be seen that the amount of movement of the mouse 810 is the same, but the amount of movement of the pointer P differs depending on the resolution of the pixel value (horizontal axis).

  For example, from the position where the pointer P is pointed to the pixel value 24 with a movement amount equal to the movement amount when the mouse 810 is moved so that the pointer P changes from the state indicating the pixel value 32 to the state indicating the pixel value 24. It can be moved to a position that points to the pixel value 16. At this time, the pointer P moves about 1/3 of the horizontal axis. On the other hand, the pointer P has a movement amount equal to the movement amount when the mouse 810 is moved from the state where the pointer P points to the pixel value 32 to the state where the pointer P points to the pixel value 24. Move to a position that points to pixel value 36. At this time, the pointer P moves about 1/5 of the horizontal axis. As described above, even when the movement amount of the mouse 810 is the same, the movement amount (operation sensitivity) of the pointer P is adjusted to be different according to the dynamic range adjustment value.

  As the dynamic range adjustment method using the lower graph of the window 800, the method described in the first embodiment can be used. In step S708, the dynamic range adjustment detection unit 202 determines whether the user has instructed the end of dynamic range adjustment. The dynamic range adjustment end instruction can be given, for example, from the UI menu of the activated image editing program. When there is no dynamic range adjustment end instruction (NO in S708), the dynamic range adjustment detection unit 202 returns the process to Step S706, and when it is determined that there is a dynamic range adjustment end instruction (S708: YES), this process End.

  As described above, according to the present embodiment, when the user starts adjustment of the dynamic range for a predetermined image, the display size near the dynamic range adjustment value is changed to display the dynamic range adjustment value. Change the adjustment sensitivity accordingly. Thereby, in the dynamic range adjustment using the histogram with the logarithmic axis as the input axis, the user can always perform the adjustment with the same feeling without worrying about the accuracy difference due to the scale width.

  In the present embodiment, the configuration in which the dynamic range adjustment value is designated by the mouse has been described. However, the present invention is not limited to this. For example, when the display device 102 is a touch panel, the specification of the dynamic range adjustment value and the dynamic range adjustment can be performed by a swipe operation or the like on the touch panel.

<Other embodiments>
Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. Furthermore, each embodiment mentioned above shows only one embodiment of this invention, and it is also possible to combine each embodiment suitably.

  For example, in the second embodiment, a plurality of graphs are displayed in step S705 without determining the display area size of the display device 102. However, the same processing as step S306 may be performed after step S704. Thereby, even when it changes to a single graph display, the process of step S706-S708 can be performed similarly to the case of a multiple graph display.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Image processing apparatus 101 CPU
DESCRIPTION OF SYMBOLS 102 Display apparatus 103 Memory 104 Storage apparatus 105 Input apparatus 200 Image reading part 201 Histogram calculation part 202 Dynamic range adjustment detection part 203 Dynamic range adjustment value acquisition part 204 Display area size acquisition part 205 Histogram display part

Claims (8)

An image processing apparatus that performs image processing on an image captured by an image sensor,
Calculating means for calculating a histogram of the image;
Display means for displaying a histogram calculated by the calculating means using the input axis as a logarithmic axis;
Adjustment value acquisition means for acquiring a dynamic range adjustment value for adjusting the dynamic range of the image;
An image processing apparatus comprising: display control means for displaying a histogram obtained by resizing a specific range including the dynamic range adjustment value on the display means based on the dynamic range adjustment value acquired by the adjustment value acquisition means. . Operation means operated by a user;
Detecting means for detecting that adjustment of a dynamic range for the image has been started by the user through the operation means,
The adjustment value acquisition unit acquires the value designated by the user through the operation unit as the dynamic range adjustment value when the detection unit detects that the adjustment of the dynamic range for the image is started. The image processing apparatus according to claim 1, wherein: The display control means includes
A plurality of graph displays for displaying the entire histogram of the image and the specific range on the display means in a separate graph; and a single graph display for expanding the specific range in the entire histogram and displaying it on the display means; The image processing apparatus according to claim 2, wherein the image processing apparatus is switched. A size acquisition means for acquiring a size of a display area capable of displaying the histogram in the display means;
The display control means performs the multiple graph display when the size of the display area acquired by the size acquisition means is smaller than a certain size, and displays when the size of the display area is larger than the certain size. The image processing apparatus according to claim 3, wherein a single graph display is performed. Sensitivity change means for changing operation sensitivity when the user designates the dynamic range adjustment value by the operation means,
3. The sensitivity changing unit changes the operation sensitivity according to a resolution of the histogram in a dynamic range adjustment value acquired by the adjustment value acquisition unit when adjusting a dynamic range for the image. 5. The image processing apparatus according to any one of 4 above. Storage means for storing an image captured by the image sensor;
Reading means for reading a predetermined image stored in the storage means,
The image processing apparatus according to claim 1, wherein the calculating unit calculates a histogram of the image read by the reading unit. An image processing method comprising:
A reading step of reading an image picked up by the image pickup device and stored in the storage means;
A calculation step of calculating a histogram of the image read in the reading step;
A display step of displaying on the display means the histogram calculated in the calculation step using the input axis as a logarithmic axis;
A detection step of detecting that adjustment of the dynamic range for the image is started by the user through the operation means;
An acquisition step of acquiring a dynamic range adjustment value designated by the user through the operation means;
An image processing method comprising: a display control step of displaying a histogram obtained by resizing a specific range including the dynamic range adjustment value on the display unit based on the dynamic range adjustment value acquired in the acquisition step.   A program for causing a computer to execute each step of the image processing method according to claim 7.