WO2016084253A1 - 画像処理装置及び画像処理方法 - Google Patents
画像処理装置及び画像処理方法 Download PDFInfo
- Publication number
- WO2016084253A1 WO2016084253A1 PCT/JP2014/081629 JP2014081629W WO2016084253A1 WO 2016084253 A1 WO2016084253 A1 WO 2016084253A1 JP 2014081629 W JP2014081629 W JP 2014081629W WO 2016084253 A1 WO2016084253 A1 WO 2016084253A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- gradation value
- gradation
- image
- value
- pixel
- Prior art date
Links
- 238000012545 processing Methods 0.000 title claims abstract description 142
- 238000003672 processing method Methods 0.000 title claims description 15
- 238000006243 chemical reaction Methods 0.000 claims abstract description 56
- 238000007781 pre-processing Methods 0.000 claims description 44
- 238000012805 post-processing Methods 0.000 claims description 23
- 238000000034 method Methods 0.000 description 28
- 230000008569 process Effects 0.000 description 21
- 230000002776 aggregation Effects 0.000 description 8
- 238000004220 aggregation Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000001771 impaired effect Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
- G09G3/22—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources
- G09G3/28—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using luminous gas-discharge panels, e.g. plasma panels
- G09G3/2803—Display of gradations
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
- G09G3/2092—Details of a display terminals using a flat panel, the details relating to the control arrangement of the display terminal and to the interfaces thereto
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
Definitions
- One aspect of the present invention relates to an image processing apparatus and an image processing method for converting an original image into an easily viewable image.
- Patent Document 1 it is easy to see a screen in which extremely bright portions and dark portions are mixed in the same screen in an image (still image or moving image frame) taken by a surveillance camera or the like An image processing method for converting to a screen is known.
- the image processing apparatus captures image data in units of pixels from an image obtained by photographing a subject, and generates a histogram of luminance after decomposing the captured image data into a specific color space.
- An image processing apparatus acquires a setting unit that sets an operator including a plurality of pixels including a target pixel in a processing target image, a gradation value of each pixel in the operator, A totalizing unit that generates a histogram indicating the number of pixels corresponding to each gradation value from 0 to a natural number N based on the gradation value; and each gradation value from gradation value 0 to the gradation value of the target pixel in the histogram A new gradation value is calculated by adding the number of pixels corresponding to, or by subtracting the number of pixels corresponding to each gradation value from the gradation value N to the gradation value of the target pixel from the value N And a conversion unit that converts the gradation value of the target pixel into the new gradation value.
- An image processing method is an image processing method executed by an apparatus, and includes a setting step for setting an operator including a plurality of pixels including a target pixel in a processing target image, and each pixel in the operator
- a new value is obtained by adding or subtracting the number of pixels corresponding to each tone value from one end (tone value 0 or tone value N) of the tone value histogram in the operator to the tone value of the target pixel.
- the gradation value of the target pixel is converted into the new gradation value.
- An image processing apparatus generates an overall histogram indicating the number of pixels corresponding to each gradation value from 0 to a natural number N based on the gradation value of each pixel in the input image,
- the gradation value of each pixel in the input image is converted based on the characteristics, the pre-processing unit that delivers the converted input image to the setting unit as the processing target image, and the target pixel immediately after being converted by the pre-processing unit
- a post-processing unit that outputs a gradation value calculated by adding the gradation value and the gradation value of the target pixel immediately after being converted by the conversion unit by a predetermined mixing ratio; Good.
- the preprocessing unit determines a useful gradation range in the overall histogram based on the slope in the overall histogram, and the useful gradation range and the useful gradation range are determined.
- the gradation value of each pixel of the input image may be converted based on a gain determined according to the range.
- the post-processing unit may determine the mixing ratio based on the histogram generated by the totaling unit.
- the number of pixels included in the operator may match the number of gradation values that can be expressed in the processing target image.
- an image processing apparatus and an image processing method that are suitable for hardware processing and capable of obtaining an appropriately sharpened image.
- the image processing apparatus 1 captures an image such as each frame of a still image and a moving image, and converts the gradation of each pixel in the captured image to make it clear and easy to see.
- the image processing apparatus 1 is not limited to a specific application, for example, by executing the above-described processing on each frame of a video shot by a surveillance camera or the like, it is possible to obtain a video in which each frame is clear and easy to see. it can.
- the sharpening is image processing for making it easy to see a portion that is difficult to see due to a deviation in gradation in an image.
- sharpening is image processing for adjusting the gradation of each pixel in an image.
- the sharpening for example, it is possible to obtain an image in which a portion that is difficult to see because it is extremely bright (or extremely dark) in the original image becomes easy to see.
- the gradation is a numerical value expressing gradations such as color and brightness in stages.
- an image to be processed by the image processing apparatus 1 includes information such as resolution, gradation value in pixel (pixel) units, gradation resolution, and the like.
- the gradation resolution is the number of bits prepared for expressing gradation in stages. For example, when the gradation resolution is 8 bits, the gradation can be expressed by 256 gradation values from 0 to 255.
- FIG. 1 is a diagram illustrating an example of a hardware configuration of the image processing apparatus 1.
- the image processing apparatus 1 includes one or more CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102 and a ROM (Read Only Memory) 103 which are main storage devices, a keyboard and a mouse.
- a communication device 105 for performing data communication with an external device such as a database device for accumulating image data after storing the input device 104 and a device such as a monitoring camera from which an input image is acquired;
- An FPGA (Field-programmable gate array) 106 which is an integrated circuit for executing gradation value conversion processing to be described later, and an output device 107 such as a display are provided.
- each functional component of the image processing apparatus 1 described later reads predetermined computer software on hardware such as the RAM 102, and the input device 104, the communication device 105, the FPGA 106, and the output device under the control of the CPU 101. This is realized by operating 107 and the like, and reading and writing data in the RAM 102 and the ROM 103.
- the FPGA 106 is preliminarily mounted with a circuit (see FIGS. 4 and 5) for executing a gradation value conversion process, which will be described later, and the gradation value conversion process can be executed by hardware processing. Yes.
- FIG. 2 shows that the image processing apparatus 1 is configured as one computer, the functions of the image processing apparatus 1 may be distributed to a plurality of computers.
- the image processing apparatus 1 may be configured as an appliance specialized for a function that executes image processing such as sharpening of an input image, or as a function of the computer in a computer that executes a plurality of functions. May be incorporated.
- FIG. 2 is a block diagram showing a functional configuration of the image processing apparatus 1.
- the image processing apparatus 1 includes a preprocessing unit 11, an operator setting unit 12, a totaling unit 13, a gradation value conversion unit 14, and a postprocessing unit 15.
- the preprocessing unit 11 includes a totaling unit 11A, an image quality adjusting unit 11B, and a resolution adjusting unit 11C.
- the post-processing unit 15 includes a mixing unit 15A and a mixing ratio adjusting unit 15B.
- the preprocessing unit 11 sequentially captures images taken by a surveillance camera or the like, and executes preprocessing on each frame of the captured images.
- a processing target image is generated.
- the preprocessing unit 11 performs preprocessing using each of the captured video frames as an input image, and generates a processing target image.
- the processing target image is an image to be subjected to a gradation value conversion process to be described later.
- the preprocessing performed by the preprocessing unit 11 is not essential and can be omitted in order to sharpen an image.
- the preprocessing by the preprocessing unit 11 is omitted, the processing target image matches the input image.
- the operator setting unit 12, the totaling unit 13, the gradation value conversion unit 14, and the post-processing unit 15 execute each process by using each pixel of the processing target image output from the preprocessing unit 11 as a processing target pixel. . That is, the processes of the operator setting unit 12, the totaling unit 13, the gradation value conversion unit 14, and the post-processing unit 15 are executed for each pixel of the processing target image.
- the gradation value of each pixel of the input image is converted, and an image in which both image quality improvement and sharpening image processing are executed is obtained.
- the post-processing unit 15 adjusts the degree of sharpening according to the state of the image, the user's preference, and the like, and outputs the adjusted image as an output image.
- an image (output image) in which the input image is appropriately sharpened can be obtained.
- the pre-processing unit 11 performs pre-processing on the input image prior to sharpening by a gradation value conversion unit 14 described later.
- the preprocessing by the preprocessing unit 11 is not essential and can be omitted in order to sharpen an image.
- the pre-processing unit 11 When the pre-processing unit 11 acquires each frame of the video captured by the surveillance camera or the like as the input image, the pre-processing unit 11 converts the gradation value of each pixel of the input image based on the characteristics of the input image. In the present embodiment, as an example, the preprocessing unit 11 converts the gradation value of each pixel based on the shape of a histogram related to the gradation value of each pixel of the input image.
- the function of the preprocessing unit 11 is realized by a totaling unit 11A, an image quality adjusting unit 11B, and a resolution adjusting unit 11C.
- the aggregation unit 11A generates an overall histogram indicating the number of pixels corresponding to each gradation value from 0 to a natural number N based on the gradation value of each pixel in the input image.
- “natural number N” is the maximum gradation value determined by the gradation resolution of the input image. For example, when the gradation resolution of the input image is 8 bits, since the gradation of each pixel is expressed in 256 levels from 0 to 255, the natural number N is 255.
- the graph (a) in FIG. 3 shows an example of the entire histogram generated by the counting unit 11A.
- a bar graph is obtained in which the horizontal axis indicates the gradation value and the vertical axis indicates the number of pixels.
- the graph (a) in FIG. 3 shows the whole histogram of such a bar graph. It is expressed as a line graph connecting adjacent vertices.
- a mountain-shaped distribution is formed in each of the area where the gradation value is from point S1 to point E1 and the area where the gradation value is from point S2 to point E2. Note that an image in which two peaks are formed in the entire histogram in this way is often an image taken in backlight.
- the image quality adjustment unit 11B converts the gradation value of each pixel in the input image based on the characteristics of the shape of the entire histogram generated by the totalization unit 11A. Specifically, first, the image quality adjustment unit 11B extracts an area (useful gradation area) including information useful for constructing an image from the entire histogram.
- the information useful for composing an image means information useful for recognizing an object reflected in the image.
- an area having a low level (an index indicating the number of pixels corresponding to a gradation) and a small slope indicated by the rate of change in the number of pixels between adjacent gradation values can be determined as an area (unnecessary gradation area) that has little useful information.
- the peak portion in the entire histogram in the example of the graph (a) in FIG. 3 is useful in constructing an image for the area from the point S1 to the point E1 and the area from the point S2 to the point E2. It can be determined as a useful gradation area containing a lot of information.
- the image quality adjustment unit 11B specifies an area where the level and the inclination are each equal to or less than a predetermined threshold in the entire histogram as an unnecessary gradation area, and sets the other areas as useful gradation areas. Can be extracted.
- the image quality adjustment unit 11B specifies an area whose inclination is equal to or less than a predetermined threshold as an unnecessary gradation area in the entire histogram without considering the level, and other areas. May be extracted as a useful gradation area.
- the image quality adjustment unit 11B acquires the number of pixels corresponding to each gradation value from the gradation value 0 toward the direction in which the gradation value increases, and is spaced between two adjacent gradation values or a predetermined interval. It may also be determined whether the rate of change in the number of pixels is equal to or less than a threshold value between the two gradation values. Specifically, when the rate of change in the number of pixels between two gradation values is equal to or less than a threshold value, it is determined that the area between the two gradation values is an unnecessary gradation area, and otherwise Alternatively, it may be determined that the area between the two gradation values is a useful gradation area.
- the image quality adjustment unit 11B sets the point S1 shown in the graph (a) of FIG. 3 as the end point of the unnecessary gradation area having the smaller gradation value, that is, the first useful gradation. It can be determined as the start point of the area. Further, the image quality adjustment unit 11B can determine the point E1 shown in the graph (a) of FIG. 3 as the end point of the first useful gradation area by executing the same processing. Further, the image quality adjustment unit 11B further performs the same processing to determine the points S2 and E2 shown in the graph (a) of FIG. 3 as the start point and the end point of the second useful gradation area, respectively. it can.
- the image quality adjustment unit 11B converts the gradation value of each pixel of the input image based on the useful gradation area and a gain (degree of amplification) determined according to the range of the useful gradation area. Specifically, the image quality adjustment unit 11B, in the entire histogram after converting the gradation value of each pixel of the input image, displays the entire gradation range in which the useful gradation area can be expressed by the gradation resolution of the input image. Determine the gain to occupy.
- the image quality adjustment unit 11B determines the gain by the following equation (1), and the gradation of each pixel of the input image by the following equations (2) and (3). Gain can be applied to the value.
- Vi represents a gradation value before conversion
- Vo represents a gradation value after conversion.
- the unnecessary gradation area is deleted as shown in the graph (b) of FIG.
- the useful gradation area is converted so as to occupy the entire gradation range that can be expressed by the gradation resolution of the input image.
- the processing for separating the useful gradation area from the unnecessary gradation area is executed by software processing.
- the calculation processing based on the above formulas (1) to (3) may be executed by hardware processing. Since it is necessary to execute the processing of the above formulas (2) and (3) for each pixel of the input image, the processing efficiency can be improved by executing the processing for each pixel in parallel by hardware processing.
- the resolution adjustment unit 11C adjusts the gradation resolution of the input image. Specifically, the gradation resolution is adjusted so that the number of pixels included in the operator set by the operator setting unit 12 described later matches the number of gradation values that can be expressed. As will be described in detail later, the operator is an area composed of a plurality of pixels including the target pixel in the processing target image. For example, when the operator set by the operator setting unit 12 is a rectangular area composed of 16 ⁇ 16 256 pixels, the resolution adjustment unit 11C can represent the gradation resolution of the input image with 256 step values. Adjust to 8 bits.
- the resolution adjustment unit 11C shifts the bit pattern of the original 10-bit gradation value to the right with respect to the gradation value of each pixel of the input image.
- the gradation resolution of the input image can be changed from 10 bits to 8 bits.
- the image processed by the preprocessing unit 11 is transferred to the operator setting unit 12 as a processing target image.
- the operator setting unit 12, the totaling unit 13, and the gradation value converting unit 14 for executing the sharpening on the processing target image will be described.
- the processing of the operator setting unit 12, the totaling unit 13, the gradation value conversion unit 14, and the post-processing unit 15 is executed for each pixel in the processing target image.
- the operator setting unit 12, the totaling unit 13, the gradation value conversion unit 14, and the post-processing unit 15 scan each pixel in the processing target image in units of pixels, and process target pixels (targets)
- the following processing is executed on the target pixel of the movement destination while sequentially moving the pixel) within the processing target image.
- the gradation value of each pixel in the processing target image is converted, and as a result, a sharpened image is obtained.
- the operator setting unit 12 sets an operator composed of a plurality of pixels including the target pixel in the processing target image.
- the target pixel is a pixel designated as a pixel to be processed by the above-described scanning process, and is a pixel for which a gradation value is converted by the gradation value conversion process.
- the operator is typically set as a rectangular area having the target pixel as a central pixel.
- the operator may be anything as long as it includes a target pixel and a plurality of pixels existing in the vicinity of the target pixel, and may be, for example, a circle or any other shape.
- the operator setting unit 12 sets, as an operator, a rectangular area made up of 16 ⁇ 16 256 pixels with the target pixel as the central pixel.
- the operator is set in this way, for example, when a pixel close to the boundary such as the upper side, the lower side, the left side, and the right side of the processing target image is set as the target pixel, the operator having the target pixel as the central pixel cannot be set. Therefore, the operator setting unit 12 may exclude pixels close to these boundaries from the processing target by the gradation value conversion unit 14, or may set an operator having an arbitrary shape suitable for these pixels. May be.
- the totaling unit 13 acquires the gradation value of each pixel in the operator set by the operator setting unit 12, and based on the gradation value of each pixel, the number of pixels corresponding to each gradation value from 0 to a natural number N Is generated. For example, the totalization unit 13 scans each pixel in the operator in a predetermined order in units of one pixel, and executes a counting process that increases the number corresponding to the gradation value of the operation target pixel by one. Alternatively, the totaling unit 13 may acquire the gradation value of each pixel in the operator in parallel and count the number corresponding to each gradation value. Thereby, a histogram indicating the number of pixels corresponding to each gradation value is generated.
- the histogram here is information in which the corresponding pixel numbers X0 to X255 are associated with each gradation value from gradation value 0 to gradation value 255.
- the histogram generated by the totaling unit 13 and the gradation value of the target pixel are transferred to the gradation value converting unit 14.
- the gradation value conversion unit 14 adds a new gradation value by adding the number of pixels corresponding to each gradation value from the gradation value 0 to the gradation value of the target pixel in the histogram generated by the totalization unit 13.
- the gradation value of the target pixel is calculated and converted to the new gradation value. Note that since the original gradation value of the target pixel and the new gradation value may match, the “conversion” here means that the gradation value of the target pixel is different from the original gradation value. This includes not only changing to a tone value but also updating to a tone value that matches the original tone value.
- the gradation value conversion unit 14 performs the above-described arithmetic processing using, for example, a logic circuit configured in the FPGA 106.
- a logic circuit configured in the FPGA 106.
- two circuit examples for executing such arithmetic processing will be described.
- the gradation resolution of the processing target image is adjusted to 8 bits by the resolution adjusting unit 11C described above.
- the gradation of each pixel of the processing target image takes any value from gradation value 0 to gradation value 255.
- FIG. 4 is a diagram illustrating a circuit of the gradation value conversion unit 14 according to the first example.
- the circuit shown in FIG. 4 is a vector adder configured to execute in series the first stage arithmetic processing by the first addition module A1 to the 255th stage arithmetic processing by the 255th addition module A255. Yes, the result is output every 1 pixel clock.
- the pixel clock is the number of clocks necessary to process one pixel. The specific operation of the circuit shown in FIG. 4 will be described below.
- the first addition module A1 includes registers A11 to A13, a zero register A14 in which 0 is stored, a subtracter A15, a selector A16, and an adder A17.
- the registers A11 and A12 store the number of pixels corresponding to the gradation value 0 and the gradation value 1 acquired from the aggregation unit 13, respectively, and the register A13 stores the gradation of the target pixel acquired from the aggregation unit 13. Stores the value.
- the subtracter A15 subtracts one gradation value of the target pixel stored in the register A13.
- the selector A16 compares the value N after being subtracted by the subtractor A15 with 0, and if the value N is greater than 0, the register A12 is connected to the adder A17, and if the value N is 0 or less, The zero register A14 is connected to the adder A17.
- the adder A17 adds the number of pixels corresponding to the gradation value 0 stored in the register A11 and the value stored in the register A12 or the zero register A14 connected by the selector A16, and adds the operation result to the second value.
- the data is transferred to the addition module A2.
- the arithmetic processing of the first addition module A1 when the gradation value of the target pixel is 2 or more, the subtraction result by the subtracter A15 becomes larger than 0, so the register A12 is connected to the adder A17 by the selector A16. Then, the result of adding the number of pixels corresponding to the gradation value 0 and the number of pixels corresponding to the gradation value 1 is passed to the second addition module A2.
- the result of subtraction by the subtracter A15 is 0 or less, so that the zero register A14 is connected to the adder A17 by the selector A16 and corresponds to the gradation value 0.
- the result of adding the number of pixels and 0 (that is, the number of pixels corresponding to the gradation value 0) is directly transferred to the second addition module A2.
- the second addition module A2 includes registers A21 to A23, a zero register A24 storing 0, a subtractor A25, a selector A26, and an adder A27. Yes.
- the register A21 stores the calculation result by the first addition module A1.
- the pixel value of the gradation value 2 is stored in synchronization with the timing at which the arithmetic processing in the first addition module A1 is completed by the delay processing in the delay circuit Z2.
- the register N23 stores the value N after being subtracted by the subtracter A15.
- the subtracter A25 subtracts one value N stored in the register A23.
- the selector A26 compares the value N after being subtracted by the subtractor A25 with 0, and if the value N is greater than 0, the register A22 is connected to the adder A27, and if the value N is 0 or less, The zero register A24 is connected to the adder A27.
- the adder A27 adds the value stored in the register A21 and the value stored in the register A22 or the zero register A24 connected by the selector A26, and passes the addition result to the third addition module A3.
- the register A22 is connected to the adder A27 by the selector A26.
- the result of adding the value stored in the register A22 (the sum of the number of pixels corresponding to the gradation value 0 and the number of pixels corresponding to the gradation value 1) and the number of pixels corresponding to the gradation value 2 (that is, The sum of the number of pixels corresponding to each gradation value from gradation value 0 to gradation value 2) is passed to the third addition module A3.
- the subtraction result by the subtracter A25 is 0 or less, so the zero register A24 is connected to the adder A27 by the selector A26, and the value stored in the register A21 And the result of adding 0 and 0 (that is, the value stored in the register A21) is directly transferred to the third addition module A3.
- the same processing as that of the first and second addition modules A1 and A2 described above is executed in series.
- the subtraction by the subtracter is performed in the addition module that executes the arithmetic processing after the number of pixels corresponding to each gradation value from the gradation value 0 to “the gradation value of the target pixel ⁇ 1” is added.
- the result becomes 0 or less, and the zero register is connected to the adder by the selector, so that the addition of the number of pixels is stopped. Therefore, in the above circuit, the value finally outputted from the 255th addition module A255 is obtained by adding the number of pixels corresponding to each gradation value from gradation value 0 to “gradation value-1 of target pixel”. Result.
- the determination process of the selector in each addition module whether or not to include 0 in the boundary condition can be arbitrarily determined. For example, in the determination process of the selector in each addition module, when the subtraction result by the subtracter is less than 0, a zero register is connected to the adder, and when the subtraction result by the subtractor is 0 or more, the corresponding gradation A register storing a value may be connected to the adder. In this case, the value output from the 255th addition module A255 is the result of adding the number of pixels corresponding to each gradation value from the gradation value 0 to “the gradation value of the target pixel”.
- the circuit described above is configured to sequentially add the number of pixels corresponding to each gradation value from the gradation value 0 to the gradation value of the target pixel.
- a circuit modified to sequentially subtract the number of pixels corresponding to each gradation value from the gradation value 255 to the gradation value of the target pixel may be used. That is, the gradation value conversion unit 14 may calculate a new gradation value by subtracting from the value N the number of pixels corresponding to each gradation value from the gradation value 255 to the gradation value of the target pixel. .
- the circuit that executes such processing can output the result of adding the number of pixels corresponding to each gradation value from the gradation value 0 to the gradation value of the target pixel, as in the above-described circuit.
- FIG. 5 is a diagram illustrating a circuit of the gradation value conversion unit 14 according to the second example.
- the circuit shown in FIG. 5 includes 255 adders B1 to B255 and a selector B256.
- a plurality of delay circuits Z are provided so that the gradation value of the target pixel and the calculation results by the adders B1 to B255 are output to the selector 256 at the same timing.
- the number of pixels corresponding to the gradation values 0 to 255 is assumed to be X0 to X255.
- the gradation value of the target pixel is input to the selector B 256 via the delay circuit Z. Further, the number of pixels X0 corresponding to the gradation value 0 is input to the selector B256 via the delay circuit Z and to the adder B1. Further, the number of pixels X1 corresponding to the gradation value 1 is input to the adder B1. Then, in the adder B1, the number of pixels X0 corresponding to the gradation value 0 and the number of pixels X1 corresponding to the gradation value 1 are added.
- the addition result in the adder B1 is input to the selector B256 via the delay circuit Z and also input to the adder B2 in the next stage.
- the adder B2 receives the number of pixels X2 corresponding to the gradation value 2 in synchronization with the timing at which the addition result in the adder B1 is input to the adder B2 by the delay processing in the delay circuit Z.
- the In the adder B2 the addition result (X0 + X1) in the adder B1 and the number of pixels (X2) corresponding to the gradation value 2 are added, and the addition result is input to the selector B256 via the delay circuit Z. At the same time, it is input to the adder B3 in the next stage.
- the same processing is sequentially executed in the adders B3 to B255.
- the gradation value N of the target pixel, the number of pixels X0 corresponding to the gradation value 0, and the addition results of the adders B1 to B255 are sent to the selector B256 at the same timing. Is output.
- the selector B256 generates a conversion table storing 256 values of “X0”, “X0 + X1”, “X0 + X1 + X2”,..., “X0 + X1 +.
- a value corresponding to the gradation value N is selected and output.
- the selector B256 selects and outputs the fourth value “X0 + X1 + X2 + X3” in the conversion table.
- the result of adding the number of pixels corresponding to each gradation value from the gradation value 0 to the gradation value of the target pixel is finally output by the selector B256.
- a circuit modified to execute a subtracter using a subtracter instead of an addition process using an adder may be used for the second circuit example. That is, assuming that the initial value is 255 which is the maximum gradation value, the result of sequentially subtracting the number of pixels corresponding to each gradation value from the gradation value 255 to the gradation value of the target pixel is output to the selector B 256, A circuit configured to generate a conversion table similar to the conversion table may be used.
- the gradation value conversion unit 14 outputs the converted gradation value of the target pixel to the post-processing unit 15.
- the implementation method of the gradation value conversion unit 14 is not limited to the circuit example described above, and is a result of adding the number of pixels corresponding to each gradation value from the gradation value 0 to the gradation value of the target pixel.
- any circuit configuration can be used as long as the result of subtracting the number of pixels corresponding to each gradation value from the gradation value 255 to the gradation value of the target pixel from 255 can be obtained as a new gradation. be able to.
- the gradation value conversion unit 14 has been described as executing the gradation value conversion process by hardware processing (for example, the first and second circuit examples described above).
- the value conversion process may be executed by software processing.
- various disadvantages of software processing can be eliminated by executing gradation value conversion processing by hardware processing.
- the effect of adjusting the gradation resolution so that the number of pixels included in the operator and the number of gradation values that can be expressed by the resolution adjusting unit 11C will be described. If the number of pixels included in the operator and the number of gradation values that can be expressed do not match, the number of pixels corresponding to each gradation value can be expressed as the number of pixels included in the operator. Adjustment such as multiplying by a magnification according to the ratio to the number of tone values is required. On the other hand, when the gradation resolution is adjusted so that the number of pixels included in the operator matches the number of gradation values that can be expressed as in this embodiment, the above adjustment is not necessary. It becomes. As a result, the gradation value of the target pixel can be converted by simple addition and subtraction processing as in the above circuit example.
- the post-processing unit 15 includes the tone value of the target pixel immediately after being converted by the pre-processing unit 11 (hereinafter referred to as “the tone value before sharpening”) and the target immediately after being converted by the tone value converting unit 14. Based on the gradation value of the pixel (hereinafter referred to as “the gradation value after sharpening”), post-processing for adjusting the sharpening intensity is executed. Specifically, the mixing unit 15A determines in advance the gradation value of the target pixel immediately after being converted by the preprocessing unit 11 and the gradation value of the target pixel immediately after being converted by the gradation value conversion unit 14. The gradation value calculated by adding the mixture ratios is output as the final gradation value of the target pixel.
- the sharpening intensity is the sharpening in the case where the final gradation value of the target pixel is determined by adding the gradation value before sharpening and the gradation value after sharpening by a predetermined mixing ratio. Indicates the degree of importance of the post-tone value.
- the sharpening intensity can be adjusted by adding the gradation value before sharpening and the gradation value after sharpening by the mixing ratio corresponding to the sharpening intensity.
- the mixing unit 15A adjusts the sharpening strength by the following procedure, for example.
- the adjustment level Li of the sharpening intensity can be arbitrarily set between “0” having the minimum intensity and “255” having the maximum intensity.
- the mixing unit 15A executes the calculation process of the following formula (4), thereby The gradation value Vo after pixel adjustment can be acquired.
- Vo ((V1 / 255) ⁇ (255 ⁇ Li)) + ((V2 / 255) ⁇ Li) (4)
- the mixing unit 15A mixes the pre-sharpening tone value of the target pixel and the post-sharpening tone value of the target pixel at a ratio corresponding to the adjustment level, thereby adjusting the sharpening intensity.
- the gradation value Vo of the pixel is output.
- the adjustment level Li of the sharpening intensity may be set manually by the user, or may be automatically adjusted according to the distribution of gradation values in the operator. Good.
- the mixture ratio adjustment unit 15B has a function of executing the latter automatic adjustment, acquires the histogram generated by the aggregation unit 13, and the gradation value in the operator indicated in the histogram
- the adjustment level Li is appropriately set based on the distribution of.
- the mixture ratio adjustment unit 15B may forcibly set the adjustment level Li to 0 when the distribution of gradation values in the operator is biased toward a specific gradation value.
- the gradation value V1 immediately after the preprocessing is output by the mixing unit 15A.
- the mixture ratio adjustment unit 15B determines whether or not the distribution of gradation values in the operator is flat based on a predetermined criterion. If it is determined that the distribution is flat, the adjustment level Li is not flat. May be set to a value lower than the value set when it is determined. Thereby, in the situation where noise may occur due to sharpening, the sharpening strength can be lowered and the occurrence of fluctuations can be suppressed.
- the preprocessing unit 11 acquires each frame of a video shot by a monitoring camera or the like as an input image, and executes preprocessing for converting the gradation value of each pixel of the input image based on the characteristics of the input image. (Step S1). Subsequently, the processing from step S2 to step S5 is executed for each pixel of the processing target image after the gradation value of each pixel is converted by the preprocessing unit 11.
- an operator composed of a plurality of pixels including the target pixel in the processing target image is set by the operator setting unit 12 (step S2, setting step).
- the totaling unit 13 acquires the gradation value of each pixel in the operator, and generates a histogram indicating the number of pixels corresponding to each gradation value from 0 to a natural number N based on the gradation value of each pixel. (Step S3, aggregation step).
- the gradation value conversion unit 14 adds the number of pixels corresponding to each gradation value from the gradation value 0 to the gradation value of the target pixel in the histogram generated by the aggregation unit 13, or By subtracting the number of pixels corresponding to each gradation value from the gradation value 255 to the gradation value of the target pixel, a new gradation value is calculated, and the gradation value of the target pixel is changed to the new gradation value. Conversion is performed (step S4, conversion step).
- the processing by the gradation value conversion unit 14 is realized by mounting, for example, the circuits shown in the first circuit example and the second circuit example described above on the FPGA 106.
- a post-process for adjusting the sharpening intensity is executed (step S5).
- the gradation value of the target pixel whose sharpening intensity is appropriately adjusted according to the user's preference, the illuminance distribution in the operator, or the like is output.
- a new gradation value is calculated by adding or subtracting numbers, and the gradation value of the target pixel is converted into the new gradation value.
- the processing for converting the gradation value of the target pixel is executed by software processing, in order to realize high-speed processing, it is relatively expensive such as a GPU (Graphics Processing Unit) and a memory that operates at high speed. A device is required. Further, when these devices are provided as image processing devices, the size of the device increases, and the convenience for the user is impaired. In addition, since a high operation clock is required to realize high-speed software processing, there is a problem that power consumption increases.
- the gradation value of the target pixel can be converted and sharpened by hardware processing as described above. Disadvantages in processing can be eliminated.
- FIG. 7A illustrates an example of an original image.
- An original image is an unprocessed image that has not been subjected to the above-described sharpening, pre-processing, and post-processing.
- 7B omits the processing of the pre-processing unit 11 and the post-processing unit 15 with respect to the original image shown in FIG. 7A, and sets the operator setting unit 12, the totaling unit 13, and the gradation values. It is a figure which shows the image obtained by performing only the sharpening by the conversion part.
- FIG. 7A illustrates an example of an original image.
- An original image is an unprocessed image that has not been subjected to the above-described sharpening, pre-processing, and post-processing.
- 7B omits the processing of the pre-processing unit 11 and the post-processing unit 15 with respect to the original image shown in FIG. 7A, and sets the operator setting unit 12, the totaling unit 13, and the gradation values. It is a figure which shows the
- FIG. 8 shows, in addition to the sharpening by the operator setting unit 12, the totaling unit 13, and the gradation value conversion unit 14, the pre-processing unit 11 and the post-processing for the original image shown in FIG. It is a figure which shows the image obtained by performing the pre-processing and post-processing by the part 15.
- SYMBOLS 1 DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 11 ... Pre-processing part, 11A ... Aggregation part, 11B ... Image quality adjustment part, 11C ... Resolution adjustment part, 12 ... Operator setting part, 13 ... Aggregation part, 14 ... Tone value conversion part, 15 ... Post-processing unit, 15A ... mixing unit, 15B ... mixing ratio adjusting unit, 101 ... CPU, 102 ... RAM, 103 ... ROM, 104 ... input device, 105 ... communication device, 106 ... FPGA, 107 ... output device.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Plasma & Fusion (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
Description
Gain=(256/((E1-S1)+(E2-S2)))・・・(1)
If Vi<E1:Vo=(Vi-S1)×Gain・・・(2)
If Vi>S2:Vo=(Vi-S1-(S2-E1))×Gain・・・(3)
図4は、第1の例に係る階調値変換部14の回路を示す図である。図4に示す回路は、第1の加算モジュールA1による第1段目の演算処理から第255の加算モジュールA255による第255段目の演算処理を直列に実行するように構成されたベクトル加算器であり、1ピクセルクロック毎に結果を出力する。ピクセルクロックとは、1画素を処理するのに必要なクロック数である。以下、図4に示す回路の具体的な動作について説明する。
図5は、第2の例に係る階調値変換部14の回路を示す図である。
図5に示す回路は、255個の加算器B1~B255と、セレクタB256とを備えて構成されている。また、対象画素の階調値及び各加算器B1~B255による演算結果が同じタイミングでセレクタ256に出力されるように複数の遅延回路Zが設けられている。ここで、階調値0~255に対応する画素数をX0~X255とする。
Vo=((V1/255)×(255-Li))+((V2/255)×Li)・・・(4)
Claims (6)
- 処理対象画像内の対象画素を含む複数画素からなるオペレータを設定する設定部と、
前記オペレータ内の各画素の階調値を取得し、当該各画素の階調値に基づいて0から自然数Nまでの各階調値に対応する画素数を示すヒストグラムを生成する集計部と、
前記ヒストグラムにおいて、階調値0から前記対象画素の階調値までの各階調値に対応する画素数を加算することにより、又は、階調値Nから前記対象画素の階調値までの各階調値に対応する画素数を値Nから減算することにより、新たな階調値を算出し、前記対象画素の階調値を当該新たな階調値に変換する変換部と、
を備える画像処理装置。 - 入力画像内の各画素の階調値に基づいて0から自然数Nまでの各階調値に対応する画素数を示す全体ヒストグラムを生成し、前記全体ヒストグラムの所定の特徴に基づいて前記入力画像内の各画素の階調値を変換し、変換後の前記入力画像を前記処理対象画像として前記設定部に受け渡す前処理部と、
前記前処理部により変換された直後の前記対象画素の階調値と前記変換部により変換された直後の前記対象画素の階調値とを予め定められた混合比で足し合わせることにより算出される階調値を出力する後処理部と、を更に備える、
請求項1に記載の画像処理装置。 - 前記前処理部は、前記全体ヒストグラム内での傾斜に基づいて当該全体ヒストグラムにおける有用階調の範囲を決定し、当該有用階調の範囲と当該有用階調の範囲に応じて定まるゲインとに基づいて前記入力画像の各画素の階調値を変換する、
請求項2に記載の画像処理装置。 - 前記後処理部は、前記集計部により生成されたヒストグラムに基づいて前記混合比を決定する、
請求項2又は3に記載の画像処理装置。 - 前記オペレータに含まれる画素数と前記処理対象画像において表現し得る階調値の個数とが一致する、
請求項1~4のいずれか一項に記載の画像処理装置。 - 装置により実行される画像処理方法であって、
処理対象画像内の対象画素を含む複数画素からなるオペレータを設定する設定ステップと、
前記オペレータ内の各画素の階調値を取得し、当該各画素の階調値に基づいて0から自然数Nまでの各階調値に対応する画素数を示すヒストグラムを生成する集計ステップと、
前記ヒストグラムにおいて、階調値0から前記対象画素の階調値までの各階調値に対応する画素数を加算することにより、又は、階調値Nから前記対象画素の階調値までの各階調値に対応する画素数を値Nから減算することにより、新たな階調値を算出し、前記対象画素の階調値を当該新たな階調値に変換する変換ステップと、
を含む画像処理方法。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/529,972 US20170330504A1 (en) | 2014-11-28 | 2014-11-28 | Image processing apparatus and image processing method |
JP2015534707A JP5826980B1 (ja) | 2014-11-28 | 2014-11-28 | 画像処理装置及び画像処理方法 |
PCT/JP2014/081629 WO2016084253A1 (ja) | 2014-11-28 | 2014-11-28 | 画像処理装置及び画像処理方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2014/081629 WO2016084253A1 (ja) | 2014-11-28 | 2014-11-28 | 画像処理装置及び画像処理方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016084253A1 true WO2016084253A1 (ja) | 2016-06-02 |
Family
ID=54776788
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/081629 WO2016084253A1 (ja) | 2014-11-28 | 2014-11-28 | 画像処理装置及び画像処理方法 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20170330504A1 (ja) |
JP (1) | JP5826980B1 (ja) |
WO (1) | WO2016084253A1 (ja) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6779695B2 (ja) * | 2016-07-28 | 2020-11-04 | キヤノン株式会社 | 画像処理装置及びその制御方法、表示装置 |
CN111415292B (zh) * | 2020-03-23 | 2022-07-26 | 厦门理工学院 | 一种提高dmd灰度成像刷新率的方法、装置、设备及存储介质 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS63185165A (ja) * | 1987-01-28 | 1988-07-30 | Canon Inc | 多階調デ−タの圧縮/伸長方式 |
JPH0378094A (ja) * | 1989-08-22 | 1991-04-03 | Toshiba Corp | 画像階調変換装置 |
JPH07296160A (ja) * | 1994-04-27 | 1995-11-10 | Matsushita Electron Corp | 画像補正方法 |
JP2008227945A (ja) * | 2007-03-13 | 2008-09-25 | Olympus Corp | 画像処理装置および画像処理プログラム |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011124800A (ja) * | 2009-12-10 | 2011-06-23 | Sony Corp | 画像処理装置、画像処理方法及びプログラム |
-
2014
- 2014-11-28 JP JP2015534707A patent/JP5826980B1/ja not_active Expired - Fee Related
- 2014-11-28 US US15/529,972 patent/US20170330504A1/en not_active Abandoned
- 2014-11-28 WO PCT/JP2014/081629 patent/WO2016084253A1/ja active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS63185165A (ja) * | 1987-01-28 | 1988-07-30 | Canon Inc | 多階調デ−タの圧縮/伸長方式 |
JPH0378094A (ja) * | 1989-08-22 | 1991-04-03 | Toshiba Corp | 画像階調変換装置 |
JPH07296160A (ja) * | 1994-04-27 | 1995-11-10 | Matsushita Electron Corp | 画像補正方法 |
JP2008227945A (ja) * | 2007-03-13 | 2008-09-25 | Olympus Corp | 画像処理装置および画像処理プログラム |
Also Published As
Publication number | Publication date |
---|---|
JP5826980B1 (ja) | 2015-12-02 |
US20170330504A1 (en) | 2017-11-16 |
JPWO2016084253A1 (ja) | 2017-04-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP2007293431A (ja) | 画像処理装置 | |
JP6623832B2 (ja) | 画像補正装置、画像補正方法及び画像補正用コンピュータプログラム | |
JP5870598B2 (ja) | 画像のホワイトバランス処理方法及び装置 | |
JP2010034964A (ja) | 画像合成装置、画像合成方法及び画像合成プログラム | |
JP5585494B2 (ja) | 画像処理装置、画像処理プログラム及び画像処理方法 | |
JP2007249436A (ja) | 画像信号処理装置及び画像信号処理方法 | |
JP5826980B1 (ja) | 画像処理装置及び画像処理方法 | |
JP2016177500A (ja) | 画像処理装置、画像処理システムおよびプログラム | |
JP5911633B1 (ja) | 画像処理装置 | |
US10311550B2 (en) | Image processing device for eliminating graininess of image | |
JP2014192859A (ja) | 色補正方法、プログラム及び装置 | |
JP6056511B2 (ja) | 画像処理装置、方法、及びプログラム、並びに撮像装置 | |
KR102470242B1 (ko) | 영상 처리 장치, 영상 처리 방법, 및 프로그램 | |
JP6344651B2 (ja) | フォーカスアシスト機能付き撮像装置 | |
JP5178933B1 (ja) | 画像処理装置 | |
JP2008021219A (ja) | 画像処理装置及び方法 | |
CN112446848A (zh) | 图像处理方法、装置及电子设备 | |
JP6826472B2 (ja) | 画像処理装置およびその制御方法 | |
JP2017126885A (ja) | 画像補正装置、画像補正方法及び画像補正用コンピュータプログラム | |
EP3709254A1 (en) | Image processing device, image processing method, and image processing program | |
EP3719740B1 (en) | Image processing device, image processing method, and image processing program | |
JP2012205190A (ja) | 画像処理装置およびプログラム | |
US20240155248A1 (en) | Method and apparatus for generating high-dynamic-range image, and electronic device | |
JP2010026690A (ja) | 画像処理装置 | |
JP2014086957A (ja) | 画像処理装置及び画像処理方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2015534707 Country of ref document: JP Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14906857 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15529972 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14906857 Country of ref document: EP Kind code of ref document: A1 |