WO2022259323A1 - Image processing device, image processing method, and image processing program - Google Patents

Image processing device, image processing method, and image processing program Download PDF

Info

Publication number
WO2022259323A1
WO2022259323A1 PCT/JP2021/021592 JP2021021592W WO2022259323A1 WO 2022259323 A1 WO2022259323 A1 WO 2022259323A1 JP 2021021592 W JP2021021592 W JP 2021021592W WO 2022259323 A1 WO2022259323 A1 WO 2022259323A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image processing
correction parameter
types
histogram
Prior art date
Application number
PCT/JP2021/021592
Other languages
French (fr)
Japanese (ja)
Inventor
和樹 出口
俊明 久保
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to JP2023527167A priority Critical patent/JPWO2022259323A1/ja
Priority to PCT/JP2021/021592 priority patent/WO2022259323A1/en
Publication of WO2022259323A1 publication Critical patent/WO2022259323A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing

Definitions

  • the present disclosure relates to an image processing device, an image processing method, and an image processing program.
  • Images with various resolutions and various compression methods are input to the device.
  • the various resolutions are 8K, 4K, full HD (High Definition), SD (Standard Definition), and the like.
  • various compression schemes are specified in H.264. 265/HEVC, H.I. H.264/AVC, MPEG2, and the like.
  • the device may be equipped with an image quality enhancement function. Then, it is desired that the apparatus performs optimal image quality enhancement. For example, when sharpening a blurry image, a process is performed to sharpen the image. Also, for example, when a video with a lot of noise is input, processing is executed to prevent the noise from being emphasized.
  • an image quality control device for controlling image quality has been proposed (see Patent Document 1).
  • the image quality control device of Patent Document 1 generates histograms of all luminance information, chromaticity information, color information, and frequency information obtained from an image included in a video signal.
  • the image quality control device extracts histogram patterns corresponding to all histograms from a reference table having a plurality of preset histogram patterns.
  • the image quality control device controls the image quality of the video signal based on control parameters corresponding to the extracted histogram pattern.
  • the purpose of this disclosure is to optimize image quality for various images.
  • the image processing apparatus includes an acquisition unit that acquires an image and a trained model, a plurality of feature amount extraction units that extract a plurality of types of feature amounts based on the image, and the plurality of types of feature amounts and the a correction parameter detection unit that detects a correction parameter, which is a parameter for correcting the image quality of the image, using the trained model; and a process for correcting the image quality of the image using the correction parameter. and an image processing unit for processing.
  • image quality optimization can be performed for various images.
  • FIG. 2 illustrates hardware included in the image processing apparatus according to the first embodiment
  • FIG. 2 is a block diagram showing functions of the image processing apparatus according to Embodiment 1
  • FIG. 4 is a diagram showing an example (part 1) of horizontal and vertical edge histograms according to Embodiment 1
  • FIG. 10 is a diagram showing an example (part 2) of horizontal and vertical edge histograms according to the first embodiment
  • FIG. 4 is a diagram showing a specific example of a method for extracting a frame difference histogram according to Embodiment 1
  • FIG. 5 is a diagram showing a specific example of a method for extracting the number of changes in brightness according to the first embodiment
  • FIG. 4 is a diagram showing a specific example of a saturation histogram extraction method according to the first embodiment; 1 is a diagram showing an example of a neural network according to Embodiment 1; FIG. FIG. 2 is a diagram showing an example of a trained model that constitutes a random forest according to Embodiment 1; FIG. 1 is a diagram showing an example of a support vector machine according to Embodiment 1; FIG. 4 is a flowchart showing an example of processing executed by the image processing apparatus according to Embodiment 1; 3 is a block diagram showing functions of an image processing apparatus according to a second embodiment; FIG. 9 is a flow chart showing an example of processing executed by the image processing apparatus according to the second embodiment;
  • FIG. 1 illustrates hardware included in an image processing apparatus according to a first embodiment.
  • the image processing device 100 is a device that executes an image processing method.
  • the image processing apparatus 100 has a processor 101 , a volatile memory device 102 and a nonvolatile memory device 103 .
  • the processor 101 controls the image processing apparatus 100 as a whole.
  • the processor 101 is a CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), or the like.
  • Processor 101 may be a multiprocessor.
  • the image processing apparatus 100 may have a processing circuit.
  • the processing circuit may be a single circuit or multiple circuits.
  • the volatile storage device 102 is the main storage device of the image processing device 100 .
  • the volatile memory device 102 is RAM (Random Access Memory).
  • a nonvolatile storage device 103 is an auxiliary storage device of the image processing apparatus 100 .
  • the nonvolatile storage device 103 is a HDD (Hard Disk Drive) or an SSD (Solid State Drive).
  • FIG. 2 is a block diagram showing functions of the image processing apparatus according to the first embodiment.
  • the image processing apparatus 100 includes a storage unit 110, an acquisition unit 120, feature quantity extraction units 130_1, 130_2, .
  • n is a positive integer.
  • the storage unit 110 may be implemented as a storage area secured in the volatile storage device 102 or the nonvolatile storage device 103 .
  • a part or all of the acquisition unit 120, the feature amount extraction unit 130, the correction parameter detection unit 140, the image processing unit 150, and the output unit 160 may be realized by a processing circuit.
  • Part or all of the acquisition unit 120, the feature amount extraction unit 130, the correction parameter detection unit 140, the image processing unit 150, and the output unit 160 may be implemented as modules of a program executed by the processor 101.
  • the program executed by the processor 101 is also called an image processing program.
  • an image processing program is recorded on a recording medium.
  • the image processing device 100 may acquire analog images.
  • the image processing device 100 has an A (Analog)/D (Digital) converter.
  • the storage unit 110 may store images and trained models. Moreover, the storage unit 110 may store a video.
  • Acquisition unit 120 acquires an image. For example, the acquisition unit 120 acquires an image from the storage unit 110. FIG. Also, for example, the acquisition unit 120 acquires an image from a camera. Note that the illustration of the camera is omitted. Acquisition unit 120 also acquires video. In other words, the acquisition unit 120 acquires multiple images.
  • the acquisition unit 120 acquires a learned model.
  • the acquisition unit 120 acquires the trained model from the storage unit 110.
  • the trained model may be stored in an external device (for example, cloud server). If the trained model is stored in the external device, the obtaining unit 120 obtains the trained model from the external device.
  • the feature amount extraction unit 130 extracts a plurality of types of feature amounts based on the image. In other words, the feature amount extraction unit 130 extracts a plurality of feature amounts of different types based on the image. A specific description will be given of the feature amount extraction processing.
  • the feature quantity extraction unit 130 extracts a horizontal edge histogram, which is one feature quantity, based on the image. Specifically, the feature amount extraction unit 130 calculates a luminance difference value between adjacent pixels in the horizontal direction as edge enhancement based on the luminance value of the image. For example, the feature amount extraction unit 130 extracts a horizontal edge histogram expressed by dividing edge enhancement of 1024 gradations into 16 pieces. 4 illustrates a horizontal edge histogram;
  • FIG. 3 is a diagram showing an example (part 1) of a horizontal edge histogram according to the first embodiment.
  • the vertical axis indicates the appearance frequency.
  • the horizontal axis indicates edge strength.
  • the feature amount extraction unit 130 extracts a horizontal edge amount, which is one feature amount. Specifically, a method for extracting the horizontal edge amount will be described.
  • FIG. 4 is a diagram showing an example (part 2) of the horizontal edge histogram according to the first embodiment. Assume that the horizontal edge histogram shown in FIG. 4 has been extracted.
  • the feature quantity extraction unit 130 calculates the total frequency of appearance between the minimum value (edge_left in FIG. 4) and the maximum value (edge_right in FIG. 4) of the edge strength in the specified range in the horizontal edge histogram as a horizontal edge. Extract as quantity. Note that the range may be a preset range.
  • the feature quantity extraction unit 130 extracts a vertical edge histogram, which is one feature quantity, based on the image. Specifically, the feature quantity extraction unit 130 calculates a luminance difference value between adjacent pixels in the vertical direction as edge enhancement based on the luminance value of the image. For example, the feature amount extraction unit 130 extracts a vertical edge histogram expressed by dividing edge enhancement of 1024 gradations into 16 pieces. For example, a vertical edge histogram can be expressed as shown in FIG. Therefore, FIG. 3 may be considered as a diagram showing an example of a vertical edge histogram.
  • the feature quantity extraction unit 130 extracts a vertical edge quantity, which is one feature quantity.
  • the method for extracting the vertical edge amount is the same as the method for extracting the horizontal edge amount. For example, when considering FIG. 4 as an example of a vertical edge histogram, the feature quantity extraction unit 130 calculates the total frequency of appearance between the minimum value and the maximum value of edge strength in a specified range in the vertical edge histogram. is extracted as the vertical edge quantity.
  • the feature quantity extraction unit 130 uses an image to extract a frame difference histogram, which is one feature quantity. A method for extracting a frame difference histogram will be specifically described.
  • FIG. 5 is a diagram showing a specific example of a frame difference histogram extraction method according to the first embodiment.
  • FIG. 5 shows a luminance histogram based on the previously acquired image (for example, n ⁇ 1th frame) and a luminance histogram based on the currently acquired image (for example, nth frame).
  • the vertical axis of these luminance histograms indicates the number of pixels.
  • the horizontal axis of these luminance histograms indicates luminance.
  • a brightness histogram is obtained by dividing the brightness into 16 based on the maximum value of RGB (Red, Green, Blue) of each pixel. That is, the luminance histogram is obtained by summarizing the luminance in 16 gradations. Note that the luminance histogram may be grouped by 1 gradation and 8 gradation.
  • the feature amount extraction unit 130 extracts a frame difference histogram by calculating the difference between the brightness histogram based on the image acquired last time and the brightness histogram based on the image acquired this time.
  • FIG. 5 shows a frame difference histogram.
  • the vertical axis of the frame difference histogram indicates the number of pixels.
  • the horizontal axis of the frame difference histogram indicates luminance. Note that the frame difference histogram may also be called a difference luminance histogram.
  • the feature quantity extraction unit 130 uses an image to extract a frequency histogram, which is one feature quantity. Moreover, the feature quantity extraction unit 130 may extract the number of luminance changes as a feature quantity. A method for extracting the number of luminance changes will be specifically described.
  • FIG. 6 is a diagram showing a specific example of a method for extracting the number of luminance changes according to the first embodiment.
  • the vertical axis of the graph in FIG. 6 indicates the luminance value.
  • the horizontal axis of the graph in FIG. 6 indicates pixel coordinates.
  • FIG. 6 shows nine pixels in the horizontal direction and luminance values of the nine pixels.
  • the feature amount extraction unit 130 extracts the number of pixels whose luminance difference values between adjacent pixels in the horizontal direction are equal to or greater than a threshold as the number of luminance changes.
  • the feature amount extraction unit 130 extracts the N+3 pixel as a pixel whose luminance has changed.
  • pixels whose brightness has changed are represented by colored circles.
  • FIG. 6 shows that the brightness change number is five.
  • the feature amount extraction unit 130 may extract the total value of the number of luminance changes as the feature amount.
  • the feature amount extraction unit 130 uses an image to extract a saturation histogram, which is one feature amount. A method for extracting a saturation histogram will be specifically described.
  • FIG. 7 is a diagram showing a specific example of a saturation histogram extraction method according to the first embodiment.
  • the vertical axis indicates the appearance frequency.
  • the horizontal axis indicates saturation.
  • the feature amount extraction unit 130 extracts saturation based on the maximum value when the U value and V value calculated in the CIE UVW color space are replaced with absolute values.
  • the feature amount extraction unit 130 extracts a saturation histogram expressed by dividing the saturation of 512 gradations into 16 parts.
  • the feature amount may be a feature amount other than the above.
  • the feature amount is the maximum luminance of the image, the minimum luminance of the image, the average luminance of the image, the value of the black area in the image, the value of the white area in the image, and the like.
  • the correction parameter detection unit 140 detects correction parameters using a plurality of types of feature quantities and learned models. That is, the correction parameter detection unit 140 detects correction parameters output by the learned model by inputting a plurality of types of feature amounts to the learned model.
  • the correction parameter is a parameter for correcting image quality of an image.
  • the correction parameter may be expressed as a parameter for improving the image quality of the image.
  • the trained model may consist of multiple layers of neural networks. Illustrate a neural network. 8 is a diagram showing an example of a neural network according to Embodiment 1.
  • FIG. A neural network consists of an input layer, an intermediate layer, and an output layer.
  • FIG. 8 shows that the number of intermediate layers is three.
  • the number of intermediate layers is not limited to three.
  • the number of neurons is not limited to the number in the example of FIG.
  • Multiple types of features are assigned to multiple neurons in the input layer.
  • one neuron in the input layer is assigned a horizontal edge amount.
  • the correction parameter y is calculated by Equation (1) based on multiple types of feature amounts input to multiple neurons.
  • n is the number of neurons in the input layer.
  • x1 to xn are a plurality of types of feature amounts.
  • b is the bias.
  • w is the weight. Bias b and weight w are determined by learning.
  • s indicates a function.
  • Function s(a) is the activation function.
  • the activation function s(a) may be a step function that outputs 0 if a is less than or equal to 0 and outputs 1 if a is not zero.
  • the activation function s(a) may be a ReLU function that outputs 0 if a is 0 or less and outputs the input value a if a is other than 0, or it outputs the input value a as it is. It may be an identity function or a sigmoid function.
  • the input layer neurons output the input values as they are. Therefore, it can be said that the activation function used in the neurons of the input layer is the identity function.
  • a step function or a sigmoid function may be used in the intermediate layer. It may be considered that the ReLU function is used in the output layer. Also, different functions may be used between neurons in the same layer.
  • the weight w is determined by learning.
  • An example of a method of calculating the weight w will be described.
  • Image quality correction processing in machine learning is defined such that noise reduction processing, sharpening processing, contrast improvement processing, and color conversion processing are performed.
  • the values used in the noise reduction process, sharpening process, contrast improvement process, and color conversion process are changed to improve the image quality.
  • a difference between an output value corresponding to an input pattern and a prepared correction parameter is calculated. The weight w is determined so that the difference becomes small.
  • the trained model may be composed of a random forest.
  • An example of a trained model that constitutes a random forest is shown.
  • 9 is a diagram showing an example of a trained model that constitutes the random forest according to Embodiment 1.
  • FIG. FIG. 9 shows a random forest 200.
  • FIG. Random forest 200 includes decision trees 210_1, 210_2, . . . , 210_n. n is a positive integer.
  • Decision tree 210 comprises a plurality of nodes. In the decision tree 210, tree-structured classification rules are created by stacking nodes.
  • Each of the multiple decision trees 210 is assigned one of multiple types of feature quantities.
  • decision tree 210_1 is assigned a horizontal edge amount.
  • the feature amount is input to the first layer node of the decision tree 210 .
  • the horizontal edge amount is input to the first layer node of the decision tree 210_1.
  • a branch condition is defined for the node.
  • the branch condition stipulates that the horizontal edge amount is 1000 or more.
  • the branch condition is determined by the condition that maximizes the information gain of the input data.
  • Information gain refers to the classification degree (eg, also referred to as impurity) of data when classifying data to the next node of a certain node x. Maximization of information gain means maximizing "(impurity before classification) - (impurity after classification)" at a certain node x, and minimizing the impurity after classification. do.
  • Impurity g(T) is calculated using equation (2).
  • g is the Gini coefficient.
  • the Gini coefficient is an index that indicates whether data classification is successful.
  • T indicates data input to the node.
  • t) indicates the number of data in a certain class for input data.
  • c indicates the number of classes.
  • the information gain IG is calculated using equation (3).
  • Tb indicates data before classification.
  • z indicates the number of nodes after classification.
  • Ni indicates the number of data in node i after classification.
  • Np indicates the number of data before classification.
  • Di indicates the data at node i after classification.
  • branching conditions are defined for nodes.
  • Branching conditions are defined by machine learning.
  • An example of a method of calculating a branching condition in machine learning will be described.
  • Image quality correction processing in machine learning is defined such that noise reduction processing, sharpening processing, contrast improvement processing, and color conversion processing are performed.
  • the values used in the noise reduction process, sharpening process, contrast improvement process, and color conversion process are changed to improve the image quality.
  • a difference between an output value corresponding to an input pattern and a prepared correction parameter is calculated. A branching condition is determined so that the difference becomes small.
  • the correction parameters are output by voting the results output from the plurality of decision trees 210 by majority.
  • the trained model may be a trained model generated by using a support vector machine. Therefore, the trained model reflects the support vector machine technology.
  • the learned model is exemplified.
  • FIG. 10 is a diagram showing an example of the support vector machine of Embodiment 1.
  • Support vector machines utilize linear input elements.
  • a support vector machine receives input of multiple types of features.
  • Each of the plurality of types of feature quantities is classified into two classes by linear input elements.
  • a linear input element is calculated using equation (4).
  • Equation (5) indicates input data (that is, feature amount).
  • w ⁇ T(x)+b denotes a linear straight line separating the data. Equations (5) and (6) are used to calculate Equation (4). Equation (5) is expressed as follows.
  • w indicates the slope of the linear input element.
  • ⁇ indicates a variable that weakens the constraint of Eq.
  • C denotes a positive value regularization factor.
  • t is a variable (ie, 1 or -1) for making "(w ⁇ T(x)+b)" a positive value.
  • a kernel function may be used when the input data cannot be classified into two by the linear input element.
  • a kernel function By using a kernel function, the input data is expanded into a multidimensional space and a plane that can be linearly classified with linear input elements is calculated.
  • the kernel function may be an RBF kernel, a polynomial kernel, a linear kernel, or a sigmoid kernel.
  • the linear input elements are calculated by machine learning.
  • An example of a method for calculating linear input elements in machine learning will be described.
  • Image quality correction processing in machine learning is defined such that noise reduction processing, sharpening processing, contrast improvement processing, and color conversion processing are performed.
  • the values used in the noise reduction process, sharpening process, contrast improvement process, and color conversion process are changed to improve the image quality.
  • a difference between an output value corresponding to an input pattern and a prepared correction parameter is calculated.
  • a linear input element is calculated so that the difference is small.
  • the image processing unit 150 uses the correction parameters to perform processing for correcting the image quality of the image. For example, the image processing unit 150 uses the correction parameters to perform processing for reducing noise in the image. For example, the process for reducing noise is low-pass filtering. For example, the image processing unit 150 corrects an image with much noise to an image with less noise using the correction parameters. Further, for example, the image processing unit 150 corrects an image including fine pattern noise to an image with less noise using a correction parameter.
  • the image processing unit 150 executes processing for sharpening the image using the correction parameters.
  • processing for sharpening an image is high-pass filtering.
  • the image processing unit 150 corrects a noisy image to a sharpened image using correction parameters.
  • the image processing unit 150 corrects a blurred image to a sharpened image using correction parameters.
  • the image processing unit 150 may use the correction parameters to perform processing for improving the contrast of the image, processing for converting the color of the image, and the like.
  • the output unit 160 outputs the corrected image.
  • the output unit 160 outputs the corrected image to a display. Thereby, the user can visually recognize the image with the optimum image quality. Illustration of the display is omitted.
  • the output unit 160 may output the corrected image to an external device.
  • the output section 160 may output the corrected image to the storage section 110 .
  • FIG. 11 is a flowchart illustrating an example of processing executed by the image processing apparatus according to Embodiment 1.
  • FIG. (Step S11) Acquisition unit 120 acquires an image.
  • Step S12) The feature amount extraction unit 130 extracts a plurality of types of feature amounts based on the image.
  • the correction parameter detection unit 140 detects correction parameters using a plurality of types of feature amounts and learned models.
  • the image processing unit 150 uses the correction parameters to perform processing for correcting the image quality of the image.
  • the output unit 160 outputs the corrected image.
  • the image processing apparatus 100 corrects the image using the correction parameters output from the trained model.
  • various images are input as learning data, and learning is performed to output appropriate correction parameters corresponding to the various images.
  • the learned model is generated. Therefore, the image processing apparatus 100 using the learned model can optimize the image quality of various images.
  • multiple types of feature values are extracted before multiple types of feature values are input to the trained model. That is, the trained model does not perform processing for extracting a plurality of types of feature amounts from the image. Therefore, according to Embodiment 1, it is possible to reduce the weight of the trained model and speed up the processing in the trained model.
  • Embodiment 2 Next, Embodiment 2 will be described. In Embodiment 2, mainly matters different from Embodiment 1 will be described. In the second embodiment, descriptions of items common to the first embodiment are omitted.
  • FIG. 12 is a block diagram showing functions of the image processing apparatus according to the second embodiment. 12 that are the same as those shown in FIG. 2 are assigned the same reference numerals as those shown in FIG.
  • the image processing device 100 further has a determination section 170 .
  • a part or all of the determination unit 170 may be implemented by a processing circuit. Also, part or all of the determination unit 170 may be implemented as a program module executed by the processor 101 . The function of the determination unit 170 will be described later in detail.
  • FIG. 13 is a flowchart illustrating an example of processing executed by the image processing apparatus according to the second embodiment; FIG. The process of FIG. 13 differs from the process of FIG. 11 in that step S13a is executed. Therefore, FIG. 13 demonstrates step S13a. A description of the processes other than step S13a is omitted.
  • Step S13a The determination unit 170 determines whether or not the difference between the detected correction parameter and the preset correction parameter is equal to or less than a preset threshold.
  • the acquisition unit 120 can acquire a plurality of images (for example, videos) for a certain period of time. Then, the feature quantity extraction unit 130 and the correction parameter detection unit 140 detect multiple correction parameters based on the multiple images. As a result, a plurality of correction parameters detected over a certain period of time are detected.
  • the preset correction parameter is an average value based on a plurality of correction parameters detected over a certain period of time.
  • step S14 If the difference is equal to or less than the threshold, the process proceeds to step S14. If the difference is greater than the threshold, the process ends.
  • the image processing apparatus 100 uses the threshold to determine whether or not to perform correction in order to suppress sudden changes in image quality. Therefore, according to the second embodiment, the image processing apparatus 100 can suppress sudden changes in image quality.
  • image processing device 101 processor, 102 volatile storage device, 103 non-volatile storage device, 110 storage unit, 120 acquisition unit, 130, 130_1, 130_2, ..., 130_n feature amount extraction unit, 140 correction parameter detection unit, 150 image processing unit, 160 output unit, 170 determination unit, 200 random forest, 210, 210_1, 210_2, ..., 210_n decision tree.

Abstract

An image processing device (100) includes: an acquisition unit (120) that acquires an image and a trained model; a plurality of feature amount extraction units (130_1 to 130_n) that extract a plurality of types of feature amounts on the basis of the image; a correction parameter detection unit (140) that uses the plurality of types of feature amounts and the trained model to detect a correction parameter, which is a parameter for correcting the quality of the image; and an image processing unit (150) that uses the correction parameter to execute a process for correcting the quality of the image.

Description

画像処理装置、画像処理方法、及び画像処理プログラムImage processing device, image processing method, and image processing program
　本開示は、画像処理装置、画像処理方法、及び画像処理プログラムに関する。 The present disclosure relates to an image processing device, an image processing method, and an image processing program.
　装置には、様々な解像度及び様々な圧縮方式の映像が入力される。例えば、様々な解像度は、８Ｋ、４Ｋ、フルＨＤ（Ｈｉｇｈ　Ｄｅｆｉｎｉｔｉｏｎ）、ＳＤ（Ｓｔａｎｄａｒｄ　Ｄｅｆｉｎｉｔｉｏｎ）などである。また、例えば、様々な圧縮方式は、Ｈ．２６５／ＨＥＶＣ、Ｈ．２６４／ＡＶＣ、ＭＰＥＧ２などである。また、装置には、高画質化機能が搭載されていることがある。そして、装置が最適な高画質化を行うことが、望まれる。例えば、ぼやけた映像を鮮鋭化する場合、当該映像を鮮鋭化するための処理が実行される。また、例えば、ノイズが多い映像が入力された場合、ノイズが強調されないための処理が実行される。ここで、画質を制御する画質制御装置が提案されている（特許文献１を参照）。例えば、特許文献１の画質制御装置は、映像信号に含まれる画像から得られる輝度情報、色度情報、色彩情報、及び周波数情報の全てのヒストグラムを生成する。画質制御装置は、予め設定された複数のヒストグラムパターンを有する参照テーブルから、全てのヒストグラムに対応するヒストグラムパターンを抽出する。画質制御装置は、抽出されたヒストグラムパターンに対応する制御パラメータに基づいて映像信号の画質を制御する。 Images with various resolutions and various compression methods are input to the device. For example, the various resolutions are 8K, 4K, full HD (High Definition), SD (Standard Definition), and the like. Also, for example, various compression schemes are specified in H.264. 265/HEVC, H.I. H.264/AVC, MPEG2, and the like. In addition, the device may be equipped with an image quality enhancement function. Then, it is desired that the apparatus performs optimal image quality enhancement. For example, when sharpening a blurry image, a process is performed to sharpen the image. Also, for example, when a video with a lot of noise is input, processing is executed to prevent the noise from being emphasized. Here, an image quality control device for controlling image quality has been proposed (see Patent Document 1). For example, the image quality control device of Patent Document 1 generates histograms of all luminance information, chromaticity information, color information, and frequency information obtained from an image included in a video signal. The image quality control device extracts histogram patterns corresponding to all histograms from a reference table having a plurality of preset histogram patterns. The image quality control device controls the image quality of the video signal based on control parameters corresponding to the extracted histogram pattern.
特許５６５１３４０号公報Japanese Patent No. 5651340
　上記の技術では、参照テーブルを用いて画質が制御される。しかし、様々な画像が入力される場合、参照テーブルに登録されている情報だけでは、画質の最適化は、困難である。 In the above technology, image quality is controlled using a lookup table. However, when various images are input, it is difficult to optimize image quality with only the information registered in the reference table.
　本開示の目的は、様々な画像に対して画質の最適化を行うことである。 The purpose of this disclosure is to optimize image quality for various images.
　本開示の一態様に係る画像処理装置が提供される。画像処理装置は、画像と学習済モデルとを取得する取得部と、前記画像に基づいて、複数の種類の特徴量を抽出する複数の特徴量抽出部と、前記複数の種類の特徴量と前記学習済モデルとを用いて、前記画像の画質を補正するためのパラメータである補正パラメータを検出する補正パラメータ検出部と、前記補正パラメータを用いて、前記画像の画質を補正するための処理を実行する画像処理部と、を有する。 An image processing device according to one aspect of the present disclosure is provided. The image processing apparatus includes an acquisition unit that acquires an image and a trained model, a plurality of feature amount extraction units that extract a plurality of types of feature amounts based on the image, and the plurality of types of feature amounts and the a correction parameter detection unit that detects a correction parameter, which is a parameter for correcting the image quality of the image, using the trained model; and a process for correcting the image quality of the image using the correction parameter. and an image processing unit for processing.
　本開示によれば、様々な画像に対して画質の最適化を行うことができる。 According to the present disclosure, image quality optimization can be performed for various images.
実施の形態１の画像処理装置が有するハードウェアを示す図である。2 illustrates hardware included in the image processing apparatus according to the first embodiment; FIG. 実施の形態１の画像処理装置の機能を示すブロック図である。2 is a block diagram showing functions of the image processing apparatus according to Embodiment 1; FIG. 実施の形態１の水平及び垂直エッジヒストグラムの例（その１）を示す図である。FIG. 4 is a diagram showing an example (part 1) of horizontal and vertical edge histograms according to Embodiment 1; 実施の形態１の水平及び垂直エッジヒストグラムの例（その２）を示す図である。FIG. 10 is a diagram showing an example (part 2) of horizontal and vertical edge histograms according to the first embodiment; 実施の形態１のフレーム差分ヒストグラムの抽出方法の具体例を示す図である。FIG. 4 is a diagram showing a specific example of a method for extracting a frame difference histogram according to Embodiment 1; 実施の形態１の輝度変化数の抽出方法の具体例を示す図である。FIG. 5 is a diagram showing a specific example of a method for extracting the number of changes in brightness according to the first embodiment; 実施の形態１の彩度ヒストグラムの抽出方法の具体例を示す図である。FIG. 4 is a diagram showing a specific example of a saturation histogram extraction method according to the first embodiment; 実施の形態１のニューラルネットワークの例を示す図である。1 is a diagram showing an example of a neural network according to Embodiment 1; FIG. 実施の形態１のランダムフォレストを構成する学習済モデルの例を示す図である。FIG. 2 is a diagram showing an example of a trained model that constitutes a random forest according to Embodiment 1; FIG. 実施の形態１のサポートベクターマシーンの例を示す図である。1 is a diagram showing an example of a support vector machine according to Embodiment 1; FIG. 実施の形態１の画像処理装置が実行する処理の例を示すフローチャートである。4 is a flowchart showing an example of processing executed by the image processing apparatus according to Embodiment 1; 実施の形態２の画像処理装置の機能を示すブロック図である。3 is a block diagram showing functions of an image processing apparatus according to a second embodiment; FIG. 実施の形態２の画像処理装置が実行する処理の例を示すフローチャートである。9 is a flow chart showing an example of processing executed by the image processing apparatus according to the second embodiment;
　以下、図面を参照しながら実施の形態を説明する。以下の実施の形態は、例にすぎず、本開示の範囲内で種々の変更が可能である。 Embodiments will be described below with reference to the drawings. The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.
実施の形態１．
　図１は、実施の形態１の画像処理装置が有するハードウェアを示す図である。画像処理装置１００は、画像処理方法を実行する装置である。画像処理装置１００は、プロセッサ１０１、揮発性記憶装置１０２、及び不揮発性記憶装置１０３を有する。 Embodiment 1.
FIG. 1 illustrates hardware included in an image processing apparatus according to a first embodiment. The image processing device 100 is a device that executes an image processing method. The image processing apparatus 100 has a processor 101 , a volatile memory device 102 and a nonvolatile memory device 103 .
　プロセッサ１０１は、画像処理装置１００全体を制御する。例えば、プロセッサ１０１は、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）、ＦＰＧＡ（Ｆｉｅｌｄ　Ｐｒｏｇｒａｍｍａｂｌｅ　Ｇａｔｅ　Ａｒｒａｙ）などである。プロセッサ１０１は、マルチプロセッサでもよい。また、画像処理装置１００は、処理回路を有してもよい。処理回路は、単一回路又は複合回路でもよい。 The processor 101 controls the image processing apparatus 100 as a whole. For example, the processor 101 is a CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), or the like. Processor 101 may be a multiprocessor. Also, the image processing apparatus 100 may have a processing circuit. The processing circuit may be a single circuit or multiple circuits.
　揮発性記憶装置１０２は、画像処理装置１００の主記憶装置である。例えば、揮発性記憶装置１０２は、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）である。不揮発性記憶装置１０３は、画像処理装置１００の補助記憶装置である。例えば、不揮発性記憶装置１０３は、ＨＤＤ（Ｈａｒｄ　Ｄｉｓｋ　Ｄｒｉｖｅ）、又はＳＳＤ（Ｓｏｌｉｄ　Ｓｔａｔｅ　Ｄｒｉｖｅ）である。 The volatile storage device 102 is the main storage device of the image processing device 100 . For example, the volatile memory device 102 is RAM (Random Access Memory). A nonvolatile storage device 103 is an auxiliary storage device of the image processing apparatus 100 . For example, the nonvolatile storage device 103 is a HDD (Hard Disk Drive) or an SSD (Solid State Drive).
　次に、画像処理装置１００が有する機能を説明する。
　図２は、実施の形態１の画像処理装置の機能を示すブロック図である。画像処理装置１００は、記憶部１１０、取得部１２０、特徴量抽出部１３０＿１，１３０＿２，・・・，１３０＿ｎ、補正パラメータ検出部１４０、画像処理部１５０、及び出力部１６０を有する。なお、ｎは、正の整数である。ここで、特徴量抽出部１３０＿１，１３０＿２，・・・，１３０＿ｎの総称は、特徴量抽出部１３０とする。 Next, functions of the image processing apparatus 100 will be described.
FIG. 2 is a block diagram showing functions of the image processing apparatus according to the first embodiment. The image processing apparatus 100 includes a storage unit 110, an acquisition unit 120, feature quantity extraction units 130_1, 130_2, . Note that n is a positive integer. Here, the feature quantity extraction units 130_1, 130_2, .
　記憶部１１０は、揮発性記憶装置１０２又は不揮発性記憶装置１０３に確保した記憶領域として実現してもよい。
　取得部１２０、特徴量抽出部１３０、補正パラメータ検出部１４０、画像処理部１５０、及び出力部１６０の一部又は全部は、処理回路によって実現してもよい。また、取得部１２０、特徴量抽出部１３０、補正パラメータ検出部１４０、画像処理部１５０、及び出力部１６０の一部又は全部は、プロセッサ１０１が実行するプログラムのモジュールとして実現してもよい。例えば、プロセッサ１０１が実行するプログラムは、画像処理プログラムとも言う。例えば、画像処理プログラムは、記録媒体に記録されている。 The storage unit 110 may be implemented as a storage area secured in the volatile storage device 102 or the nonvolatile storage device 103 .
A part or all of the acquisition unit 120, the feature amount extraction unit 130, the correction parameter detection unit 140, the image processing unit 150, and the output unit 160 may be realized by a processing circuit. Part or all of the acquisition unit 120, the feature amount extraction unit 130, the correction parameter detection unit 140, the image processing unit 150, and the output unit 160 may be implemented as modules of a program executed by the processor 101. For example, the program executed by the processor 101 is also called an image processing program. For example, an image processing program is recorded on a recording medium.
　ここで、以下の説明では、画像処理装置１００がデジタル画像を取得する場合を説明する。画像処理装置１００は、アナログ画像を取得してもよい。画像処理装置１００は、アナログ画像を取得する場合、画像処理装置１００は、Ａ（Ａｎａｌｏｇ）／Ｄ（Ｄｉｇｉｔａｌ）変換器を有する。 Here, in the following description, a case where the image processing apparatus 100 acquires a digital image will be described. The image processing device 100 may acquire analog images. When the image processing device 100 acquires an analog image, the image processing device 100 has an A (Analog)/D (Digital) converter.
　記憶部１１０は、画像と学習済モデルを記憶してもよい。また、記憶部１１０は、映像を記憶してもよい。
　取得部１２０は、画像を取得する。例えば、取得部１２０は、画像を記憶部１１０から取得する。また、例えば、取得部１２０は、画像をカメラから取得する。なお、カメラの図示は、省略されている。また、取得部１２０は、映像を取得する。言い換えれば、取得部１２０は、複数の画像を取得する。 The storage unit 110 may store images and trained models. Moreover, the storage unit 110 may store a video.
Acquisition unit 120 acquires an image. For example, the acquisition unit 120 acquires an image from the storage unit 110. FIG. Also, for example, the acquisition unit 120 acquires an image from a camera. Note that the illustration of the camera is omitted. Acquisition unit 120 also acquires video. In other words, the acquisition unit 120 acquires multiple images.
　また、取得部１２０は、学習済モデルを取得する。例えば、取得部１２０は、学習済モデルを記憶部１１０から取得する。ここで、学習済モデルは、外部装置（例えば、クラウドサーバ）に格納されてもよい。学習済モデルが外部装置に格納されている場合、取得部１２０は、外部装置から学習済モデルを取得する。 Also, the acquisition unit 120 acquires a learned model. For example, the acquisition unit 120 acquires the trained model from the storage unit 110. Here, the trained model may be stored in an external device (for example, cloud server). If the trained model is stored in the external device, the obtaining unit 120 obtains the trained model from the external device.
　特徴量抽出部１３０は、画像に基づいて、複数の種類の特徴量を抽出する。言い換えれば、特徴量抽出部１３０は、画像に基づいて、異なる種類の複数の特徴量を抽出する。特徴量の抽出処理を具体的に説明する。 The feature amount extraction unit 130 extracts a plurality of types of feature amounts based on the image. In other words, the feature amount extraction unit 130 extracts a plurality of feature amounts of different types based on the image. A specific description will be given of the feature amount extraction processing.
　特徴量抽出部１３０（例えば、特徴量抽出部１３０＿１）は、画像に基づいて、１つの特徴量である水平エッジヒストグラムを抽出する。詳細には、特徴量抽出部１３０は、画像の輝度値に基づいて、水平方向の隣接画素同士の輝度差分値をエッジ強調として算出する。例えば、特徴量抽出部１３０は、１０２４階調のエッジ強調を１６個に分割することにより表現された水平エッジヒストグラムを抽出する。水平エッジヒストグラムを例示する。 The feature quantity extraction unit 130 (for example, the feature quantity extraction unit 130_1) extracts a horizontal edge histogram, which is one feature quantity, based on the image. Specifically, the feature amount extraction unit 130 calculates a luminance difference value between adjacent pixels in the horizontal direction as edge enhancement based on the luminance value of the image. For example, the feature amount extraction unit 130 extracts a horizontal edge histogram expressed by dividing edge enhancement of 1024 gradations into 16 pieces. 4 illustrates a horizontal edge histogram;
　図３は、実施の形態１の水平エッジヒストグラムの例（その１）を示す図である。縦軸は、出現頻度を示している。横軸は、エッジ強度を示している。 FIG. 3 is a diagram showing an example (part 1) of a horizontal edge histogram according to the first embodiment. The vertical axis indicates the appearance frequency. The horizontal axis indicates edge strength.
　特徴量抽出部１３０は、１つの特徴量である水平エッジ量を抽出する。具体的に、水平エッジ量の抽出方法を説明する。 The feature amount extraction unit 130 extracts a horizontal edge amount, which is one feature amount. Specifically, a method for extracting the horizontal edge amount will be described.
　図４は、実施の形態１の水平エッジヒストグラムの例（その２）を示す図である。図４に示す水平エッジヒストグラムが、抽出されたものとする。特徴量抽出部１３０は、水平エッジヒストグラムの中の指定された範囲のエッジ強度の最小値（図４のｅｄｇｅ＿ｌｅｆｔ）と最大値（図４のｅｄｇｅ＿ｒｉｇｈｔ）との間の出現頻度の総量を、水平エッジ量として抽出する。なお、範囲は、予め設定された範囲でもよい。 FIG. 4 is a diagram showing an example (part 2) of the horizontal edge histogram according to the first embodiment. Assume that the horizontal edge histogram shown in FIG. 4 has been extracted. The feature quantity extraction unit 130 calculates the total frequency of appearance between the minimum value (edge_left in FIG. 4) and the maximum value (edge_right in FIG. 4) of the edge strength in the specified range in the horizontal edge histogram as a horizontal edge. Extract as quantity. Note that the range may be a preset range.
　また、特徴量抽出部１３０（例えば、特徴量抽出部１３０＿２）は、画像に基づいて、１つの特徴量である垂直エッジヒストグラムを抽出する。詳細には、特徴量抽出部１３０は、画像の輝度値に基づいて、垂直方向の隣接画素同士の輝度差分値をエッジ強調として算出する。例えば、特徴量抽出部１３０は、１０２４階調のエッジ強調を１６個に分割することにより表現された垂直エッジヒストグラムを抽出する。例えば、垂直エッジヒストグラムは、図３のように表現することができる。よって、図３は、垂直エッジヒストグラムの例を示す図と考えてもよい。 Also, the feature quantity extraction unit 130 (for example, the feature quantity extraction unit 130_2) extracts a vertical edge histogram, which is one feature quantity, based on the image. Specifically, the feature quantity extraction unit 130 calculates a luminance difference value between adjacent pixels in the vertical direction as edge enhancement based on the luminance value of the image. For example, the feature amount extraction unit 130 extracts a vertical edge histogram expressed by dividing edge enhancement of 1024 gradations into 16 pieces. For example, a vertical edge histogram can be expressed as shown in FIG. Therefore, FIG. 3 may be considered as a diagram showing an example of a vertical edge histogram.
　特徴量抽出部１３０は、１つの特徴量である垂直エッジ量を抽出する。垂直エッジ量の抽出方法は、水平エッジ量の抽出方法と同じである。例えば、図４を、垂直エッジヒストグラムの例と考えた場合、特徴量抽出部１３０は、垂直エッジヒストグラムの中の指定された範囲のエッジ強度の最小値と最大値との間の出現頻度の総量を、垂直エッジ量として抽出する。 The feature quantity extraction unit 130 extracts a vertical edge quantity, which is one feature quantity. The method for extracting the vertical edge amount is the same as the method for extracting the horizontal edge amount. For example, when considering FIG. 4 as an example of a vertical edge histogram, the feature quantity extraction unit 130 calculates the total frequency of appearance between the minimum value and the maximum value of edge strength in a specified range in the vertical edge histogram. is extracted as the vertical edge quantity.
　また、特徴量抽出部１３０（例えば、特徴量抽出部１３０＿３）は、画像を用いて、１つの特徴量であるフレーム差分ヒストグラムを抽出する。フレーム差分ヒストグラムの抽出方法を具体的に説明する。 Also, the feature quantity extraction unit 130 (for example, the feature quantity extraction unit 130_3) uses an image to extract a frame difference histogram, which is one feature quantity. A method for extracting a frame difference histogram will be specifically described.
　図５は、実施の形態１のフレーム差分ヒストグラムの抽出方法の具体例を示す図である。図５は、前回取得された画像（例えば、ｎ－１番目のフレームとも言う）に基づく輝度ヒストグラムと、今回取得された画像（例えば、ｎ番目のフレームとも言う）に基づく輝度ヒストグラムとを示している。 FIG. 5 is a diagram showing a specific example of a frame difference histogram extraction method according to the first embodiment. FIG. 5 shows a luminance histogram based on the previously acquired image (for example, n−1th frame) and a luminance histogram based on the currently acquired image (for example, nth frame). there is
　これらの輝度ヒストグラムの縦軸は、ピクセル数を示している。これらの輝度ヒストグラムの横軸は、輝度を示している。輝度ヒストグラムは、各画素のＲＧＢ（Ｒｅｄ、Ｇｒｅｅｎ、Ｂｌｕｅ）の最大値に基づいて、輝度を１６個に分割することにより得られる。すなわち、輝度ヒストグラムは、輝度を１６階調でまとめることで得られる。なお、輝度ヒストグラムは、１階調、８階調でまとめてもよい。 The vertical axis of these luminance histograms indicates the number of pixels. The horizontal axis of these luminance histograms indicates luminance. A brightness histogram is obtained by dividing the brightness into 16 based on the maximum value of RGB (Red, Green, Blue) of each pixel. That is, the luminance histogram is obtained by summarizing the luminance in 16 gradations. Note that the luminance histogram may be grouped by 1 gradation and 8 gradation.
　特徴量抽出部１３０は、前回取得された画像に基づく輝度ヒストグラムと、今回取得された画像に基づく輝度ヒストグラムとの差分を算出することにより、フレーム差分ヒストグラムを抽出する。図５は、フレーム差分ヒストグラムを示している。フレーム差分ヒストグラムの縦軸は、ピクセル数を示している。フレーム差分ヒストグラムの横軸は、輝度を示している。なお、フレーム差分ヒストグラムは、差分輝度ヒストグラムと呼んでもよい。 The feature amount extraction unit 130 extracts a frame difference histogram by calculating the difference between the brightness histogram based on the image acquired last time and the brightness histogram based on the image acquired this time. FIG. 5 shows a frame difference histogram. The vertical axis of the frame difference histogram indicates the number of pixels. The horizontal axis of the frame difference histogram indicates luminance. Note that the frame difference histogram may also be called a difference luminance histogram.
　また、特徴量抽出部１３０（例えば、特徴量抽出部１３０＿４）は、画像を用いて、１つの特徴量である周波数ヒストグラムを抽出する。また、特徴量抽出部１３０は、輝度変化数を特徴量として、抽出してもよい。輝度変化数の抽出方法を具体的に説明する。 Also, the feature quantity extraction unit 130 (for example, the feature quantity extraction unit 130_4) uses an image to extract a frequency histogram, which is one feature quantity. Moreover, the feature quantity extraction unit 130 may extract the number of luminance changes as a feature quantity. A method for extracting the number of luminance changes will be specifically described.
　図６は、実施の形態１の輝度変化数の抽出方法の具体例を示す図である。図６のグラフの縦軸は、輝度値を示している。図６のグラフの横軸は、ピクセル座標を示している。図６は、水平方向の９つの画素と、９つの画素の輝度値とを示している。特徴量抽出部１３０は、水平方向の隣接画素同士の輝度差分値が閾値以上である個数を、輝度変化数として抽出する。例えば、特徴量抽出部１３０は、Ｎ＋２の画素の輝度値と、Ｎ＋３の画素の輝度値との差分が閾値以上である場合、Ｎ＋３の画素を、輝度が変化した画素として抽出する。図６は、輝度が変化した画素を色付き丸で表現している。そして、図６は、輝度変化数が５であることを示している。
　また、特徴量抽出部１３０は、輝度変化数の合計値を特徴量として、抽出されてもよい。 FIG. 6 is a diagram showing a specific example of a method for extracting the number of luminance changes according to the first embodiment. The vertical axis of the graph in FIG. 6 indicates the luminance value. The horizontal axis of the graph in FIG. 6 indicates pixel coordinates. FIG. 6 shows nine pixels in the horizontal direction and luminance values of the nine pixels. The feature amount extraction unit 130 extracts the number of pixels whose luminance difference values between adjacent pixels in the horizontal direction are equal to or greater than a threshold as the number of luminance changes. For example, when the difference between the luminance value of the N+2 pixel and the luminance value of the N+3 pixel is equal to or greater than the threshold, the feature amount extraction unit 130 extracts the N+3 pixel as a pixel whose luminance has changed. In FIG. 6, pixels whose brightness has changed are represented by colored circles. FIG. 6 shows that the brightness change number is five.
Further, the feature amount extraction unit 130 may extract the total value of the number of luminance changes as the feature amount.
　特徴量抽出部１３０（例えば、特徴量抽出部１３０＿５）は、画像を用いて、１つの特徴量である彩度ヒストグラムを抽出する。彩度ヒストグラムの抽出方法を具体的に説明する。 The feature amount extraction unit 130 (for example, feature amount extraction unit 130_5) uses an image to extract a saturation histogram, which is one feature amount. A method for extracting a saturation histogram will be specifically described.
　図７は、実施の形態１の彩度ヒストグラムの抽出方法の具体例を示す図である。縦軸は、出現頻度を示している。横軸は、彩度を示している。例えば、特徴量抽出部１３０は、ＣＩＥ　ＵＶＷ色空間で算出されるＵ値及びＶ値を絶対値に置き換えたときの最大値に基づいて、彩度を抽出する。例えば、特徴量抽出部１３０は、５１２階調の彩度を１６個に分割することにより表現された彩度ヒストグラムを抽出する。 FIG. 7 is a diagram showing a specific example of a saturation histogram extraction method according to the first embodiment. The vertical axis indicates the appearance frequency. The horizontal axis indicates saturation. For example, the feature amount extraction unit 130 extracts saturation based on the maximum value when the U value and V value calculated in the CIE UVW color space are replaced with absolute values. For example, the feature amount extraction unit 130 extracts a saturation histogram expressed by dividing the saturation of 512 gradations into 16 parts.
　上記では、特徴量の例として、水平エッジ量、水平エッジヒストグラム、垂直エッジ量、垂直エッジヒストグラム、フレーム差分ヒストグラム、周波数ヒストグラム、及び彩度ヒストグラムを示した。特徴量は、上記以外の特徴量でもよい。例えば、特徴量は、画像の最大輝度、画像の最小輝度、画像の平均輝度、画像における黒面積の値、画像における白面積の値などである。 Above, horizontal edge amount, horizontal edge histogram, vertical edge amount, vertical edge histogram, frame difference histogram, frequency histogram, and saturation histogram are shown as examples of feature amounts. The feature amount may be a feature amount other than the above. For example, the feature amount is the maximum luminance of the image, the minimum luminance of the image, the average luminance of the image, the value of the black area in the image, the value of the white area in the image, and the like.
　補正パラメータ検出部１４０は、複数の種類の特徴量と学習済モデルとを用いて、補正パラメータを検出する。すなわち、補正パラメータ検出部１４０は、複数の種類の特徴量を学習済モデルに入力することで、学習済モデルが出力する補正パラメータを検出する。なお、補正パラメータは、画像の画質を補正するためのパラメータである。また、補正パラメータは、画像の画質を向上させるためのパラメータであると表現してもよい。 The correction parameter detection unit 140 detects correction parameters using a plurality of types of feature quantities and learned models. That is, the correction parameter detection unit 140 detects correction parameters output by the learned model by inputting a plurality of types of feature amounts to the learned model. Note that the correction parameter is a parameter for correcting image quality of an image. Also, the correction parameter may be expressed as a parameter for improving the image quality of the image.
　次に、学習済モデルを説明する。学習済モデルは、多層のニューラルネットワークで構成されてもよい。ニューラルネットワークを例示する。
　図８は、実施の形態１のニューラルネットワークの例を示す図である。ニューラルネットワークは、入力層、中間層、及び出力層で構成されている。図８は、中間層の数が３つであることを示している。中間層の数は、３つに限らない。また、ニューロンの数は、図８の例の数に限らない。 Next, the trained model will be explained. The trained model may consist of multiple layers of neural networks. Illustrate a neural network.
8 is a diagram showing an example of a neural network according to Embodiment 1. FIG. A neural network consists of an input layer, an intermediate layer, and an output layer. FIG. 8 shows that the number of intermediate layers is three. The number of intermediate layers is not limited to three. Also, the number of neurons is not limited to the number in the example of FIG.
　入力層の複数のニューロンには、複数の種類の特徴量が割り当てられる。例えば、入力層の１つのニューロンには、水平エッジ量が割り当てられる。
　例えば、補正パラメータｙは、複数のニューロンに入力された複数の種類の特徴量に基づいて、式（１）によって算出される。 Multiple types of features are assigned to multiple neurons in the input layer. For example, one neuron in the input layer is assigned a horizontal edge amount.
For example, the correction parameter y is calculated by Equation (1) based on multiple types of feature amounts input to multiple neurons.
Figure JPOXMLDOC01-appb-M000001
Figure JPOXMLDOC01-appb-M000001
　なお、ｎは、入力層のニューロンの数である。ｘ１～ｘｎは、複数の種類の特徴量である。ｂは、バイアスである。ｗは、重みである。バイアスｂ及び重みｗは、学習により定められる。 Note that n is the number of neurons in the input layer. x1 to xn are a plurality of types of feature amounts. b is the bias. w is the weight. Bias b and weight w are determined by learning.
　また、ｓは、関数を示している。当該関数を、関数ｓ（ａ）とする。関数ｓ（ａ）は、活性化関数である。例えば、活性化関数ｓ（ａ）は、ａが０以下であれば０を出力し、ａが０以外であれば１を出力するステップ関数でもよい。また、例えば、活性化関数ｓ（ａ）は、ａが０以下であれば０を出力し、ａが０以外ならば入力値ａを出力するＲｅＬＵ関数でもよいし、入力値ａをそのまま出力する恒等関数でもよいし、ジグモイド関数でもよい。 Also, s indicates a function. Let this function be function s(a). Function s(a) is the activation function. For example, the activation function s(a) may be a step function that outputs 0 if a is less than or equal to 0 and outputs 1 if a is not zero. Further, for example, the activation function s(a) may be a ReLU function that outputs 0 if a is 0 or less and outputs the input value a if a is other than 0, or it outputs the input value a as it is. It may be an identity function or a sigmoid function.
　なお、入力層のニューロンは、入力値をそのまま出力する。そのため、入力層のニューロンで用いられる活性化関数は、恒等関数であると言える。中間層では、ステップ関数又はジグモイド関数が用いられると考えてもよい。出力層では、ＲｅＬＵ関数が用いられると考えてもよい。また、同じ層内のニューロン相互間では、異なる関数が用いられてもよい。 Note that the input layer neurons output the input values as they are. Therefore, it can be said that the activation function used in the neurons of the input layer is the identity function. A step function or a sigmoid function may be used in the intermediate layer. It may be considered that the ReLU function is used in the output layer. Also, different functions may be used between neurons in the same layer.
　ここで、上記したように、重みｗは、学習により定められる。重みｗの算出方法の一例を説明する。まず、複数の画像と、当該複数の画像の画質を補正するための補正パラメータとが用意される。機械学習における画質の補正処理では、ノイズ軽減処理、鮮鋭化処理、コントラスト改善処理、及び色変換処理が行われるように定義される。また、当該補正処理では、ノイズ軽減処理、鮮鋭化処理、コントラスト改善処理、及び色変換処理で用いられる値を変化して画質が向上するように、処理が行われる。機械学習では、入力パターンに対応する出力値と、用意された補正パラメータとの差が算出される。当該差が小さくなるように、重みｗが定められる。 Here, as described above, the weight w is determined by learning. An example of a method of calculating the weight w will be described. First, a plurality of images and correction parameters for correcting the image quality of the plurality of images are prepared. Image quality correction processing in machine learning is defined such that noise reduction processing, sharpening processing, contrast improvement processing, and color conversion processing are performed. Further, in the correction process, the values used in the noise reduction process, sharpening process, contrast improvement process, and color conversion process are changed to improve the image quality. In machine learning, a difference between an output value corresponding to an input pattern and a prepared correction parameter is calculated. The weight w is determined so that the difference becomes small.
　また、学習済モデルは、ランダムフォレストで構成されてもよい。ランダムフォレストを構成する学習済モデルを例示する。
　図９は、実施の形態１のランダムフォレストを構成する学習済モデルの例を示す図である。図９は、ランダムフォレスト２００を示している。ランダムフォレスト２００は、決定木２１０＿１，２１０＿２，・・・，２１０＿ｎを含む。ｎは、正の整数である。ここで、決定木２１０＿１，２１０＿２，・・・，２１０＿ｎの総称を、決定木２１０と呼ぶ。決定木２１０は、複数のノードを構成する。決定木２１０では、ノードを積み重ねることで、木構造の分類ルールが作成される。 Also, the trained model may be composed of a random forest. An example of a trained model that constitutes a random forest is shown.
9 is a diagram showing an example of a trained model that constitutes the random forest according to Embodiment 1. FIG. FIG. 9 shows a random forest 200. FIG. Random forest 200 includes decision trees 210_1, 210_2, . . . , 210_n. n is a positive integer. Here, the decision trees 210_1, 210_2, . Decision tree 210 comprises a plurality of nodes. In the decision tree 210, tree-structured classification rules are created by stacking nodes.
　複数の決定木２１０のそれぞれには、複数の種類の特徴量のいずれかが割り当てられる。例えば、決定木２１０＿１には、水平エッジ量が割り当てられる。そして、特徴量は、決定木２１０の１層目のノードに入力される。例えば、水平エッジ量は、決定木２１０＿１の１層目のノードに入力される。ノードには、分岐条件が定められている。例えば、分岐条件には、水平エッジ量が１０００以上であることという条件が定められている。分岐条件は、入力されたデータの情報利得が最大となる条件によって決定される。情報利得は、あるノードｘの次のノードにデータを分類する際のデータの分類度（例えば、不純度とも言う）を指す。情報利得の最大化は、あるノードｘにおいて、“（分類前の不純度）－（分類後の不純度）”を最大化することを意味し、分類後の不純度を最小化することを意味する。不純度ｇ（Ｔ）は、式（２）を用いて算出される。 Each of the multiple decision trees 210 is assigned one of multiple types of feature quantities. For example, decision tree 210_1 is assigned a horizontal edge amount. Then, the feature amount is input to the first layer node of the decision tree 210 . For example, the horizontal edge amount is input to the first layer node of the decision tree 210_1. A branch condition is defined for the node. For example, the branch condition stipulates that the horizontal edge amount is 1000 or more. The branch condition is determined by the condition that maximizes the information gain of the input data. Information gain refers to the classification degree (eg, also referred to as impurity) of data when classifying data to the next node of a certain node x. Maximization of information gain means maximizing "(impurity before classification) - (impurity after classification)" at a certain node x, and minimizing the impurity after classification. do. Impurity g(T) is calculated using equation (2).
Figure JPOXMLDOC01-appb-M000002
Figure JPOXMLDOC01-appb-M000002
　ｇは、ジニ係数である。ジニ係数は、データの分類が成功しているか否かを示す指標である。Ｔは、ノードに入力されるデータを示す。ｐ（ｉ｜ｔ）は、入力されるデータに対する、あるクラスのデータ数を示す。ｃは、クラスの数を示す。
　情報利得ＩＧは、式（３）を用いて算出される。 g is the Gini coefficient. The Gini coefficient is an index that indicates whether data classification is successful. T indicates data input to the node. p(i|t) indicates the number of data in a certain class for input data. c indicates the number of classes.
The information gain IG is calculated using equation (3).
Figure JPOXMLDOC01-appb-M000003
Figure JPOXMLDOC01-appb-M000003
　Ｔｂは、分類前のデータを示す。ｚは、分類後のノード数を示す。Ｎｉは、分類後のノードｉにおけるデータ数を示す。Ｎｐは、分類前のデータ数を示す。Ｄｉは、分類後のノードｉにおけるデータを示す。
　ノードｘに入力されたデータの不純度と、ノードｘにおける分類後の不純度との差が大きくなる時、情報利得は、最大化され、当該データを最もよく分類することができる。
　式（３）では、ジニ係数ｇが用いられた。式（３）には、エントロピー又は分類誤差が用いられてもよい。 Tb indicates data before classification. z indicates the number of nodes after classification. Ni indicates the number of data in node i after classification. Np indicates the number of data before classification. Di indicates the data at node i after classification.
When the difference between the impurity of the data input to node x and the impurity after classification at node x is large, the information gain is maximized and the data can be best classified.
In equation (3), the Gini coefficient g was used. Entropy or classification error may be used in equation (3).
　ここで、上記したように、ノードには、分岐条件が定められている。分岐条件は、機械学習で定められる。機械学習における分岐条件の算出方法の一例を説明する。まず、複数の画像と、当該複数の画像の画質を補正するための補正パラメータとが用意される。機械学習における画質の補正処理では、ノイズ軽減処理、鮮鋭化処理、コントラスト改善処理、及び色変換処理が行われるように定義される。また、当該補正処理では、ノイズ軽減処理、鮮鋭化処理、コントラスト改善処理、及び色変換処理で用いられる値を変化して画質が向上するように、処理が行われる。機械学習では、入力パターンに対応する出力値と、用意された補正パラメータとの差が算出される。当該差が小さくなるように、分岐条件が定められる。 Here, as described above, branching conditions are defined for nodes. Branching conditions are defined by machine learning. An example of a method of calculating a branching condition in machine learning will be described. First, a plurality of images and correction parameters for correcting the image quality of the plurality of images are prepared. Image quality correction processing in machine learning is defined such that noise reduction processing, sharpening processing, contrast improvement processing, and color conversion processing are performed. Further, in the correction process, the values used in the noise reduction process, sharpening process, contrast improvement process, and color conversion process are changed to improve the image quality. In machine learning, a difference between an output value corresponding to an input pattern and a prepared correction parameter is calculated. A branching condition is determined so that the difference becomes small.
　ランダムフォレスト２００では、複数の決定木２１０から出力された結果を多数決することで、補正パラメータが出力される。 In the random forest 200, the correction parameters are output by voting the results output from the plurality of decision trees 210 by majority.
　また、学習済モデルは、サポートベクターマシーンを用いることで生成された学習済モデルでもよい。よって、学習済モデルには、サポートベクターマシーンの技術が反映されている。当該学習済モデルを例示する。 Also, the trained model may be a trained model generated by using a support vector machine. Therefore, the trained model reflects the support vector machine technology. The learned model is exemplified.
　図１０は、実施の形態１のサポートベクターマシーンの例を示す図である。サポートベクターマシーンでは、線形入力素子が利用される。サポートベクターマシーンには、複数の種類の特徴量が入力される。複数の種類の特徴量のそれぞれは、線形入力素子により、２つのクラスに分類される。線形入力素子は、式（４）を用いて算出される。 FIG. 10 is a diagram showing an example of the support vector machine of Embodiment 1. FIG. Support vector machines utilize linear input elements. A support vector machine receives input of multiple types of features. Each of the plurality of types of feature quantities is classified into two classes by linear input elements. A linear input element is calculated using equation (4).
Figure JPOXMLDOC01-appb-M000004
Figure JPOXMLDOC01-appb-M000004
　ｘは、入力されるデータ（すなわち、特徴量）を示す。“ｗ＾Ｔ（ｘ）＋ｂ”は、データを分離する線形な直線を示す。式（４）の算出には、式（５）と式（６）が用いられる。式（５）は、次のように表される。 x indicates input data (that is, feature amount). "w^T(x)+b" denotes a linear straight line separating the data. Equations (5) and (6) are used to calculate Equation (4). Equation (5) is expressed as follows.
Figure JPOXMLDOC01-appb-M000005
Figure JPOXMLDOC01-appb-M000005
　式（６）は、次のように表される。 Formula (6) is expressed as follows.
Figure JPOXMLDOC01-appb-M000006
Figure JPOXMLDOC01-appb-M000006
　ｗは、線形入力素子の傾きを示す。εは、式６の制約条件を弱める変数を示す。Ｃは、正の値の正則化係数を示す。ｔは、“（ｗ＾Ｔ（ｘ）＋ｂ）”を正の値にするための変数(すなわち、１又は－１)である。εとＣによって、線形入力素子の分類失敗を許容することで、最適な線形入力素子が算出される。 w indicates the slope of the linear input element. ε indicates a variable that weakens the constraint of Eq. C denotes a positive value regularization factor. t is a variable (ie, 1 or -1) for making "(w^T(x)+b)" a positive value. With ε and C, the optimal linear input element is calculated by allowing the classification failure of the linear input element.
　線形入力素子によって、入力されるデータを２つに分類できない場合、カーネル関数が用いられてもよい。カーネル関数を用いることで、入力データが多次元空間に拡張され、線形に線形入力素子で分類できる平面が算出される。また、カーネル関数は、ＲＢＦカーネル、多項式カーネル、線形カーネル、又はシグモイドカーネルでもよい。 A kernel function may be used when the input data cannot be classified into two by the linear input element. By using a kernel function, the input data is expanded into a multidimensional space and a plane that can be linearly classified with linear input elements is calculated. Also, the kernel function may be an RBF kernel, a polynomial kernel, a linear kernel, or a sigmoid kernel.
　ここで、線形入力素子は、機械学習で算出される。機械学習における線形入力素子の算出方法の一例を説明する。まず、複数の画像と、当該複数の画像の画質を補正するための補正パラメータとが用意される。機械学習における画質の補正処理では、ノイズ軽減処理、鮮鋭化処理、コントラスト改善処理、及び色変換処理が行われるように定義される。また、当該補正処理では、ノイズ軽減処理、鮮鋭化処理、コントラスト改善処理、及び色変換処理で用いられる値を変化して画質が向上するように、処理が行われる。機械学習では、入力パターンに対応する出力値と、用意された補正パラメータとの差が算出される。当該差が小さくなるように、線形入力素子が算出される。 Here, the linear input elements are calculated by machine learning. An example of a method for calculating linear input elements in machine learning will be described. First, a plurality of images and correction parameters for correcting the image quality of the plurality of images are prepared. Image quality correction processing in machine learning is defined such that noise reduction processing, sharpening processing, contrast improvement processing, and color conversion processing are performed. Further, in the correction process, the values used in the noise reduction process, sharpening process, contrast improvement process, and color conversion process are changed to improve the image quality. In machine learning, a difference between an output value corresponding to an input pattern and a prepared correction parameter is calculated. A linear input element is calculated so that the difference is small.
　画像処理部１５０は、補正パラメータを用いて、画像の画質を補正するための処理を実行する。
　例えば、画像処理部１５０は、補正パラメータを用いて、画像のノイズを軽減する処理を実行する。例えば、ノイズを軽減する処理は、ローパスフィルタ処理である。例えば、画像処理部１５０は、補正パラメータを用いて、ノイズの多い画像を、ノイズの少ない画像に補正する。また、例えば、画像処理部１５０は、補正パラメータを用いて、細かい模様のノイズを含む画像を、当該ノイズの少ない画像に補正する。 The image processing unit 150 uses the correction parameters to perform processing for correcting the image quality of the image.
For example, the image processing unit 150 uses the correction parameters to perform processing for reducing noise in the image. For example, the process for reducing noise is low-pass filtering. For example, the image processing unit 150 corrects an image with much noise to an image with less noise using the correction parameters. Further, for example, the image processing unit 150 corrects an image including fine pattern noise to an image with less noise using a correction parameter.
　また、例えば、画像処理部１５０は、補正パラメータを用いて、画像を鮮鋭化するための処理を実行する。例えば、画像を鮮鋭化するための処理は、ハイパスフィルタ処理である。例えば、画像処理部１５０は、補正パラメータを用いて、ノイズの多い画像を、鮮鋭化された画像に補正する。また、例えば、画像処理部１５０は、補正パラメータを用いて、ぼやけた画像を、鮮鋭化された画像に補正する。 Also, for example, the image processing unit 150 executes processing for sharpening the image using the correction parameters. For example, processing for sharpening an image is high-pass filtering. For example, the image processing unit 150 corrects a noisy image to a sharpened image using correction parameters. Also, for example, the image processing unit 150 corrects a blurred image to a sharpened image using correction parameters.
　画像処理部１５０は、補正パラメータを用いて、画像のコントラストを改善するための処理、画像の色を変換するための処理などを実行してもよい。 The image processing unit 150 may use the correction parameters to perform processing for improving the contrast of the image, processing for converting the color of the image, and the like.
　出力部１６０は、補正された画像を出力する。例えば、出力部１６０は、補正された画像を、ディスプレイに出力する。これにより、ユーザは、最適な画質の画像を視認することができる。なお、ディスプレイの図示は、省略されている。また、出力部１６０は、補正された画像を、外部装置に出力してもよい。出力部１６０は、補正された画像を、記憶部１１０に出力してもよい。 The output unit 160 outputs the corrected image. For example, the output unit 160 outputs the corrected image to a display. Thereby, the user can visually recognize the image with the optimum image quality. Illustration of the display is omitted. Also, the output unit 160 may output the corrected image to an external device. The output section 160 may output the corrected image to the storage section 110 .
　次に、画像処理装置１００が実行する処理を、フローチャートを用いて、説明する。
　図１１は、実施の形態１の画像処理装置が実行する処理の例を示すフローチャートである。
　（ステップＳ１１）取得部１２０は、画像を取得する。
　（ステップＳ１２）特徴量抽出部１３０は、画像に基づいて、複数の種類の特徴量を抽出する。 Next, processing executed by the image processing apparatus 100 will be described using a flowchart.
11 is a flowchart illustrating an example of processing executed by the image processing apparatus according to Embodiment 1. FIG.
(Step S11) Acquisition unit 120 acquires an image.
(Step S12) The feature amount extraction unit 130 extracts a plurality of types of feature amounts based on the image.
　（ステップＳ１３）補正パラメータ検出部１４０は、複数の種類の特徴量と学習済モデルとを用いて、補正パラメータを検出する。
　（ステップＳ１４）画像処理部１５０は、補正パラメータを用いて、画像の画質を補正するための処理を実行する。
　（ステップＳ１５）出力部１６０は、補正された画像を出力する。 (Step S13) The correction parameter detection unit 140 detects correction parameters using a plurality of types of feature amounts and learned models.
(Step S14) The image processing unit 150 uses the correction parameters to perform processing for correcting the image quality of the image.
(Step S15) The output unit 160 outputs the corrected image.
　実施の形態１によれば、画像処理装置１００は、学習済モデルから出力された補正パラメータを用いて、画像を補正する。当該学習済モデルの学習フェーズでは、様々な画像が学習データとして入力され、当該様々な画像に対応する適切な補正パラメータを出力するための学習が行われる。このような学習により、当該学習済モデルが生成される。そのため、当該学習済モデルを用いる画像処理装置１００は、様々な画像に対して画質の最適化を行うことができる。 According to Embodiment 1, the image processing apparatus 100 corrects the image using the correction parameters output from the trained model. In the learning phase of the learned model, various images are input as learning data, and learning is performed to output appropriate correction parameters corresponding to the various images. Through such learning, the learned model is generated. Therefore, the image processing apparatus 100 using the learned model can optimize the image quality of various images.
　また、複数の種類の特徴量が学習済モデルに入力される前に、複数の種類の特徴量が抽出される。すなわち、学習済モデルでは、複数の種類の特徴量を画像から抽出する処理が行われない。よって、実施の形態１によれば、学習済モデルの軽量化及び学習済モデルにおける処理の高速化が実現できる。 In addition, multiple types of feature values are extracted before multiple types of feature values are input to the trained model. That is, the trained model does not perform processing for extracting a plurality of types of feature amounts from the image. Therefore, according to Embodiment 1, it is possible to reduce the weight of the trained model and speed up the processing in the trained model.
実施の形態２．
　次に、実施の形態２を説明する。実施の形態２では、実施の形態１と相違する事項を主に説明する。そして、実施の形態２では、実施の形態１と共通する事項の説明を省略する。 Embodiment 2.
Next, Embodiment 2 will be described. In Embodiment 2, mainly matters different from Embodiment 1 will be described. In the second embodiment, descriptions of items common to the first embodiment are omitted.
　図１２は、実施の形態２の画像処理装置の機能を示すブロック図である。図２に示される構成と同じ図１２の構成は、図２に示される符号と同じ符号を付している。
　画像処理装置１００は、さらに、判定部１７０を有する。判定部１７０の一部又は全部は、処理回路によって実現してもよい。また、判定部１７０の一部又は全部は、プロセッサ１０１が実行するプログラムのモジュールとして実現してもよい。
　判定部１７０の機能は、後で詳細に説明する。 FIG. 12 is a block diagram showing functions of the image processing apparatus according to the second embodiment. 12 that are the same as those shown in FIG. 2 are assigned the same reference numerals as those shown in FIG.
The image processing device 100 further has a determination section 170 . A part or all of the determination unit 170 may be implemented by a processing circuit. Also, part or all of the determination unit 170 may be implemented as a program module executed by the processor 101 .
The function of the determination unit 170 will be described later in detail.
　次に、画像処理装置１００が実行する処理を、フローチャートを用いて説明する。
　図１３は、実施の形態２の画像処理装置が実行する処理の例を示すフローチャートである。図１３の処理は、ステップＳ１３ａが実行される点が図１１の処理と異なる。そのため、図１３では、ステップＳ１３ａを説明する。そして、ステップＳ１３ａ以外の処理の説明は、省略する。 Next, processing executed by the image processing apparatus 100 will be described using a flowchart.
13 is a flowchart illustrating an example of processing executed by the image processing apparatus according to the second embodiment; FIG. The process of FIG. 13 differs from the process of FIG. 11 in that step S13a is executed. Therefore, FIG. 13 demonstrates step S13a. A description of the processes other than step S13a is omitted.
　（ステップＳ１３ａ）判定部１７０は、検出された補正パラメータと予め設定された補正パラメータとの差分が予め設定された閾値以下であるか否かを判定する。 (Step S13a) The determination unit 170 determines whether or not the difference between the detected correction parameter and the preset correction parameter is equal to or less than a preset threshold.
　ここで、取得部１２０は、一定期間、複数の画像（例えば、映像）を取得することができる。そして、特徴量抽出部１３０及び補正パラメータ検出部１４０は、複数の画像に基づいて、複数の補正パラメータを検出する。これにより、一定期間に検出された複数の補正パラメータが検出される。例えば、予め設定された補正パラメータは、一定期間に検出された複数の補正パラメータに基づく平均値である。 Here, the acquisition unit 120 can acquire a plurality of images (for example, videos) for a certain period of time. Then, the feature quantity extraction unit 130 and the correction parameter detection unit 140 detect multiple correction parameters based on the multiple images. As a result, a plurality of correction parameters detected over a certain period of time are detected. For example, the preset correction parameter is an average value based on a plurality of correction parameters detected over a certain period of time.
　当該差分が当該閾値以下である場合、処理は、ステップＳ１４に進む。当該差分が当該閾値よりも大きい場合、処理は、終了する。 If the difference is equal to or less than the threshold, the process proceeds to step S14. If the difference is greater than the threshold, the process ends.
　ここで、ステップＳ１３ａでＮｏのときに、画像処理装置１００が画像を補正した場合、急激な画質の変化が起こる。そこで、画像処理装置１００は、急激な画質の変化を抑えるために、当該閾値を用いて、補正するか否かを判定する。よって、実施の形態２によれば、画像処理装置１００は、急激な画質の変化を抑えることができる。 Here, if the image processing apparatus 100 corrects the image when the answer to step S13a is No, the image quality will change abruptly. Therefore, the image processing apparatus 100 uses the threshold to determine whether or not to perform correction in order to suppress sudden changes in image quality. Therefore, according to the second embodiment, the image processing apparatus 100 can suppress sudden changes in image quality.
　以上に説明した各実施の形態における特徴は、互いに適宜組み合わせることができる。 The features of each embodiment described above can be combined as appropriate.
　１００　画像処理装置、　１０１　プロセッサ、　１０２　揮発性記憶装置、　１０３　不揮発性記憶装置、　１１０　記憶部、　１２０　取得部、　１３０，１３０＿１，１３０＿２，・・・，１３０＿ｎ　特徴量抽出部、　１４０　補正パラメータ検出部、　１５０　画像処理部、　１６０　出力部、　１７０　判定部、　２００　ランダムフォレスト、　２１０，２１０＿１，２１０＿２，・・・，２１０＿ｎ　決定木。 100 image processing device, 101 processor, 102 volatile storage device, 103 non-volatile storage device, 110 storage unit, 120 acquisition unit, 130, 130_1, 130_2, ..., 130_n feature amount extraction unit, 140 correction parameter detection unit, 150 image processing unit, 160 output unit, 170 determination unit, 200 random forest, 210, 210_1, 210_2, ..., 210_n decision tree.

Claims (7)

  1. 　画像と学習済モデルとを取得する取得部と、
    　前記画像に基づいて、複数の種類の特徴量を抽出する複数の特徴量抽出部と、
    　前記複数の種類の特徴量と前記学習済モデルとを用いて、前記画像の画質を補正するためのパラメータである補正パラメータを検出する補正パラメータ検出部と、
    　前記補正パラメータを用いて、前記画像の画質を補正するための処理を実行する画像処理部と、
    　を有する画像処理装置。     an acquisition unit that acquires an image and a trained model;
    a plurality of feature quantity extraction units for extracting a plurality of types of feature quantity based on the image;
    a correction parameter detection unit that detects a correction parameter, which is a parameter for correcting image quality of the image, using the plurality of types of feature amounts and the learned model;
    an image processing unit that executes processing for correcting image quality of the image using the correction parameter;
    An image processing device having
  2. 　検出された前記補正パラメータと、予め設定された補正パラメータとの差分が予め設定された閾値以下であるか否かを判定する判定部をさらに有し、
    　前記画像処理部は、前記差分が前記閾値以下である場合、検出された前記補正パラメータを用いて、前記画像の画質を補正するための処理を実行する、
    　請求項１に記載の画像処理装置。     further comprising a determination unit that determines whether a difference between the detected correction parameter and a preset correction parameter is equal to or less than a preset threshold;
    When the difference is equal to or less than the threshold, the image processing unit uses the detected correction parameter to perform processing for correcting the image quality of the image.
    The image processing apparatus according to claim 1.
  3. 　前記学習済モデルは、ニューラルネットワーク、ランダムフォレスト、サポートベクターマシーンのいずれかで構成される、
    　請求項１又は２に記載の画像処理装置。     The trained model consists of either a neural network, a random forest, or a support vector machine,
    The image processing apparatus according to claim 1 or 2.
  4. 　前記複数の種類の特徴量は、水平エッジ量、水平エッジヒストグラム、垂直エッジ量、垂直エッジヒストグラム、フレーム差分ヒストグラム、周波数ヒストグラム、彩度ヒストグラム、前記画像の最大輝度、前記画像の最小輝度、前記画像の平均輝度、前記画像における黒面積の値、及び前記画像における白面積の値のうちの２以上の種類の特徴量である、
    　請求項１から３のいずれか１項に記載の画像処理装置。     The plurality of types of feature amounts include horizontal edge amount, horizontal edge histogram, vertical edge amount, vertical edge histogram, frame difference histogram, frequency histogram, saturation histogram, maximum luminance of the image, minimum luminance of the image, and the image. Two or more types of feature quantities among the average luminance of the image, the value of the black area in the image, and the value of the white area in the image,
    The image processing apparatus according to any one of claims 1 to 3.
  5. 　補正された前記画像を出力する出力部をさらに有する、
    　請求項１から４のいずれか１項に記載の画像処理装置。     further comprising an output unit for outputting the corrected image;
    The image processing apparatus according to any one of claims 1 to 4.
  6. 　画像処理装置が、
    　画像と学習済モデルとを取得し、
    　前記画像に基づいて、複数の種類の特徴量を抽出し、
    　前記複数の種類の特徴量と前記学習済モデルとを用いて、前記画像の画質を補正するためのパラメータである補正パラメータを検出し、
    　前記補正パラメータを用いて、前記画像の画質を補正するための処理を実行する、
    　画像処理方法。     The image processing device
    Get an image and a trained model,
    Based on the image, extracting a plurality of types of feature amounts,
    detecting a correction parameter, which is a parameter for correcting the image quality of the image, using the plurality of types of feature amounts and the learned model;
    using the correction parameter to perform processing for correcting the image quality of the image;
    Image processing method.
  7. 　画像処理装置に、
    　画像と学習済モデルとを取得し、
    　前記画像に基づいて、複数の種類の特徴量を抽出し、
    　前記複数の種類の特徴量と前記学習済モデルとを用いて、前記画像の画質を補正するためのパラメータである補正パラメータを検出し、
    　前記補正パラメータを用いて、前記画像の画質を補正するための処理を実行する、
    　処理を実行させる画像処理プログラム。
          image processing equipment,
    Get an image and a trained model,
    Based on the image, extracting a plurality of types of feature amounts,
    detecting a correction parameter, which is a parameter for correcting the image quality of the image, using the plurality of types of feature amounts and the learned model;
    using the correction parameter to perform processing for correcting the image quality of the image;
    An image processing program that executes processing.
PCT/JP2021/021592 2021-06-07 2021-06-07 Image processing device, image processing method, and image processing program WO2022259323A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023527167A JPWO2022259323A1 (en) 2021-06-07 2021-06-07
PCT/JP2021/021592 WO2022259323A1 (en) 2021-06-07 2021-06-07 Image processing device, image processing method, and image processing program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/021592 WO2022259323A1 (en) 2021-06-07 2021-06-07 Image processing device, image processing method, and image processing program

Publications (1)

Publication Number Publication Date
WO2022259323A1 true WO2022259323A1 (en) 2022-12-15

Family

ID=84424995

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/021592 WO2022259323A1 (en) 2021-06-07 2021-06-07 Image processing device, image processing method, and image processing program

Country Status (2)

Country Link
JP (1) JPWO2022259323A1 (en)
WO (1) WO2022259323A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003187215A (en) * 2001-12-18 2003-07-04 Fuji Xerox Co Ltd Image processing system and image processing server
JP2009151350A (en) * 2007-12-18 2009-07-09 Nec Corp Image correction method and device
WO2018150685A1 (en) * 2017-02-20 2018-08-23 ソニー株式会社 Image processing device, image processing method, and program
JP2020188484A (en) * 2016-06-02 2020-11-19 ソニー株式会社 Image processing device and image processing method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003187215A (en) * 2001-12-18 2003-07-04 Fuji Xerox Co Ltd Image processing system and image processing server
JP2009151350A (en) * 2007-12-18 2009-07-09 Nec Corp Image correction method and device
JP2020188484A (en) * 2016-06-02 2020-11-19 ソニー株式会社 Image processing device and image processing method
WO2018150685A1 (en) * 2017-02-20 2018-08-23 ソニー株式会社 Image processing device, image processing method, and program

Also Published As

Publication number Publication date
JPWO2022259323A1 (en) 2022-12-15

Similar Documents

Publication Publication Date Title
US10339643B2 (en) Algorithm and device for image processing
US7003153B1 (en) Video contrast enhancement through partial histogram equalization
US7853095B2 (en) Apparatus, method, recording medium and program for processing signal
Nithyananda et al. Review on histogram equalization based image enhancement techniques
US9478017B2 (en) Guided image filtering for image content
US20070036429A1 (en) Method, apparatus, and program for object detection in digital image
JP2020160616A (en) Generation device and computer program and generation method
JP2016505186A (en) Image processor with edge preservation and noise suppression functions
US8335375B2 (en) Image processing apparatus and control method thereof
CN112465727A (en) Low-illumination image enhancement method without normal illumination reference based on HSV color space and Retinex theory
US7327504B2 (en) Method of detecting clipped image pixels
CN111047543A (en) Image enhancement method, device and storage medium
CN111292269A (en) Image tone mapping method, computer device and computer readable storage medium
JP6396066B2 (en) Image quality improvement system, image quality improvement method, and program
CN111340732A (en) Low-illumination video image enhancement method and device
CN114998122A (en) Low-illumination image enhancement method
Kansal et al. Trade-off between mean brightness and contrast in histogram equalization technique for image enhancement
WO2020107308A1 (en) Low-light-level image rapid enhancement method and apparatus based on retinex
WO2022259323A1 (en) Image processing device, image processing method, and image processing program
US11082613B2 (en) Image adjusting method and image adjusting device
Ramadan Monochromatic-based method for impulse noise detection and suppression in color images
Lin et al. Tri-histogram equalization based on first order statistics
Okado et al. Fast and high-quality regional histogram equalization
CN113256533A (en) Self-adaptive low-illumination image enhancement method and system based on MSRCR
Sari et al. Preprocessing of tomato images captured by smartphone cameras using color correction and V-channel Otsu segmentation for tomato maturity clustering

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21945004

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023527167

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE