WO2018216280A1

WO2018216280A1 - Image processing apparatus, image processing program, recording medium, and image processing method

Info

Publication number: WO2018216280A1
Application number: PCT/JP2018/006525
Authority: WO
Inventors: 熊倉　威; 塩見　誠
Original assignee: シャープ株式会社
Priority date: 2017-05-25
Filing date: 2018-02-22
Publication date: 2018-11-29

Abstract

The present invention realizes an image processing apparatus or the like capable of appropriately expressing the texture of an image included in a picture. An image processing apparatus (100) is provided with: a division unit (11) that divides an image into a plurality of divided areas; a score calculation unit (12) that calculates a score indicating the level of a specific texture in each of the divided areas; and a texture adjustment unit (14) that adjusts the specific texture according to the scores of the divided areas.

Description

Image processing apparatus, image processing program, recording medium, and image processing method

The following disclosure relates to an image processing apparatus that adjusts the texture of an image.

Patent Document 1 discloses an image forming apparatus that switches a gradation processing method based on a gloss addition amount for each pixel.

Japanese Patent Publication “JP 2013-15640 A (published January 24, 2013)”

In the image forming apparatus disclosed in Patent Document 1, since the gloss addition amount is determined for each pixel, there is a possibility that the gloss addition amount cannot be appropriately determined for the image of the object included in the image.

An object of one embodiment of the present disclosure is to realize an image processing device that can appropriately express the texture of an image included in an image.

In order to solve the above-described problem, an image processing apparatus according to an aspect of the present disclosure is an image processing apparatus that adjusts the texture of an image, the dividing unit that divides the image into a plurality of divided regions, and the plurality A score calculating unit that calculates a score indicating the degree of the specific texture in each of the divided regions, and a texture adjusting unit that adjusts the specific texture according to the scores of the plurality of divided regions.

An image processing method according to an aspect of the present disclosure is an image processing method for adjusting the texture of an image, and includes a division step of dividing the image into a plurality of divided regions, and identification in each of the plurality of divided regions. A score calculation step of calculating a score indicating the degree of the texture of the material, and a texture adjustment step of adjusting the specific texture according to the score of the plurality of divided regions.

According to one aspect of the present disclosure, the texture of the image included in the image can be appropriately expressed.

1 is a block diagram illustrating a configuration of an image display system including an image processing apparatus according to Embodiment 1. FIG. It is a figure which shows the example which divided | segmented the input image hierarchically, Comprising: (a) shows the example of the division | segmentation of the 0th hierarchy, (b) shows the example of the division | segmentation of the 1st hierarchy, (c) is , An example of division of the second layer, (d) shows an example of division of the third layer, (e) shows an example of division of the fourth layer, and (f) A four-level hierarchical structure is shown. It is a figure which shows the example of the process in a score calculation part. (A) is a figure which shows the example of an input image, (b) is a figure which shows the brightness | luminance image produced | generated by extracting brightness | luminance information from the input image shown to (a), (c) is a figure. It is a figure which shows the brightness | luminance image after performing a blurring process to the brightness | luminance image shown to (b). It is a flowchart which shows the flow of the process which an image process part performs. (A) is a figure which shows an example of an input image, (b) is a figure which shows the score mask corresponding to the input image shown to (a), (c) was shown to (a) FIG. 6 is a diagram illustrating a luminance image that has been subjected to a blurring process corresponding to an input image, where (d) illustrates only a region where glossiness is enhanced, and (e) illustrates an output image that has been subjected to texture adjustment. FIG. It is a figure which shows the timing of the process in an image process part. (A) is a conceptual diagram of an input image input to the image processing device, (b) is a conceptual diagram of a score mask generated by the score calculation unit, and (c) is an image (output) output from the image processing device. FIG. 6 is a block diagram illustrating a configuration of an image display system including an image processing apparatus according to a second embodiment. FIG. It is a figure which shows the structure of a general portable information terminal.

Embodiment 1
Hereinafter, embodiments of the present disclosure will be described in detail. FIG. 1 is a block diagram illustrating a configuration of an image display system 1 including an image processing apparatus 100 according to the present embodiment. The image display system 1 includes an image processing device 100 and a display device 200. The image processing apparatus 100 is an apparatus that executes a process of adjusting a specific texture of an image (input image) input to the image processing apparatus 100. In the following description, it is assumed that the image processing apparatus 100 adjusts glossiness as the specific texture. The texture to be adjusted by the image processing apparatus 100 is not limited to a glossy feeling, and may be, for example, a material feeling (for example, roughness), a three-dimensional feeling, or the like. The input image may be a moving image or a still image.

As shown in FIG. 1, the image processing apparatus 100 includes an image processing unit 10, a receiving unit 20, and a storage unit 30. The display device 200 is a device that displays an image processed by the image processing device 100.

The image processing unit 10 includes a dividing unit 11, a score calculating unit 12, a score correcting unit 13, and a texture adjusting unit 14. Each unit included in the image processing unit 10 will be described later. The receiving unit 20 receives an image transmitted by broadcasting as an input image. Further, the receiving unit 20 executes buffer processing for outputting the received input image to the dividing unit 11, the score correcting unit 13, and the texture adjusting unit 14 at an appropriate timing. Note that the receiving unit 20 may read the data from a storage medium in which the data of the input image is stored. The storage unit 30 stores information necessary for processing by the image processing unit 10.

(Division unit 11)
The dividing unit 11 divides the input image into a plurality of divided areas. In the present embodiment, the dividing unit 11 divides the image in a plurality of pattern division modes having different numbers of divided areas. In the following description, each of a plurality of different patterns of division modes is referred to as a hierarchy.

FIG. 2 is a diagram showing an example in which an input image is divided hierarchically. 2A shows an example of division of the 0th layer, FIG. 2B shows an example of division of the first layer, and FIG. 2C shows division of the second layer. An example is shown, (d) of FIG. 2 shows an example of the division of the third hierarchy, and (e) of FIG. 2 shows an example of the division of the fourth hierarchy. FIG. 2F shows the hierarchical structure of the 0th to 4th hierarchies.

In the example shown in FIGS. 2A to 2F, the input image is divided as follows in each layer.
Layer 0 (L0): Not divided (the number of divided areas is 1).
First layer (L1): Divided into two vertically and horizontally (the number of divided areas is 4).
Second layer (L2): Divided into 4 parts vertically and horizontally (the number of divided areas is 16).
Third layer (L3): Divided into 8 parts vertically and horizontally (the number of divided areas is 64).
Fourth layer (L4): Divided into 16 vertically and horizontally (the number of divided areas is 256). Note that in one embodiment of the present disclosure, the number of layers may be 2 or more, 4 or less, or 6 or more. Further, the number of divided areas in each layer may be set as appropriate.

In this way, by dividing an image in a plurality of layers, it is possible to flexibly cope with the position and size of an object included in the image.

(Score calculator 12)
The score calculation unit 12 calculates a score indicating the degree of specific texture (for example, glossiness) in each of the plurality of divided regions. A learned model having a neural network can be used as the score calculation unit 12. In other words, the score calculation unit 12 includes a learned model learned by using a plurality of images of an object having a specific texture.

The neural network is provided between an input layer that is an uppermost layer to which input data is input, an output layer that is a lowermost layer that outputs output data (the score), and the input layer and the output layer. A three-layer neural network including an intermediate layer. The number of intermediate layers is, for example, 20 to 25 layers, but is not limited to this example.

When the score calculation unit 12 determines glossiness, the neural network is made to read a large number of images (referred to as a learning data set) with glossy or non-glossy tags and learn glossiness. Let (supervised learning). The number of images included in the learning data set is about 20,000 to 30,000, and each image includes images of various types of objects.

In the neural network, the node value is transmitted from the upper layer to the lower layer, and weighting is performed using the weight parameter set for each connection between the nodes. These weights can be learned by the error back propagation method (back propagation). The error back propagation method is a method of adjusting (learning) each weight parameter so that the difference between the output y when the input x is input and the true output y (teacher) is reduced for each neuron. . By learning using the learning data set, each weight parameter is optimized, and a neural network capable of generating a score indicating the degree of glossiness is constructed.

When a learned model constructed by such a learning method (deep learning) was checked for the presence or absence of gloss in an image without a tag, a correct answer rate of about 90% was obtained.

When the neural network determines a texture other than a glossy feeling (for example, a three-dimensional effect), a learning data set including an image with a tag with or without the texture is read into the neural network and learned. That's fine.

In addition, a learned model obtained by performing machine learning other than deep learning may be used as the score calculation unit 12.

FIG. 3 is a diagram illustrating an example of processing in the score calculation unit 12. The score calculation unit 12 according to the present embodiment calculates a score for each of a total of 341 divided regions in the 0th to 4th layers generated by dividing the input image. In the example illustrated in FIG. 3, the score calculation unit 12 determines that (i, i) = [0, 1] divided region (lower left region of the four divided input image) in the first hierarchy of the input image. ) A recognition process 12a for calculating the score, and (ii) a position specifying process 12b for acquiring position information ([x, y] = [0, 1]) of the divided area. The position information of the divided areas is generated by the dividing unit 11 along with the process of dividing the input image and is stored in the storage unit 30.

The score calculation unit 12 performs the same processing for each divided region of the input image, and creates data in which the score and the position information are linked. The score in the present embodiment is the probability that the divided area has glossiness, and is calculated in the range of 0% or more and 100% or less.

Further, the score calculation unit 12 further adds the scores of the divided areas having a correspondence relationship between the plurality of divided images (between hierarchies), so that each division in the divided image (fourth hierarchy) having the largest number of divided areas is performed. The area score is calculated. The set of scores is referred to as a score mask. The score mask will be specifically described below.

The score of each divided area is represented as Sn [x, y]. n is a hierarchy (0 to 4). In the n-th layer, the image is divided into 4 ⁿ divided regions, and the score is calculated for each divided region. Therefore, the number of scores in the layer is 4 ⁿ . x and y indicate the positions of the divided areas. [X, y] = [0, 0] indicates the position of the divided area at the upper left corner of the image, and [x, y] = [2 ⁿ −1, 2 ⁿ −1] indicates the divided area at the lower right corner of the image. Indicates the position. For example, the scores at the four corners of the fourth layer are S4 [0,0], S4 [15,0], S4 [0,15], and S4 [15,15], respectively.

The score included in the score mask is obtained by normalizing the scores in the divided areas of each hierarchy by adding the scores between the hierarchies. For this reason, the number of scores included in the score mask is the same as that of the hierarchy having the largest number of scores. In the example illustrated in FIG. 2, the fourth hierarchy having 256 scores is the hierarchy having the largest number of divided areas. For this reason, the score mask also has 256 scores.

When the score value in the score mask of 16 × 16 division is SM [x, y], for example, the scores at the four corners of the score mask are expressed as follows.
SM [0,0] = S0 [0,0] + S1 [0,0] + S2 [0,0] + S3 [0,0] + S4 [0,0]
SM [15,0] = S0 [0,0] + S1 [1,0] + S2 [3,0] + S3 [7,0] + S4 [15,0]
SM [0,15] = S0 [0,0] + S1 [0,1] + S2 [0,3] + S3 [0,7] + S4 [0,15]
SM [15,15] = S0 [0,0] + S1 [1,1] + S2 [3,3] + S3 [7,7] + S4 [15,15]
Generalizing this, SM [x, y] is expressed by the following equation.
SM [x, y] = (S0 [0,0] + S1 [int (x / 8), int (y / 8)] + S2 [int (x / 4), int (y / 4)] + S3 [int ( x / 2), int (y / 2)] + S4 [x, y]) / 5
Here, int () means integerization by rounding down the fractional part.

As described above, in the present embodiment, the score of each divided area is a probability that the divided area is glossy, and takes a value of 0% or more and 100% or less. For this reason, the sum of the scores of the divided areas of the 0th to 4th hierarchies takes a value of 0% to 500%. In the above formula, in order to normalize the score, the sum of the scores of the divided areas of each layer is divided by 5 which is the number of layers.

In calculating SM [x, y], the scores in each layer are linearly added. Therefore, for example, even if the score in the lower divided region (large divided region) is low, the glossiness in the divided region becomes stronger if the score in the upper divided region (small divided region) is high.

The score calculation unit 12 can flexibly cope with the position and size of the object in the image by calculating the scores for the divided areas with different sizes divided by the dividing unit 11. That is, the score calculation unit 12 can calculate a score according to the presence or absence of glossiness regardless of the size of the object in the image. In general, when the proportion of the object in the divided region is high, the recognition processing by the score calculation unit 12 tends to be performed correctly as compared with the case where the proportion is low.

(Score correction unit 13)
The score correction unit 13 extracts information related to a specific texture level from the input image, and corrects the score calculated by the score calculation unit 12 based on the extracted information. In the present embodiment, since the specific texture is glossiness, the score correction unit 13 extracts luminance information as information related to the degree of glossiness, and luminance information corresponding to each of the plurality of divided regions. Is used to correct the score of the divided area.

Specifically, the score correction unit 13 performs the following processing.
When the input image is expressed in the RGB format, the score correction unit 13 extracts the luminance information (Y) after converting the input image into the YUV format. In other words, the score correction unit 13 converts the input image into a luminance image indicating luminance information.
The score correction unit 13 calculates a correction value that is a value obtained by normalizing the luminance of the divided region for each of the divided regions similar to the score mask in the luminance image.

The luminance of each divided region can be obtained as an average value of luminance values indicated by a plurality of pixels included in the divided region, for example. A set of combinations of the calculated correction value and information for specifying a divided area of the score mask to which each correction value is applied is referred to as a correction mask.

Further, the score correction unit 13 corrects the score mask using the correction mask. Specifically, the score correction unit 13 multiplies each of the scores of the divided areas included in the score mask by a correction value indicated by the divided area of the correction mask corresponding to the divided area of the score mask. That is, when the correction value at the position [x, y] of the correction mask is F [x, y] and the score of the score mask at the position is SF [x, y], SF [x, y] It is expressed by a formula.
SF [x, y] = SM [x, y] × F [x, y]
The score calculated by the score calculation unit 12 may include a score calculated based on misrecognition. For this reason, in the score mask, there is a possibility that a divided area which does not actually have a glossy feeling has a high score. Therefore, the score correction unit 13 generates a correction mask and corrects the score mask.

Generally, in an image, it is considered that there is no glossiness in a divided area with low luminance. Therefore, the score mask reliability can be improved by correcting the score mask using the luminance for each divided region of the input image.

It should be noted that since there may be no glossiness even when the brightness value is high, such as a blank portion of a text font, it is not appropriate to use the brightness value itself as a score indicating glossiness. Therefore, the luminance value is used only for correcting the score.

Further, the score correction unit 13 may correct (blur processing) the score of each divided region so that the difference between the scores of the adjacent divided regions becomes small. As such a correction method, for example, there is a method of correcting the luminance of each divided region so that the difference in luminance between adjacent divided regions in the luminance image becomes small. This example will be described below.

FIG. 4A shows an example of an input image. FIG. 4B is a diagram showing a luminance image generated by extracting luminance information from the input image shown in FIG. The luminance image is generated by extracting luminance information (Y) from a YUV format image.

(C) of FIG. 4 is a diagram showing a luminance image after performing the blurring process on the luminance image shown in (b) of FIG. The edges of the input image shown in FIG. 4A and the luminance image shown in FIG. 4B are clear. By performing blurring processing on the luminance image shown in FIG. 4B, a luminance image with blurred edges is obtained as shown in FIG. 4C.

In one aspect of the present disclosure, the score correction unit 13 does not necessarily need to execute the blurring process. However, since the edge portion of the image is blurred when the blurring process is executed, a rapid change in the correction value in the divided region corresponding to the edge portion and the score corrected by the correction value is suppressed. As a result, the image quality failure at the edge portion is suppressed. Therefore, it is preferable that the score correction unit 13 performs a blurring process on the luminance image.

(Texture adjustment unit 14)
The texture adjustment unit 14 adjusts a specific texture according to the score calculated by the score calculation unit 12 and corrected by the score correction unit 13 for a plurality of divided regions. In the present embodiment, the texture adjusting unit 14 emphasizes a specific texture for a divided area having a score equal to or higher than a predetermined value among the plurality of divided areas. The predetermined value may be appropriately set according to the type of texture to be emphasized.

As described above, in this embodiment, the specific texture is gloss. For this reason, specifically, the texture adjustment unit 14 performs luminance adjustment, contrast adjustment, edge enhancement processing, or a combination thereof in each divided region of the input image for a divided region having a score equal to or greater than a predetermined value. About the grade which performs a brightness | luminance adjustment, contrast adjustment, and an edge emphasis process, a multi-step may be set so that it may select, as mentioned later.

According to subjective evaluation, contrast enhancement and edge enhancement greatly affect the glossiness felt by humans. Therefore, the texture adjusting unit 14 can enhance the glossiness of the divided region by executing processing including contrast enhancement and edge enhancement for the divided region having a score equal to or greater than a predetermined value.

Also, when the specific texture is a three-dimensional effect, for example, by increasing the γ value and contrast, the change in the output luminance gradation with respect to the input image luminance gradation can be made steep. By making the change in the gradation of the output luminance steep, the shadow of the object in the output image appears to be relatively dark, so that the stereoscopic effect can be enhanced.

Also, when a specific texture is used as a material feeling, for example, by emphasizing the edges, the pattern of the input image can be clearly felt, so that the material feeling can be emphasized.

(Process flow)
FIG. 5 is a flowchart showing a flow of processing (image processing method) executed by the image processing unit 10. FIG. 6A is a diagram illustrating an example of an input image. FIG. 6B is a diagram illustrating a score mask corresponding to the input image. (C) of FIG. 6 is a figure which shows the brightness | luminance image which performed the blurring process corresponding to the said input image. FIG. 6D shows only the divided areas where the glossiness is emphasized. FIG. 6E is a diagram illustrating an output image whose texture has been adjusted.

In the image processing unit 10, the dividing unit 11 divides an input image as shown in FIG. 6A in a plurality of divided patterns having a plurality of divided regions, and thereby has a plurality of divided regions. (S1, division process). Next, the score calculation unit 12 calculates a score for each divided region (S2, score calculation step), and generates a score mask by adding the scores of the divided regions that are in a correspondence relationship between layers (S3). FIG. 6B shows an image of the score mask visualized so that the divided region having a higher score becomes closer to white.

In parallel with steps S1 to S3, the score correction unit 13 generates a correction mask including a correction value based on the luminance value indicated by the luminance image as shown in FIG. 6C (S4). Further, the score correction unit 13 corrects the score mask using the correction mask (S5).

After that, the texture adjusting unit 14 executes a process of enhancing the texture (glossiness) for the divided areas whose score is a predetermined value or more in the score mask (S6, texture adjusting process). Specifically, as shown in FIG. 6D, the texture adjusting unit 14 generates an enhanced image in which glossiness is enhanced only in the divided areas having a score equal to or higher than a predetermined value. Furthermore, the texture adjusting unit 14 generates an output image shown in FIG. 6E by synthesizing the input image and the emphasized image.

In the example shown in FIG. 5, the processing by the score calculation unit 12 (S2, S3) and part of the processing by the score correction unit 13 (S4) are executed in parallel. However, steps S2 to S4 may be executed in this order, and step S4 may be executed before steps S2 and S3.

FIG. 7 is a diagram illustrating processing timing in the image processing unit 10. In FIG. 7, t1 to t6 are times when some processing is executed in the image processing unit 10.

First, at time t1, the receiving unit 20 receives an input image and stores it in a buffer. The receiving unit 20 outputs the buffered input image to the dividing unit 11 at time t2. The dividing unit 11 divides the image into a plurality of divided areas. The score calculation unit 12 calculates a score for each of the divided regions and generates a score mask.

Also, the receiving unit 20 outputs the input image to the score correction unit 13 at time t3 after time t2. This is because the time required for the processing in the division unit 11 and the score calculation unit 12 is longer than the time required for the processing in the score correction unit 13. Thus, by providing a time difference between time t2 and time t3, both the score mask and the correction mask are generated at time t4.

At time t5, the score correction unit 13 corrects the score mask using the correction mask. The receiving unit 20 outputs the input image to the texture adjusting unit 14 at the same timing as the correction. Thereby, at time t6, the texture adjusting unit 14 can adjust the glossiness of the input image using the corrected score mask.

(Example)
FIG. 8 is a diagram for explaining an example of processing by the image processing apparatus 100. 8A is a conceptual diagram of an image (input image) input to the image processing apparatus 100, FIG. 8B is a conceptual diagram of a score mask generated by the score calculation unit 12, and FIG. c) is a conceptual diagram showing an image (output image) output from the image processing apparatus 100. FIG.

For the sake of simplicity, in FIG. 8B, the score mask will be described as a set of scores for each of the divided areas obtained by dividing the input image into eight parts in the vertical and horizontal directions. For each score mask score, for the sake of convenience, 1 is set when there is glossiness, and 0 when there is no glossiness.

The input image shown in FIG. 8A includes images of a metal spoon and a ceramic spoon. The image of the metal spoon is glossy. On the other hand, the image of a ceramic spoon has no gloss.

In such a case, as shown in FIG. 8B, the score calculation unit 12 sets the score of the divided area corresponding to the image of the metal spoon to 1, and sets the score of the other divided areas to 0. Generate a mask.

The texture adjusting unit 14 executes a process of emphasizing glossiness in a divided area where the score in each divided area of the texture mask shown in FIG. 8B is 0.5 or more, and the score is less than 0.5. Processing for enhancing glossiness is not performed on the divided areas. Thereby, as shown in FIG. 8C, the glossiness is emphasized only in the divided areas corresponding to the image of the metal spoon, and the divided areas corresponding to the other divided areas, for example, the image of the ceramic spoon. In the area, an image that is not emphasized is output to the display device 200.

The score may take a decimal value between 0 and 1, for example. In this case, for example, the score calculation unit 12 takes a value of 0.5 or more and 1 or less in the divided area having glossiness, and the score in the divided area not having the glossiness is 0 or more and less than 0.5. A score mask like this is generated.

Also, the score may take a value in a range different from 0 or more and 1 or less, for example. Furthermore, the texture adjusting unit 14 may execute a process of enhancing glossiness for a divided region having a score equal to or higher than the predetermined value with a predetermined value different from 0.5 as a reference.

Further, the texture adjusting unit 14 may set a plurality of stages of processing for enhancing the glossiness, and may determine which stage of processing is performed on the divided area according to the score of the divided area. In this case, parameters used for the brightness adjustment process, contrast adjustment process, or edge enhancement process are associated in advance for each stage.

The parameter in the brightness adjustment process is a γ value. The brightness adjustment process is executed by the following expression, for example.
p = 255 × (p0 / 255) ^ (1 / γ)
Here, p is a pixel value in the output image of the pixel to be processed, and p0 is a pixel value in the input image of the pixel. “^ (1 / γ)” means 1 / γ power. As γ increases from 1, the luminance after the luminance adjustment processing becomes larger than the luminance before the processing.

The parameter in the contrast adjustment process is a coefficient α that is multiplied by the difference between the pixel value of the pixel to be processed and the intermediate value. The contrast adjustment process is executed by the following formula, for example.
p = p0 + (p0−pth) × α
Here, pth is an intermediate value of pixel values (that is, a reference value for increasing or decreasing the pixel value by contrast adjustment processing). As α increases, pixel values larger than the intermediate value become larger, and pixel values smaller than the intermediate value become smaller.

The parameter in the edge enhancement process is a ratio β2 / β1 of a coefficient β2 that multiplies a pixel value around the pixel to be processed with respect to a coefficient β1 that multiplies the pixel value of the pixel to be processed for the edge enhancement process. . The edge enhancement process is executed by the following formula, for example.
p = (β1 × p0−β2 × (p1 + p2 + p3 + p4)) / (β1-4 × β2)
Here, p1 to p4 are pixel values of pixels adjacent to the four sides of the pixel to be processed. As β2 / β1 increases, more edges are emphasized.

Suppose that the glossiness enhancement stage is set in, for example, three stages of “none”, “medium”, and “strong”. In this case, when the enhancement stage is “none”, the above parameters are set (through setting) so that the amount of change in the pixel value before and after the process is zero. Specifically, for example, the value of γ is set to 1, and α and β2 / β1 are set to 0.

When the emphasis stage is “medium”, the above parameters are set to values that emphasize glossiness. Specifically, for example, the value of γ is set to a value greater than 1, and α and β2 / β1 are set to a value greater than 0. When the emphasis stage is “strong”, the amount of change in the parameter is set to a value that further enhances the glossiness (that is, the maximum value in the image processing apparatus 100) than when the emphasis stage is “medium”.

強調 The gloss enhancement step may be set in four or more steps. For example, when there are a large number of stages, such as 256 stages, a table showing the values of the parameters corresponding to the respective stages may be referred to, and the parameters may be interpolated arithmetically (for example, linearly).

(Modification)
In addition, the texture adjusting unit 14 does not limit the divided areas whose glossiness is to be adjusted based on the score, but may set all the divided areas as the target. After that, the texture adjustment unit 14 adjusts the texture of each divided region of the input image by increasing or decreasing the parameter of the texture to be changed for the texture adjustment according to the score calculated by the score calculation unit 12. May be.

Specifically, the texture adjusting unit 14 employs a value obtained by multiplying the maximum change amount, which is the difference between the minimum value and the maximum value that can be taken by the parameter, by the score as a weighting factor, as the value of the parameter.

For example, for the above-mentioned γ, when the minimum value is 1 and the maximum value is 1.5 (maximum change amount is 0.5), the γ value for the score S is expressed by the following equation.
γ = 1 + 0.5 × S
Further, for the above α and β2 / β1 having values of 0 in the through setting, the value of the parameter used for texture adjustment is obtained by multiplying the maximum value by the score.

Note that the brightness adjustment process, the contrast adjustment process, and the edge enhancement process are not limited to the above example, and the above parameters vary depending on the content of the process.

Further, in one aspect of the present disclosure, the image processing apparatus 100 does not necessarily include the score correction unit 13. When the image processing apparatus 100 does not include the score correction unit 13, the texture adjustment unit 14 adjusts a specific texture according to the score calculated by the score calculation unit 12.

(effect)
According to the image processing apparatus 100, it is possible to perform a process of extracting a glossy divided area in the input image and enhancing the glossiness only in the divided area. Therefore, it is possible to reduce the possibility that the processing causes the image quality failure without considering the influence of the processing for enhancing the glossiness to the divided area having no glossiness.

[Embodiment 2]
Other embodiments of the present disclosure are described below. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted.

FIG. 9 is a block diagram showing a configuration of an image display system 1A including the image processing apparatus 100A according to the present embodiment. The image display system 1A is different from the image display system 1 in that it includes the image processing apparatus 100A instead of the image processing apparatus 100.

The image processing apparatus 100A includes a communication unit 40 in addition to the components included in the image processing apparatus 100. The communication unit 40 communicates with an external device in order to update the learned model included in the score calculation unit 12. By updating the learned model via the communication unit 40, the image processing apparatus 100A can improve the accuracy of the recognition process in which the score calculation unit 12 calculates the score.

Note that the image processing apparatus 100A illustrated in FIG. 9 includes the receiving unit 20 and the communication unit 40 separately. However, the receiving unit 20 and the communication unit 40 may be a common member.

[Embodiment 3]
Other embodiments of the present disclosure are described below.

FIG. 10 is a diagram showing a configuration of a general information terminal 2. The information terminal 2 may be, for example, a workstation, a personal computer, a smartphone, or a tablet. The information terminal 2 includes a control CPU (Central Processing Unit) 201, an image processing GPU (Graphics Processing Unit) 202 capable of parallel arithmetic processing, a RAM (Random Access Memory) 203, a storage unit 204, a display 205, A camera 206 that captures images and videos, a communication unit 207 that performs wireless communication, an audio input / output 208, a sensor 209 such as a touch panel or buttons, and a connector 210 are connected to each other via a bus 211.

Such an information terminal 2 can function as an information processing apparatus according to an aspect of the present disclosure. For example, the input image stored in the storage unit 204 is divided into a plurality of divided regions by the process of step S1 shown in FIG. Next, the GPU 202 executes the processes of steps S2 and S3 and the process of step S4 in parallel, and further executes the process of step S5. Thereafter, the CPU 201 executes the process of step S6. The processed output image is displayed on the display 205. Alternatively, the output image may be stored in the storage unit 204 or may be output to the outside via the communication unit 207 or the connector 210.

[Embodiment 4]
Other embodiments of the present disclosure are described below. Note that the image processing apparatus of the present embodiment has the same configuration as that of the image processing apparatus 100 shown in FIG. 1, and therefore, in the following description, the same reference numerals as those in FIG.

As described in the modification of the first embodiment, the image processing apparatus 100 according to the present embodiment increases or decreases the texture parameter to be changed for texture adjustment according to the score calculated by the score calculation unit 12. The texture of each divided area of the input image is adjusted. In addition, the image processing apparatus 100 according to the present embodiment is configured such that the user can adjust the manner or degree in which the score correction unit 13 corrects the score. For example, a plurality of stages are set for the degree to which the score correction unit 13 corrects the score, and the user may select a desired stage among the plurality of stages. The above selection may be performed using, for example, a menu displayed on the display unit provided in the image processing apparatus. Alternatively, the above-described selection dial or lever may be provided in the image processing apparatus.

Magnification for multiplying the score is defined for each of the above-mentioned plurality of stages. The score correction unit 13 multiplies the score value in each divided region by a magnification according to the stage selected by the user. By multiplying the above magnification, the degree of glossiness adjustment according to the score is increased or decreased.

Also, a stage where the score value becomes negative may be set. When the score value is negative, as the score value approaches −1, the adjustment amount for reducing the glossiness increases. For this reason, the glossiness of the divided area having the glossiness is reduced.

According to the image processing apparatus 100 of the present embodiment, the user can adjust the manner and degree in which the texture adjustment unit 14 adjusts the texture by adjusting the manner or degree in which the score correction unit 13 corrects the score. it can.

[Example of software implementation]
The control blocks (particularly, the dividing unit 11, the score calculating unit 12, the score correcting unit 13, and the texture adjusting unit 14) of the

image processing apparatuses

100 and 100A are formed by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. It may be realized or may be realized by software using a CPU (Central Processing Unit).

In the latter case, the

image processing apparatuses

100 and 100A have a CPU that executes instructions of an image processing program, which is software that realizes each function, and the image processing program and various data recorded in a computer (or CPU) so that they can be read. A ROM (Read Only Memory) or a storage device (these are called “recording media”), a RAM (Random Access Memory) for developing the image processing program, and the like are provided. Then, the computer (or CPU) reads the image processing program from the recording medium and executes it, thereby achieving the object of one aspect of the present disclosure. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, the image processing program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the image processing program. Note that one aspect of the present disclosure can also be realized in the form of a data signal embedded in a carrier wave, in which the image processing program is embodied by electronic transmission.

[Summary]
An image processing apparatus according to aspect 1 of the present disclosure is an image processing apparatus that adjusts the texture of an image, and includes a dividing unit that divides the image into a plurality of divided areas, and a specific texture in each of the plurality of divided areas. A score calculation unit that calculates a score indicating the degree of the texture, and a texture adjustment unit that adjusts the specific texture according to the scores of the plurality of divided regions.

According to the above configuration, the dividing unit divides the input image into a plurality of divided regions, and the score calculating unit calculates a score indicating the degree of specific texture in each of the plurality of divided regions. The texture adjusting unit executes a process of adjusting the specific texture according to the score of the divided area. Therefore, it is possible to recognize a specific texture for each divided region and adjust the specific texture.

In the image processing apparatus according to aspect 2 of the present disclosure, in the aspect 1, the texture adjustment unit emphasizes the specific texture for a divided area in which the score is a predetermined value or more among the plurality of divided areas. It is preferable.

According to the above configuration, the specific texture can be emphasized only for the divided areas having a score equal to or higher than a predetermined value.

In the image processing apparatus according to aspect 3 of the present disclosure, in the

aspect

1 or 2, the dividing unit divides the image in a plurality of pattern division modes having different numbers of the divided regions, and the score calculating unit includes: It is possible to calculate the score of each divided region in the divided image having the largest number of divided regions by adding the scores of the divided regions corresponding to each other between the plurality of divided images divided in the above-described division pattern of the plurality of patterns. preferable.

According to the above configuration, the score of a plurality of divided images divided in a plurality of pattern division modes having different numbers of divided regions can be used for final score calculation. Therefore, an appropriate score can be calculated flexibly corresponding to various sizes and positions of objects in the image.

The image processing apparatus according to Aspect 4 of the present disclosure, in any one of Aspects 1 to 3, extracts information related to the specific texture level from the image, and based on the extracted information, the score It is preferable to further include a score correction unit that corrects the score calculated by the calculation unit.

According to the above configuration, even if the score calculated by the score calculation unit is not appropriate, the score correction unit can correct the score to an appropriate value.

In the image processing device according to aspect 5 of the present disclosure, in the aspect 4, the specific texture is glossiness, the score correction unit extracts luminance information as the relevant information, and It is preferable to correct the score of the divided area using luminance information corresponding to each of the divided areas.

According to the above configuration, the score correction unit can correct the score of the divided area using the luminance information of the divided area.

In the image processing apparatus according to Aspect 6 of the present disclosure, in the Aspect 4 or 5, the score correction unit may correct the score of each divided region so that the difference between the scores of the adjacent divided regions is small. Good.

According to the above configuration, a rapid change in score between adjacent divided regions is suppressed, and image quality failure is suppressed.

In the image processing device according to aspect 7 of the present disclosure, in any one of aspects 1 to 6, the score calculation unit includes a learned model learned using a plurality of images of the object having the specific texture. It is preferable.

According to the above configuration, the presence / absence of a specific texture in the divided region can be appropriately determined by the learned model.

In the aspect 8, the image processing apparatus according to the aspect 8 of the present disclosure preferably further includes a communication unit that communicates with an external apparatus in order to update the learned model.

According to the above configuration, by updating the learned model via the communication unit, it is possible to improve the accuracy of determination of the presence or absence of a specific texture.

In the image processing apparatus according to the ninth aspect of the present disclosure, in the fourth or fifth aspect, the user may be able to adjust the manner or degree in which the score correction unit corrects the score.

According to the above configuration, the manner or degree by which the score correction unit corrects the score can be adjusted according to the subjectivity of the user.

In the image processing apparatus according to the tenth aspect of the present disclosure, in any one of the first to ninth aspects, it is preferable that the texture adjusting unit performs brightness adjustment, contrast adjustment, or edge enhancement processing of the image.

According to the above configuration, the glossiness of the image can be adjusted by brightness adjustment, contrast adjustment, or edge enhancement processing.

The image processing method according to the eleventh aspect of the present disclosure is an image processing method for adjusting the texture of an image, and includes a dividing step of dividing the image into a plurality of divided regions, and a specific texture in each of the plurality of divided regions. A score calculation step of calculating a score indicating the degree of the texture, and a texture adjustment step of adjusting the specific texture according to the scores of the plurality of divided regions.

According to the above configuration, the same effect as in the first aspect is obtained.

The image processing apparatus according to each aspect of the present disclosure may be realized by a computer. In this case, the image processing apparatus is operated on each computer by causing the computer to operate as each unit (software element) included in the image processing apparatus. An image processing program of an image processing apparatus to be realized in this manner and a computer-readable recording medium that records the image processing program also fall within the category of one aspect of the present disclosure.

The present disclosure is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments can be obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of one embodiment of the present disclosure. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.
(Cross-reference of related applications)
This application claims the benefit of priority to the Japanese patent application filed on May 25, 2017: Japanese Patent Application No. 2017-103895. Included in this document.

DESCRIPTION OF SYMBOLS 11 Dividing part 12 Score calculating part 13 Score correcting part 14 Texture adjusting part 40

Communication part

100, 100A Image processing apparatus

Claims

An image processing apparatus for adjusting the texture of an image,
A dividing unit for dividing the image into a plurality of divided regions;
A score calculation unit for calculating a score indicating the degree of a specific texture in each of the plurality of divided regions;
An image processing apparatus comprising: a texture adjusting unit that adjusts the specific texture according to the scores of the plurality of divided regions.
2. The image processing apparatus according to claim 1, wherein the texture adjusting unit emphasizes the specific texture for a divided area having a score equal to or higher than a predetermined value among the plurality of divided areas.
The division unit divides the image in a plurality of pattern division modes having different numbers of the divided regions,
The score calculation unit adds the scores of the divided areas corresponding to each other between the plurality of divided images divided in the division pattern of the plurality of patterns, so that each divided area in the divided image having the largest number of divided areas is added. The image processing apparatus according to claim 1, wherein a score is calculated.
The information processing apparatus further comprises a score correction unit that extracts information related to the specific texture level from the image, and corrects the score calculated by the score calculation unit based on the extracted information. The image processing apparatus according to any one of 1 to 3.
The specific texture is glossy,
The score correction unit extracts luminance information as the relevant information, and corrects the score of the divided region using luminance information corresponding to each of the plurality of divided regions. An image processing apparatus according to 1.
6. The image processing apparatus according to claim 4, wherein the score correction unit corrects the score of each divided region so that a difference in scores between adjacent divided regions becomes small.
The image processing apparatus according to claim 1, wherein the score calculation unit includes a learned model learned by using a plurality of images of the object having the specific texture.
The image processing apparatus according to claim 7, further comprising a communication unit that communicates with an external apparatus in order to update the learned model.
The image processing apparatus according to claim 4 or 5, wherein the user can adjust the manner or degree by which the score correction unit corrects the score.
The image processing apparatus according to claim 1, wherein the texture adjustment unit performs brightness adjustment, contrast adjustment, or edge enhancement processing of the image.
An image processing program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 10, wherein the image processing program causes the computer to function as each unit.
A computer-readable recording medium on which the image processing program according to claim 11 is recorded.
An image processing method for adjusting the texture of an image,
A dividing step of dividing the image into a plurality of divided regions;
A score calculation step of calculating a score indicating the degree of the specific texture in each of the plurality of divided regions;
And a texture adjustment step of adjusting the specific texture according to the scores of the plurality of divided regions.