JP4798446B2 - Photographed image correction method and photographed image correction module - Google Patents

Description

  The present invention relates to a captured image correction technique for correcting an input captured image in consideration of a captured scene of the input captured image.

  Currently, not only digitally photographed images acquired by digital cameras but also photographed images formed on photographic films such as negative films and reversal films are converted into digital images through film scanners, and light emitted from the print head based on the digital data 2. Description of the Related Art Photo printing apparatuses that control beams and ink droplets, print and scan photosensitive materials and recording paper with the controlled light beams and ink droplets to form images, and output photographic prints have become widespread. Note that, here, unless otherwise specifically required, the captured image data as digital data, the print captured image based on the captured image data, and the monitor display captured image are collectively referred to simply as a captured image. . In such a photographic printing apparatus, correction processing such as density correction and contrast correction is performed on a photographic image converted into digital data in order to finally realize high-quality image formation. For example, as a density correction technique for a digitized and inputted photographed image, each RGB component of the RGB color components is grayed out when the pixel values (density values; luminance values) of all the pixels for each RGB color are averaged. A method for correcting a histogram is known. This method is based on Evans' theorem that "when a general outdoor landscape is photographed, all the colors recorded in the image are combined to become gray". However, with this conventional captured image correction technique, for example, an image of a person photographed without using a strobe against a bright background, an image of a person photographed using a flash against a dark background, or a person photographed with backlight When a photographic image such as an image having a biased density on the entire screen is subjected to density correction by the above technique, there has been a problem that a person who is a main subject is not properly corrected due to the influence of the background.

  In order to solve the above problem, a shooting scene (for example, backlight shooting, flash shooting, etc.) that is a situation at the time of shooting of a shot image as a correction target is determined, and appropriate correction is performed according to the determined shooting scene. By determining the method (degree of correction) and correcting the captured image based on the determined degree of correction, the main subject can be expressed appropriately in both backlight shooting and flash shooting. Become. For this reason, image reproduction means for reproducing an image on a monitor screen using image data composed of a plurality of pixel data, and range selection for selecting a specific range among images reproduced on the monitor screen Means, a backlight setting means for setting a backlight condition when the image reproduced on the monitor screen is in a backlight or a state close to backlight, and pixel data included in the selected range when the backlight condition is set When a predetermined ratio of pixel data is extracted from the lower luminance of the pixel, and the backlight condition is not set, the predetermined percentage of pixel data from the higher luminance of the pixel data included in the selected range is extracted. Pixel data extracting means for extracting, average luminance calculating means for calculating the average value of the luminance in the selected range using the pixel data extracted by the pixel data extracting means, and the obtained brightness Correction amount calculation means for calculating a correction amount for correcting the average value of the predetermined value to a predetermined luminance value set in advance, image data correction means for correcting all pixel data using the calculated correction amount, and correction There is known an image forming apparatus including an image forming unit that forms an image using the image data thus obtained (see, for example, Patent Document 1). However, in this prior art, the backlight setting unit is composed of a mouse, a keyboard, a CPU, a ROM, a RAM, and the like. If the prescan image reproduced on the display is in a backlight or a state close to backlight, an operator or user If a judgment is made, the mouse or keyboard is operated to input a predetermined signal corresponding to the backlight state. The judgment of the shooting scene depends on the experience and knowledge of the operator and the user, and is skilled in handling. There is a problem that requires.

  It has also been proposed to employ a neural network in order to automatically determine a shooting scene such as backlight shooting. For example, frequency values (normalized values) of sampling density values determined in a normalized density distribution histogram are input to each nonlinear element of the input layer, and “proper exposure” and “under” are output patterns. Build a neural network with “exposure”, “overexposure”, and “backlight” settings, and properly extract the reference density value used to determine image correction calculation processing based on the output result of this neural network Thus, an image processing apparatus that performs appropriate image processing regardless of the exposure state (shooting scene) of a multi-tone image is known (for example, see Patent Document 2). However, in a single neural network that determines four photographic scenes using image feature values obtained from the histogram as input, various histogram forms occur even in the same photographic scene, and when a person is the subject, the person is the best. Since prioritized image correction is required, there is a problem that sufficient output accuracy cannot be obtained depending on the captured image.

JP 2002-185793 (paragraph numbers 0006, 0024, FIG. 3) JP-A-11-154235 (paragraph numbers 0054-0057, FIG. 7)

  In view of the above situation, an object of the present invention is to provide a shooting that enables a shooting scene of an input shooting image to be estimated with high accuracy using a neutral network, and that the input shooting image can be accurately corrected based on the estimation result. It is to provide image correction technology.

In order to solve the above-described problem in the captured image correction method for correcting the input captured image in consideration of the captured scene of the input captured image, the captured image correction method according to the present invention includes an image feature based on a histogram of the input captured image. A step of obtaining various histogram feature amounts as quantities, a step of obtaining various region-dependent density feature amounts as image feature amounts depending on the region from the region set in the input photographed image, and the input photographed image image determining a variety of color feature as an image feature amount, the set of the various histogram feature amounts and the various regions dependent density feature quantity and the variety of color characteristic amounts as a pre-outs image feature quantity characterized quantity set, a plurality of images JP to encompass different image feature quantity is at least partially while being selected from the image characteristic quantity set A step of calculating an output relating to a photographic scene by a plurality of neural network units each receiving each of the quantity subsets, and an estimation value for estimating a photographic scene of the input photographic image using the outputs from the plurality of neural network units Step.

In addition, a captured image correction module according to the present invention for solving the above problem in a captured image correction module for correcting the input captured image in consideration of a captured scene of the input captured image is based on a histogram of the input captured image. Histogram feature amount calculation unit for obtaining various histogram feature amounts as image feature amounts, and region dependence for obtaining various region-dependent density feature amounts as image feature amounts depending on this region from the region set in the input photographed image a density feature quantity calculation section, various color feature calculation unit for obtaining a color feature, the various regions and the various histogram feature amounts of the front Kiga image feature amount of an image feature quantity from the input captured image a set of dependent density feature quantity and the variety of color feature data and image feature amount set small with is selected from the image characteristic quantity set In addition, a plurality of neural network units that calculate outputs related to a shooting scene using each of a plurality of image feature amount subsets including image feature amounts that are partially different from each other, and using outputs from the plurality of neural network units, And an output processing unit that calculates an estimated value for estimating a captured scene of the input captured image.

  According to such a feature configuration of the present invention, a plurality of neural network units for outputting a conclusion for estimating a shooting scene of a processing target shooting image by using a plurality of types of image feature amounts obtained from the processing target shooting image In addition, the image feature quantity subsets input to the respective neural network units are at least partially different in content, so each neural network unit has an independent algorithm and a conclusion about the shooting scene. Can be output. Since an estimated value for estimating the shooting scene of the final captured image is calculated from a plurality of conclusions output from the plurality of neural network units, this estimated value is a conclusion based on a kind of collaborative system, which is quite extreme. Even if it is a photographed image photographed under photographing conditions, the disadvantage of calculating extremely different estimated values is suppressed. By obtaining an accurate estimated value of the shooting scene, it is possible to determine an accurate correction degree for the input captured image to be processed.

The number of neural network units used is determined based on conditions such as the reliability of the final conclusion and the construction cost of the neural network unit. As a preferred embodiment, the present invention provides three different neural network units. Proposed to do. That is, before calculating the first output related to a shooting scene by the first neural network unit which receives the first image characteristic amount subset of selected image features from Kiga image characteristic quantity set, before Kiga image feature quantity A shooting scene by a second neural network unit that receives a second image feature quantity subset that is selected from the set and at least partially comprises an image feature quantity different from the image feature quantity included in the first image feature quantity subset. second calculating output differs from the image feature amount is at least partially included in the first image characteristic amount subset and the second image characteristic amount subset with are selected from pre-outs image characteristic quantity set image related A scene shot by a third neural network unit that receives a third image feature subset consisting of features Configured to calculate a third output related. In this way, the three neural network configurations have the advantage of being able to incorporate all-matching rules and majority rules when calculating the estimated values from the first, second, and third outputs that are the respective conclusions. It is done.

Preferred embodiment, by to include pre-Symbol always histogram feature amount and the area-dependent density features and the color feature in each image feature amount subset, the concentration of the input captured image the whole image of the present invention The shooting scene of the input shot image can be estimated from the viewpoints of the characteristics, the density characteristics of the image specific area, and the color characteristics of the image. This is an effective image considering that the backlight scene, which is the most important shooting scene, can be well estimated using the density value of the entire image, the density value of the central area, and the skin color distribution as parameters. This is the selection of feature quantities. Therefore, when the photographic scene to be estimated backlit scenes, flash photography scene, in a preferred embodiment of the present invention, as a pre Kihi Sutoguramu feature amount, obtained by dividing the density gradation into a plurality of zones average density value of each region is used, and as the pre-Symbol area dependent density feature quantity, and the center average density value is the average density value in the central area of the input captured image, enlarged region in which the magnified from the central region mean is the concentration value of at expansion and the center average density value, and the entire average density value is the average density value of the entire region, and the concentration standard deviation is the concentration value standard deviation of the entire region is used, and prior Symbol As the color feature amount, the average saturation of the input photographed image, the high density average saturation which is the average saturation of pixels having a density value equal to or higher than the average density value of the input photographed image, and the average density of the input photographed image Pixel with density value less than the value A lightly doped average saturation is an average saturation, and skin color pixel number is the number of pixels skin color with an average density value or density value of the input captured image is used.

  A photographic image correction module for each photographic scene according to the present invention, which is used to correct an input photographic image in consideration of a photographic scene at the time of shooting (backlight photographic scene, strobe photographic scene, normal photographic scene, etc.), particularly density adjustment correction. The basic configuration is shown in FIG. The feature of this photographed scene correction module 60 is that it includes a neural network module 62 having a plurality of neural network sections or creating a plurality of neural network sections. In the element group of the input layer of each neural network unit, several image feature amounts selected in advance from a plurality of types of image feature amounts obtained by the image feature amount calculation unit 61 from the input photographed image are input. . At that time, at least one of the set of image feature values given to the input layer of each neural network unit, that is, the contents of the image feature value subset is different from the image feature value subset given to the input layer of the other neural network unit. Has features. Furthermore, the photographic image correction module 60 for each photographic scene is calculated by an output processing unit 63 that calculates an estimated value for estimating a photographic scene of the input photographic image using the output from each neural network unit, and the output processing unit 63. A correction degree determination unit 64 that determines a correction degree for the input photographed image based on the estimated value. Since the output value output from each neural network unit is a numerical value such as the probability that the shooting scene of the input shot image is one of the predefined shooting scenes, for example, the output processing unit 63 outputs a plurality of neural network units. By simply averaging the output values to be output or averaging the weighted values, an estimated value for each specific shooting scene assigned to the input shot image is calculated. For example, with respect to a certain input photographed image, an estimated value (estimated vector) is calculated such that what is the probability that the photographed scene is a backlight photographed scene and what is the probability that the photographed scene is a flash photographed scene. After that, from the estimated value, the correction degree determination unit 64 determines the correction degree based on knowledge obtained from many experimental results and the like. The correction degree determined by the correction degree determination unit 64 is used in the image correction unit 70 that has conventionally performed gamma correction and contrast correction, and image correction is performed in consideration of the shooting scene.

  Next, a photographic printing apparatus equipped with the above-described photographic image correction module 60 for each photographic scene will be described. FIG. 2 is an external view showing the photographic printing apparatus. This photographic printing apparatus includes a printing station 1B as a photographic printer that performs exposure processing and development processing on the photographic paper P, and a developed photographic film 2a and digital The operation station 1A is configured to process a captured image taken from an image input medium such as a camera memory card 2b and generate / transfer print data used in the print station 1B.

  This photo printing apparatus is also called a digital minilab. As can be understood from FIG. 3, the printing station 1B pulls out the roll-shaped printing paper P stored in the two printing paper magazines 11 and prints it with the sheet cutter 12. The back print unit 13 prints print processing information such as color correction information and frame number on the back side of the photographic paper P, and the print exposure unit 14 cuts the print paper P into the size. A photographed image is exposed on the surface of the photographic paper P, and the exposed photographic paper P is sent to a processing tank unit 15 having a plurality of development processing tanks for development processing. After drying, the photographic paper P, that is, the photographic prints P, sent to the sorter 17 from the transverse feed conveyor 16 at the upper part of the apparatus, is collected in a plurality of trays of the sorter 17 in a state of being sorted in order units (see FIG. 2). ).

  A photographic paper transport mechanism 18 is laid to transport the photographic paper P at a transport speed in accordance with various processes for the photographic paper P described above. The photographic paper transport mechanism 18 is composed of a plurality of nipping and transporting roller pairs including a chucker type photographic paper transport unit 18a disposed before and after the print exposure unit 14 in the photographic paper transport direction.

  The print exposure unit 14 applies R (red), G (green), and B (blue) to the printing paper P conveyed in the sub-scanning direction based on print data from the operation station 1A along the main scanning direction. A line exposure head for irradiating laser beams of the three primary colors (1) is provided. The processing tank unit 15 includes a color developing tank 15a for storing a color developing processing liquid, a bleach-fixing tank 15b for storing a bleach-fixing processing liquid, and a stabilizing tank 15c for storing a stable processing liquid.

  At the upper position of the desk-like console of the operation station 1A, a film capable of acquiring photographed image data (hereinafter simply referred to as image data or photographed image) from a photographed image frame of the photographic film 2a with a resolution exceeding 2000 dpi. A media reader 21 that is provided with a scanner 20 and obtains image data from various semiconductor memories, CD-Rs, and the like used as a photographic image recording medium 2b mounted on a digital camera or the like is a controller of this photographic printing apparatus. It is built in a general-purpose personal computer that functions as 3. The general-purpose personal computer is also connected with a monitor 23 for displaying various information, and a keyboard 24 and a mouse 25 as operation input devices used as an operation input unit used for various settings and adjustments.

  The controller 3 of this photographic printing apparatus uses a CPU as a core member and constructs a functional unit for performing various operations of the photographic printing apparatus by hardware and / or software, as shown in FIG. As described above, the functional unit particularly related to the present invention includes an image input unit 31 that takes in image data read by the film scanner 20 and the media reader 21 and performs preprocessing necessary for the next processing, Graphic user interface (hereinafter abbreviated as GUI) that generates a control command from creation of a graphic operation screen including a window, various operation buttons, and the like, and user operation input through such a graphic operation screen (using the keyboard 24, mouse 25, etc.) Sent from the GUI unit 33 and the GUI unit 33 A print management unit 32 that performs image processing on the image data transferred from the image input unit 31 to the memory 30 in order to generate desired print data based on a control command or an operation command directly input from the keyboard 24 or the like. Video control for generating a video signal for causing the monitor 23 to display graphic data sent from the GUI source 33, a simulated image as a print source image, an expected finished print image, and a pre-judge print operation such as color correction A print data generation unit 36 for generating print data suitable for the print exposure unit 14 installed in the print station 1B based on the processed image data for which image processing has been completed, Image data or processed image data that has been processed Etc. etc. formatter 37 that formats the format for writing to CD-R can be mentioned a.

  When the photographic image recording medium is the film 2a, the image input unit 31 separately sends the scan data for the pre-scan mode and the main scan mode to the memory 30, and performs preprocessing according to each purpose. Further, when the captured image recording medium is the memory card 2b, when the captured image data includes thumbnail image data (low resolution data), the actual data of the captured image is used for the purpose of displaying a list on the monitor 23. Separately from (high resolution data), it is sent to the memory 30, but if thumbnail image data is not included, a reduced image is created from this data and sent to the memory 30 as thumbnail image data. Further, the input captured image is converted into a correction processing captured image (for example, 256 × 384 pixels) having a smaller image size in order to obtain various correction conditions and correction positions for the input captured image at high speed. It also has a function.

  The print management unit 32 includes a print order processing unit 40 that manages the print size, the number of prints, and the like, and an image processing unit 50 that performs various types of image processing on the captured image developed in the memory 30.

  The image processing unit 50 is determined with a captured image correction module 60 that calculates an estimated value for each specific captured scene from the corrected captured image developed in the memory 30 and determines a correction degree from the estimated captured scene value. An image correction unit 70 that corrects a high-resolution captured image used for printing purposes by using the degree of correction, and a photo retouch module that performs partial or full correction of the captured image according to a known image processing algorithm. 80 is basically implemented in the form of a program.

  As shown in FIG. 5, the image feature amount calculation unit 61 mounted in the captured image correction module 60 is converted into an appropriate processing size by the image input unit 31 and developed in the memory 30. A histogram feature amount calculation unit 61a for obtaining a histogram feature amount from the region, a region-dependent density feature amount calculation unit 61b for obtaining various region-dependent density feature amounts depending on this region from the region set in the correction processing photographed image, and correction A color feature amount calculation unit 61c that obtains various color feature amounts from the processing photographed image.

  As shown in FIG. 6, the histogram feature amount obtained by the histogram feature amount calculation unit 61 a of this embodiment has five areas (first area, second area, third area, Average density value of each area obtained by dividing into 4th area and 5th area (first area average density value: D1, second area average density value: D2, third area average density value: D3, first 4 zone average density value: D4, 5 zone average density value: D5). Further, as can be understood from FIG. 7, the region-dependent density feature amount obtained by the region-dependent density feature amount calculation unit 61b is a rhombus central region (major axis: 128 × minor axis) of the correction processing photographed image (256 × 384 pixels). Center average density value: E1 which is an average density value at about 192 pixels) and an enlargement which is an average density value in an enlarged area (major axis: 256 × minor axis: about 384 pixels) enlarged from the rhombus central area The central average density value: E2, the whole area average density value: E3, which is the average density value in all areas, and the density standard deviation: E4, which is the density value standard deviation in all areas.

このＲＧＢ／ＰＱ変換によって作り出されるＰＱ色空間は図８に示すように、縦軸がＱ軸で、横軸がＰ軸となっている。このＰＱ座標で決定されるＰＱ色空間は、極座標で考察すると理解し易い。極座標で考察すると、その角度で色相が表され、原点からの距離で彩度が表される。つまり、原点領域が無彩色領域となり、π／６の角度（位相角）がシアン、π／２の角度が青、５π／６の角度がマゼンタ、−５π／６の角度が赤、−π／２の角度が黄、−π／６の角度が緑となっているので、ＰＱ値から簡単に彩度と色相を求めることができる。例えば、写真撮影における重要な被写体である人物の肌色の領域は、図８で斜線で示しているように、角度が−５π／６〜−２π／３の領域となる。 Further, the color feature value obtained by the color feature value computing unit 61c is a feature value related to saturation and hue. In order to obtain this feature value, the captured image for correction processing expressed in the RGB color space is converted into PQ in advance. It is converted into a form expressed in a color space (a kind of hue circle). This RGB / PQ conversion will be described below.
First, a conversion formula from a value in the R, G, B color space to a PQ coordinate value in the PQ color space can be obtained by the matrix calculation shown below.

In the PQ color space created by this RGB / PQ conversion, the vertical axis is the Q axis and the horizontal axis is the P axis, as shown in FIG. The PQ color space determined by the PQ coordinates is easy to understand when considered in polar coordinates. When considered in polar coordinates, hue is represented by the angle, and saturation is represented by the distance from the origin. That is, the origin region is an achromatic region, the angle of π / 6 (phase angle) is cyan, the angle of π / 2 is blue, the angle of 5π / 6 is magenta, the angle of −5π / 6 is red, −π / Since the angle 2 is yellow and the angle −π / 6 is green, saturation and hue can be easily obtained from the PQ value. For example, the skin color area of a person who is an important subject in photography is an area having an angle of −5π / 6 to −2π / 3 as shown by hatching in FIG.

  The color feature amount calculated by the color feature amount calculation unit 61c of this embodiment is the total saturation of the average chromaticity of the entire captured image: S1, and the average saturation of pixels having a density value equal to or higher than the average density value. High density average saturation: S2, the average density of pixels having a density value less than the average density value, low density average saturation: S3, the number of skin color pixels, the number of skin color pixels having a density value greater than or equal to the average density value : S4.

  As shown in FIG. 5, the neural network module 62 operates in this embodiment by a first neural network unit 62 a, a second neural network unit 62 b, and a third neural network unit 62 c. Each neural network unit itself employs a well-known configuration, and a teacher signal is given through the GUI unit 33, and the neural network management unit 62z uses an algorithm known per se using this teacher signal. To derive the weighting factor for each element.

  As shown in FIG. 9, the first neural network unit 62a includes an input layer having nine input elements, an intermediate layer having five intermediate elements, and an output layer having three output elements. Each element of the input layer includes a first zone average density value: D1, a second zone average density value: D2, a third zone average density value: D3, and a fourth zone average density value: D4. And the fifth area average density value: D5, and the area-dependent density feature amount calculation unit 61b gives the central average density value: E1, the enlarged central average density value: E2, and the entire area average density value: E3, and the color. The total average saturation: S1 is given from the feature amount calculation unit 61c. From each element of the output layer, the strobe degree indicating that the shooting scene is a strobe shooting scene: α1, the backlighting degree indicating that the shooting scene is a backlight shooting scene: β1, and the shooting scene is a normal shooting scene The normality indicating the possibility of γ1 is output. α1, β1, and γ1 are represented by numerical values from 0 to 10, and the larger the numerical value, the higher the possibility.

  As shown in FIG. 10, the second neural network unit 62b includes an input layer having ten input elements, an intermediate layer having five intermediate elements, and an output layer having three output elements. Each element of the input layer includes a first zone average density value: D1, a second zone average density value: D2, a third zone average density value: D3, and a fourth zone average density value: D4. And the fifth area average density value: D5, and the area-dependent density feature amount calculation unit 61b gives the central average density value: E1, the enlarged central average density value: E2, and the entire area average density value: E3, and the color. The feature amount calculation unit 61c gives the total average saturation: S1 and the number of skin color pixels: S4. Each element in the output layer outputs the strobe degree: α2, the backlight intensity: β2, and the normality: γ2 as in the first neural network unit 62a.

  As shown in FIG. 11, the third neural network unit 62c includes an input layer having 12 input elements, an intermediate layer having 6 intermediate elements, and an output layer having 3 output elements. Each element of the input layer includes a first zone average density value: D1, a second zone average density value: D2, a third zone average density value: D3, and a fourth zone average density value: D4. And the fifth area average density value: D5, and the area-dependent density feature value calculation unit 61b gives the central average density value: E1, the enlarged central average density value: E2, the whole area average density value: E3, and the density standard deviation: E4. And the color feature amount calculation unit 61c provides high density average saturation: S2, low density average saturation: S3, and the number of skin color pixels: S4. Each element of the output layer outputs the strobe degree: α3, the backlight intensity: β3, and the normality: γ3, similarly to the first neural network unit 62a.

  As is apparent from the above description, the neural network module 62 outputs three calculation results, that is, [α1, β1, γ1] as the first calculation result (output) and [α2, β2 as the second calculation result (output). , γ2] and [α3, β3, γ3] are output as the third calculation result (output). The output processing unit 63 calculates estimated values (estimated vectors) [α, β, γ] for estimating the shooting scene of the final input shot image based on these three calculation results. Here, α is the estimated value of the flash photography scene, β is the estimated value of the backlight photography scene, γ is the estimated value of the backlight photography scene, and takes a numerical value from 0 to 10. For example, if [9,1,1], it is estimated that the photographed image to be processed is a strobe photographing scene with a probability of almost 100%. In the simplest case, the final estimated value can be obtained by arithmetically averaging the three calculation results, but more preferably, the statistical image characteristics of the photographed image (such as the average density value and the density intermediate value). The weighted average using the weight set in accordance with is performed. For example, for a photographed image having an average density value that makes it difficult to determine the flash photography scene, the weighted average is performed by reducing the weight for α which is the calculation result of the flash photography scene and increasing the weight for β and γ. Become.

  With reference to the photographed scene estimated values [α, β, γ] calculated in this way, the correction degree determination unit 64 determines the correction degree for the photographed image. When the correction degree determined by the correction degree determination unit 64 is sent to the image correction unit 70, the image correction unit 70 uses this correction degree to perform inclination or shift of the reference correction curve or partial deformation of the correction curve. Then, a high-resolution captured image used for actual printing is corrected.

  In this way, a calculation result for estimating a shooting scene is output from each using a plurality of neural network units having different image feature values set in the input layer, and based on these calculation results, a so-called consensus system is used. Since the final shooting scene estimation value is calculated, it is possible to identify shooting scenes that are more reliable than the conventional technology, and as a result, it is possible to correct shot images that can produce photographic prints that will please high-quality customers. To do.

  In the description of the embodiment described above, an example in which the photographed image correction technique according to the present invention is incorporated in a photo printing apparatus called a minilab installed in a DP shop has been taken up. It may be incorporated into various photographic printing apparatuses such as an installed self-service photographic printing apparatus, or may be incorporated into a program of the image processing software as one of the photographed image correction functions of the image processing software. That is, the scope of rights of the present invention extends to a captured image correction program that causes a computer to execute the above-described captured image correction method.

  In the above-described embodiment, the print station 1B as the image output means exposes the photographic image to the photographic paper P by the print exposure unit 14 equipped with a laser exposure engine, and the photographic paper after the exposure. A so-called silver salt photographic printing method in which P is subjected to a plurality of development processings has been adopted. Of course, the printing method of a photographed image corrected by the present invention is not limited to such a method. The present invention extends to various photographic printing methods such as an ink jet printing method in which an image is formed by ejecting ink onto a film or paper and a thermal transfer method using a thermal transfer sheet.

Explanatory drawing explaining the principle of the picked-up image correction module which calculates the correction degree for correct | amending an input picked-up image in consideration of the picked-up scene by this invention. 1 is an external view of a photographic printing apparatus equipped with a photographic image correction module according to the present invention. Schematic diagram schematically showing the configuration of the print station of the photo printing device Functional block diagram explaining the functional elements built in the controller of the photo printing device The functional block diagram which shows the function structure in one Embodiment of the picked-up image correction module of this invention. Explanatory drawing explaining the histogram feature amount Explanatory drawing explaining area dependent density feature quantity Explanatory drawing explaining color feature amount Schematic diagram showing the configuration of the first neural network section Schematic diagram showing the configuration of the second neural network section Schematic diagram showing the configuration of the third neural network section

Explanation of symbols

31: Image input unit 60: Captured image correction module 61: Image feature amount calculation unit 61a: Histogram feature amount calculation unit 61b: Region-dependent density feature amount calculation unit 61c: Color feature amount calculation unit 62: Neural network module (neural network unit) )
62a: first neural network unit 62b: second neural network unit 62c: third neural network unit 63: output processing unit 64: correction degree determining unit 70: image correcting unit

Claims (8)

In the captured image correction method for correcting the input captured image in consideration of the captured scene of the input captured image,
Obtaining various histogram feature values as image feature values from the histogram of the input photographed image;
Obtaining various region-dependent density feature amounts as image feature amounts depending on the region from the region set in the input photographed image;
Obtaining various color feature values as image feature values from the input photographed image;
A set of said various histogram feature amounts and the various regions dependent density feature quantity and the variety of color characteristic amounts as a pre-outs image feature amount and the image feature quantity set, together are selected from the image characteristic quantity set Calculating an output relating to a photographic scene by a plurality of neural network units each receiving a plurality of image feature amount subsets including at least partially different image feature amounts;
Calculating an estimated value for estimating a captured scene of the input captured image using outputs from the plurality of neural network units;
A photographed image correction method comprising: In the step of calculating the output related to the shooting scene,
Calculating a first output for the photographic scene by the first neural network unit which receives the first image characteristic amount subset of the previous image feature quantities selected from Kiga image characteristic quantity set,
The second to at least partially together are selected from pre-outs image feature amount set as an input the second image characteristic amount subset of different image feature quantity from the image feature amount included in the first image characteristic amount subset Calculating a second output related to the photographic scene by the neural network unit;
Third image features of different image feature quantity from the image feature amount is at least partially included in the first image characteristic amount subset and the second image characteristic amount subset with are selected from pre-outs image characteristic quantity set Calculating a third output related to the photographic scene by a third neural network unit having the quantity subset as input;
The captured image correction method according to claim 1, wherein: Captured image correction method according to claim 1 or 2, characterized in that before Symbol respective image feature amount subset contains histogram feature amount and the area-dependent density features and the color feature. As before Kihi Sutoguramu feature amount, the average density value of each zone obtained by dividing the density gradation into a plurality of zones are used, and as the pre-Symbol area dependent density feature quantity, the central region of the input captured image A central average density value that is an average density value in the area, an enlarged central average density value that is an average density value in an enlarged area that is enlarged from the central area, and an overall average density value that is an average density value in all areas. , a density standard deviation is the concentration value standard deviation of the entire region is used, and as the pre-Symbol color feature, the average and saturation of the input captured image, an average density value or density value of the input captured image A high density average saturation which is an average saturation of pixels having a low density average saturation which is an average saturation of pixels having a density value less than the average density value of the input photographed image, and an average density of the input photographed image Skin color, which is the number of skin color pixels with density value greater than or equal to And a prime number is used,
The method for correcting a photographed image according to any one of claims 1 to 3 , wherein: In the captured image correction module for correcting the input captured image in consideration of the captured scene of the input captured image,
A histogram feature amount calculation unit for obtaining various histogram feature amounts as image feature amounts from the histogram of the input photographed image;
A region-dependent density feature amount calculating unit for obtaining various region-dependent density feature amounts as image feature amounts depending on the region from the region set in the input photographed image;
A color feature quantity computing unit for obtaining various color feature quantities as image feature quantities from the input photographed image;
A set of said various histogram feature amounts and the various regions dependent density feature quantity and the variety of color characteristic amounts as a pre-outs image feature amount and the image feature quantity set, together are selected from the image characteristic quantity set A plurality of neural network units for calculating an output relating to a shooting scene by inputting each of a plurality of image feature amount subsets including at least partially different image feature amounts;
An output processing unit that calculates an estimated value for estimating a shooting scene of the input captured image using outputs from the plurality of neural network units;
A photographed image correction module comprising: The front Symbol multiple of the neural network unit,
A first neural network unit for calculating a first output for the photographic scene a first image characteristic amount subset of front Kiga image characteristic quantity set image feature quantities selected from the input,
Related to a shooting scene as input a second image characteristic amount subset of different image feature quantity is the image feature amount and the at least partially included in the first image characteristic amount subset with are selected from pre-outs image characteristic quantity set A second neural network unit for calculating a second output;
Third image features of different image feature quantity from the image feature amount is at least partially included in the first image characteristic amount subset and the second image characteristic amount subset with are selected from pre-outs image characteristic quantity set A third neural network unit for calculating a third output related to the shooting scene with the quantity subset as an input;
The photographed image correction module according to claim 5, wherein: Photographed image correction module according to claim 5 or 6, characterized in that it contains the histogram feature amount and the area-dependent density features and the color feature before Symbol respective image feature amount subset. As before Kihi Sutoguramu feature amount, the average density value of each zone obtained by dividing the density gradation into a plurality of zones are used, and as the pre-Symbol area dependent density feature quantity, the central region of the input captured image A central average density value that is an average density value in the area, an enlarged central average density value that is an average density value in an enlarged area that is enlarged from the central area, and an overall average density value that is an average density value in all areas. , a density standard deviation is the concentration value standard deviation of the entire region is used, and as the pre-Symbol color feature, the average and saturation of the input captured image, an average density value or density value of the input captured image A high density average saturation which is an average saturation of pixels having a low density average saturation which is an average saturation of pixels having a density value less than the average density value of the input photographed image, and an average density of the input photographed image Skin color, which is the number of skin color pixels with density value greater than or equal to And a prime number is used,
The photographed image correction module according to any one of claims 5 to 7 , wherein

Images