JP5961390B2 - Lookup table acquisition device, lookup table generation support method, lookup table production method, and program - Google Patents

Description

  The present invention relates to a lookup table acquisition device that acquires a lookup table, which is information for generating an output image, using an input image.

  Conventionally, in a digital image signal interpolating apparatus for interpolating and generating image data of a predetermined pixel between pixels corresponding to actual input image data and existing input image data, the actual input image data is received and the predetermined pixel The least square method is used to minimize the sum of squares of the error between the interpolation value and the true value, using means for extracting a predetermined number of actual pixel data existing in the peripheral position of the pixel and pixel data for obtaining the coefficient. And a means for interpolating and generating pixel data of a predetermined pixel by linear linear combination of the predetermined number of coefficients determined in advance by the extracted predetermined number of existing pixel data. There has been an interpolation device for image signals (see Patent Document 1).

  Also, there has been a digital video signal conversion device that generates a second digital video signal having a larger number of pixels than the first digital video signal from the first digital video signal (see Patent Document 2).

Japanese Patent No. 2825482 (first page, FIG. 1 etc.) Japanese Patent No. 3072306 (first page, FIG. 1 etc.)

  However, in the prior art, information for outputting an image in which the brain load is adjusted according to the purpose cannot be generated.

The lookup table acquisition device according to the first aspect of the present invention includes a first image storage unit that can store one or more first images, and one or more images in which brain load is adjusted with respect to the first image. A second image storage unit capable of storing each second image in association with one or more first images, all or part of the one or more first images, and one or more second images. A storage unit capable of storing a look-up table having one or more correspondence information, which is information for taking correspondence with all or a part of the first, and one or more first items stored in the first image storage unit A lookup table configuration unit that configures a lookup table having one or more correspondence information from all or part of an image and all or part of one or more second images stored in the second image storage unit And lookup table component A look-up table arrangement, a look-up table obtaining apparatus comprising a look-up table storage section for storing in the storage unit.
With this configuration, it is possible to generate information for outputting an image in which the brain load is adjusted according to the purpose.
In the lookup table acquisition apparatus according to the second aspect of the present invention, the correspondence information corresponds to the first aspect, wherein the correspondence information is information related to all or a part of the one or more first images, and one or more pieces. Information relating to all or part of the second image of the second image-related information, which is information corresponding to the first image-related information.
First information related to one or more first images stored in the first image storage unit is acquired from all or a part of one or more first images. One or more second image related information, which is information about all or part of one or more second images stored in the one image related information acquisition unit and the second image storage unit. A second image related information acquisition unit that acquires from all or a portion of the first image related information acquired by the first image related information acquisition unit, and a first image related information Information corresponding to the information and having the second image related information acquired by the second image related information acquisition unit, and all or a part of the one or more first images and one or more second images. Indicates correspondence with all or part The correspondence information that is information, a lookup table acquiring apparatus constituting the look-up table having one or more.

With this configuration, it is possible to generate information for outputting an image in which the brain load is adjusted according to the purpose.
In the lookup table acquisition apparatus according to the third aspect of the present invention, the correspondence information includes a feature vector corresponding to all or a part of the one or more first images and one or more first tables. An image conversion filter for converting all or a portion of one image into all or a portion of one or more second images, and the look-up table component includes all or one or more of the one or more first images. Acquiring a feature vector from a part, obtaining a conversion filter for converting all or a part of one or more first images into all or a part of one or more second images, and the feature vector and the conversion filter; Is a lookup table acquisition device that constitutes a lookup table having one or two or more correspondence information.
With this configuration, it is possible to generate information for outputting an image in which the brain load is adjusted according to the purpose.

  The lookup table acquisition device according to the fourth aspect of the present invention is the same as the first or more first images in association with the first or more first images with respect to the first to third aspects of the invention. A second image information candidate set storage unit capable of storing one or more second image information candidate sets, which is a set of two or more second image information candidates for the object, and one or more second image information candidate sets. Select one or more second image information candidates from a candidate output unit to be output and two or more second image information candidates of each of the one or more second image information candidate sets output by the candidate output unit An instruction receiving unit that receives an instruction to receive the image, and one or more second image information candidates corresponding to the instruction received by the instruction receiving unit are acquired, and the second image information candidate set is acquired. The one or more second images are associated with the corresponding first image and the second image A look-up table obtaining device further comprising a second image storage section for storing the paying unit.

  With this configuration, it is possible to generate information for outputting an image in which the brain load is adjusted for a specific user.

  According to the lookup table acquisition device of the present invention, it is possible to generate information for outputting an image in which the brain load is adjusted according to the purpose.

Block diagram of image processing apparatus 1 according to Embodiment 1 A flowchart for explaining the operation of the image processing apparatus 1 Flowchart explaining the image analysis processing Flow chart explaining the mapping process Flowchart explaining the same type identifier acquisition process Flowchart explaining the lookup table update process Block diagram of other image processing apparatus 1 Block diagram of the image processing apparatus 2 in the second embodiment A flowchart for explaining the operation of the image processing apparatus 2 Block diagram of other image processing apparatus 2 Block diagram of lookup table acquisition apparatus 3 in Embodiment 3 The flowchart explaining the operation | movement which selects the 2nd image in the lookup table acquisition apparatus 3 The flowchart explaining the operation | movement which comprises the lookup table in the lookup table acquisition apparatus 3 The figure which shows the example of the same 2nd image information candidate set The figure which shows the example of an output of such second image information candidate set The figure which shows the example of the 1st image in Embodiment 4. The figure which shows the example of the second image The figure which shows the example of the first spatiotemporal block The figure which shows the example of the second spatiotemporal block Conceptual diagram explaining the operation of the lookup table component Conceptual diagram of processing of the lookup table configuration unit Conceptual diagram of processing of the lookup table configuration unit The figure which shows the example of the same lookup table Block diagram of image processing apparatus 4 according to Embodiment 5 A figure showing an example of an image with the same shallow depth of field Conceptual diagram of processing of the mapping unit Block diagram of image processing apparatus 5 in Embodiment 6 The figure which shows the example of the restraint information A flowchart for explaining the operation of the image processing apparatus 5 Flow chart explaining the mapping process Flowchart explaining the constraint condition determination process Block diagram of image processing apparatus 6 in Embodiment 7 The figure which shows the example of the same LUT change information management table FIG. 34 is a flowchart for explaining the operation of the image processing apparatus 6. Flow chart explaining the mapping process Flow chart explaining the LUT determination process Block diagram of image learning apparatus 7 in Embodiment 8 Overview of the computer system in the above embodiment Block diagram of the computer system

  Hereinafter, embodiments of an image processing apparatus and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment 1)
In the present embodiment, an image processing apparatus 1 that acquires an image in which the brain load is adjusted with respect to an received image according to the purpose using a lookup table described later will be described. In addition, the image in which the brain load is adjusted according to the purpose is, for example, an image that can further reduce the brain load during viewing or an image that stimulates the brain during viewing and increases entertainment properties. An image that can further reduce the load on the brain at the time of viewing is, for example, an image that makes it easier to recognize the target. The purpose is usually the purpose of the viewer when viewing the image, for example, the purpose of relaxing and watching, the purpose of watching for a long time without getting tired as much as possible, and the purpose of enjoying entertainment. .

  Moreover, in this Embodiment, the process which acquires the image by which the brain load was adjusted according to the objective performs a process for every spatiotemporal block, for example. The description of the spatiotemporal block will be described later.

  In the present embodiment, the look-up table can be customized.

  In the present embodiment, the image processing apparatus 1 has a function of converting an output candidate image so as to satisfy the predetermined condition when the output candidate image does not satisfy the predetermined condition.

  FIG. 1 is a block diagram of an image processing apparatus 1 in the present embodiment.

  The image processing apparatus 1 includes a storage unit 101, a constraint condition storage unit 102, an instruction reception unit 103, an image reception unit 104, a division unit 105, an image analysis unit 106, a mapping unit 107, a synthesis unit 108, an image output unit 109, a look An uptable update unit 110 is provided.

  The mapping unit 107 includes, for example, a type identifier acquisition unit 1071, an output image candidate acquisition unit 1072, a determination unit 1073, a change unit 1074, and an output image acquisition unit 1075.

  The storage unit 101 can store one or more lookup tables. The storage unit 101 may store two or more sets of a type identifier for identifying the type of the lookup table and a lookup table corresponding to the type identifier. Note that the data structure of the lookup table is not limited to the table structure. The data structure of the lookup table does not matter. Needless to say, the type of lookup table may be the lookup table itself.

  Here, the lookup table has one or more correspondence information. The correspondence information is information for taking correspondence between all or part of one or more first images and all or part of one or more second images. When the lookup table has one correspondence information, the correspondence information is, for example, an arithmetic expression for converting an image or one or more parameter groups given to an arithmetic expression for converting an image. The correspondence information may be, for example, an arithmetic expression that calculates the median of the pixel values of the spatiotemporal block.

  Further, all or a part of the one or more first images and all or a part of the one or more second images may be an output one-screen image or a part of one-screen block image. One pixel may be used, a plurality of temporally continuous screens may be used, or a part of a temporally continuous multiple screen space (a spatiotemporal block described later) may be used. It may be a set of one pixel. That is, the unit of the information amount of the first image and the second image is not limited. Note that the spatio-temporal block generally refers to a part obtained by dividing an image spatially, temporally, or spatially and temporally. However, the spatiotemporal block may be a set of pixels. That is, the spatiotemporal block is not necessarily a rectangular continuous area. The spatiotemporal block may be a non-rectangular and discontinuous image area. In addition, the plurality of spatiotemporal blocks obtained by dividing the image may have partially overlapping pixels.

  The second image is an image that can further reduce the load on the brain when viewed from the first image. In addition, the second image is an image that can reduce the load on the brain in the case of normally recognizing one object, as compared to the first image. Images that can further reduce the load on the brain are the texture, perspective, presence, presence, weight, warmth, chilliness, speed, dynamics, smoothness of movement, freshness, luxury It is an image that more feels one or more attributes of feeling. Note that the attribute varies depending on the image. The texture is a texture of a target included in an image such as a material feeling, a glossy feeling, a transparent feeling, a soft feeling, and a moist feeling. In addition, the sense of perspective can be said to be a sense of resolution, a stereoscopic effect, a sense of depth, a feeling of air, and the like.

  Note that an image that can reduce the load on the brain may be paraphrased as an image that is suitably felt by the brain.

  In addition, the one or more second images that can reduce the brain load from the one or more first images are, for example, one or more images that can be acquired in a different environment from the one or more first images. That is, in such a case, for example, one or more first images and one or more second images have different environments for acquiring images.

  A 1st image is an image about one object obtained in one environment, for example. The second image is, for example, an image about the one target obtained in another environment. The first image is, for example, an image obtained by photographing one object in one environment. The second image is, for example, an image obtained by photographing the first target in another environment. In such a case, the second image and the first image usually have the same number of pixels, but may have different numbers of pixels. In addition, the image which image | photographed one object may be the produced image equivalent to having image | photographed one object.

  The other environment is an environment different from the one environment, and is an environment in which an image that matches the purpose can be taken from the one environment. In addition, the other environment is an environment in which, for example, an image that can reduce the brain load from one environment can be taken. The other environment is an environment such as the following (1) to (9). (1) an environment where the amount of light of the illumination (light source) is larger than that of one environment, (2) an environment of the position of the light source where the shade of the object on the image is well expressed compared to the one environment, (3) one The environment of the light source position that expresses the stereoscopic effect of the object on the image better than the environment, and (4) The camera position that expresses the stereoscopic effect of the object on the image better than the one environment. Environment, (5) Environment with different lighting color from one environment, (6) Environment with different weather from one environment, (7) Environment with different temperature and humidity from one environment, (8) Different from one environment An environment with a depth of field, and (9) an environment with a shutter speed different from the one environment.

  That is, the shooting environment such as the amount of light, the position of the light source, the color of the light source, the position of the camera, the camera settings, and the like are different between the one environment and the other environment. The environment is usually a shooting environment, but may be anything as long as it relates to an environment for acquiring an image.

  Note that the second image does not include an image that simply has more pixels than the first image. Such an image having only a large number of pixels is an image based on an image simply taken by a camera having a large number of pixels (for example, an HD camera). The image is based on how the brain feels (it may be said that the person feels), and makes a clear difference.

  In addition, the second image in which the brain load is adjusted with respect to the first image includes, for example, an image in which entertainment properties are improved as compared with the first image. An image with improved entertainment properties is an image that is suitable for the user who prefers entertainment properties and is suitable for the brain. It can be said that an image that can be suitably felt in the brain is an image in which stimulation in the brain can be enjoyed. In addition, the plurality of images (videos) with improved entertainment properties are, for example, videos in which a flash screen is inserted in part.

  In addition, when the second image in which the brain load is adjusted is an image that is more entertaining than the first image, the second image can more enjoy stimulation to the brain, In some cases, the load may further increase.

  Further, the second image in which the brain load is adjusted according to the purpose from the first image is, for example, the first image obtained by photographing one object with respect to the first image. This is a deformed image. The second image obtained by deforming the first image is, for example, a portrait image of a person shown in the first image. It can be said that a deformed image such as a portrait image is an image in which the brain load is adjusted so that the user can easily recognize the target and the brain load is reduced when the target is recognized.

  In addition, the second image in which the brain load is adjusted according to the purpose from the first image, for example, describes the one target with respect to the first image obtained by photographing the one target. It is an image of a picture taken. By describing the target, the concept of the target may be expressed more clearly. That is, it can be said that an image obtained by photographing a picture depicting a target is an image in which the brain load is adjusted so that the brain load can be reduced, rather than an image obtained by directly photographing the target.

  In addition, the second image in which the brain load is adjusted according to the purpose from the first image is, for example, the first image obtained by photographing one object with respect to the first image. It is a CG image. It can be said that the CG image is an image in which the brain load is adjusted so that the brain load can be reduced rather than the captured image depending on the object.

  The correspondence information is, for example, a pair of first image related information and second image related information. The first image related information is information related to all or a part of one or more first images. The second image related information is information related to all or a part of one or more second images. The first image related information is, for example, one or more feature amounts extracted from all or a part of one or more first images. The second image related information is, for example, all or a part of one or more second images. The second image related information is, for example, an arithmetic expression that generates all or a part of one or more second images. The second image related information is, for example, a pair of parameter groups given to an arithmetic expression for generating all or a part of one or more second images. In such a case, the arithmetic expression is held by the mapping unit 107. The one or more feature amounts extracted from the image include, for example, a neighborhood feature amount that is a partial feature amount of the one or more images and an overall feature amount that is a feature amount of the entire one or more images. . The neighborhood feature amount includes, for example, a pixel value, activity, spatiotemporal correlation, motion vector, frequency distribution, and the like. The activity is, for example, a maximum value and a minimum value of a plurality of pixels, a dynamic range (DR), a difference value between the plurality of pixels, and the like. The difference value between a plurality of pixels may be a difference value between a plurality of pixels in the space, a difference value between a plurality of pixels in the time direction, or a difference value between a plurality of pixels in the space and the time direction. . Further, the total feature amount relates to, for example, a pixel value distribution of the entire image, a motion vector distribution of the entire image, a spatiotemporal correlation of one or more images, a noise amount of the entire image, an image format, and editing of the entire image. Information (whether it is PinP, whether it is PoutP, presence / absence of a telop, etc.), content information related to content that is one or more images, pattern detection results (face, etc.), and the like. Here, the content information is, for example, the program genre, title, performer name, and the like included in the electronic program guide (EPG). One or more feature quantities are also referred to as feature quantity vectors as appropriate.

  The storage unit 101 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. In particular, the storage unit 101 is preferably a nonvolatile RAM. Further, the process of storing the lookup table in the storage unit 101 does not matter. For example, a lookup table may be stored in the storage unit 101 via a recording medium, or a lookup table transmitted via a communication line or the like may be stored in the storage unit 101. Alternatively, a lookup table input via an input device may be stored in the storage unit 101.

  The constraint condition storage unit 102 can store constraint conditions. The constraint condition is a condition related to any process in the image processing apparatus 1. The conditions regarding processing are conditions for all or part of an image (input image) to be processed, for example. Moreover, the conditions regarding processing are conditions regarding the look-up table utilized for processing, for example. Moreover, the conditions regarding processing are conditions regarding an output image, for example. It can be said that the constraint condition is a condition to be satisfied by one or more types of information of the input image, the correspondence information, or the output image. The condition for all or part of the image to be processed (input image) is, for example, a condition using one or more feature quantities (for example, luminance, pixel value, etc.) to be processed as parameters. The condition for all or part of the input image is, for example, that the pixel value does not deviate from the normal range. Note that the normal range of pixel values is stored in advance. The condition relating to the lookup table is a condition relating to selection such as the selection frequency of the correspondence information included in the lookup table. The condition regarding the lookup table is, for example, that the same correspondence information as the correspondence information used immediately before is not used. The conditions regarding the lookup table are, for example, using the same correspondence information continuously under the first specific condition, and using the specific correspondence information fixedly under the second specific condition, For example, the same correspondence information is not continuously used under the third specific condition, and the specific correspondence information is not used under the fourth specific condition. Further, for example, the condition relating to the output image is a condition using one or more feature quantities (for example, luminance, pixel value, etc.) of the output image as parameters. It can be said that the condition relating to the output image is a constraint condition to be satisfied by all or a part of the one or more second images acquired by the mapping unit 107. The condition regarding the output image is, for example, that the activity is within a threshold value. The conditions relating to the output image are, for example, the maximum value of the pixel “pixel value <= 255”, the minimum value of the pixel “pixel value> = 10”, and the “pixel brightness time” for the change Δ of the pixel value. Change <= 100 ”, dynamic range“ DR <= 200 ”, and the like. Needless to say, the constraint condition may be embedded in a program for performing image processing.

  Further, the constraint condition may include two or more kinds of conditions among a condition for all or part of the input image, a condition for the lookup table, and a condition for the output image. The constraint condition is, for example, that the difference between the pixel value of the input image and the pixel value of the output image is larger than the threshold value. The threshold value is stored in advance.

  The constraint condition storage unit 102 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The process of storing the constraint condition in the constraint condition storage unit 102 is not limited. For example, the constraint condition may be stored in the constraint condition storage unit 102 via a recording medium, and the constraint condition transmitted via a communication line or the like is stored in the constraint condition storage unit 102. Alternatively, the constraint condition input via the input device may be stored in the constraint condition storage unit 102.

  The instruction receiving unit 103 receives an instruction. The instruction may be received from the user, read from an external recording medium, or received from an external device. The instruction may be embedded in a broadcast transmitted from the broadcast device. The instruction is an instruction to change the lookup table so as to match the user's preference or the like. The instruction includes, for example, the degree of stereoscopic effect of the image. In addition, the instruction is an instruction to select one or more second image candidates from two or more second image candidates included in each of the one or more second image information candidate sets that are output. . Such selection is an instruction to select an image that matches the user's preference. In addition, the instruction may include user attributes such as age, age group (30's, 60's, etc.), gender, myopia, presbyopia, and presence / absence of adjustment disorder. The instruction is, for example, a purpose, a problem, or a preference. Examples of purposes, tasks, and preferences include "I want to make the image more visible because it is difficult to see the image", "I want to make the image darker and less visible", and "I want to see a more flashy image". That is, there are various contents and types of instructions.

  In addition, reception means reception of information input from an input device such as a remote controller, keyboard, mouse, touch panel, reception of information transmitted via a wired or wireless communication line, optical disk, magnetic disk, semiconductor memory, etc. It is a concept including reception of information read from a recording medium. The instruction input means may be anything such as a remote control, a numeric keypad, a keyboard, a mouse, or a menu screen. The instruction receiving unit 103 can be realized by a device driver of an input unit such as a numeric keypad or a keyboard, control software for a menu screen, or the like.

  The image receiving unit 104 receives an image. Further, the image receiving unit 104 may receive a video that is a set of one or more images. Here, the image may be a single still image or a moving image composed of two or more still images. The image receiving unit 104 can be realized by, for example, a television tuner, a wireless or wired receiving unit, or the like. Note that accepting an image may be reading of an image from a recording medium. That is, the means for receiving an image is not limited.

  The dividing unit 105 divides the image (which may be a still image or a moving image) received by the image receiving unit 104 into two or more spatially, temporally, or spatially and temporally, and is divided into two or more first times. Get the space block. As described above, the plurality of spatiotemporal blocks obtained by dividing the image may include overlapping regions. Further, as described above, the plurality of spatiotemporal blocks obtained by dividing an image are not necessarily rectangular continuous regions. The spatiotemporal block may be a non-rectangular and discontinuous image area. For example, the dividing unit 105 may divide one screen (one field or one frame) into four, or may divide n screens (n is an integer of 2 or more) into eight. The number of divisions is not limited. Further, the dividing unit 105 may acquire one pixel at a time from the image and pass it to the image analyzing unit 106. That is, the spatiotemporal block may be one pixel.

  The dividing unit 105 can be usually realized by an MPU, a memory, or the like. The processing procedure of the dividing unit 105 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The image analysis unit 106 determines all or part of the first image corresponding to all or part of one or more images received by the image receiving unit 104. For example, the image analysis unit 106 may determine all or a part of the first image corresponding to each first spatiotemporal block for each of the two or more first spatiotemporal blocks acquired by the dividing unit 105. good. Note that the image analysis unit 106 may determine all or part of two or more first images.

  Further, for example, the image analysis unit 106 analyzes the image received by the image reception unit 104 or each of two or more first spatiotemporal blocks acquired by the division unit 105, acquires one or more feature amounts, All or a part of the first image corresponding to the above feature amount is determined. Here, the one or more feature amounts are, for example, one or more feature amounts of the above-described neighborhood feature amount and the entire feature amount. Note that the image analysis unit 106 may perform only processing for analyzing all or a part of one or more images received by the image receiving unit 104 and acquiring one or more feature amounts. Note that the feature amount acquired by the image analysis unit 106 preferably includes both the overall feature amount and the neighborhood feature amount.

  In addition, the process of determining all or part of the first image corresponding to one or more feature amounts is, for example, the acquired feature amount vector and the feature amount vector of all or part of the first image included in each correspondence information And a parameter group corresponding to the feature vector of all or a part of the first image having the smallest distance, or an arithmetic expression or the like is obtained.

  The process for determining all or part of the first image corresponding to one or more feature amounts is, for example, a process for calculating an address of a lookup table from the acquired feature amount vector. Note that the process of calculating the address of the lookup table is a process of substituting a feature quantity vector into a previously stored arithmetic expression.

  The image analysis unit 106 can usually be realized by an MPU, a memory, or the like. The processing procedure of the image analysis unit 106 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The mapping unit 107 uses one or more correspondence information in one or more correspondence information in the lookup table, and uses one or more corresponding to all or a part of one or more images received by the image receiving unit 104. All or part of the second image is acquired. Further, for example, the mapping unit 107 uses correspondence information corresponding to all or a part of the first image determined by the image analysis unit 106 (including using part of the correspondence information (for example, parameter group)). All or part of the second image corresponding to all or part of the first image determined by the image analysis unit 106 is acquired as all or part of the output image. The mapping unit 107 acquires, for example, a parameter group given to an arithmetic expression included in correspondence information corresponding to all or a part of the first image determined by the image analysis unit 106, performs an operation, and the image analysis unit 106 determines All or part of the second image corresponding to all or part of the first image is generated. In addition, the mapping unit 107 acquires, for example, all or part of the second image included in the correspondence information corresponding to all or part of the first image determined by the image analysis unit 106 from the correspondence information.

  Based on the one or more feature amounts acquired by the image analysis unit 106, the mapping unit 107 adjusts the brain load so that the one or more images received by the image reception unit 104 are more suitable for the purpose. A look-up table capable of outputting an image in which the brain load is adjusted so as to be more suitable for an operation of selecting one or more correspondence information that can be output, or one or more images received by the image receiving unit 104 One of the operations for selecting one or more pieces of correspondence information may be performed. Next, the mapping unit 107 uses all or part of the one or more second images corresponding to all or part of the one or more images received by the image receiving unit 104 using the selected one or more pieces of correspondence information. You may get

  Further, the mapping unit 107 adjusts the brain load so that the one or more images received by the image receiving unit 104 are more suitable for the purpose based on the one or more feature amounts acquired by the image analysis unit 106. For selecting two or more pieces of correspondence information that can output an image, or for one or more images received by the image receiving unit 104, a look-up that can output an image in which the brain load is adjusted so as to better match the purpose. One of the operations of selecting a table and two or more pieces of correspondence information may be performed, and all or part of one or more second images may be acquired using the two or more pieces of correspondence information selected. When acquiring all or part of one or more second images using two or more pieces of correspondence information, the mapping unit 107 converts all or part of one or more images received by the image receiving unit 104 to two or more. It is also possible to perform processing using each of the corresponding information, and fuse two or more processed results to obtain all or a part of one or more second images. In addition, when acquiring all or part of one or more second images using two or more pieces of correspondence information, the mapping unit 107 generates new correspondence information using two or more pieces of correspondence information, and the new information Such correspondence information may be used to acquire all or part of one or more second images. Note that the fusion of two or more results is, for example, generating an image using both of two or more attributes of an image or generating an image having an intermediate property between two or more attributes. It is.

The mapping unit 107 uses correspondence information corresponding to all or part of the first image determined by the image analysis unit 106 for each of the two or more first spatiotemporal blocks (using a part of the correspondence information). The second spatiotemporal block corresponding to all or a part of the first image determined by the image analysis unit 106 may be acquired.
In addition, the mapping unit 107 acquires, for example, a parameter group to be given to an arithmetic expression included in correspondence information corresponding to all or a part of the first image determined by the image analysis unit 106, performs the calculation, and the image analysis unit 106 A second space-time block corresponding to all or part of the determined first image is acquired. In addition, the mapping unit 107 corresponds to, for example, all or part of the second image (second spatiotemporal block) included in the correspondence information corresponding to all or part of the first image determined by the image analysis unit 106. Obtain from information.

  The mapping unit 107 may perform correction processing after acquiring all or a part of the second image. In the correction process, the mapping unit 107 uses the feature amount of all or a part of the image received by the image receiving unit 104 and the feature amount of all or a part of the first image included in the correspondence information, and uses the second image. Is a process of correcting all or a part of the above. Note that an image acquired by correction may also be referred to as a second image or an output image. In the correction process, for example, the mapping unit 107 acquires a difference between the feature amount of all or part of the image received by the image receiving unit 104 and the feature amount of all or part of the first image included in the correspondence information, This is a process for correcting all or part of the second image using the difference and correcting all or part of the output image so as to more closely match all or part of the image received by the image receiving unit 104. .

  The mapping unit 107 can be usually realized by an MPU, a memory, or the like. The processing procedure of the mapping unit 107 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Hereinafter, a case where the mapping unit 107 includes a type identifier acquisition unit 1071, an output image candidate acquisition unit 1072, a determination unit 1073, a change unit 1074, and an output image acquisition unit 1075 will be described. The mapping unit 107 may not include the type identifier acquisition unit 1071, the output image candidate acquisition unit 1072, the determination unit 1073, the change unit 1074, and the output image acquisition unit 1075.

  The type identifier acquisition means 1071 constituting the mapping unit 107 acquires one or more type identifiers that identify the type of the lookup table to be used. For example, when the type identifier and one or more feature quantities of all or a part of the image are associated with each other and stored in the storage unit 101, the type identifier acquisition unit 1071 has the entire image received by the image receiving unit 104. Alternatively, a part of one or more feature quantities is acquired, one or more feature quantities that are closest to the one or more feature quantities are determined, and a type identifier corresponding to the one or more feature quantities is acquired from the storage unit 101. Also good. Further, the type identifier acquisition unit 1071 may use the genre of a program (program being output) included in an EPG (electronic program guide) stored in an EPG storage unit (not shown) as the type identifier. Further, the type identifier acquisition unit 1071 may store the type identifier received from the user in a recording medium (not shown) and acquire the type identifier. The type identifier acquisition unit 1071 acquires one or more type identifiers corresponding to the instruction received by the instruction receiving unit 103. For example, when the instruction receiving unit 103 receives a purpose corresponding to “I want to make the image more visible because it is difficult to see the image”, the type identifier acquisition unit 1071 determines the type identifier of the lookup table that reduces noise. To get. Further, for example, when the instruction receiving unit 103 receives a purpose corresponding to “I want to make the image more visible because it is difficult to see the image”, the type identifier acquisition unit 1071 identifies the type of the lookup table that brightens the lighting environment. The type identifier to be acquired may be acquired. Further, for example, when the instruction receiving unit 103 receives a purpose corresponding to “I want to make the image more visible because it is difficult to see the image”, the type identifier acquisition unit 1071 includes the type identifier of the lookup table for reducing noise, The type identifier of the look-up table that brightens the lighting environment may be acquired.

  Further, for example, when the instruction receiving unit 103 receives a purpose corresponding to “I want to make the image more visible because it is difficult to see the image”, the type identifier acquisition unit 1071 detects noise in the image and detects noise above the threshold. Is detected, and if it is determined that there is noise equal to or greater than the threshold, the type identifier of the look-up table that reduces noise is acquired. Also, for example, when the instruction receiving unit 103 receives a purpose corresponding to “I want to make the image more visible because it is difficult to see the image”, the type identifier acquisition unit 1071 acquires the luminance value distribution of the image, and When it is determined that the luminance distribution is too dark to satisfy a predetermined condition, a type identifier of a lookup table that brightens the lighting environment may be acquired.

  The type identifier acquisition unit 1071 can be usually realized by an MPU, a memory, or the like. The processing procedure of the type identifier acquisition unit 1071 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output image candidate acquisition unit 1072 is determined by the image analysis unit 106 using correspondence information corresponding to all or part of the first image determined by the image analysis unit 106 (which may be part of the correspondence information). All or part of the second image corresponding to all or part of the first image is obtained. All or part of the second image acquired here is a candidate for all or part of the output image. In addition, “acquiring all or part of the second image” means, for example, calculating using the arithmetic expression possessed by the correspondence information or a set of parameters to be substituted into the arithmetic expression possessing the correspondence information, To acquire all or part of an image. In this case, the correspondence information includes an arithmetic expression for generating all or part of the second image corresponding to all or part of the first image, or a set of parameters to be substituted into the arithmetic expression. Further, “acquiring all or part of the second image” means, for example, all or all of the second image included in the correspondence information corresponding to all or part of the first image determined by the image analysis unit 106. You may acquire a part.

  The output image candidate acquisition unit 1072 can usually be realized by an MPU, a memory, or the like. The processing procedure of the output image candidate acquisition unit 1072 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The determination unit 1073 determines whether or not the constraint condition is satisfied. When the constraint condition is a condition for all or part of the image to be processed, the determination unit 1073 acquires one or more feature amounts of all or a part of the image received by the image receiving unit 104, and the one or more feature values are acquired. It is determined whether or not the feature amount satisfies a constraint condition. Further, when the constraint condition is a condition related to the lookup table, the determination unit 1073 acquires information on the past usage of the lookup table (a usage history corresponding to the condition), and whether or not the usage history satisfies the constraint condition. Determine whether. The usage history is, for example, the type identifier of the lookup table used in the processing of the latest n field (n is an integer of 1 or more) in the input image, or the recent n field ( n is information for specifying the correspondence information used in the processing of (an integer of 1 or more). When the constraint condition includes a usage history, the usage history is generated by the mapping unit 107 being accumulated in a recording medium (not shown). Further, when the constraint condition is a condition related to the output image, the determination unit 1073 determines whether the whole or part of the second image acquired by the output image candidate acquisition unit 1072 satisfies the constraint condition of the constraint condition storage unit 102, for example. Judge whether or not. For example, the determination unit 1073 acquires the attribute values (for example, the maximum value of the pixel and the temporal change in the luminance of the pixel) of all or part of the second image included in the constraint condition, and the attribute value is the constraint condition ( For example, it is determined whether or not “the maximum value of the pixel <= 255” and “the temporal change in the luminance of the pixel <= 100”) are satisfied.

  The determination unit 1073 can be usually realized by an MPU, a memory, or the like. The processing procedure of the determination unit 1073 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The changing unit 1074 changes all or part of the second image when the determining unit 1073 determines that the constraint condition is not satisfied. The changing unit 1074 may change the lookup table when the determining unit 1073 determines that the constraint condition is not satisfied.

  In addition, the changing unit 1074, when the determining unit 1073 determines that the constraint condition is not satisfied, is an input image received after the determination, a spatiotemporal block obtained after the determination, or one or more input images received after the determination. The feature amount or one or more feature amounts of the spatiotemporal block obtained after the determination may be changed so as to satisfy the constraint condition.

  If the determination unit 1073 determines that the constraint condition is not satisfied, the change unit 1074 changes, for example, all or part of the second image that is the determination target of the determination unit 1073 so that the constraint condition is satisfied. Then, the changed image is acquired. Here, in the “processing to change to satisfy the constraint condition”, for example, the maximum value of a plurality of pixels constituting all or part of the second image is set to the constraint condition (for example, “maximum value <= 255”). For example, the values of the plurality of pixels are compressed as a whole so that the maximum value of the plurality of pixels becomes 255 so as to match. Note that the process of compressing the values of a plurality of pixels as a whole may reduce the values of the plurality of pixels at the same rate, or may change the values of the plurality of pixels according to different rates or the size of the values. The value may be reduced.

  The changing unit 1074 can be usually realized by an MPU, a memory, or the like. The processing procedure of the changing means 1074 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output image acquisition unit 1075 uses correspondence information corresponding to all or a part of the first image determined by the image analysis unit 106 among a plurality of pieces of correspondence information (which may be part of the correspondence information). All or part of the second image corresponding to all or part of the first image determined by the image analysis unit 106 is acquired as all or part of the output image. Here, the plurality of pieces of correspondence information are a plurality of pieces of correspondence information of one lookup table of the storage unit 101, or a plurality of pieces of one or more lookup tables corresponding to one or more type identifiers. It is correspondence information.

  The output image acquisition unit 1075 determines that all or a part of the second image determined by the determination unit 1073 as satisfying the constraint condition, or the changed image acquired by the change unit 1074, or the determination unit 1073 satisfies the constraint condition. It is preferable to acquire all or part of the second image and the changed image acquired by the changing unit 1074 as all or part of the output image.

  The output image acquisition unit 1075 corresponds to one or more feature amounts acquired by the image analysis unit 106 in correspondence information included in one or more lookup tables corresponding to one or more type identifiers. By using one or more pieces of correspondence information, all or part of one or more second images corresponding to all or part of one or more images received by the image receiving unit 104 are acquired as all or part of the output image. Also good. The one or more type identifiers are the type identifiers acquired by the type identifier acquisition unit 1071.

  It is preferable that the output image acquisition unit 1075 perform the above-described correction process.

  The output image acquisition unit 1075 can be usually realized by an MPU, a memory, or the like. The processing procedure of the combining unit 108 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The synthesizing unit 108 converts the two or more second spatiotemporal blocks acquired by the mapping unit 107 into temporal or spatial of the first spatiotemporal blocks corresponding to the two or more second spatiotemporal blocks. Or it arrange | positions in a spatial and temporal position, and produces | generates an output image.

  The combining unit 108 can be usually realized by an MPU, a memory, or the like. The processing procedure of the combining unit 108 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The image output unit 109 outputs an output image. The image output unit 109 may output the output image generated by the synthesis unit 108. The output here is usually a display on a display or projection using a projector, but is a concept including transmission to an external device or another program.

  For example, the image output unit 109 may be considered as including or not including an output device such as a display or a projector. The image output unit 109 can be realized by, for example, output device driver software, or output device driver software and an output device.

  The lookup table update unit 110 changes any of the two or more pieces of correspondence information included in the lookup table according to the instruction received by the instruction reception unit 103. The lookup table update unit 110 may update all the correspondence information included in the lookup table, or may update a part of the correspondence information. When the correspondence information includes an arithmetic expression for generating all or part of the second image, or a parameter group for generating all or part of the second image, the look-up table update unit 110, for example, Accordingly, the parameter group included in the arithmetic expression or the parameter group for generating all or part of the second image is changed. For example, when the instruction includes the degree of stereoscopic effect of the image, the lookup table update unit 110 changes the parameter group included in the correspondence information using the degree of stereoscopic effect of the image input from the user. In such a case, the changing method (formula for changing, etc.) does not matter. For example, the look-up table update unit 110 stores a parameter group in a recording medium (not shown) in association with information on the degree of stereoscopic effect of the image, and corresponds to information on the degree of stereoscopic effect of the designated image. The parameter group is read from a recording medium (not shown). Further, for example, the lookup table update unit 110 holds an arithmetic expression for generating a parameter group using the degree of stereoscopic effect of the image as a parameter, and substitutes the degree of stereoscopic effect of the image into the arithmetic expression. The calculation is performed to obtain the parameter group.

  In addition, when the instruction receiving unit 103 receives an instruction to select one or more second image candidates, for example, the lookup table updating unit 110 selects one or more second image candidate from one or more second image candidates. A feature amount is acquired, and the one or more feature amounts are acquired as a parameter group. In addition, when the instruction receiving unit 103 receives an instruction to select one or more second image candidates, for example, the lookup table updating unit 110 selects one or more second image candidate from one or more second image candidates. A feature amount is acquired, and the one or more feature amounts are substituted into an arithmetic expression stored in advance, and calculation is performed to acquire a parameter group.

  The lookup table update unit 110 can be usually realized by an MPU, a memory, or the like. The processing procedure of the combining unit 108 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the image processing apparatus 1 will be described using the flowchart of FIG.

  (Step S201) The image receiving unit 104 determines whether one or more images have been received. If one or more images are accepted, the process proceeds to step S202. If one or more images are not accepted, the process proceeds to step S210. Note that the image here is usually a video composed of two or more still images.

  (Step S202) The dividing unit 105 divides the image received in step S201 into two or more spatially, temporally, or spatially and temporally, and acquires two or more first spatiotemporal blocks. .

  (Step S203) The image analysis unit 106 substitutes 1 for the counter i.

  (Step S204) The image analysis unit 106 determines whether or not the i-th block is present among the first spatiotemporal blocks acquired in step S202. If the i-th block exists, the process goes to step S205, and if the i-th block does not exist, the process goes to step S208.

  (Step S205) The image analysis unit 106 performs image analysis processing on the i-th first spatiotemporal block. Then, the image analysis unit 106 determines all or part of the first image corresponding to the i-th first spatiotemporal block. Such image analysis processing will be described with reference to the flowchart of FIG.

  (Step S206) The mapping unit 107 uses the correspondence information corresponding to all or part of the first image determined in Step S205, and uses the corresponding information corresponding to all or part of the first image determined in Step S205. All or part of the second image is acquired as all or part of the output image. Such processing is called mapping processing. The mapping process will be described with reference to the flowchart of FIG. Note that all or part of the output image acquired in this step is the second spatiotemporal block.

  (Step S207) The image analysis unit 106 increments the counter i by 1, and returns to step S204.

  (Step S208) The synthesizing unit 108 synthesizes the two or more second spatiotemporal blocks acquired in step S206 to generate an output image. More specifically, the two or more second spatiotemporal blocks acquired in step S206 are temporally or spatially of the first spatiotemporal blocks corresponding to the two or more second spatiotemporal blocks, Or it arrange | positions in a spatial and temporal position, and produces | generates an output image.

  (Step S209) The image output unit 109 outputs the output image generated in step S208.

  (Step S210) The instruction receiving unit 103 determines whether or not a user instruction has been received. If a user instruction is accepted, the process proceeds to step S211, and if a user instruction is not accepted, the process returns to step S201.

  (Step S211) The lookup table update unit 110 updates the lookup table according to the instruction received in step S210. Such a process is called a lookup table update process. The lookup table update process will be described with reference to the flowchart of FIG.

  In the flowchart of FIG. 2, the process is terminated by powering off or a process termination interrupt.

  In the flowchart of FIG. 2, while receiving a plurality of images and image streams, the instruction in step S210 may be received and the lookup table may be updated.

  Next, the image analysis processing in step S205 will be described using the flowchart of FIG.

  (Step S301) The image analysis unit 106 performs image analysis on the i-th first spatiotemporal block, and acquires a feature quantity vector (referred to as a first feature quantity vector) that is one or more feature quantities. . The one or more feature amounts are, for example, one or more feature amounts of the above-described neighborhood feature amount and / or the entire feature amount. For example, the one or more feature amounts are, for example, a maximum value (MAX) and a minimum value (MIN), a dynamic range (DR = MAX−MIN), a luminance frequency distribution, a difference value in the time direction, and the like of a plurality of pixels. is there. Further, since a technique for acquiring one or more feature quantities from an image is a known technique, detailed description thereof is omitted. Needless to say, the feature quantity is not limited to the above-described feature quantity.

  (Step S302) The image analysis unit 106 substitutes 1 for a counter i.

  (Step S <b> 303) The image analysis unit 106 determines whether or not the i-th correspondence information exists in the lookup table of the storage unit 101. If the i-th correspondence information exists, the process goes to step S304, and if the i-th correspondence information does not exist, the process goes to step S307. When there are two or more lookup tables in the storage unit 101, any one lookup table may be used in this step. Further, after performing a type identifier acquisition process described later, a lookup table corresponding to the type identifier acquired in the type identifier acquisition process may be used.

  (Step S304) The image analysis unit 106 acquires a feature amount vector (referred to as a second feature amount vector) included in the i-th correspondence information.

  (Step S305) The image analysis unit 106 calculates a distance between the first feature vector acquired in step S301 and the second feature vector acquired in step S304, and sets the distance to i (i-th correspondence information). Correspondingly, temporarily stored in the buffer.

  (Step S306) The image analysis unit 106 increments the counter i by 1, and returns to step S303.

  (Step S307) The image analysis unit 106 acquires correspondence information having the smallest distance accumulated in the buffer. Return to upper process. Note that the processing for acquiring the correspondence information is processing for determining all or part of the first image.

  In the flowchart of FIG. 3, the correspondence information holds an ID (may be an address). For example, one or more feature values are substituted into an arithmetic expression stored in advance, and the ID (may be an address). You may make it acquire. That is, the method of image analysis processing does not matter.

  Next, an example of the mapping process in step S206 will be described using the flowchart of FIG.

  (Step S401) The type identifier acquisition unit 1071 constituting the mapping unit 107 acquires a type identifier for identifying the type of the lookup table to be used. An example of the type identifier acquisition process will be described with reference to the flowchart of FIG.

  (Step S402) The output image candidate acquisition unit 1072 is correspondence information in the lookup table corresponding to the type identifier acquired in step S401, and is all or a part of the first image determined by the image analysis unit 106. The parameter group in the correspondence information corresponding to is acquired.

  (Step S403) The output image candidate acquisition unit 1072 uses all or part of the second image corresponding to all or part of the first image determined by the image analysis unit 106 using the parameter group acquired in step S402. Generate. More specifically, for example, the output image candidate acquisition unit 1072 executes the arithmetic expression by substituting the parameter group acquired in step S402 for the arithmetic expression stored in advance. Then, the output image candidate acquisition unit 1072 acquires all or a part of the second image.

  (Step S <b> 404) The determination unit 1073 reads the constraint condition from the constraint condition storage unit 102.

  (Step S405) The determination unit 1073 determines whether all or a part of the second image acquired in step S402 satisfies the constraint condition read in step S403, and substitutes the determination result into the buffer.

  (Step S406) If the determination result in step S404 is “restraint condition (match)”, go to step S407, and if “restraint condition is not met (do not match)”, go to step S408.

  (Step S407) The output image acquisition means 1075 obtains all or part of the second image acquired in step S402 as all or part of the output image. Return to upper process.

  (Step S408) The changing unit 1074 changes all or part of the second image acquired in step S402 so as to satisfy the constraint condition, and acquires a changed image.

  (Step S409) The output image acquisition unit 1075 obtains the changed image acquired in step S408 as all or part of the output image. Return to upper process.

  In the flowchart of FIG. 4, another method may be used as a method for generating all or part of the second image or a method for acquiring all or part of the second image.

  Further, in the flowchart of FIG. 4, when there is one lookup table in the storage unit 101, it goes without saying that the processing in step S401 is not necessary.

  Next, an example of the type identifier acquisition process in step S401 will be described using the flowchart in FIG.

  (Step S501) The type identifier acquisition unit 1071 acquires the feature quantity vector of the i-th block (first image).

  (Step S502) The type identifier acquisition unit 1071 assigns 1 to the counter i.

  (Step S503) The type identifier acquisition unit 1071 determines whether or not the i-th type identifier exists in the storage unit 101. If the i-th type identifier exists, the process goes to step S504. If the i-th type identifier does not exist, the process goes to step S507.

  (Step S504) The type identifier acquisition unit 1071 acquires a feature vector corresponding to the i-th type identifier.

  (Step S505) The type identifier acquisition unit 1071 calculates the distance between the feature quantity vector acquired in step S501 and the feature quantity vector acquired in step S504, and temporarily stores it in the buffer in association with the i th type identifier. To do.

  (Step S506) The type identifier acquisition unit 1071 increments the counter i by 1, and returns to step S503.

  (Step S507) The type identifier acquisition unit 1071 acquires a type identifier corresponding to the smallest distance calculated in step S505. Return to upper process.

  In the flowchart of FIG. 5, the type identifier acquisition process may be another method as described above. That is, in the flowchart of FIG. 5, the type identifier is determined using the block feature quantity vector (one or more neighboring feature quantities), but as described above, the type identifier is determined using the entire feature quantity or the neighboring feature quantity. Alternatively, the type identifier may be determined using information on the electronic program guide or the like, or the type identifier may be determined according to an instruction from the user.

  Next, the lookup table update process in step S211 will be described with reference to the flowchart of FIG.

  (Step S601) The lookup table update unit 110 acquires the instruction received by the instruction reception unit 103.

  (Step S602) The lookup table update unit 110 acquires one or more parameters included in the instruction acquired in step S601.

  (Step S603) The look-up table update unit 110 substitutes 1 for a counter i.

  (Step S604) The lookup table update unit 110 determines whether or not the i-th correspondence information exists in the lookup table of the storage unit 101. If the i-th correspondence information exists, the process goes to step S605, and if the i-th correspondence information does not exist, the process returns to the higher-level process.

  (Step S605) The lookup table updating unit 110 changes the parameter group included in the i-th correspondence information or the arithmetic expression included in the i-th correspondence information using the one or more parameters acquired in Step S602.

  (Step S606) The look-up table update unit 110 increments the counter i by 1, and returns to step S604.

  In the flowchart of FIG. 6, when there are a plurality of lookup tables, all the lookup tables may be updated.

  Next, a specific example of processing for dynamically changing an image or the like using constraint conditions will be described. The constraint condition storage unit 102 in the image processing apparatus 1, the determination unit 1073 of the mapping unit 107, and the change unit 1074 perform processing so as to satisfy the constraint condition, and can output an image in which the brain load is adjusted according to the purpose. .

  The first specific example is a process of correcting when the pixel value of the output image exceeds a certain value so that it falls within that value. In such a case, the constraint condition is “one or more pixel values of the output image do not exceed the threshold”. Then, the changing unit 1074 corrects the threshold value so that it does not exceed the threshold for one or more pixels that the determining unit 1073 determines to not meet the constraint condition. Then, the output image acquisition unit 1075 determines that all or a part of the second image determined by the determination unit 1073 as satisfying the constraint condition, or the changed image acquired by the change unit 1074, or the determination unit 1073 satisfies the constraint condition. An output image is acquired using all or a part of the second image and the changed image acquired by the changing unit 1074.

  The second specific example is a process of correcting the filling of the pixel of the input image using the peripheral pixels when the pixel value of the input image is out of the normal range. In such a case, the constraint condition is “the pixel value of the input image does not deviate from the normal range”. Then, for one or more pixels determined by the determining unit 1073 not to meet the constraint condition, the changing unit 1074 acquires one or more pixel values of peripheral pixels in a predetermined range of the pixel, The pixel value of the abnormal pixel is determined using the one or more pixel values. Then, the output image acquisition unit 1075 uses one or more feature amounts acquired by the changing unit 1074 or one or more feature amounts of a part of the image received by the image receiving unit 104 to output one or more second images. Get a part of.

  In the third specific example, when an output image is acquired using correspondence information, if one piece of correspondence information in the lookup table is used immediately before, the next processing is to use other correspondence information. It is. In such a case, the constraint condition is “do not use the same correspondence information continuously”. Then, the determination unit 1073 determines whether correspondence information corresponding to all or a part of the first image determined by the image analysis unit 106 has been used immediately before. In this case, at least information for identifying the correspondence information used immediately before is temporarily stored. When the determination unit 1073 determines that the correspondence information corresponding to all or a part of the first image determined by the image analysis unit 106 is the same as the correspondence information used immediately before, the changing unit 1074 includes 2 Get the closest correspondence information. That is, the changing unit 1074 removes all or a part of the determined first image from the image analyzing unit 106 and again instructs the image analyzing unit 106 to determine all or a part of the first image. The analysis unit 106 again determines all or part of the first image. The correspondence information corresponding to all or part of the first image is correspondence information corresponding to all or part of the first image closest to the second input image, the spatiotemporal block, or the like. Then, the output image acquisition unit 1075 acquires a part of one or more second images using the correspondence information acquired by the changing unit 1074.

  The fourth specific example is an example in which the parameter included in the constraint condition and the change destination are different. The fourth specific example is a case where, for example, when the pixel value of the output image exceeds a threshold value, processing is performed again using specific correspondence information. That is, the constraint condition is “one or more pixel values of the output image do not exceed the threshold”. Then, the changing unit 1074 reads predetermined correspondence information for one or more pixels determined by the determining unit 1073 not to meet the constraint condition. Then, using the correspondence information, the output image acquisition unit 1075 forms an output image again.

  The fifth specific example is an example in which a plurality of parameters are used for determination. In the fifth specific example, for example, the constraint condition is “the difference between the pixel value of the input image and the pixel value of the output image is within a threshold”. Then, for one or more pixels that the determination unit 1073 has determined not to meet the constraint condition, the change unit 1074 corrects the pixel value of the pixel of the output image, and the pixel value of the pixel of the input image The difference between is within the threshold. Then, the output image acquisition unit 1075 acquires an output image having pixels with such pixel values.

  As described above, according to the present embodiment, it is possible to provide an image processing apparatus that outputs an image in which the brain load is adjusted according to the purpose. For example, according to the present embodiment, it is possible to provide an image processing device that outputs an image that can further reduce the brain load, an image processing device that outputs an image that gives more stimulation to the brain and increases entertainment properties, and the like.

  When the image processing apparatus according to the present embodiment outputs an image that can further reduce the brain load, the image processing apparatus can output an image that can be recognized as a suitable image for the human brain.

  More specifically, according to the present embodiment, an image captured in different environments such as light quantity, light source position, light source color, and camera position can be output. A recognizable image can be output. More specifically, according to the present embodiment, the brain has a texture, perspective, presence, presence, weight, warmth, coolness, speed, dynamics, smoothness of movement, freshness Images that make you feel better and feel more luxurious can be output.

  Further, according to the present embodiment, by using an appropriate lookup table among a plurality of lookup tables, an image in which the brain load is adjusted according to the purpose can be output.

  Further, according to the present embodiment, by changing the output image so as to satisfy the constraint condition, it is possible to output an image in which the brain load is more suitably adjusted according to the purpose.

  Further, according to the present embodiment, by making the look-up table customizable, it is possible to output an image in which the brain load is more suitably adjusted according to the user's preference or the like.

  Note that the image processing apparatus 1 in the present embodiment may be a television receiver. In such a case, the image received by the image receiving unit 104 is an image constituting a television video.

  Moreover, if the image processing apparatus 1 in this Embodiment is used, the apparatus which produces | generates and outputs the moving image by which the brain load was adjusted in real time according to the objective using the received moving image is realizable.

  In the present embodiment, when the lookup table has only one piece of correspondence information, the image processing apparatus 1 may not have the image analysis unit 106. In such a case, the image processing apparatus 1 is configured as shown in FIG. In such a case, the mapping unit 107 uses one piece of correspondence information of the storage unit 101, or all or one or more second images corresponding to all or part of one or more images received by the image receiving unit 104. Get a portion. Even in such a case, the image analysis unit 106 exists, and the image analysis unit 106 acquires one or more feature amounts from all or a part of the received one or more images, and the mapping unit 107 uses the feature amounts. The above-described correction process may be performed.

  Needless to say, the image processing apparatus 1 according to the present embodiment may not include some components such as the instruction receiving unit 103 and the lookup table update unit 110.

  Furthermore, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. The software that realizes the image processing apparatus according to the present embodiment is the following program. That is, the program is an image in which the brain load is adjusted with respect to all or a part of one or more first images and the first image on the storage medium, and is the same as the first image. A lookup table having one or more correspondence information that is information for taking correspondence with all or part of a second image that is an image of the object, and storing the computer with one or more images One or more second images corresponding to all or a part of the one or more images received by the image receiving unit using the image receiving unit to receive and the corresponding information of one or more of the one or more corresponding information. A mapping unit that acquires all or part of an image, and a set of all of one or more second images acquired by the mapping unit, or a part of one or more second images acquired by the mapping unit Program for functioning as an image output unit for outputting a certain one or more output image is.

  In the above program, the look-up table has two or more pieces of correspondence information, and the computer accepts all or part of the first image corresponding to all or part of one or more images received by the image receiving unit. And the mapping unit uses the correspondence information corresponding to all or a part of the first image determined by the image analysis unit to determine the first image determined by the image analysis unit. A program for causing a computer to function as the second image corresponding to all or part of the image is preferable.

  In the above program, all or a part of the first image is a first spatiotemporal block that is a part of an input video, and all or a part of the second image is displayed on a screen. A second spatio-temporal block that is a part of the output video, wherein the image receiving unit receives a video that is a set of one or more images, and sends a computer to the video received by the image receiving unit; , Spatially, temporally, or spatially and temporally divided into two or more, and further functioning as a dividing unit that obtains two or more first spatiotemporal blocks, and the image analyzing unit includes: For each of the two or more acquired first spatio-temporal blocks, the whole or a part of the first image corresponding to each first spatio-temporal block is determined, and the mapping unit includes the two or more first spatio-temporal blocks. Space-time block A second time space corresponding to all or part of the first image determined by the image analysis unit using correspondence information corresponding to all or part of the first image determined by the image analysis unit. The block is acquired, and the two or more second spatiotemporal blocks acquired by the mapping unit are temporally or spatially of the first spatiotemporal block corresponding to the two or more second spatiotemporal blocks. Alternatively, it is preferable to arrange at a spatial and temporal position and further function as a synthesis unit that generates an output image.

  In the above program, the image analysis unit analyzes one or more images received by the image receiving unit or two or more first spatiotemporal blocks acquired by the dividing unit, and one or more feature amounts. And a program for causing a computer to function as determining all or a part of the first image corresponding to the one or more feature quantities.

  In the above program, the storage medium stores two or more sets of a type identifier for identifying a type of the lookup table and a lookup table corresponding to the type identifier, and the mapping unit uses Of the type identifier acquisition means for acquiring one or more type identifiers for identifying the type of the lookup table to be performed, and the image analysis unit of correspondence information included in the one or more lookup tables corresponding to the one or more type identifiers Using the correspondence information corresponding to all or a part of the first image determined by the image analysis unit, all or a part of the second image corresponding to all or a part of the first image determined by the image analysis unit is output as an output image. It is preferable to cause the computer to function as output image acquisition means for acquiring all or part of the output image acquisition means.

  Further, in the above program, the computer further functions as an instruction receiving unit that receives a user instruction, and a lookup table update unit that changes any one of two or more pieces of correspondence information included in the lookup table according to the instruction. It is suitable to make it.

  In addition, software that realizes the image processing apparatus according to the present embodiment stores all or a part of one or more first images and all or one of the second images different from the first image on a storage medium. A look-up table having one or more correspondence information, which is information for taking correspondence with a part, and a constraint condition that is a condition to be satisfied by one or more types of information of the input image, the correspondence information, or the output image One or more images received by the image receiving unit using the image receiving unit that receives one or more images and any one of the one or more pieces of corresponding information. A mapping unit that acquires all or a part of one or more second images corresponding to all or a part of the image so as to meet the constraint condition, and one or more second images acquired by the mapping unit. In whole or programmed to function as an image output unit for outputting an output image composed of a portion of the.

(Embodiment 2)
In the present embodiment, compared to the first embodiment, the image processing apparatus 2 does not include the dividing unit 105 and the combining unit 108.

  FIG. 8 is a block diagram of the image processing apparatus 2 in the present embodiment.

  The image processing apparatus 2 includes a storage unit 101, a constraint condition storage unit 102, an instruction reception unit 103, an image reception unit 104, an image analysis unit 106, a mapping unit 107, an image output unit 109, and a lookup table update unit 110. .

  Next, the operation of the image processing apparatus 2 will be described using the flowchart of FIG. The flowchart of FIG. 9 is a part of the flowchart of FIG. However, the image analysis processing in step S205 is performed on the entire image accepted in step S201. The mapping process in step S206 is also performed on the entire image received in step S201.

  Further, in the flowchart of FIG. 9, the process is ended by powering off or a process end interrupt.

  In the flowchart of FIG. 9, the instruction in step S210 may be received and the lookup table may be updated while a plurality of images and image streams are being received.

  As described above, according to the present embodiment, it is possible to provide an image processing apparatus that outputs an image in which the brain load is adjusted according to the purpose. For example, according to the present embodiment, it is possible to provide an image processing device that outputs an image that can further reduce the brain load, an image processing device that outputs an image that gives more stimulation to the brain and increases entertainment properties, and the like.

  When the image processing apparatus according to the present embodiment outputs an image that can further reduce the brain load, the image processing apparatus outputs an image that can be recognized as a suitable image for the human brain. Can be provided.

  More specifically, according to the present embodiment, it is possible for the human brain to output an image in which the environment for acquiring the image, such as the light amount, the position of the light source, the color of the light source, and the position of the camera, is improved. An image that can be recognized as a suitable image can be output. More specifically, according to the present embodiment, the brain has a texture, perspective, presence, presence, weight, warmth, coolness, speed, dynamics, smoothness of movement, freshness Images that make you feel better and feel more luxurious can be output.

  Note that a television receiver incorporating the image processing apparatus 2 in the present embodiment can be configured. In such a case, the image received by the image receiving unit 104 is an image constituting a television video.

  In the present embodiment, when the lookup table has only one piece of correspondence information, the image processing apparatus 2 may not have the image analysis unit 106. In such a case, the image processing apparatus 1 is configured as shown in FIG. In such a case, the mapping unit 107 uses one piece of correspondence information of the storage unit 101, or all or one or more second images corresponding to all or part of one or more images received by the image receiving unit 104. Get a portion. Even in such a case, the image analysis unit 106 exists, and the image analysis unit 106 acquires one or more feature amounts from all or a part of the received one or more images, and the mapping unit 107 uses the feature amounts. The above-described correction process may be performed.

  Needless to say, the image processing apparatus 2 according to the present embodiment may not include some components such as the instruction receiving unit 103 and the lookup table update unit 110.

  Furthermore, the software that realizes the image processing apparatus according to the present embodiment is the following program. That is, the program is an image in which the brain load is adjusted with respect to all or a part of one or more first images and the first image on the storage medium, and is the same as the first image. A lookup table having one or more correspondence information that is information for taking correspondence with all or part of a second image that is an image of the object, and storing the computer with one or more images One or more second images corresponding to all or a part of the one or more images received by the image receiving unit using the image receiving unit to receive and the corresponding information of one or more of the one or more corresponding information. A mapping unit that acquires all or part of an image, and a set of all of one or more second images acquired by the mapping unit, or a part of one or more second images acquired by the mapping unit Program for functioning as an image output unit for outputting a certain one or more output image is.

(Embodiment 3)
In the present embodiment, a lookup table acquisition device 3 that generates a lookup table will be described. The lookup table acquisition device 3 is a device for learning a lookup table used to convert all or part of the first image referred to in this specification into all or part of the second image.

  FIG. 11 is a block diagram of lookup table acquisition apparatus 3 in the present embodiment.

  The lookup table acquisition device 3 includes a storage unit 101, a first image storage unit 31, a second image storage unit 32, a second image information candidate set storage unit 33, an instruction receiving unit 103, a candidate output unit 34, and a second image accumulation. Unit 35, first image related information acquisition unit 36, second image related information acquisition unit 37, lookup table configuration unit 38, and lookup table storage unit 39.

  The instruction receiving unit 103 receives a user instruction. The instruction here is an instruction to output the second image information candidate set stored in the second image information candidate set storage unit 33, an instruction to select one or more second image information candidates, and the like. The user may be a person in charge of a company that provides the lookup table acquisition device 3, or a user (viewer) of the above-described image processing device or television receiver. The second image information candidate set and the second image information will be described later.

  The first image storage unit 31 can store one or more first images. The first image is an image assuming an image accepted by the image processing apparatus. The first image is, for example, an image to which a brain load is applied in the above-described meaning from the second image. In addition, the first image is an image that has not been improved in entertainment properties, for example.

  The first image storage unit 31 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The process in which the first image is stored in the first image storage unit 31 does not matter. The first image in the first image storage unit 31 may be generated from the second image in the second image storage unit 32, for example. In other words, for example, a means (not shown) may generate the first image by changing the properties of the second image by reducing the brightness of the second image or performing a simulation to automatically change the camera position. good.

  The second image storage unit 32 can store one or more second images. The second image is an image in which the brain load is adjusted according to the purpose with respect to the first image. For example, the second image is an image in which the brain load can be reduced as compared with the first image. In addition, for example, the second image is an image in which entertainment properties are increased compared to the first image. Furthermore, for example, the second image is an image acquired by photographing one object in another environment.

  The second image and the first image are images for the same object. The second image and the first image usually have the same number of pixels, but may have different numbers of pixels.

  The second image storage unit 32 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The process in which the second image is stored in the second image storage unit 32 does not matter. The second image in the second image storage unit 32 may be generated from the first image in the first image storage unit 31, for example. That is, for example, a means (not shown) may generate a second image by changing the properties of the first image by increasing the brightness of the first image or performing a simulation that automatically changes the camera position. good.

  The second image information candidate set storage unit 33 can store one or more second image information candidate sets in association with one or more first images. The second image information candidate set is a set of two or more second image information candidates. The second image information is information related to the second image. The second image information may be a second image, or one or more attributes of the second image. The attribute of the image is, for example, a shooting parameter or one or more feature amounts. The two or more second image information candidates are image information of the second image for the same target as the associated first image. The second image information candidate set storage unit 33 may physically include the first image storage unit 31. Further, “corresponding to the first image” may be directly associated with the first image, or may be indirectly associated with the first image. . Indirect association is associated with, for example, first image information. The first image information is information relating to the first image, such as the first image itself or the attributes of the first image.

  The second image information candidate set storage unit 33 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The process of storing the second image information candidate set in the second image information candidate set storage unit 33 does not matter.

  The candidate output unit 34 outputs one or more second image information candidate sets stored in the second image information candidate set storage unit 33. When the instruction receiving unit 103 receives an instruction to output the second image information candidate set, the candidate output unit 34 stores one or more second image information candidate sets stored in the second image information candidate set storage unit 33. May be output. The output here is usually a display on a display or projection using a projector, but is a concept including transmission to an external device or another program.

  The second image storage unit 35 corresponds one or more second images corresponding to one or more second image information candidates corresponding to the instruction received by the instruction receiving unit 103 to the second image information candidate set. The second image storage unit 32 stores the first image in association with the first image. For example, when the second image information is the second image, the second image storage unit 35 corresponds one or more second images corresponding to the instruction received by the instruction receiving unit 103 to the first image. In addition, the image is accumulated in the second image storage unit 32. Further, for example, when the second image information is an attribute of the image, the second image storage unit 35 configures the second image using the attribute of the image, associates it with the first image, Accumulate in the second image storage unit 32. The second image storage unit 35 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second image storage unit 35 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first image related information acquisition unit 36 acquires one or more first image related information from one or more first images in the first image storage unit 31. The first image related information is information relating to all or part of the first image. The first image related information is, for example, one or more feature amounts extracted from all or a part of the first image. The one or more feature amounts include a neighborhood feature amount that is a partial feature amount of the one or more first images and an overall feature amount that is a feature amount of the one or more first images as a whole. Since the example of the neighborhood feature amount and the overall feature amount has been described above, the description thereof will be omitted. Further, the first image related information may be information obtained by degenerating one or more feature amounts described above. Such degenerated information is also a feature amount.

  The first image related information acquisition unit 36 can usually be realized by an MPU, a memory, or the like. The processing procedure of the first image related information acquisition unit 36 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second image related information acquisition unit 37 acquires one or more second image related information from one or more second images in the second image storage unit 32. The second image related information is information regarding all or part of the second image. The second image related information is information for generating all or part of the second image, for example, a parameter group given to an arithmetic expression for generating all or part of the second image, It is an arithmetic expression for generating all or part of an image. The second image related information may be, for example, all or part of the second image itself. For example, when the second image related information is all or a part of the second image itself, the second image related information acquisition unit 37 performs a process of dividing the second image into spatiotemporal blocks.

  The second image related information acquisition unit 37 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second image related information acquisition unit 37 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The lookup table configuration unit 38 configures a lookup table having one or more correspondence information. As described above, the correspondence information is information for taking correspondence between all or a part of one or more first images and all or a part of one or more second images. Further, for example, the correspondence information is information that associates the first image related information with the second image related information. The correspondence information is information having first image related information and second image related information, for example. In addition, the content and structure of correspondence information are not ask | required. Here, with respect to the second image related information corresponding to the first image related information, all or a part of the first image that is the source of the first image related information and the second that is the source of the second image related information. When all or a part of the image corresponds to the spatial position on the frame image constituting the video, or corresponds to the temporal position on the video, the spatial and temporal position on the video corresponds There is a case.

  The lookup table configuration unit 38 can be usually realized by an MPU, a memory, or the like. The processing procedure of the second image related information acquisition unit 37 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The lookup table storage unit 39 stores the lookup table configured by the lookup table configuration unit 38 in the storage unit 101. Note that the accumulation here includes writing to a memory and the like.

  The lookup table storage unit 39 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second image related information acquisition unit 37 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation in which the user selects a second image suitable for himself / herself in the lookup table acquisition device 3 will be described with reference to the flowchart of FIG.

  (Step S1201) The instruction receiving unit 103 determines whether an instruction to output the second image information candidate set has been received. If such an instruction is accepted, the process goes to step S1202, and if no such instruction is accepted, the process returns to step S1201.

  (Step S1202) The candidate output unit 34 substitutes 1 for the counter i.

  (Step S1203) The candidate output unit 34 determines whether or not the i-th second image information candidate set exists in the second image information candidate set storage unit 33. If the i-th second image information candidate set exists, the process goes to step S1204. If the i-th second image information candidate set does not exist, the process ends.

  (Step S1204) The candidate output unit 34 reads the i-th second image information candidate set from the second image information candidate set storage unit 33 and outputs it.

  (Step S1205) The instruction receiving unit 103 determines whether a selection instruction for one or more second image information is received from two or more second image information candidates included in the second image information candidate set. To do. If such a selection instruction is accepted, the process goes to step S1206. If such a selection instruction is not accepted, the process returns to step S1205.

  (Step S1206) The second image storage unit 35 acquires one or more second image information candidates corresponding to the instruction received in step S1205.

  (Step S1207) The second image storage unit 35 acquires the second image using the second image information acquired in step S1206, associates it with the first image, and stores it in the second image storage unit 32. accumulate. This first image is the first image associated with the i-th second image information candidate set.

  (Step S1208) The candidate output unit 34 increments the counter i by one. The process returns to step S1203.

  Next, an operation in which the lookup table acquisition device 3 configures and stores a lookup table will be described with reference to the flowchart of FIG. It is assumed that a second image corresponding to each first image stored in the first image storage unit 31 is stored in the second image storage unit 32.

  (Step S1301) The first image related information acquisition unit 36 substitutes 1 for a counter i.

  (Step S1302) The first image related information acquisition unit 36 determines whether or not the i-th first image is stored in the first image storage unit 31. If the i-th first image is stored, the process goes to step S1303, and if not stored, the process ends.

  (Step S1303) The first image related information acquisition unit 36 reads the i-th first image from the first image storage unit 31.

  (Step S1304) The first image related information acquisition unit 36 divides the i-th first image read in step S1303 into blocks. Then, the first image related information acquisition unit 36 obtains n first spatiotemporal blocks.

  (Step S 1305) The second image related information acquisition unit 37 reads the i-th second image from the second image storage unit 32.

  (Step S1306) The second image related information acquisition unit 37 divides the i-th second image read in step S1305 into blocks. Then, the second image related information acquisition unit 37 obtains n second spatiotemporal blocks.

  (Step S1307) The lookup table configuration unit 38 substitutes 1 for the counter j.

  (Step S1308) The lookup table configuration unit 38 determines whether or not the j-th block exists in the spatiotemporal block acquired in step S1304. If the jth block exists, the process goes to step S1309, and if the jth block does not exist, the process goes to step S1314.

  (Step S 1309) The first image related information acquisition unit 36 acquires first image related information from the j th block of the i th first image.

  (Step S1310) The second image related information acquisition unit 37 acquires second image related information from the j th block of the i th second image.

  (Step S1311) The lookup table configuration unit 38 configures correspondence information including the first image related information acquired in Step S1309 and the second image related information acquired in Step S1310.

  (Step S1312) The lookup table storage unit 39 stores the correspondence information configured in step S1311 in the storage unit 101.

  (Step S1313) The lookup table configuration unit 38 increments the counter j by 1. The process returns to step S1308.

  (Step S1314) The first image related information acquisition unit 36 increments the counter i by one. The process returns to step S1302.

  Hereinafter, a specific operation of the lookup table acquisition apparatus 3 in the present embodiment will be described.

  First, an example of an operation of receiving a second image selection instruction from the user and accumulating the second image in the lookup table acquisition device 3 will be described.

  Now, a plurality of second image information candidate sets as shown in FIG. 14 are stored in the second image information candidate set storage unit 33. Here, in the second image information candidate set, it is assumed that the image of candidate 1 is the first image. In the second image information candidate set, candidates 1 to 4 are second image candidates. That is, here, the second image information is the second image itself.

  Then, the user inputs an instruction to output the second image information candidate set to the lookup table acquisition device 3. Then, the instruction receiving unit 103 receives an instruction to output the second image information candidate set.

  Next, the candidate output unit 34 reads and outputs the first second image information candidate set (the set of images of the record of “ID = 1” in FIG. 14). An output example of the second image information candidate set is shown in FIG.

  Next, the user inputs an instruction (for example, “4”) to select an image that matches his / her preference. Then, the instruction receiving unit 103 receives an instruction (for example, “4”).

  Next, the second image storage unit 35 acquires the fourth image corresponding to the instruction and stores it in the second image storage unit 32.

  The above processing is also performed on other second image information candidate sets, and the second image preferred by the user is accumulated in the second image storage unit 32.

  Next, a specific operation in which the lookup table acquisition device 3 configures and stores the lookup table will be described. Here, explanation will be given using seven specific examples.

(Specific example 1)
Specific example 1 is an example in which the same subject is photographed using light sources having different light amounts, and the acquired first image and second image are used.

  First, a light source with a light amount A is applied to one subject, and the first image is acquired by an imaging device (not shown) and accumulated in the first image storage unit 31. Further, a light source having a light amount B (light amount B> light amount A) is applied to the one subject, the image is taken with a photographing device (not shown), a second image is acquired, and stored in the second image storage unit 32. In such a case, the second image is an image that can reduce the brain load more than the first image. In addition, it is preferable that the second image has a higher quality than the first image when viewed by a person. A high-quality image is an image that expresses the texture, perspective, airiness, warmth, transparency, freshness, luxury, etc. of the subject.

  In addition, it is preferable that the subject to be photographed by changing the light amount of the light source is of many types such as nature, human beings, animals and still life. Needless to say, the degree of change in the light amount of the light source may vary depending on the subject.

  As described above, the first image storage unit 31 and the second image storage unit 32 are constructed.

  Next, the first image related information acquisition unit 36 sequentially reads the first image from the first image storage unit 31. Then, the first image related information acquisition unit 36 divides the first image into blocks and obtains n first spatiotemporal blocks. The plurality of first spatiotemporal blocks may have overlapping areas. That is, the division process performed by the first image related information acquisition unit 36 is the same process as the division process performed by the division unit 105.

  The second image related information acquisition unit 37 also reads out the second image corresponding to the first image from the second image storage unit 32. Then, the second image related information acquisition unit 37 divides the second image into blocks and obtains n second spatiotemporal blocks.

  Next, the first image related information acquisition unit 36 determines the maximum value and minimum value of a plurality of pixels, the dynamic range, the luminance frequency distribution, and the time direction difference value (for example, the previous field) from each block of the first image. And the like (difference value with the corresponding pixel). Such information is a feature vector and is first image related information. The first image related information is acquired for each block.

  Next, the second image related information acquisition unit 37 acquires one or more parameters for generating a block from each block of the second image. Specifically, the second image related information acquisition unit 37 acquires, for example, a feature vector of each block of the second image, substitutes the feature vector into an arithmetic expression, and acquires one or more parameters. One or more parameters are examples of the second image related information.

  Then, the lookup table configuration unit 38 configures correspondence information including first image related information and second image related information for each block. Then, the lookup table accumulation unit 39 accumulates the configured correspondence information in the storage unit 101.

  With the above processing, a lookup table having a lot of correspondence information is generated. Note that the same or similar first image related information may be collectively set as one correspondence information.

(Specific example 2)
Specific example 2 is an example in which the same subject is photographed using light sources having different colors, and the acquired first image and second image are used.

  First, a light source of color X (for example, white) is applied to one subject, the image is taken with a photographing apparatus (not shown), a first image is acquired, and stored in the first image storage unit 31. Further, a light source of color Y (for example, orange) is applied to the one subject, the image is taken with a photographing device (not shown), a second image is acquired, and stored in the second image storage unit 32. In such a case, it is necessary for a person to see that the second image is higher quality than the first image.

  In addition, it is preferable that the subject to be photographed by changing the color of the light source is of many types such as nature, human beings, animals and still life. Needless to say, the degree of change in the color of the light source may vary depending on the subject.

  As described above, the first image storage unit 31 and the second image storage unit 32 are constructed. Further, the processing after the first image storage unit 31 and the second image storage unit 32 are constructed is the same as that in the first specific example, and thus the description thereof is omitted.

(Specific example 3)
Specific Example 3 is an example in which the same first subject and second image are used by photographing the same subject using light sources arranged at different positions.

  First, a light source installed at a position α is applied to one subject, the image is taken with a photographing device (not shown), a first image is acquired, and stored in the first image storage unit 31. Further, the second subject is shot with a light source installed at a position β (a position where a shadow of the subject is generated more than when a light source is installed at the position α), and the second image is captured. Are stored in the second image storage unit 32. In such a case, it is necessary for the second image to have more shadow area of the subject than the first image, and to make the subject feel more stereoscopic when viewed by a person.

  In addition, it is preferable that the subject to be photographed by changing the position of the light source is of many types, such as nature, human beings, animals, and still life. It goes without saying that the degree of change in the position of the light source may vary depending on the subject.

  As described above, the first image storage unit 31 and the second image storage unit 32 are constructed. Further, the processing after the first image storage unit 31 and the second image storage unit 32 are constructed is the same as that in the first specific example, and thus the description thereof is omitted.

(Specific example 4)
Specific Example 4 is an example in which the same first subject and second image are used by photographing the same subject using light sources arranged at different positions with different amounts of light.

  First, a light source arranged at a position α is applied to one subject, a light source with a light amount A is applied, and the image is taken with a photographing device (not shown), and a first image is acquired and accumulated in the first image storage unit 31. In addition, a light source disposed at a position β is applied to the one subject, a light source having a light amount B (light amount B> light amount A) is applied, and the second image is obtained by capturing with a photographing device (not shown). Accumulate in the image storage unit 32. In such a case, it is preferable that the second image is higher in quality than the first image when viewed by a person.

  In addition, it is preferable that the subject to be photographed by changing the light amount of the light source is of many types such as nature, human beings, animals and still life.

  As described above, the first image storage unit 31 and the second image storage unit 32 are constructed. Further, the processing after the first image storage unit 31 and the second image storage unit 32 are constructed is the same as that in the first specific example, and thus the description thereof is omitted.

(Specific example 5)
Specific example 5 is an example in which the same subject is photographed in an environment where one or more of weather, temperature, and humidity are different, and the acquired first image and second image are used.

  First, in an environment of weather “cloudy”, temperature “25 ° C.”, and humidity “40%”, a light source is applied to one subject, the image is taken with a photographing device (not shown), a first image is acquired, and the first image is stored. Accumulate in part 31. Further, in an environment of weather “cloudy”, temperature “22 ° C.”, and humidity “70%”, a light source is applied to the one subject, the image is taken with a photographing device (not shown), a second image is obtained, and a second image is obtained. Accumulate in the storage unit 32. In such a case, the second image can usually feel a sense of perspective and moistness when viewed by a person.

  As described above, the first image storage unit 31 and the second image storage unit 32 are constructed. Further, the processing after the first image storage unit 31 and the second image storage unit 32 are constructed is the same as that in the first specific example, and thus the description thereof is omitted.

  Similarly, first, in an environment of weather “cloudy”, temperature “25 ° C.”, and humidity “40%”, a light source is applied to one subject, the image is taken with a photographing device (not shown), and a first image is obtained. Accumulate in the first image storage unit 31. Further, in an environment of weather “sunny”, temperature “28 ° C.”, and humidity “20%”, a light source is applied to the one subject, the image is taken with a photographing device (not shown), a second image is obtained, and a second image is obtained. Accumulate in the storage unit 32. In such a case, the second image can usually feel a sense of air, warmth, and the like when viewed by a person.

  As described above, the first image storage unit 31 and the second image storage unit 32 are constructed. Further, the processing after the first image storage unit 31 and the second image storage unit 32 are constructed is the same as that in the first specific example, and thus the description thereof is omitted.

(Specific example 6)
The specific example 6 is an example in which the first image and the second image acquired by a photographing apparatus capable of photographing images with different depths of field are used.

  First, with a photographing device that captures an image with a shallower depth of field, a light source is applied to one subject, a first image is acquired, and stored in the first image storage unit 31. In addition, a photographing apparatus that captures an image with a deeper depth of field, a light source is applied to the one subject, a photograph is taken by a photographing apparatus (not shown), a second image is acquired, and stored in the second image storage unit 32. To do. In such a case, the person sees and feels that the second image is generally more refreshed because the overall focus is more general. In addition, the second image has more sharpness due to depth, and usually provides a sense of depth and a three-dimensional effect.

  As described above, the first image storage unit 31 and the second image storage unit 32 are constructed. Further, the processing after the first image storage unit 31 and the second image storage unit 32 are constructed is the same as that in the first specific example, and thus the description thereof is omitted.

(Specific example 7)
The specific example 7 is an example in which the first image and the second image acquired by photographing apparatuses having different shutter speeds are used.

  A light source is applied to one subject, and a first image is acquired and stored in the first image storage unit 31 with a photographing device having a slower shutter speed (shutter speed = A). Further, a light source is applied to the one subject, and a second image is obtained by photographing with a photographing device (not shown) with a photographing device having a faster shutter speed (shutter speed = B, A <B), and storing a second image. Accumulate in part 32. In such a case, when viewed by a person, the second image usually feels as if the sense of speed and dynamic feeling have increased.

  As described above, the first image storage unit 31 and the second image storage unit 32 are constructed. Further, the processing after the first image storage unit 31 and the second image storage unit 32 are constructed is the same as that in the first specific example, and thus the description thereof is omitted.

  As described above, according to the present embodiment, it is possible to automatically acquire a lookup table that can be used to generate an image in which the brain load is adjusted according to the purpose.

  In addition, according to the present embodiment, if a user who is a viewer of an image creates a lookup table using the lookup table acquisition device 3, an image in which the brain load is adjusted according to the purpose for himself / herself. Can be obtained.

  In addition, according to the present embodiment, if one or more users select the second image for each user attribute (up to 30 years old, 60 years old or older, female, male, high school student, etc.), A lookup table may be created for each of the attributes. Then, an image processing apparatus (for example, a television receiver) using the lookup table created as described above can output an image in which the brain load is adjusted so as to match the user's preference for each user attribute. .

  In addition, the lookup table acquisition device 3 of the present embodiment includes some components such as the second image information candidate set storage unit 33, the instruction reception unit 103, the candidate output unit 34, and the second image storage unit 35. Needless to say, you don't have to.

  Furthermore, it is preferable that the look-up table acquired using the look-up table acquiring apparatus 3 in the present embodiment is copied and used in the image processing apparatus described in the first and second embodiments.

  Note that the software that implements the lookup table acquisition apparatus according to the present embodiment is the following program. In other words, this program is one or more first images on the storage medium and an image in which the brain load is adjusted with respect to the first images, and the same target as each of the one or more first images. One or more second images that are images of the image are stored in association with the one or more first images, and the computer is stored in the storage medium. A first image related information acquisition unit that acquires one or more first image related information, which is information about all or a part of an image, from all or a part of the one or more first images; and is stored in the storage medium A second image related information acquisition unit that acquires one or more second image related information, which is information related to all or a part of one or more second images, from all or a part of the one or more second images; Acquired by the first image related information acquisition unit. The first image related information and the information corresponding to the first image related information, and the second image related information acquired by the second image related information acquisition unit, and the one or more first images A look-up table constituting unit constituting a look-up table having one or more correspondence information, which is information indicating correspondence between all or part of an image and all or part of the one or more second images, and the look A program for causing a lookup table configured by an uptable configuration unit to function as a lookup table storage unit that stores in a storage medium.

In the above program, the storage medium is a set of two or more second image candidates for the same target as the one or more first images in association with the one or more first images. One or more second image information candidate sets are further stored, and a computer outputs a candidate output unit that outputs the one or more second image information candidate sets, and one or more second output information output by the candidate output unit. An instruction receiving unit that receives an instruction to select one or more second image candidates from two or more second image candidates included in the two-image information candidate set, and 1 corresponding to the instruction received by the instruction receiving unit A program that further functions as a second image storage unit that stores the second image candidates in the second image storage unit in association with the first image corresponding to the second image information candidate set. Is preferred.
(Embodiment 4)
In the present embodiment, a specific example of processing for generating a lookup table using the lookup table acquisition device 3 will be further described.
This specific example is a case where the correspondence information constituting the lookup table is a feature vector and an image conversion filter. The feature amount vector is information extracted from the input image (first image). The image conversion filter is, for example, a set of one or more filter coefficients.
Now, the first image storage unit 31 stores a large number of images including a first image having a shallow depth of field as shown in FIG. The second image storage unit 32 stores a large number of images including a second image having a deep depth of field as shown in FIG. A certain first image and a certain second image are associated with each other. That is, for example, the first image in FIG. 16 and the second image in FIG. 17 are associated with each other.
Also, the second image has a greater depth of field than the first image. Also, the depth of field in FIG. 16 is shallow, and the red car in front of the image is in focus. On the other hand, the depth of field in FIG. 17 is deep, and the yellow car at the back of the image is also in focus. In general, it can be said that the second image with a deep depth of field is easier to understand the entire image and is a preferable image with less brain load than the first image.
In such a situation, it is assumed that an instruction to construct a lookup table is input from the user. Next, the first image related information acquisition unit 36 reads the first image (here, for example, FIG. 16) from the first image storage unit 31. Then, the first image related information acquisition unit 36 divides the first image and acquires a plurality of spatiotemporal blocks. Here, as shown in FIG. 18, the first image related information acquisition unit 36 divides the first image into 16 and acquires 16 first spatiotemporal blocks. In FIG. 18, a block identifier is described in each first spatiotemporal block.
Next, the second image related information acquisition unit 37 reads the second image (here, for example, FIG. 17) from the second image storage unit 32. Then, the second image related information acquisition unit 37 divides the second image and acquires a plurality of spatiotemporal blocks. Here, as shown in FIG. 19, the second image related information acquisition unit 37 divides the second image into 16 and acquires 16 second space-time blocks. In FIG. 19, a block identifier is written in each second space-time block.
Next, the lookup table configuration unit 38 converts the filter coefficient of the conversion filter that converts the first first space-time block of the first image into the first second space-time block of the second image. For example, calculation is performed using least square estimation.
Similarly, the lookup table configuration unit 38 determines the nth (n is 2 to 16) first spatiotemporal block of the first image as the nth (n is 2 to 16) of the second image. The filter coefficients of the transform filter that transforms into the second space-time block are calculated sequentially using, for example, least square estimation.
As described above, filter coefficients of 16 types of image conversion filters are obtained. FIG. 20 is a conceptual diagram illustrating such an operation performed by the lookup table configuration unit 38. In FIG. 20, F ₀₄ Indicates the fourth image conversion filter, F ₁₅ Indicates the fifteenth image conversion filter.
Next, the lookup table configuration unit 38 acquires the neighborhood feature amount of each of the first first spatiotemporal blocks of the first image. For example, the lookup table configuration unit 38 calculates frequency components using discrete Fourier transform for each first spatiotemporal block, and acquires power for each band. The power for each band is used as a feature vector of neighboring feature values. The concept of such processing is shown in FIG. In FIG. 21, for example, the lookup table configuration unit 38 starts from the fourth first spatio-temporal block to the feature quantity vector “α ₀₄ = (p _04-1 , p _04-2 , p _04-3 , p _04-4 ) ”And the feature vector“ α of the neighborhood feature from the fifteenth first spatiotemporal block ₁₅ = (p _15-1 , p _15-2 , p _15-3 , p _15-4 ) ”.
Next, the lookup table configuration unit 38 calculates an overall feature amount that is a feature amount of the entire first image. For example, the look-up table configuration unit 38 calculates the variance “A” for each component of the locally obtained feature vector (neighboring feature vector). ₁ = (V ₁ , V ₂ , V _Three , V _Four ) "Is calculated as an overall feature amount. The concept of such processing is shown in FIG.
Next, the look-up table construction unit 38 connects the overall feature quantity and the neighborhood feature quantity for each region, and the feature quantity vector “(A ₁ , Α ₀₁ ) Etc.]. Also, the lookup table configuration unit 38 configures correspondence information in which feature quantity vectors and filter coefficients are associated with each other.
Then, the lookup table accumulation unit 39 accumulates 16 pieces of correspondence information in which the feature amount vectors and the filter coefficients are associated with each other in the storage unit 101.
The above processing is performed on the first image and the second image of another set. Then, the lookup table acquisition device 3 acquires the lookup table shown in FIG. The lookup table in FIG. 23 has a lot of correspondence information. Here, the correspondence information includes a feature amount vector and a filter coefficient.
As described above, in the present embodiment, a lookup table that can be used to generate an image in which the brain load is adjusted according to the purpose can be automatically acquired.
(Embodiment 5)
In the present embodiment, the image processing apparatus 4 that acquires an image in which the brain load is adjusted with respect to the received image according to the purpose using the lookup table constructed in the fourth embodiment will be described.
FIG. 24 is a block diagram of the image processing apparatus 4 in the present embodiment. The image processing apparatus 4 includes a storage unit 101, an image receiving unit 104, a dividing unit 105, an image analyzing unit 106, a mapping unit 107, a combining unit 108, and an image output unit 109.
Now, the storage unit 101 of the image processing apparatus 4 stores the lookup table shown in FIG. The look-up table is preferably indexed using the feature vector as a key. In the lookup table, it is preferable that the correspondence information is sorted using the feature vector as a key.
Further, it is assumed that the image receiving unit 104 has received the image shown in FIG. Note that the image in FIG. 25 is an image with a shallow depth of field.
Next, the image analysis unit 106 performs the entire feature amount (A _n ) Is calculated. Note that the method for calculating the entire feature amount is the same processing as that described in the fourth embodiment.
Next, the image analysis unit 106 and the like perform the neighborhood feature amount (α _n ) Is calculated. As the neighborhood feature amount here, the frequency component in the neighborhood 16 × 16 pixel block of the pixel is calculated, and the power for each band is acquired. This process is also the same as the process described in the fourth embodiment.
Next, the image analysis unit 106 synthesizes a feature amount vector (A _n , Α _n ).
Next, the image analysis unit 106 performs a feature vector (A _n , Α _n ) And two or more feature quantity vectors included in the two or more correspondence information constituting the lookup table, and a feature quantity vector “for example, (A _m , Α _m ) ”.
Then, the mapping unit 107 sets a filter coefficient “for example, F _m Is obtained.
Next, the mapping unit 107 performs a conversion filter realized by the acquired filter coefficient on each pixel of the input image, and acquires each pixel of the second image (output image).
Then, the synthesizing unit 108 configures the output image using each acquired pixel as an output pixel value at the pixel position of the input image. Next, the image output unit 109 outputs the configured output image. The concept of the above processing is shown in FIG.
As described above, according to the present embodiment, it is possible to output an image in which the brain load is suitably adjusted.
Note that, in the present embodiment, the method for obtaining the feature vector constituting the lookup table is different from the method for obtaining the feature vector of the image analysis unit 106. However, the method for obtaining the feature vector constituting the lookup table and the method for obtaining the feature vector of the image analysis unit 106 may be the same.
(Embodiment 6)
In the present embodiment, an image processing apparatus 5 that acquires an image in which the brain load is adjusted with respect to an received image according to the purpose using a lookup table will be described. In the present embodiment, the main difference from Embodiment 1 and the like is that the constraint conditions to be applied change dynamically according to the input image.
FIG. 27 is a block diagram of the image processing apparatus 5 in the present embodiment.
The image processing apparatus 5 includes a storage unit 101, a constraint condition storage unit 502, an instruction reception unit 503, an image reception unit 104, a division unit 105, an image analysis unit 106, a mapping unit 507, a synthesis unit 108, an image output unit 109, and a constraint. A condition change unit 510 is provided.
The mapping unit 507 includes, for example, an output image candidate acquisition unit 1072, a determination unit 1073, a feature amount acquisition unit 5071, a constraint condition determination unit 5072, a change unit 5074, and an output image acquisition unit 1075.
The constraint condition storage unit 502 can store two or more constraint information that is a set of condition identification information and constraint conditions. The constraint condition here will be described taking the condition relating to the output image as an example, but it may be the condition relating to the input image or the correspondence information. The condition identification information is information for identifying a constraint condition. The condition identification information is, for example, a feature vector. Further, the feature quantity constituting the vector is, for example, a neighborhood space activity of the input image. Neighboring space activities include, for example, time difference values, slopes (primary and secondary spatial differential values), local minimum values and local maximum values (first value), nth value, dynamic range, and space of pixels at the same position on the field. There may be various values such as a difference value, pixel value dispersion, power in a high spatial frequency range, a ratio of time activity to space activity, and the like. The constraint condition is an appropriate neighborhood space activity. An example of constraint information is shown in FIG. In FIG. 28, the constraint information includes a neighborhood space activity that is condition identification information and an appropriate activity that is a constraint condition.
The instruction receiving unit 503 receives an instruction for changing the constraint condition from the user. The instruction is, for example, an instruction to correct the constraint information in FIG. The instruction is, for example, an instruction to change the amount of stimulation that the user receives from the image.
A feature amount acquisition unit 5071 constituting the mapping unit 507 acquires a feature amount vector of an image (input image) received by the image receiving unit 104. The feature amount acquisition unit 5071 may acquire the feature amount vector of the entire image received by the image receiving unit 104, or acquire the feature amount vector of the spatio-temporal block constituting the image received by the image receiving unit 104. Also good.
The constraint condition determination unit 5072 applies the feature quantity vector acquired by the feature quantity acquisition unit 5071 to the constraint information in the constraint condition storage unit 502, and selects a constraint condition. More specifically, the constraint condition determination unit 5072 selects condition identification information that is closest to the feature amount vector acquired by the feature amount acquisition unit 5071 (when the condition identification information is also a feature amount vector), and A constraint condition paired with the selected condition identification information is acquired from the constraint condition storage unit 502.
When the determination unit 1073 determines that the constraint condition is not satisfied, the change unit 5074 changes all or part of the second image using the constraint condition acquired by the constraint condition determination unit 5072. Note that the changing unit 5074 may change the lookup table when the determining unit 1073 determines that the constraint condition is not satisfied. Further, the changing unit 5074, when the determining unit 1073 determines that the constraint condition is not satisfied, the input image received after the determination, the spatio-temporal block obtained after the determination, or one or more of the input images received after the determination The feature amount or one or more feature amounts of the spatiotemporal block obtained after the determination may be changed.
The constraint condition changing unit 510 changes the constraint information in accordance with the instruction received by the instruction receiving unit 503. Further, the constraint condition changing unit 510 may automatically change the constraint information. In other words, the constraint condition changing unit 510 includes, for example, user attributes (age, sex, address, etc.), information superimposed on the broadcast (information included in the EPG, etc.), time, time zone, external sensor information (eg, viewing and listening). The constraint information may be automatically changed according to the brightness, temperature, etc. of the room. In such a case, the constraint condition changing unit 510 preliminarily stores information on how to change the constraint information according to the user attribute or the like (for example, a correspondence table of expressions indicating the user attribute and the algorithm to be changed). keeping. When using external sensor information, the constraint condition changing unit 510 obtains information (room brightness, temperature, etc.) from a sensor (not shown).
The constraint condition storage unit 502 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The process in which the constraint information is stored in the constraint condition storage unit 502 does not matter. For example, the constraint information may be stored in the constraint condition storage unit 502 via a recording medium, and the constraint information transmitted via a communication line or the like is stored in the constraint condition storage unit 502. Alternatively, the constraint information input via the input device may be stored in the constraint condition storage unit 502.
The instruction receiving unit 503 can be realized by a device driver for input means such as a keyboard, control software for a menu screen, or the like.
The mapping unit 507, the constraint condition changing unit 510, the feature amount acquiring unit 5071, the constraint condition determining unit 5072, and the changing unit 5074 can be usually realized by an MPU, a memory, or the like. The processing procedure of the mapping unit 507 and the like is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).
Next, the operation of the image processing apparatus 5 will be described using the flowchart of FIG. In the flowchart of FIG. 29, steps different from the flowchart of FIG. 2 will be described.
(Step S2901) The mapping unit 507 performs mapping processing. The mapping process will be described with reference to the flowchart of FIG.
(Step S2902) The instruction receiving unit 503 determines whether a user instruction has been received. If a user instruction is accepted, the process proceeds to step S2903, and if a user instruction is not accepted, the process returns to step S201.
(Step S2903) The constraint condition changing unit 510 changes the constraint information in accordance with the instruction received by the instruction receiving unit 503.
In the flowchart of FIG. 29, the input image is divided and processing is performed for each space-time block. However, in the flowchart of FIG. 29, processing may be performed on the entire input image.
Next, the mapping process of step S2901 will be described using the flowchart of FIG. In the flowchart of FIG. 30, steps different from the flowchart of FIG. 4 will be described.
(Step S3001) The feature amount acquisition unit 507 constituting the mapping unit 507 acquires the feature amount vector of the image received by the image receiving unit 104.
(Step S3002) The constraint condition determination unit 5072 applies the feature quantity vector acquired by the feature quantity acquisition unit 5071 to the constraint information in the constraint condition storage unit 502, and determines the constraint condition. Such constraint condition determination processing will be described with reference to the flowchart of FIG.
Next, the constraint condition determination processing in step S3002 will be described using the flowchart in FIG.
(Step S3101) The constraint condition determining means 5072 substitutes 1 for the counter i.
(Step S 3102) The constraint condition determining means 5072 determines whether or not the i th constraint information exists in the constraint condition storage unit 502. If the i-th constraint information exists, the process goes to step S3103, and if not, the process goes to step S3106.
(Step S3103) The constraint condition determination unit 5072 acquires the condition identification information included in the i-th constraint information from the constraint condition storage unit 502.
(Step S3104) The constraint condition determination unit 5072 calculates the distance between the feature quantity vector acquired in step S3001 and the i-th condition identification information. Then, the constraint condition determining unit 5072 temporarily stores the distance in the buffer in association with the i-th condition identification information.
(Step S3105) The restraint condition determination means 5072 increments the counter i by 1, and returns to step S3102.
(Step S3106) The constraint condition determination unit 5072 acquires from the constraint condition storage unit 502 a constraint condition that is paired with the condition identification information having the smallest distance from the feature vector acquired in step S3001. Return to upper process.
As described above, according to the present embodiment, it is possible to output an extremely high quality image by changing the constraint condition to be applied according to the input image and constructing an output image that matches the constraint condition.
In the present embodiment, the constraint condition determining unit 5072 acquires the constraint condition to be applied according to the input image. However, the constraint condition determination unit 5072 may change the constraint condition to be applied according to the output image. Further, the constraint condition determination unit 5072 may change the constraint condition to be applied according to the input image and the output image.
In this embodiment, the dividing unit 105 divides the image received by the image receiving unit 104. However, the image received by the image receiving unit 104 may not be divided. In such a case, the dividing unit 105 is not necessary in the image processing apparatus 5. The same applies to other embodiments.
(Embodiment 7)
In the present embodiment, an image processing apparatus 6 that acquires an image in which the brain load is adjusted with respect to an received image according to the purpose using a lookup table will be described. In the present embodiment, the main difference from Embodiment 1 and the like is that the applied lookup table is dynamically changed according to the input image and the output image.
FIG. 32 is a block diagram of the image processing apparatus 6 in the present embodiment.
The image processing apparatus 6 includes a storage unit 101, a constraint condition storage unit 502, an instruction reception unit 603, an image reception unit 104, a division unit 105, an image analysis unit 106, a mapping unit 607, a synthesis unit 108, an image output unit 109, and an LUT. A change information change unit 610 is provided.
The mapping unit 607 also includes, for example, an output image candidate acquisition unit 1072, a determination unit 1073, a first feature amount acquisition unit 6071, a second feature amount acquisition unit 6072, an LUT change information storage unit 6073, an LUT determination unit 6074, and a change unit. 5074 and output image acquisition means 1075.
Here, the storage unit 101 stores two or more lookup tables (hereinafter referred to as “LUT” where appropriate).
The instruction receiving unit 603 receives an instruction for changing LUT change information described later from the user. The instruction is, for example, an instruction to change the amount of stimulation that the user receives from the image.
A first feature amount acquisition unit 6071 constituting the mapping unit 607 acquires a feature amount vector of an image received by the image receiving unit 104. The first feature amount acquisition unit 6071 performs the same operation as the feature amount acquisition unit 5071 described above. Note that the feature quantity vector acquired by the first feature quantity acquisition unit 6071 is referred to as a first feature quantity vector.
The second feature amount acquisition unit 6072 acquires the feature amount vector of the output image acquired by the output image acquisition unit 1075. The second feature quantity acquisition unit 6072 performs the same operation as the first feature quantity acquisition unit 6071, although the information acquisition source is different from the first feature quantity acquisition unit 6071. The one or more feature amounts acquired by the first feature amount acquisition unit 6071 and the one or more feature amounts acquired by the second feature amount acquisition unit 6072 may all be the same feature amount, or may be partially the same. May be included, or different feature amounts may be included. Note that the feature quantity vector acquired by the second feature quantity acquisition unit 6072 is referred to as a second feature quantity vector.
The LUT change information storage unit 6073 stores LUT change information, which is information for changing the LUT used when acquiring the image candidates output by the output image candidate acquisition unit 1072. The LUT change information is, for example, a pair of a first feature vector and a type identifier that identifies a lookup table. Further, the LUT change information may be a pair of a second feature vector and a type identifier, for example. Further, the LUT change information may be, for example, a pair of a first feature vector, a second feature vector, and a type identifier. The LUT change information is, for example, a pair of a first feature vector, two or more type identifiers, and two or more ratios corresponding to each type identifier. The LUT change information may be, for example, a pair of a second feature quantity vector, two or more type identifiers, and two or more ratios corresponding to each type identifier. The LUT change information may be, for example, a pair of a first feature vector, a second feature vector, two or more type identifiers, and two or more ratios corresponding to each type identifier. The LUT change information storage unit 6073 normally stores two or more LUT change information. The ratio is a ratio corresponding to each LUT when two or more LUTs are mixed to form a new LUT.
An example of the LUT change information management table stored in the LUT change information storage unit 6073 is shown in FIG. In FIG. 33, the LUT change information management table has a plurality of LUT change information. The LUT change information includes “ID”, “first feature vector”, “second feature vector”, “LUT”, and “ratio”. “LUT” is a type identifier. The “ratio” is information indicating the mixing ratio of each LUT when a new LUT is configured. The “ratio” is not limited to a ratio, and may be any information as long as it is information for configuring a new LUT using a plurality of LUTs. The information for constructing a new LUT is, for example, an arithmetic expression using parameters constituting elements of a plurality of LUTs. The elements constituting the LUT are, for example, filter coefficients and feature vector elements.
The LUT determination unit 6074 determines the LUT used by the output image candidate acquisition unit 1072 using the LUT change information in the LUT change information storage unit 6073. More specifically, the LUT determination unit 6074 applies the first feature vector, the second feature vector, or the first feature vector and the second feature vector to the LUT change information, and outputs an image candidate. The LUT used by the acquisition unit 1072 is determined. The LUT determination unit 6074 may configure the LUT by applying the first feature vector, the second feature vector, or the first feature vector and the second feature vector to the LUT change information. .
The operation of the LUT determination unit 6074 when the LUT change information management table of FIG. 33 is stored in the LUT change information storage unit 6073 will be described below. The LUT determination unit 6074 calculates the distance between the first feature amount vector acquired by the first feature amount acquisition unit 6071 and the first feature amount vector of each record in the LUT change information management table. Next, the LUT determination unit 6074 calculates the distance between the second feature amount vector acquired by the second feature amount acquisition unit 6072 and the second feature amount vector of each record in the LUT change information management table. Then, the LUT determination unit 6074 acquires, for example, the sum of two distances for each record of the LUT change information management table. Next, the LUT determination unit 6074 determines the record having the smallest sum of the two distances. Next, the LUT determination unit 6074 acquires the attribute value of “LUT” included in the determined record or the attribute value of “LUT” and “ratio” included in the determined record. When the attribute value of “LUT” included in the determined record is one type identifier, the LUT determination unit 6074 determines the LUT identified by the type identifier as the LUT used by the output image candidate acquisition unit 1072. On the other hand, when the attribute value of the “LUT” included in the determined record is two or more type identifiers, the LUT determination unit 6074 uses two or more type identifiers and the attribute value of “ratio” (hereinafter also simply referred to as a ratio). ) And a LUT identified by each of the two or more type identifiers is mixed so as to match the ratio, thereby forming a new LUT.
The LUT change information changing unit 610 changes the LUT change information according to the instruction received by the instruction receiving unit 603. Further, the LUT change information changing unit 610 may automatically change the LUT change information. That is, the LUT change information changing unit 610 includes, for example, user attributes (age, sex, address, etc.), information superimposed on the broadcast (information included in the EPG), time, time zone, external sensor information (eg, viewing) The LUT change information may be automatically changed according to the brightness of the room, temperature, etc. In such a case, the LUT change information changing unit 610 holds in advance information on how to change the LUT change information according to user attributes and the like. When using external sensor information, the constraint condition changing unit 510 obtains information (room brightness, temperature, etc.) from a sensor (not shown).
The instruction receiving unit 603 can be realized by a device driver for input means such as a keyboard, control software for a menu screen, or the like.
The mapping unit 607, the LUT change information change unit 610, the first feature quantity acquisition unit 6071, the second feature quantity acquisition unit 6072, and the LUT determination unit 6074 can be normally realized by an MPU, a memory, or the like. The processing procedure of the mapping unit 607 or the like is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).
The LUT change information storage means 6073 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The process in which the LUT change information is stored in the LUT change information storage unit 6073 does not matter. For example, the LUT change information may be stored in the LUT change information storage unit 6073 via a recording medium, and the LUT change information transmitted via a communication line or the like is stored in the LUT change information storage unit 6073. Alternatively, the LUT change information input via the input device may be stored in the LUT change information storage unit 6073.
Next, the operation of the image processing apparatus 6 will be described using the flowchart of FIG. In the flowchart of FIG. 34, steps different from the flowchart of FIG. 2 will be described.
(Step S3401) The mapping unit 607 performs mapping processing. The mapping process will be described with reference to the flowchart in FIG.
(Step S3402) The instruction receiving unit 603 determines whether a user instruction has been received. If a user instruction is accepted, the process proceeds to step S3403. If a user instruction is not accepted, the process returns to step S201.
(Step S3403) The LUT change information changing unit 610 changes the LUT change information in accordance with the instruction received by the instruction receiving unit 503.
Next, the mapping process of step S3401 will be described using the flowchart of FIG. In the flowchart of FIG. 35, steps different from the flowchart of FIG. 4 will be described.
(Step S3501) The first feature amount acquisition unit 6071 of the mapping unit 607 acquires a first feature amount vector from the input image received by the image receiving unit 104.
(Step S3502) The second feature amount acquisition unit 6072 acquires a second feature amount vector from the output image acquired by the output image acquisition unit 1075. Here, the output image from which the second feature vector is acquired is usually the output image output immediately before.
(Step S3503) The LUT determination unit 6074 determines the LUT using the first feature vector acquired in step S3501 and the second feature vector acquired in step S3502. The LUT determination process which is such a process will be described with reference to the flowchart of FIG.
Next, the LUT determination processing in step S3503 will be described using the flowchart in FIG.
(Step S3601) The LUT determination means 6074 substitutes 1 for the counter i.
(Step S3602) The LUT determination unit 6074 determines whether or not the i-th LUT change information exists. If the i-th LUT change information exists, the process goes to step S3603, and if not, the process goes to step S3607.
(Step S3603) The LUT determination unit 6074 acquires the first feature vector included in the i-th LUT change information. Then, the LUT determination unit 6074 calculates a distance (d1) between the acquired first feature value vector and the first feature value vector acquired in step S3501.
(Step S3604) The LUT determination means 6074 acquires the second feature amount vector included in the i-th LUT change information. Then, the LUT determination unit 6074 calculates the distance (d2) between the acquired second feature value vector and the second feature value vector acquired in step S3502.
(Step S3605) The LUT determination unit 6074 substitutes the distance (d1) calculated in step S3603 and the distance (d2) calculated in step S3604 for the calculation formula, and executes the calculation formula. Then, the LUT determination unit 6074 acquires the cost (C) of the i-th LUT change information. Then, the LUT determination unit 6074 stores at least temporarily the cost (C) in association with the i-th LUT change information. The calculation formula is a calculation formula using two distances as parameters, and is a function using each distance as a parameter. Such a function is usually an increasing function with each distance as a parameter. The calculation formula is, for example, “C = a1 × d1 + (1−a1) × d2”. In addition, a1 is 0.5, 0.8, etc., for example.
(Step S3606) The LUT determination means 6074 increments the counter i by one.
(Step S3607) The LUT determination unit 6074 acquires one or more type identifiers included in the LUT change information with the smallest cost calculated in Step S3605.
(Step S3608) The LUT determination means 6074 determines whether or not there are a plurality of type identifiers acquired in step S3607. If there is more than one, go to step S3609, and if there is one, go to step S3611.
(Step S3609) The LUT determination unit 6074 acquires, from the LUT change information storage unit 6073, a plurality of ratios included in the LUT change information with the lowest cost in association with the type identifier.
(Step S3610) The LUT determination means 6074 acquires the LUT corresponding to the plurality of type identifiers from the storage unit 101. Then, the LUT determination unit 6074 mixes the acquired LUTs according to the ratio acquired in step S3609 to generate a new LUT. The new LUT is an LUT used by the output image candidate acquisition unit 1072.
The LUT mixing is, for example, the following processing. For example, it is assumed that the correspondence information constituting the LUT is a feature vector and a parameter group (for example, a set of filter coefficients). The plurality of LUTs have the same feature vector. Then, the LUT determination unit 6074 acquires a plurality of LUT parameter groups corresponding to the same feature vector, and mixes the parameters constituting the parameter group according to the ratio. For example, it is assumed that the feature quantity vector included in the correspondence information with the LUT “A” is V1 and the parameter group is (4, 10, 7,...). Further, it is assumed that the feature quantity vector included in the correspondence information with the LUT “B” is V1 and the parameter group is (10, 8, 17,...). It is assumed that the ratio corresponding to the LUT “A” is “a1” and the ratio corresponding to the LUT “A” is “1-a1”. In such a case, the LUT determination unit 6074 acquires a parameter group (4 × a1 + 10 × (1−a1), 10 × a1 + 8 × (1−a1), 7 × a1 + 17 × (1−a1),...). Such processing is performed for all correspondence information.
(Step S3611) The LUT determination means 6074 acquires the LUT identified by the type identifier of the LUT change information with the lowest cost from the LUT change information storage means 6073. Such an LUT is an LUT used by the output image candidate acquisition unit 1072.
As described above, according to the present embodiment, an extremely high quality image is output by changing the look-up table to be applied and configuring the output image according to the input image, the output image, or the input image and the output image. it can.
(Embodiment 8)
In the present embodiment, the image learning device 7 will be described. The image learning device 7 is a device that acquires a first image and a second image that are used by the lookup table acquisition device 3 to form a lookup table. The image learning device 7 here is mainly a device for a look-up table used in an image processing device that improves entertainment properties.
FIG. 37 is a block diagram of the image learning device 7. The image learning device 7 includes a first image storage unit 31, a second image storage unit 32, an image generation unit 71, and an image storage unit 72. The first image storage unit 31 and the second image storage unit 32 constitute a device such as the lookup table acquisition device 3.
The first image storage unit 31 can store one or more first images. The first image storage unit 31 may store a first image that is a moving image.
The second image storage unit 32 can store one or more second images. The second image storage unit 32 may store a second image that is a moving image.
The image generation unit 71 generates a second image from the first image stored in the first image storage unit 31. The image generation unit 71 may generate the first image from the second image stored in the second image storage unit 32.
For example, the image generation unit 71 generates the following second image or first image.
(1) Generation of a second image with a flash image that is a white image inserted
The image generation unit 71 reads the first image that is a moving image from the first image storage unit 31. Next, the image generation unit 71 inserts a white image frame at an arbitrary frame position of the first image, and acquires a second image.
(2) Generation of a second image in which a flash image that is an image with an increased luminance value is inserted
The image generation unit 71 reads the first image that is a moving image from the first image storage unit 31. Next, the image generation unit 71 increases the luminance value of the frame at an arbitrary frame position of the first image, and acquires the second image.
(3) Generation of a second image with a flash image as a high contrast image inserted
The image generation unit 71 reads the first image that is a moving image from the first image storage unit 31. Next, the image generation unit 71 inserts a high-contrast image at an arbitrary frame position of the first image, and acquires the second image.
(4) Generation of second image with non-uniformly enhanced image
The image generation unit 71 reads the first image that is a moving image from the first image storage unit 31. Next, the image generation unit 71 performs edge enhancement, contrast change, and the like on a frame at an arbitrary position of the first image, and acquires a second image. Since processing such as edge enhancement and contrast change is a known technique, description thereof is omitted.
(5) Film-like image processing
The image generation unit 71 reads a second image that is a moving image of the film source from the second image storage unit 32. Next, the second image is image-converted to obtain the first image.
The image storage unit 72 stores the second image generated by the image generation unit 71 in the second image storage unit 32. In addition, the image storage unit 72 stores the first image generated by the image generation unit 71 in the first image storage unit 31.
Next, mapping processing of the image processing apparatus will be described. The mapping process is performed by the mapping unit of the image processing apparatus described above.
When the second image is a flash image generated in any one of the above (1) to (3), the mapping unit 107 calculates a feature vector (for example, luminance distribution or full screen motion amount) of the input image. Corresponding information corresponding to the feature vector is acquired from the LUT using a vector). Then, the mapping unit 107 inserts a special frame using the acquired correspondence information. Note that the feature vector of the input image is acquired by the image analysis unit 106. The mapping unit 107 may insert a special frame when the feature vector of the input image satisfies a predetermined condition (may be correspondence information). A special frame is a frame of a flash image.
In addition, when the second image is the nonuniformly emphasized image generated in the above (4), the mapping unit 107 performs the above-described processing so that the enhancement method is uniform over the entire screen. Instead, for example, different enhancement methods are automatically selected according to the waveform.
When the second image is a film-like image generated in the above (5), the mapping unit 107 can output an output image converted into a film-like image by performing the above-described processing.
Note that, for example, a first image and a second image obtained by photographing the same subject under different illumination environments may be accumulated in the first image storage unit 31 and the second image storage unit 32, respectively. As described above, the different illumination environment is an environment in which the brightness of the illumination is varied depending on the object to be photographed, for example. Further, the different illumination environment is an environment in which the position of the light source is shifted, for example.
In such a case, the mapping unit 107 performs the above-described processing, thereby improving the texture of the image or obtaining an image with completely different surface reflection characteristics. For example, a new string instrument in the input image is output like a string instrument with a sober varnish color like Old.

  FIG. 38 shows the external appearance of a computer that executes the program described in this specification and realizes the lookup table acquisition apparatus and the like of the above-described embodiment. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 38 is an overview of the computer system 340, and FIG. 39 is a block diagram of the computer system 340.

  38, the computer system 340 includes a computer 341 including an FD drive and a CD-ROM drive, a keyboard 342, a mouse 343, and a monitor 344.

  In FIG. 39, a computer 341 stores an MPU 3413, a bus 3414 connected to the CD-ROM drive 3412 and the FD drive 3411, and a program such as a bootup program, in addition to the FD drive 3411 and the CD-ROM drive 3412. A RAM 3416 for temporarily storing application program instructions and providing a temporary storage space; and a hard disk 3417 for storing application programs, system programs, and data. Although not shown here, the computer 341 may further include a network card that provides connection to the LAN.

  A program that causes the computer system 340 to execute the functions of the lookup table acquisition apparatus and the like of the above-described embodiment is stored in the CD-ROM 3501 or FD 3502, inserted into the CD-ROM drive 3412 or FD drive 3411, and It may be transferred to the hard disk 3417. Alternatively, the program may be transmitted to the computer 341 via a network (not shown) and stored in the hard disk 3417. The program is loaded into the RAM 3416 at the time of execution. The program may be loaded directly from the CD-ROM 3501, the FD 3502, or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 341 to execute functions such as the lookup table acquisition device of the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 340 operates is well known and will not be described in detail.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the image processing apparatus according to the present invention has an effect of outputting an image in which the brain load is adjusted, and is useful as a television receiver or the like.

1, 2, 4, 5, 6 Image processing device 3 Look-up table acquisition device 7 Image learning device 31 First image storage unit 32 Second image storage unit 33 Second image information candidate set storage unit 34 Candidate output unit 35 Second Image storage unit 36 First image related information acquisition unit 37 Second image related information acquisition unit 38 Lookup table configuration unit 39 Lookup table storage unit 71 Image generation unit 72 Image storage unit 101 Storage unit 102, 502 Constraint condition storage unit 103 , 503, 603 Instruction receiving unit 104 Image receiving unit 105 Dividing unit 106 Image analyzing unit 107, 507, 607 Mapping unit 108 Combining unit 109 Image output unit 110 Look-up table updating unit 507, 5071 Feature quantity acquisition unit 510 Constraint condition changing unit 610 LUT change information change unit 1071 Type identifier acquisition means 10 2 output image candidate obtaining unit 1073 determining unit 1074,5074 changing means 1075 outputs the image acquiring unit 5072 constraint determination unit 6071 first feature amount acquisition unit 6072 second feature amount acquisition unit 6073 LUT change information storage unit 6074 LUT determining means

Claims (13)

A first image storage unit capable of storing one or more first images;
A second image storage unit capable of storing one or more second images that are different images of the same object as the first image in association with the one or more first images;
Information for taking correspondence between all or one or more of the one or more first images and one or more of the one or more second images and one or more of the two or more pixels A storage unit capable of storing a lookup table having two or more correspondence information;
All of one or more first images stored in the first image storage unit or part of two or more pixels, and all of one or more second images stored in the second image storage unit Or a lookup table configuration unit that configures a lookup table having one or more correspondence information from a portion that is two or more pixels ;
A lookup table storage unit that stores the lookup table configured by the lookup table configuration unit in the storage unit ;
The second image has a relationship that the second image is an image obtained in another environment that is different from the one environment in which the first image is obtained, or the second image The image has a relationship that a flash screen is inserted in part with respect to the first image, or the second image has the relationship with the first image with respect to the first image. The second image is a deformed image, or the second image is an image of a picture depicting the first image with respect to the first image, or the second image The image has a relationship that is a CG image of the first image with respect to the first image;
Compared to the one environment, the other environment is an environment of the position of the light source where the shadow of the object on the image appears well, or the object on the image has a better three-dimensional effect than the one environment. The environment of the position of the light source to be exposed, the environment of the camera position where the stereoscopic effect of the object on the image is well expressed compared to the one environment, the environment of the weather different from the one environment, or the one A look-up table acquisition apparatus, which is an environment having a temperature and humidity different from the environment, an environment having a depth of field different from the one environment, or an environment having a shutter speed different from the one environment . The correspondence information is
First image related information that is information about all or part of the one or more first images, and information about all or part of the one or more second images, information corresponding to the first image related information Information including the second image related information,
One or more first image related information, which is information about all or part of one or more first images stored in the first image storage unit, is acquired from all or part of the one or more first images. A first image related information acquisition unit
One or more second image related information, which is information related to all or part of one or more second images stored in the second image storage unit, is acquired from all or part of the one or more second images. And a second image related information acquisition unit.
The lookup table configuration unit includes:
The first image related information acquired by the first image related information acquisition unit, and the second image related information acquired by the second image related information acquisition unit, which is information corresponding to the first image related information. A look-up table having one or more correspondence information, which is information and indicates correspondence between all or part of the one or more first images and all or part of the one or more second images look-up table acquiring apparatus according to claim 1 wherein the configuration. The correspondence information is
A feature vector corresponding to all or part of the one or more first images, and all or part of the one or more first images for converting all or part of the one or more second images to An image conversion filter,
The lookup table configuration unit includes:
A conversion filter that obtains a feature vector from all or part of the one or more first images and converts all or part of the one or more first images into all or part of the one or more second images. The lookup table acquisition apparatus according to claim 1 or 2, wherein a lookup table having one or more correspondence information having the feature quantity vector and the conversion filter is configured. A second that can store one or more second image information candidate sets that are sets of two or more second image information candidates that are information related to the second image in association with one or more first images. An image information candidate set storage unit;
A candidate output unit that outputs the one or more second image information candidate sets;
Instruction reception for receiving an instruction to select one or more second image information candidates from two or more second image information candidates included in each of the one or more second image information candidate sets output by the candidate output unit And
Acquires one or more second images using one or more second image information candidates corresponding to the instruction received by the instruction receiving unit, and corresponds to the first image corresponding to the second image information candidate set The lookup table acquisition device according to any one of claims 1 to 3 , further comprising a second image storage unit that stores the one or more second images in the second image storage unit. On the storage medium,
One or more first images;
One or more second images, which are different images of the same object as the first image, are stored in association with the one or more first images;
A lookup table generation unit, and a lookup table generation support method that can be realized by a lookup table storage unit,
The look-up table configuration unit includes the one or more of the one or more first images or a portion that is two or more pixels and the one or more of the one or more second images or a portion that is two or more pixels. Lookup that constitutes a lookup table having one or more correspondence information that is information for taking correspondence between all or part of the first image and all or part of the one or more second images. Table composition steps;
The lookup table storage unit includes a lookup table storage step of storing the lookup table configured in the lookup table configuration step in a storage medium ;
The second image has a relationship that the second image is an image obtained in another environment that is different from the one environment in which the first image is obtained, or the second image The image has a relationship that a flash screen is inserted in part with respect to the first image, or the second image has the relationship with the first image with respect to the first image. The second image is a deformed image, or the second image is an image of a picture depicting the first image with respect to the first image, or the second image The image has a relationship that is a CG image of the first image with respect to the first image;
Compared to the one environment, the other environment is an environment of the position of the light source where the shadow of the object on the image appears well, or the object on the image has a better three-dimensional effect than the one environment. The environment of the position of the light source to be exposed, the environment of the camera position where the stereoscopic effect of the object on the image is well expressed compared to the one environment, the environment of the weather different from the one environment, or the one A method for assisting generation of a look-up table, which is an environment having a temperature and humidity different from the one environment, an environment having a depth of field different from the one environment, or an environment having a shutter speed different from the one environment . The correspondence information is
First image related information that is information about all or part of the one or more first images, and information about all or part of the one or more second images, information corresponding to the first image related information Information including the second image related information,
The lookup table generation support method according to claim 5, further comprising a first image related information acquisition unit and a second image related information acquisition unit,
The first image related information acquisition unit acquires one or more first image related information, which is information related to all or a part of the one or more first images, from all or a part of the one or more first images. A first image related information acquisition step,
The second image related information acquisition unit acquires one or more second image related information, which is information about all or a part of the one or more second images, from all or a part of the one or more second images. And a second image related information acquisition step.
In the lookup table construction step,
First image related information acquired in the first image related information acquisition step, information corresponding to the first image related information, and second image related information acquired in the second image related information acquisition step Lookup that includes one or more correspondence information that is information indicating correspondence between all or part of the one or more first images and all or part of the one or more second images. 6. The lookup table generation support method according to claim 5, wherein the table is configured. The correspondence information is
A feature vector corresponding to all or part of the one or more first images, and all or part of the one or more first images for converting all or part of the one or more second images to An image conversion filter,
In the lookup table construction step,
A conversion filter that obtains a feature vector from all or part of the one or more first images and converts all or part of the one or more first images into all or part of the one or more second images. The lookup table generation support method according to claim 5 or 6, wherein a lookup table having one or more correspondence information having the feature quantity vector and the transformation filter is configured. On the storage medium,
One or more second image information candidate sets that are sets of two or more second image information candidates that are information relating to the second image are further stored in association with each of the one or more first images. ,
The lookup table generation support method according to claim 3, further comprising a candidate output unit, an instruction receiving unit, and a second image storage unit.
A candidate output step in which the candidate output unit outputs the one or more second image information candidate sets;
The instruction receiving unit selects one or more second image information candidates from two or more second image information candidates included in each of the one or more second image information candidate sets output in the candidate output step. An instruction receiving step for receiving an instruction to perform,
The second image storage unit acquires one or more second images using one or more second image information candidates corresponding to the instruction received in the instruction receiving step, and a second image information candidate set The lookup table generation support method according to any one of claims 5 to 7 , wherein the one or more second images are stored in the storage medium in association with a first image corresponding to. On the storage medium,
One or more first images;
One or more second images, which are different images of the same object as the first image, are stored in association with the one or more first images;
A method for producing a lookup table that can be realized by a lookup table configuration unit and a lookup table storage unit,
The look-up table configuration unit includes the one or more of the one or more first images or a portion that is two or more pixels and the one or more of the one or more second images or a portion that is two or more pixels. Lookup that constitutes a lookup table having one or more correspondence information that is information for taking correspondence between all or part of the first image and all or part of the one or more second images. Table composition steps;
The lookup table storage unit includes a lookup table storage step of storing the lookup table configured in the lookup table configuration step in a storage medium ;
The second image has a relationship that the second image is an image obtained in another environment that is different from the one environment in which the first image is obtained, or the second image The image has a relationship that a flash screen is inserted in part with respect to the first image, or the second image has the relationship with the first image with respect to the first image. The second image is a deformed image, or the second image is an image of a picture depicting the first image with respect to the first image, or the second image The image has a relationship that is a CG image of the first image with respect to the first image;
Compared to the one environment, the other environment is an environment of the position of the light source where the shadow of the object on the image appears well, or the object on the image has a better three-dimensional effect than the one environment. The environment of the position of the light source to be exposed, the environment of the camera position where the stereoscopic effect of the object on the image is well expressed compared to the one environment, the environment of the weather different from the one environment, or the one A method for producing a look-up table, which is an environment having a temperature and humidity different from the one environment, an environment having a depth of field different from the one environment, or an environment having a shutter speed different from the one environment . The correspondence information is
First image related information that is information about all or part of the one or more first images, and information about all or part of the one or more second images, information corresponding to the first image related information Information including the second image related information,
The method for producing a lookup table according to claim 5, further comprising a first image related information acquisition unit and a second image related information acquisition unit,
The first image related information acquisition unit acquires one or more first image related information, which is information related to all or a part of the one or more first images, from all or a part of the one or more first images. A first image related information acquisition step,
The second image related information acquisition unit acquires one or more second image related information, which is information about all or a part of the one or more second images, from all or a part of the one or more second images. And a second image related information acquisition step.
In the lookup table construction step,
First image related information acquired in the first image related information acquisition step, information corresponding to the first image related information, and second image related information acquired in the second image related information acquisition step Lookup that includes one or more correspondence information that is information indicating correspondence between all or part of the one or more first images and all or part of the one or more second images. The method for producing a look-up table according to claim 9 , comprising a table. The correspondence information is
A feature vector corresponding to all or part of the one or more first images, and all or part of the one or more first images for converting all or part of the one or more second images to An image conversion filter,
In the lookup table construction step,
A conversion filter that obtains a feature vector from all or part of the one or more first images and converts all or part of the one or more first images into all or part of the one or more second images. The lookup table production method according to claim 9 or 10, wherein a lookup table having one or more correspondence information having the feature quantity vector and the transformation filter is configured. On the storage medium,
One or more second image information candidate sets that are sets of two or more second image information candidates that are information relating to the second image are further stored in association with each of the one or more first images. ,
The method for producing a lookup table according to any one of claims 9 to 11 , further comprising a candidate output unit, an instruction receiving unit, and a second image storage unit,
A candidate output step in which the candidate output unit outputs the one or more second image information candidate sets;
The instruction receiving unit selects one or more second image information candidates from two or more second image information candidates included in each of the one or more second image information candidate sets output in the candidate output step. An instruction receiving step for receiving an instruction to perform,
The second image storage unit acquires one or more second images using one or more second image information candidates corresponding to the instruction received in the instruction receiving step, and a second image information candidate set first in association with the image, the one or more second claim 11 the method of producing a lookup table according to any one of claims 9 to an image accumulated in the storage medium corresponding to the. On the storage medium,
One or more first images;
One or more second images, which are different images of the same object as the first image, are stored in association with the one or more first images;
Computer
And a whole or a two or more part which is a pixel of said one or more first of all or more than one portion is a pixel and one or more second images of images, all of the one or more first image or A lookup table configuration unit that configures a lookup table having one or more correspondence information, which is information for taking correspondence between a portion and all or a portion of the one or more second images,
A program for causing a lookup table configured by the lookup table configuration unit to function as a lookup table storage unit for storing in a storage medium ,
The second image has a relationship that the second image is an image obtained in another environment that is different from the one environment in which the first image is obtained, or the second image The image has a relationship that a flash screen is inserted in part with respect to the first image, or the second image has the relationship with the first image with respect to the first image. The second image is a deformed image, or the second image is an image of a picture depicting the first image with respect to the first image, or the second image The image has a relationship that is a CG image of the first image with respect to the first image;
Compared to the one environment, the other environment is an environment of the position of the light source where the shadow of the object on the image appears well, or the object on the image has a better three-dimensional effect than the one environment. The environment of the position of the light source to be exposed, the environment of the camera position where the stereoscopic effect of the object on the image is well expressed compared to the one environment, the environment of the weather different from the one environment, or the one A program that is an environment having a temperature or humidity different from that of the environment, an environment having a depth of field different from that of the one environment, or an environment having a shutter speed different from that of the one environment .