JP2012029004A - Image generating device, image generating program, and image generating method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image generating device, an image generating program, and an image generating method that can form a lens array image with a small amount of calculation.SOLUTION: An image generating device 100 comprises a setting information acquisition part 110 for acquiring setting information representing the focal distance of a lens array including arrangement of plural lenses for forming an optical image by a main lens of an original image. The image generating device 100 comprises a trace specifying part 150 for specifying the trace of light imaged at a position of an image pickup element away from the lens array by the focal distance by tracking the trace from the position of the image pickup element. The image generating device 100 further comprises an image generation part 160 for, by calculating the pixel value output from the image pickup element based on the specified trace, forming a lens array image representing the optical image by the main lens of the original image formed by the lens array.

Description

  The present invention relates to an image generation apparatus, an image generation program, and an image generation method.

  In Non-Patent Document 1, the original image IO formed by the main lens LM as shown in FIG. 14 is formed by a plurality of microlenses arranged in the main scanning direction and the sub-scanning direction (that is, the vertical direction and the horizontal direction). There is disclosed a technique for generating an image (hereinafter referred to as a lens array image) further imaged by a configured microlens array LA based on various set parameters and an original image IO.

Tom E. Bishop and two others, “Light Field Superresolution”, ICCP 09 (IEEE International Conference on Computational Photography), April 2009

  Here, in the technique according to Non-Patent Document 1, for all the pixels of the original image IO, a blur (that is, subject blur) generated on the image plane PI when light emitted from each pixel passes through the main lens LM. BM and blurs BD1 to B3 generated on the image pickup element surface DP when light further passes through the lens array LA are calculated. In the technology according to Non-Patent Document 1, whether or not each element is included in the blur is calculated for all the image sensors that form the image sensor surface DP, and the image sensor included in the range of the blur is calculated. The pixel value of the original image IO used for calculating the blur is substituted. For this reason, in order to generate a lens array image, there is a problem that the amount of calculation is large because it is necessary to perform an operation for the number of times of integration of the number of pixels of the original image IO and the number of pixels of the image sensor.

  In view of the above, the present invention has an object to provide an image generation apparatus, an image generation program, and an image generation method capable of generating a lens array image with a small amount of calculation.

In order to achieve the above object, an image generation apparatus according to a first aspect of the present invention provides:
Image information acquisition means for acquiring original image information representing the original image;
A lens array configured by an array of a plurality of lenses, which represents a focal length of a lens array that forms an optical image of a main image of an original image represented by original image information acquired by the image information acquisition unit Setting information acquisition means for acquiring setting information;
A trajectory that identifies the trajectory of light that is imaged at the position of the image sensor separated from the lens array by the focal length represented by the setting information acquired by the setting information acquisition means from the position of the image sensor. Specific means,
Based on the locus specified by the locus specifying means, a pixel value output from the image sensor is calculated, thereby generating a lens array image in which the lens array forms an optical image of the original image by the main lens. Image generating means,
It is characterized by that.

In the image generation device according to the first aspect,
The setting information acquisition means further acquires setting information indicating the size of the lenses constituting the lens array,
The locus specifying means includes
Based on the focal length and size of the lens array represented by the setting information acquired by the setting information acquisition means, a first specification for specifying an imaging range in which light is imaged by the lenses constituting the lens array Means,
The pixel range of the original image including the starting point of the light imaged at the position of the image sensor included in the imaging range specified by the first specifying means has passed through the center of the lenses constituting the lens array. A second specifying means for specifying a trajectory of light imaged later at the position of the image sensor from the position of the image sensor to the original image based on the size of the lens;
The image generating means calculates a pixel value of the image sensor based on a value of a pixel of the original image included in a pixel range specified by the second specifying means;
It is also good.

In the image generating apparatus according to the first aspect,
Corresponding to the imaging range information representing the imaging range in which the lens of the size forms light, the focal length information representing the focal length of the lens array and the size information representing the size of the lens constituting the lens array Further comprising information storage means for storing information;
The first specifying means included in the trajectory specifying means is stored in the information storage means based on focal length information indicating a focal length set by the setting information acquired by the setting information acquiring means and size information indicating a size. Identify the imaged range information
It is also good.

Furthermore, in the image generation device according to the first aspect,
Based on the focal length and size of the lens constituting the lens array represented by the setting information acquired by the setting information acquisition means, the imaging range in which light is imaged by the lenses constituting the lens array is calculated. First calculating means for
The image forming range information representing the image forming range calculated by the first calculating means is stored in the information storing means in association with the focal length information representing the focal distance and the size information representing the size of the lens. 1 storage means is further provided,
It is also good.

Furthermore, in the image generating device according to the first aspect,
Information retrieval means for retrieving imaging range information stored in the information storage means based on the focal length and size of the lens represented by the setting information obtained by the setting information obtaining means;
The pixel range of the original image including the starting point of the light imaged at the position of the image sensor included in the imaging range represented by the imaging range information searched by the information search means is set to the center of the lens. Second calculation means for calculating a trajectory of light imaged at the position of the image sensor after passing based on a result of tracing from the position of the image sensor to the original image and a size of the lens;
A second storage unit that stores the pixel range information representing the pixel range calculated by the second calculation unit and the element identification information for identifying the imaging element in association with each other and stored in the information storage unit;
The second specifying means included in the locus specifying means includes pixel range information stored in the information storage means based on element identification information for identifying an image pickup element included in the imaging range specified by the first specifying means. Identify the pixel range represented by
It is also good.

Furthermore, in the image generating device according to the first aspect,
Image evaluation means for evaluating the visibility of the lens array image based on the contrast of the lens array image generated by the image generation means;
Any two or more of the evaluation result by the image evaluation unit, the lens array image evaluated by the image evaluation unit, and the focal length of the lens array used to generate the lens array image are displayed in association with each other. A display means,
It is also good.

In order to achieve the above object, an image generation program according to the second aspect of the present invention provides:
A computer for controlling the image generating device;
Image information acquisition means for acquiring original image information representing the original image;
A lens array configured by an array of a plurality of lenses, which represents a focal length of a lens array that forms an optical image of a main image of an original image represented by original image information acquired by the image information acquisition unit Setting information acquisition means for acquiring setting information;
A trajectory that identifies the trajectory of light that is imaged at the position of the image sensor separated from the lens array by the focal length represented by the setting information acquired by the setting information acquisition means from the position of the image sensor. Specific means,
Based on the locus specified by the locus specifying means, a pixel value output from the image sensor is calculated, thereby generating a lens array image in which the lens array forms an optical image of the original image by the main lens. Function as image generation means,
It is characterized by that.

In order to achieve the above object, an image generation method according to the third aspect of the present invention includes:
An image generation method in an image generation apparatus,
An image information acquisition step for acquiring original image information representing the original image;
A lens array configured by an array of a plurality of lenses, which represents a focal length of a lens array that forms an optical image of a main image of an original image represented by original image information acquired by the image information acquisition unit A setting information acquisition step for acquiring setting information;
A trajectory that identifies the trajectory of light that is imaged at the position of the image sensor separated from the lens array by the focal length represented by the setting information acquired in the setting information acquisition step from the position of the image sensor Specific steps,
A pixel value output from the image sensor is calculated based on the locus specified in the locus specifying step, thereby generating a lens array image in which the lens array forms an optical image of the main image of the main lens. An image generation step,
It is characterized by that.

  According to the image generation apparatus, the image generation program, and the image generation method according to the present invention, a lens array image can be generated with a small amount of calculation.

It is a figure showing the example of 1 composition of the imaging device which the image generating device concerning this embodiment simulates. (A) is a figure showing an example of an original image, (b) is a figure showing an example of the lens array image produced | generated based on the original image, (c) is the partial expansion of a lens array image FIG. It is a hardware block diagram showing the example of 1 structure of an image generation apparatus. It is a flowchart showing an example of the image generation process which an image generation apparatus performs. (A) is a functional block diagram showing an example of the functions of the image generation apparatus, (b) is a diagram showing an example of the configuration of the processing target pixel table generation unit, and (c) is an integration target pixel table generation unit. The figure showing one structural example, (d) is a figure showing the one structural example of a locus | trajectory specific | specification part. (A) is a flowchart showing an example of the 1st setting information acquisition process which a setting information acquisition part performs, (b) is a flowchart showing an example of the 2nd setting information acquisition process which a setting information acquisition part performs. (A) is a figure showing an example of a processing object pixel table, (b) is a figure showing an example of the 1st setting table, (c) is a figure showing an example of an integration object pixel table, (d) is a figure. It is a figure showing an example of the 2nd setting table. It is a flowchart showing an example of the process target pixel table production | generation process which a process target pixel table production | generation part performs. It is a figure for demonstrating an example of the calculation method of 1st projection size. (A) is a figure for demonstrating an example of the production | generation method of a 1st relative range table, (b) is a figure for demonstrating the other example of the production | generation method of a 1st relative range table. It is a figure showing an example of a process target pixel relative range table (1st relative range table). It is a flowchart showing an example of the integration object pixel table production | generation process which an integration object pixel table production | generation part performs. It is a flowchart showing an example of the array image generation process which an image generation part performs. It is a figure for demonstrating an example of the production | generation method of the lens array image in the past.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described with reference to the accompanying drawings.

  The image generation apparatus 100 according to the embodiment of the present invention uses an optical image of a subject formed by a main lens LM as shown in FIG. 1 as a micro that has a plurality of microlenses arranged in the main scanning direction and the sub-scanning direction. An image capturing apparatus that captures an image further formed using a lens array (hereinafter simply referred to as a lens array) LA is simulated, and a captured image (hereinafter referred to as a lens array image) obtained by the simulated image capturing apparatus It is generated based on parameters representing the original image IO to be represented and the focal length fa of the lens array LA.

  Specifically, the image generation apparatus 100 is configured to wait until light starting from each pixel of the original image IO as illustrated in FIG. 2A reaches the image sensor surface DP via the main lens LM and the lens array LA. The locus is identified by tracing from the imaging element surface DP side (that is, the side opposite to the light traveling direction). Next, the image generation apparatus 100 specifies the pixel value output from the image sensor that is the end point of the trajectory based on the pixel value of the original image IO that is the start point of the specified trajectory, thereby FIG. A lens array image as shown is generated. FIG. 2C is a partially enlarged view of part A shown in FIG.

  As shown in FIG. 3, the image generation apparatus 100 includes a CPU (Central Processing Unit) 100a, a ROM (Read Only Memory) 100b, a RAM (Random Access Memory) 100c, a hard disk 100d, a media controller 100e, and a LAN (Local Area Network). A card 100f, a video card 100g, an LCD (Liquid Crystal Display) 100h, and a keyboard 100i are provided.

  The CPU 100a performs overall control of the image generation apparatus 100 by executing software processing according to a program stored in the ROM 100b or the hard disk 100d. The RAM 100c temporarily stores information (data) to be processed when the CPU 100a executes the program.

  The media controller 100e reads data representing various images from a recording medium such as a flash memory card. The recording medium to / from which data is read / written by the media controller 100e and the hard disk 100d store image data representing an image, a program, and various data tables referred to when the program is executed. The LAN card 100f receives (that is, downloads) image data from a device (or the Internet) connected to the image generation device 100. The video card 100g is controlled by the CPU 100a to control the display on the LCD 100h. The LCD 100h and the keyboard 100i provide a user interface.

  The image generation apparatus 100 executes image generation processing for generating a lens array image as shown in FIG. 4 using the hardware shown in FIG. The CPU 100a of the image generation device 100 illustrated in FIG. 3 executes the image generation processing, thereby setting information acquisition unit 110, processing target pixel table generation unit 121, and integration target pixel table generation unit as illustrated in FIG. 122, functions as the image information acquisition unit 140, the trajectory identification unit 150, the image generation unit 160, the image evaluation unit 170, and the image display control unit 180, and cooperates with the hard disk 100d in FIG. It functions as an information storage unit 130 as shown in FIG.

  When the execution of the image generation process in FIG. 4 is started, the setting information acquisition unit 110 in FIG. 5A executes a first setting information acquisition process as shown in FIG. 6A (step S01).

  When the first setting information acquisition process in FIG. 6A is started, the setting information acquisition unit 110, based on the signal output from the keyboard 100i in FIG. 3, the main lens LM and the lens array LA set by the user. Distance information representing the distance Zma is acquired (step S11). Next, the setting information acquisition unit 110 similarly acquires focal length information indicating the focal length fa (hereinafter, simply referred to as the focal length of the lens array LA) fa of the microlens constituting the lens array LA set by the user. (Step S12). Similarly, the setting information acquisition unit 110 acquires size information indicating the diameters d of the lenses constituting the lens array LA (step S13). Thereafter, similarly, the setting information acquisition unit 110 acquires lens number information indicating the number N of lenses constituting the lens array LA (step S14).

  Thereafter, the setting information acquisition unit 110 uses the information (distance information, focal length information, size information, and lens number information) acquired in steps S11 to S14 as the first setting information, and the information storage unit 130 in FIG. (Step S15), the first setting acquisition process is terminated.

  Here, the information storage unit 130 in FIG. 5A stores information representing a range of pixels to be processed (hereinafter referred to as processing target pixels) for processing to calculate the pixel value of the lens array image. A processing target pixel table as shown in FIG. 7 (a) is stored. Further, the information storage unit 130 stores the first setting information and the table number of the processing target pixel table used when the setting represented by the first setting information is performed in association with each other, as illustrated in FIG. The first setting table as shown in FIG.

  Therefore, after step S01 in FIG. 4, has the processing target pixel table generation unit 121 in FIG. 5A searched for the table number from the first setting table based on the first setting information acquired in step S01? In other words, it is determined whether or not there is a processing target pixel table corresponding to the acquired first setting information (step S02). At this time, if it is determined that there is no processing target pixel table corresponding to the first setting information (step S02; No), the processing target pixel table generation unit 121 executes processing target pixel table generation processing as shown in FIG. (Step S03). 5A includes a first calculation unit 121a and a first storage unit 121b as illustrated in FIG. 5B.

  When the execution of the processing target pixel table generation process of FIG. 8 is started, the first calculation unit 121a sets the distance Zad between the lens array LA and the imaging element surface DP to the focal distance fa of the lens array LA, and then the set distance. Based on Zad and the distance Zma acquired in step S11 of FIG. 6A, the distance Zmd between the main lens LM and the imaging element surface DP is calculated (step S21).

  Next, as shown in FIG. 9, the first calculation unit 121a passes through the center position OM of the main lens LM for one of the lenses constituting the lens array LA and then passes through the center position OM without passing through the lens Li. The size (hereinafter referred to as the first projection size) dp of the lens Li projected by the light reaching the DP is calculated (step 26). Specifically, the first calculation unit 121a acquires the distance Zma acquired in step S11 of FIG. 6B, the distance Zmd calculated in step S21, and the step S13 of FIG. 6B. The first projection size dp is calculated by using the micro lens size d in the following equation (1) (step S22).

  dp = d × Zmd / Zma (1)

  Here, the range in which the light that has passed through the main lens LM is imaged by the microlens Li (hereinafter simply referred to as the imaging range) is the position where the center position OA of the microlens Li is projected onto the imaging element surface DP. Since this is a range at a distance of the first projection size dp from OD (hereinafter referred to as the first projection position), the pixels included in this range are set as processing target pixels, and the range including the processing target pixels is set as a processing target pixel range. To do.

  For this reason, the first calculation unit 121a calculates the relative range of the processing target pixel using the coordinate axes Xr and Yr of the relative coordinate system with the first projection position as the origin as shown in FIGS. A processing target pixel relative range table (hereinafter referred to as a first relative range table) that stores coordinate values to be expressed is generated using the first projection size dp (step S23).

  Specifically, as shown in FIG. 10A, the first calculation unit 121a is one of the regions obtained by dividing the image sensor DC into 16 regions, and the center position OA is located in the region identified by the index number “1”. If the first projection position OD is included, the distance Lce between the center Oc of the region including the first projection position OD and the center Oe of the image sensor DE is smaller than half of the first projection size dp. In other words, since the image sensor DE satisfies the following conditional expression (2), it is determined that the image sensor DE is included in the processing target pixel range. Therefore, the first calculation unit 121a identifies an area in which the coordinate value (−2, −3) representing the position of the image sensor DE with the position of the image sensor DC as the origin is included in the first projection position OD. The index “1” is associated and stored in the first relative range table as shown in FIG. Note that the first calculation unit 121a also performs the same processing as described above for other image sensors that are at a predetermined distance from the image sensor DC.

  Lce <dp / 2 (2)

  On the other hand, if the first projection position OD is included in the region as shown in FIG. 10B, the first calculation unit 121a determines the center Oc of the region including the first projection position OD. Since the distance Lce from the center Oe of the image sensor DE is more than half of the projection size dp (that is, the above conditional expression (2) is not satisfied), the image sensor DE is not included in the processing target pixel range. Determine. Note that the first calculation unit 121a performs the same process for the remainder of the 16-divided area.

  After step S23, the first calculation unit 121a initializes the target lens number for identifying the microlens to be processed in steps S26 to S29 as the value “1” (step S24). The microlens is identified by any one of the numbers from “1” to “N” acquired in step S14 in FIG.

  Next, the first calculation unit 121a determines whether or not the value of the target lens number is “N”, which is the maximum value (step S25). At this time, if it is determined that the value of the target lens number is not “N” (step S25; No), the first calculation unit 121a first identifies the first microlens (hereinafter simply referred to as the target lens) identified by the target lens number. The projection position OD is calculated. Specifically, the first calculation unit 121a sets, as the first projection position OD, a point where a straight line passing through the center position OA of the target lens and the center position OM of the main lens LM intersects the imaging element surface DP. The position coordinates of the OD are calculated using the following formula (3) using the distance Zma acquired in step S11 of FIG. 6A and the distance Zmd calculated in step S21.

(X _OD , Y _OD ) = (X _OA , Y _OA ) × Zmd / Zma (3)

However, (X _OD , Y _OD ) represents the XY coordinates of the first projection position OD, and (X _OA , Y _OA ) represents the XY coordinates of the center position OA. As shown in FIG. 9, the origin of the XYZ coordinate axes is the center position OM of the main lens LM, the X-axis direction is the main scanning direction (that is, the horizontal direction), and the Y-axis direction is the sub-scanning direction (that is, that is, (Vertical direction). Note that the coordinate axes Xr and Yr in the relative coordinate systems of FIGS. 10A and 10B are parallel to the X axis and the Y axis of FIG. 9, respectively.

  Thereafter, the first calculation unit 121a specifies the processing target pixel range of the target lens based on the calculated first projection position OD and the first relative range table generated in step S23 (step S27). Specifically, the first calculation unit 121a identifies the image sensor DC at the first projection position OD of the target lens, and then the first projection position OD is in any of the regions obtained by dividing the identified image sensor DC into 16 parts. Specify whether it is included. Next, the first calculation unit 121a searches for one or more coordinate values representing the processing target pixel relative range from the first relative range table generated in step S23 based on the index number for identifying the specified region. . Thereafter, the first calculation unit 121a acquires one or more coordinate values representing the processing target pixel range by adding the coordinate value of the first projection position OD to the coordinate value representing the processing target pixel relative range. .

  After step S27, the first storage unit 121b in FIG. 5B associates the lens number for identifying the microlens with the coordinate value representing the processing target pixel range of the microlens, and the process in FIG. As illustrated, the information is stored in a processing target pixel table stored in the information storage unit 130. Next, the first storage unit 121b stores the first setting information used for generating the processing target pixel table in association with the table number of the processing target pixel table, and the information storage unit 130 stores the first setting information in FIG. (Step S28).

  Thereafter, the first calculation unit 121a increments the target lens number by the value “1” (step S29), and then repeats the above-described processing from step S25.

  If it is determined in step S25 that the value of the target lens number is “N” (step S25; Yes), the first calculation unit 121a ends the execution of the processing target pixel table generation process.

  In step S02 of FIG. 4, after determining that there is a processing target pixel table corresponding to the first setting information (step S02; Yes) and after step S03, the setting information acquisition of FIG. The unit 110 executes a second setting information acquisition process as shown in FIG. 6B (step S04).

  When the second setting information acquisition process of FIG. 6B is started, the setting information acquisition unit 110, from the original image IO representing the subject to the main lens LM, based on the signal output from the keyboard 100i of FIG. Distance information representing the distance Zim set by the user is acquired (step S41). Similarly, the setting information acquisition unit 110 acquires distance information indicating the distance Zmp between the main lens LM and the imaging plane PI of the main lens LM (step S42). Thereafter, the setting information acquisition unit 110 stores the distance information acquired in steps S41 and S42 as second setting information in the information storage unit 130 of FIG. 5A (step S43), and then performs the second setting acquisition process. End execution.

  Here, the information storage unit 130 in FIG. 5A uses the pixel range of the pixels of the original image IO (hereinafter referred to as integration target pixels) to be subjected to integration calculation in the process of calculating the pixel value of the processing target pixel. Is stored in association with information indicating the position of the processing target pixel as shown in FIG. 7C. Further, the information storage unit 130 is an integration used when the settings represented by the first setting information and the second setting information, and the first setting information and the second setting information (hereinafter simply referred to as setting information) are performed. A second setting table as shown in FIG. 7D is stored, which is stored in association with the table number of the target pixel table.

  Therefore, after step S04 in FIG. 4, the integration target pixel table generation unit 122 in FIG. 5A has searched the table number from the second setting table based on the setting information acquired in steps S01 and S04. It is determined whether there is an integration target pixel table corresponding to the acquired setting information (step S05). At this time, if it is determined that there is no integration target pixel table corresponding to the setting information (step S05; No), the integration target pixel table generation unit 122 executes integration target pixel table generation processing as shown in FIG. Step S06). The integration target pixel table generation unit 122 in FIG. 5A includes an information search unit 122a, a second calculation unit 122b, and a second storage unit 122c as illustrated in FIG.

  When the execution of the integration target pixel table generation process shown in FIG. 12 is started, the second calculation unit 122b sets the focal length fa set by the first setting information as the distance Zad between the lens array LA and the imaging element surface D ( Step S51). Thereafter, the second calculation unit 122b determines the distance Zma between the main lens LM and the lens array LA set by the first setting information, and the main lens LM and the imaging plane PI acquired in step S42 in FIG. 6B. The distance Zpa between the image plane PI and the lens array LA is calculated from the distance Zmp to (step S52).

  Next, as shown in FIG. 1, the second calculation unit 122 b generates an image having the same size d as the lens Li from the parallel light emitted from the lenses Li constituting the lens array LA, and the image plane PI of the main lens LM. After that, an image size di (hereinafter referred to as a second projection size) di that passes through the center of the main lens LM and is projected onto the original image IO is calculated (step 53). Specifically, the second calculation unit 122b uses the distances Zim and Zmp acquired in steps S41 and S42 in FIG. 6B and the lens represented by the first setting information acquired in step S01 in FIG. The second projection size di of the lens Li is calculated using the size d of Li in the following equation (4).

  di = d × Zim / Zmp (4)

  Here, a position where light starting from a certain imaging element DC reaches the original image IO through the center position OA of the microlens Li (that is, a position where the center position OA of the microlens Li is projected onto the original image IO). Therefore, hereinafter, a range at a distance from the OI to the second projection size di (hereinafter referred to as a second projection position) is defined as a pixel range including the integration target pixel of the image sensor DC (hereinafter referred to as an integration target pixel range). This is because light starting from the integration target pixel range forms an image on the image sensor DC.

  For this reason, the second calculation unit 122b stores an integration target pixel relative range table (hereinafter referred to as a second integration target pixel relative range table) that stores coordinate values representing the relative range of the integration target pixel in the relative coordinate systems Xr and Yr with the second projection position OI as the origin. A relative range table is generated (step S54). Note that the method for generating the second relative range table is the same as the method for generating the first relative range table described in step S23 of FIG.

  Thereafter, the second calculation unit 122b initializes the target lens number for identifying the microlens to be processed in steps S56 to S62 as the value “1” (step S55). Next, the second calculator 122b determines whether or not the value of the target lens number is the maximum value “N” set by the first setting information read in Step S51 (Step S56). At this time, if it is determined that the value of the target lens number is not “N” (step S56; No), the information search unit 122a is based on the first setting information acquired in step S01 of FIG. The table number of the processing target pixel table is searched from the first setting table of a). Thereafter, the information search unit 122a searches the processing target pixel range based on the target lens number from the processing target pixel table of the searched table number (step S57).

  Thereafter, the second calculation unit 122b determines whether or not the following steps S59 to S62 have been executed for all the processing target pixels included in the detected processing target pixel range (step S58). At this time, when it is determined that the processing of steps S59 to S62 has not been executed for all the processing target pixels (step S58; No), the second calculation unit 122b selects one of the unprocessed processing target pixels. Select (step S59). Next, the second calculation unit 122b calculates the second projection position OI of the target lens identified by the target lens number (step S60).

  Specifically, the second calculation unit 122b determines the position (hereinafter referred to as a point) where a straight line passing through the selected processing target pixel (hereinafter referred to as a selected pixel) PD and the center position OA of the target lens Li intersects the imaging plane PI. OP (referred to as the projection position on the image plane PI) is calculated using the distances Zad and Zpa calculated in S51 and the following equation (5).

(X _OP , Y _OP ) = (X _OA , Y _OA ) + ((X _OA , Y _OA ) − (X _PD , Y _PD )) / Zad × Zpa (5)
However, (X _Op , Y _OP ) represents the XY coordinates of the projection position OP, (X _OA , Y _OA ) represents the XY coordinates of the center position OA, and (X _PD , Y _PD ) represents the XY coordinates of the selected pixel PD. Represents.

  Next, the second calculation unit 122b sets the second projection position OI as a point where a straight line passing through the projection position OP on the image plane PI and the center position OM of the main lens LM intersects the original image IO, and performs the second projection. The position coordinates of the position OI are calculated using the following formula (6) and the distances Zim and Zmp acquired in steps S41 and S42 in FIG.

(X _OI , Y _OI ) = − (X _OP , Y _OP ) × Zim / Zmp (6)
However, (X _OI , Y _OI ) represents the XY coordinates of the second projection position OI, and (X _OP , Y _OP ) represents the XY coordinates of the projection position OP.

  After that, the second calculation unit 122b selects based on the second projection position OI calculated in step S60 and the second relative range table generated in step S54, similarly to the process in step S27 of FIG. The integration target pixel range of the pixel PD is specified (step S61).

  Thereafter, the second storage unit 122c in FIG. 5C associates the coordinate value of the selected pixel PD with the coordinate value representing the integration target pixel range of the selected pixel PD, as illustrated in FIG. 7C. The information is stored in the integration target pixel table stored in the information storage unit 130. Next, in the second storage unit 122c, the information storage unit 130 stores the first setting information and the second setting information used for generating the processing target pixel table in association with the table number of the integration target pixel table. The second setting table as shown in FIG. 7D is stored (step S62). Thereafter, the process returns to step S58 and the above process is repeated.

  If it is determined in step S58 that the processing of steps S59 to S62 has been executed for all the processing target pixels (step S58; Yes), the second calculation unit 122b increments the target lens number by the value “1”. Later (step S63), the above processing is repeated from step S56.

  If it is determined in step S56 that the value of the target lens number is “N” (step S56; Yes), the first calculation unit 121a ends the integration target pixel table generation process.

  After step S06 in FIG. 4, the image information acquisition unit 140 in FIG. 5A acquires original image information representing the original image IO from any of the hard disk 100d, the media controller 100e, and the LAN card 100f in FIG. (Step S07). Thereafter, the trajectory specifying unit 150 and the image generating unit 160 in FIG. 5A generate an array image based on the acquired setting information and original image information representing the original image, as shown in FIG. Generation processing is executed (step S08). The trajectory specifying unit 150 includes a first specifying unit 151 and a second specifying unit 152 as shown in FIG.

  When the execution of the array image shown in FIG. 13 is started, the trajectory specifying unit 150 in FIG. 5A initializes the target lens number for identifying the microlens to be processed in steps S73 to S78 as the value “1”. (Step S71). Next, the trajectory specifying unit 150 determines whether or not the value of the target lens number is the maximum value “N” (step S72). At this time, if it is determined that the value of the target lens number is not “N” (step S72; No), the first specifying unit 151 included in the trajectory specifying unit 150 is based on the first setting information, as shown in FIG. The table number of the processing target pixel table is searched from the first setting table of FIG. 7, and the table number of the integration target pixel table is searched from the second setting table of FIG. 7D based on the setting information. Thereafter, the first specifying unit 151 searches the processing target pixel range based on the target lens number from the processing target pixel table of the searched table number (step S73).

  Thereafter, the second specifying unit 152 included in the trajectory specifying unit 150 determines whether or not the following steps S75 to S78 have been performed for all the processing target pixels included in the detected processing target pixel range (step S75). S74). At this time, when it is determined that the processing of steps S75 to S78 has not been performed for all the processing target pixels (step S74; No), the second specifying unit 152 selects one of the unprocessed processing target pixels. Select (step S75). Next, the second specifying unit 152 searches the integration target pixel range from the integration target pixel table of the table number searched above based on the coordinate value of the selected processing target pixel (hereinafter, selected pixel). (Step S76).

  Thereafter, the image generation unit 160 performs an integration operation on the pixel values of the original image IO included in the searched integration target pixel range (step S77), and uses the calculated integration value as the pixel value of the selected pixel ( Step S78). Thereafter, the above process is repeated from step S74.

  If it is determined in step S74 that the processing of steps S75 to S78 has been executed for all the processing target pixels (step S74; Yes), the trajectory specifying unit 150 increments the target lens number by the value “1”. (Step S79), the above process is repeated from Step S72.

  If it is determined in step S72 that the value of the target lens number is “N” (step S72; Yes), the trajectory specifying unit 150 ends the execution of the array image generation process.

  After step S08 in FIG. 4, the image evaluation unit 170 evaluates the visibility of the lens array generated in step S08 (step S09). Specifically, the image evaluation unit 170 calculates the contrast for each imaging range of each microlens, and evaluates the array image using the calculated average value of the plurality of contrasts. The image evaluation unit 170 determines that setting information used for generating an array image having a higher average contrast value is used for designing an imaging apparatus having a lens array preferentially, and gives priority to the calculated average contrast value. Rank. This is because the higher the contrast, the easier it is for the user to visually recognize the lens array image. Thereafter, the image evaluation unit 170 includes array image information representing the lens array image, original image information representing the original image IO, priority order information representing the priority order, and setting information used for generating the lens array image. The information is stored in the information storage unit 130 in FIG.

  After that, the image display control unit 180 in FIG. 5A performs the original image information representing the original image acquired in step S07, the image information representing the array image generated in step S08, and the current image. Any two or more of the priority order calculated in step S09 (that is, the evaluation result) and the setting information acquired in steps S01 and S04 of this time are associated with each other and displayed on the LCD 100h in FIG. S10). Further, the image display control unit 180 associates the array image information stored in the information storage unit 130 with the priority information indicating the priority higher than the user-specified value input from the keyboard 100i in FIG. Any two or more of the image information, the priority information, the first setting information, and the second setting information may be associated with each other and displayed on the LCD 100h. Thereafter, the image display control unit 180 ends the execution of the image generation process.

  According to these configurations, the distance Zad between the microlens array LA and the imaging element surface DP is set as the focal length fa of the microlens, and the image of the light that forms an image of the locus of light incident in parallel to the microlens. Since tracing is performed from the position of the element DP, a lens array image can be generated with a small amount of calculation. This is because light rays incident parallel to the lens are imaged at the focal length of the incident lens.

  Further, according to these configurations, after the imaging range (that is, the processing target pixel range) of the lenses constituting the lens array LA is specified, an image is formed at the position of the image sensor included in the specified imaging range. A pixel range (that is, an integration target pixel range) including the starting point of the light locus is specified, and a pixel value of the image sensor is calculated based on the pixels of the original image IO included in the specified pixel range. For this reason, the number of times of the arithmetic processing can be reduced to an integrated value of the number of pixels of the image sensor and the number of pixels of the original image IO included in the integration target pixel range. Therefore, for example, based on the trajectory result of tracing the trajectory of light from the pixels of the original image IO, it is determined whether or not each of the imaging elements is included in the blurs BD1 to BD3 of the lens array LA as shown in FIG. Unlike the conventional method that requires processing to be performed, it is not necessary to perform the calculation by the number of times of integration of the number of pixels of the original image IO and the number of pixels of the image sensor, so that it is necessary to generate a lens array image. Calculation amount and calculation time can be reduced.

  Further, according to these configurations, after the first calculation unit 121a calculates the imaging range (that is, the processing target pixel range) of the microlens LA that constitutes the lens array LA, the information indicating the calculated imaging range is temporarily obtained. If the first storage unit 121b associates the information indicating the focal length fa and the information indicating the lens size d with each other and stores them in the data table shown in FIGS. 7A and 7B, the same focus will be obtained next time. When the distance fa and the lens size d are set, it is not necessary to calculate the imaging range again. For this reason, it is possible to reduce the amount of calculation required to generate the lens array image.

  Further, according to these configurations, the trajectory specifying unit 150 uses the data table shown in FIGS. 7A and 7B based on the information indicating the focal length fa and the information indicating the lens size d, that is, the imaging range (that is, Since the trajectory of the light imaged in the searched imaging range is specified after searching for information representing the processing target pixel range), the amount of calculation required to specify the trajectory can be reduced. Furthermore, according to these configurations, after the second calculation unit 122b calculates the pixel range (that is, the integration target pixel range) for each image sensor included in the imaging range, the information and the focus indicating the calculated pixel range. If the second storage unit 122c associates the information indicating the distance fa and the information indicating the lens size d and stores them in the data table shown in FIGS. 7C and 7D, the same focal length fa will be stored next time. When the lens size d is set, it is not necessary to calculate the pixel range again. For this reason, it is possible to reduce the amount of calculation required to generate the lens array image.

  Further, according to these configurations, the lens array image, the evaluation result of the lens array image, and the setting information indicating the focal length fa used for generating the lens array image are displayed in association with each other. The setting information of the lens array LA that can generate a lens array image with good visibility can be easily confirmed.

  It should be noted that not only can an image generating apparatus provided with a configuration for realizing the functions according to the present invention be provided in advance, but also an existing image generating apparatus can be made to function as the image generating apparatus according to the present invention by applying a program. . That is, by applying an image generation program for realizing each functional configuration by the image generation apparatus 100 exemplified in the above embodiment so that a computer (such as a CPU) that controls the existing image generation apparatus can be executed. The image generating apparatus 100 according to the invention can function. The image generation method according to the present invention can be implemented using the image generation apparatus 100.

  Such a program distribution method is arbitrary. For example, the program can be distributed by being stored in a recording medium such as a memory card, a CD-ROM, or a DVD-ROM, or via a communication medium such as the Internet. .

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

IO ... Original image, LM ... Main lens, PI ... Imaging surface, LA ... Lens array, DP ... Image sensor surface, 100 ... Image generating device, 100a ... CPU , 100b ... ROM, 100c ... RAM, 100d ... hard disk, 100e ... media controller, 100f ... LAN card, 100g ... video card, 100h ... LCD, 100i ... Keyboard: 110 ... Setting information acquisition unit, 121 ... Processing target pixel table generation unit, 121a ... First calculation unit, 121b ... First storage unit, 122 ... Integration target pixel table generation unit , 122a... Information search unit, 122b... Second calculation unit, 122c... Second storage unit, 130... Information storage unit, 140. · Locus identification unit 151 first specific portion ..., 152 ... second specific section, 160 ... image generation unit, 170 ... image evaluation unit, 180 ... image display control unit

Claims (8)

Image information acquisition means for acquiring original image information representing the original image;
A lens array configured by an array of a plurality of lenses, which represents a focal length of a lens array that forms an optical image of a main image of an original image represented by original image information acquired by the image information acquisition unit Setting information acquisition means for acquiring setting information;
A trajectory that identifies the trajectory of light that is imaged at the position of the image sensor separated from the lens array by the focal length represented by the setting information acquired by the setting information acquisition means from the position of the image sensor. Specific means,
Based on the locus specified by the locus specifying means, a pixel value output from the image sensor is calculated, thereby generating a lens array image in which the lens array forms an optical image of the original image by the main lens. Image generating means,
An image generation apparatus characterized by that. The setting information acquisition means further acquires setting information indicating the size of the lenses constituting the lens array,
The locus specifying means includes
Based on the focal length and size of the lens array represented by the setting information acquired by the setting information acquisition means, a first specification for specifying an imaging range in which light is imaged by the lenses constituting the lens array Means,
The pixel range of the original image including the starting point of the light imaged at the position of the image sensor included in the imaging range specified by the first specifying means has passed through the center of the lenses constituting the lens array. A second specifying means for specifying a trajectory of light imaged later at the position of the image sensor from the position of the image sensor to the original image based on the size of the lens;
The image generating means calculates a pixel value of the image sensor based on a value of a pixel of the original image included in a pixel range specified by the second specifying means;
The image generating apparatus according to claim 1. Corresponding to the imaging range information representing the imaging range in which the lens of the size forms light, the focal length information representing the focal length of the lens array and the size information representing the size of the lens constituting the lens array Further comprising information storage means for storing information;
The first specifying means included in the trajectory specifying means is stored in the information storage means based on focal length information indicating a focal length set by the setting information acquired by the setting information acquiring means and size information indicating a size. Identify the imaged range information
The image generating apparatus according to claim 2. Based on the focal length and size of the lens constituting the lens array represented by the setting information acquired by the setting information acquisition means, the imaging range in which light is imaged by the lenses constituting the lens array is calculated. First calculating means for
The image forming range information representing the image forming range calculated by the first calculating means is stored in the information storing means in association with the focal length information representing the focal distance and the size information representing the size of the lens. 1 storage means is further provided,
The image generating apparatus according to claim 3. Information retrieval means for retrieving imaging range information stored in the information storage means based on the focal length and size of the lens represented by the setting information obtained by the setting information obtaining means;
The pixel range of the original image including the starting point of the light imaged at the position of the image sensor included in the imaging range represented by the imaging range information searched by the information search means is set to the center of the lens. Second calculation means for calculating a trajectory of light imaged at the position of the image sensor after passing based on a result of tracing from the position of the image sensor to the original image and a size of the lens;
A second storage unit that stores the pixel range information representing the pixel range calculated by the second calculation unit and the element identification information for identifying the imaging element in association with each other and stored in the information storage unit;
The second specifying means included in the locus specifying means includes pixel range information stored in the information storage means based on element identification information for identifying an image pickup element included in the imaging range specified by the first specifying means. Identify the pixel range represented by
The image generation apparatus according to claim 3, wherein the image generation apparatus is an image generation apparatus. Image evaluation means for evaluating the visibility of the lens array image based on the contrast of the lens array image generated by the image generation means;
Any two or more of the evaluation result by the image evaluation unit, the lens array image evaluated by the image evaluation unit, and the focal length of the lens array used to generate the lens array image are displayed in association with each other. A display means,
The image generation apparatus according to claim 1, wherein the image generation apparatus is an image generation apparatus. A computer for controlling the image generating device;
Image information acquisition means for acquiring original image information representing the original image;
A lens array configured by an array of a plurality of lenses, which represents a focal length of a lens array that forms an optical image of a main image of an original image represented by original image information acquired by the image information acquisition unit Setting information acquisition means for acquiring setting information;
A trajectory that identifies the trajectory of light that is imaged at the position of the image sensor separated from the lens array by the focal length represented by the setting information acquired by the setting information acquisition means from the position of the image sensor. Specific means,
Based on the locus specified by the locus specifying means, a pixel value output from the image sensor is calculated, thereby generating a lens array image in which the lens array forms an optical image of the original image by the main lens. Function as image generation means,
An image generation program characterized by that. An image generation method in an image generation apparatus,
An image information acquisition step for acquiring original image information representing the original image;
A lens array configured by an array of a plurality of lenses, which represents a focal length of a lens array that forms an optical image of a main image of an original image represented by original image information acquired by the image information acquisition unit A setting information acquisition step for acquiring setting information;
A trajectory that identifies the trajectory of light that is imaged at the position of the image sensor separated from the lens array by the focal length represented by the setting information acquired in the setting information acquisition step from the position of the image sensor Specific steps,
A pixel value output from the image sensor is calculated based on the locus specified in the locus specifying step, thereby generating a lens array image in which the lens array forms an optical image of the main image of the main lens. An image generation step,
An image generation method characterized by the above.

Images