JP2012043004A - Light source direction specifying device and light source direction specifying program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To specify the direction of a light source with few components and a simple process.SOLUTION: A digital photo frame includes: a cylindrical lens hood to which light emitted from a light source 40 enters from an opening of one end and has a peripheral inner wall of the opening lit by the incident light; an image pickup apparatus 30 disposed at the other end side of the lens hood and capturing an image of the opening of the lens hood and the inner wall connected to the opening; and a light source direction specifying unit specifying a bright region R3 having brightness of a prescribed standard or more in the inner wall in a captured image 90c based on the captured image 90c captured by the image pickup apparatus 30 and specifying the direction of the light source 40 based on the specified bright region R3. Specifically, the direction of the light source 40 is specified based on relative coordinates of a point B3 of the bright region R3 to a central point P.

Description

  The present invention relates to a light source direction specifying device and a light source direction specifying program for specifying the direction of a light source.

  For example, in order to add a shadow or the like to an image according to the position of the light source, a technique for estimating the position of the light source is required. As a technique for estimating the position of a light source, for example, in Patent Document 1, in a light source position estimation device that estimates the position of a light source arranged in an illumination space, a light source and a calibration tool composed of a calibration pattern and a three-dimensional object are used. A camera parameter calculation unit that inputs an image of the configured illumination space and calculates a camera parameter from a camera calibration pattern image included in the illumination space image, and a two-dimensional image in the shadow of the three-dimensional object using the camera parameter A light source position estimation device comprising shadow coordinate calculation means for calculating three-dimensional world coordinates from coordinates, and light source position estimation means for estimating world coordinates of a light source based on the world coordinates of a solid object and the world coordinates of the shadow. It is disclosed.

JP 2008-107087 A

  However, in the light source position estimation device disclosed in Patent Document 1, a component tool composed of a plurality of three-dimensional objects must be arranged at a position where light emitted from the light source strikes, and the direction of the light source is specified. There were restrictions on the configuration of the device.

  The present invention has been made in view of such a problem, and an object thereof is to specify the direction of a light source with a small number of components.

In order to solve the above-described problem, a light source direction identification device according to the first aspect of the present invention includes:
A light introducing portion having a hole at both ends and light emitted from a light source is incident on the inside of the hole from an opening at one end, and the incident light illuminates the periphery of the opening at the one end on the inner wall of the hole When,
An imaging device that is disposed on the opening side of the other end of the light introducing portion and that images the inner wall connected to the opening of the one end and the opening of the one end of the light introducing portion;
Based on the first image captured by the imaging device, a bright area having a brightness equal to or higher than a predetermined reference in the inner wall in the first image is identified, and the light source is based on the identified bright area. A light source direction specifying part for specifying the direction of
Is provided.

Moreover, the light source direction specifying program according to the second aspect of the present invention includes:
A light introducing portion having a hole at both ends and light emitted from a light source is incident on the inside of the hole from an opening at one end, and the incident light illuminates the periphery of the opening at the one end on the inner wall of the hole When,
A computer that controls an apparatus including: an imaging device that is disposed on an opening side of the other end of the light introducing unit and that images the inner wall connected to the opening of the one end of the light introducing unit and the opening of the one end;
Based on the first image captured by the imaging device, a bright area having a brightness equal to or higher than a predetermined reference in the inner wall in the first image is identified, and the light source is based on the identified bright area. A light source direction specifying step for specifying the direction of
To do.

  According to the present invention, the direction of the light source can be specified with a small number of components.

1 is a perspective view showing a digital photo frame according to an embodiment of the present invention. It is a schematic sectional drawing for demonstrating an imaging device and a lens hood in the digital photo frame of FIG. It is a figure for demonstrating the structure of a digital photo frame. It is a figure for demonstrating the hardware constitutions of a digital photo frame. (A) And (B) is a schematic diagram which shows an imaging device and a lens hood in FIG. (A) is a schematic diagram which shows the light emission state of the light source arrange | positioned so that the angle (alpha) 1 may be comprised with respect to the axis line of a lens hood. (B) is a schematic diagram illustrating an image captured by the imaging device in the state illustrated in (A). (C) is a schematic diagram showing a light emission state of a light source arranged so as to form an angle α2 with respect to the axis line of the lens hood. (D) is a schematic diagram illustrating an image captured by the imaging device in the state illustrated in (C). (A) is a figure which shows the image which binarized the image shown by FIG. 6 (B). FIG. 6B is a diagram illustrating an image obtained by binarizing the image illustrated in FIG. It is a schematic diagram for demonstrating the state which has a light source in the position of azimuth | direction angle (beta) 1. (A) is a schematic diagram which shows the relationship between the light source, imaging device, and lens hood which are arrange | positioned at azimuth | direction angle (beta) 2. (B) is a schematic diagram illustrating an image captured by the imaging device in the state illustrated in (A). (C) is a diagram showing an image obtained by binarizing the image shown in (B). It is a schematic diagram which shows the image imaged by the imaging device. It is a flowchart for demonstrating the light source direction specific process which a digital photo frame performs.

  Hereinafter, an embodiment of the present invention will be specifically described with reference to the accompanying drawings.

  The digital photo frame (light source direction specifying device) 10 according to the present embodiment specifies the direction of the light source that illuminates the digital photo frame 10, and displays the painting-like image displayed on the monitor 20 according to the specified direction of the light source. The following description will be given on the assumption that it has a function of adding shading or the like (for example, shading and gloss, or only shading).

  As shown in FIGS. 1 and 2, the digital photo frame 10 according to the present embodiment includes a housing 100 and a monitor (display unit) incorporated in the housing 100 so as to be exposed on the surface 100 a side of the housing 100. ) 20 and the imaging device 30 incorporated in the housing 100 at a position above the monitor 20.

  The imaging device 30 is disposed in a hole 100 b formed above the monitor 20 of the housing 100. As will be described in detail later, the imaging device 30 captures an image serving as a basis for specifying the direction of the light source and the like. The imaging direction of the imaging device 30 is directed to the surface 100a side of the housing 100 (see FIG. 2).

  In addition, a lens hood 32 (light introducing portion) having an opening at both ends whose inner wall 328 extends from the imaging device 30 to the surface 100 a of the housing 100 is disposed in the hole 100 b of the housing 100. In the present embodiment, the lens hood 32 is configured by a cylindrical body made of synthetic resin or the like. The inner wall of the cylindrical body constitutes the inner wall 328 of the hole. The color of the inner wall 328 of the lens hood 32 is desirably white in order to make it easier to specify the color emitted by the light source in the processing described later. The opening 324 on one end side of the lens hood 32 reaches the surface 100 a of the housing 100, and the opening 326 on the other end side surrounds the lens of the imaging device 30. The lens hood 32 is disposed in the housing 100 by being fitted into the hole 100b.

  Further, as shown in FIG. 3, the digital photo frame 10 further includes a painting tone conversion unit 52, a light source direction specifying unit 62, and an optical processing unit 60. Although not particularly mentioned in the following description, the data format of image data is appropriately changed (compressed, decompressed, etc.) in processing performed by each unit.

  The light source direction specifying unit 62 controls the imaging device 30 to cause the imaging device 30 to perform imaging, and captured image data representing the captured image captured by the imaging device 30 (for example, image data in a YUV (luminance color difference) data format). To get. The light source direction specifying unit 62 generates data such as the direction of the light source based on the acquired captured image data. In this way, the light source direction specifying unit 62 acquires the captured image, and specifies the direction of the light source based on the acquired captured image.

  The painting style conversion unit 52 acquires original image data representing an original image described later, performs a known image conversion process on the acquired original image data, and converts the original image into a painting style (for example, oil painting style). Painting image data representing the converted painting image is generated. Note that the painting-like image data here also includes three-dimensional data that is data such as the concavo-convex shape of the surface of the image (for example, the concavo-convex shape of an oil painting) (note that this three-dimensional data is the original image data). It may be prepared in advance corresponding to the original image). In this way, the original image is converted into a painting-like image.

  The optical processing unit 60 acquires the light source information (information on the direction of the light source and the illuminance of light emitted from the light source) generated by the light source direction specifying unit 62 and the painting-like conversion generated by the painting-like conversion unit 52. Get image data. Then, based on the acquired data (particularly, three-dimensional data), predetermined known processing is performed, and shaded painting-like image data representing a shaded painting-like image obtained by adding a shadow or the like to the painting-like image is obtained. Generate. In this way, the optical processing unit 60 adds a shadow or the like to the painting-like image converted by the painting-like conversion unit based on the position in the light source direction specified by the light source direction specifying unit 62.

  The optical processing unit 60 supplies the shaded painting-like image data generated above to the monitor 20. The monitor 20 performs a predetermined operation based on the shaded painting-like image data supplied from the optical processing unit 60, and displays the shaded image represented by the shaded painting-like image data. In this way, the optical processing unit 60 displays the shaded image on the monitor 20 based on the shaded painting-like image data.

  As shown in FIG. 4, the digital photo frame 10 includes a CPU (Central Processing Unit) 64, a storage unit (storage unit) 66, an imaging device 30, a monitor 20, and a system bus 70 as hardware configurations. . The CPU 64 and the like are connected to the system bus 70 as appropriate.

  The storage unit 66 stores data and programs used by the CPU 64, data generated by the CPU 64, and the like. The storage unit 66 includes one or more necessary combinations such as a RAM (Random Access Memory) that functions as a main memory of the CPU 64, a flash memory that functions as an auxiliary storage device, and a hard disk. The storage unit 66 may include an external storage medium (for example, a memory card) connected to the digital photo frame 10 via a reading / writing device (for example, a card slot) (not shown), for example. That is, at least a part of the storage unit 66 may not be a component of the digital photo frame 10.

  The CPU 64 controls each component of the digital photo frame 10 according to a program recorded in the storage unit 66 (for example, read out to the RAM) and using various data recorded in the storage unit 66. Then, the light source direction specifying process described later is performed. In the present embodiment, the painting style conversion unit 52, the light source direction specifying unit 62, and the optical processing unit 60 are each configured by a CPU 64 that performs a light source direction specifying process described later according to the program. Each of the painting-like conversion unit 52, the light source direction specifying unit 62, and the optical processing unit 60 may be configured by various dedicated circuits.

  The imaging device 30 is based on an imaging device (CCD (Charge Coupled Device) image sensor, CMOS (Complementary Metal Oxide Semiconductor) image sensor, etc.) that outputs an imaging signal representing an image obtained by imaging, and an imaging signal output by the imaging device. Various electronic circuits (AFE (Analog Front End), DSP (Digital Signal Processor), etc.) that perform predetermined processing to generate the captured image data and supply the generated captured image data to the light source direction specifying unit 62; The imaging device 30 is controlled by the light source direction specifying unit 62 and supplies captured image data representing the captured image to the light source direction specifying unit 62. The processing performed by the DSP is a light source. The direction specifying unit 62 may perform as appropriate, in which case, the light source direction specifying unit 62 is supplied with data generated by AFE or the like from the imaging device 30, and This data is acquired by performing the same processing as the DSP on the supplied data and generating the captured image data.

  Next, a concept for specifying the direction of the light source by the light source direction specifying unit 62 and processing performed by the light source direction specifying unit 62 will be described.

  For example, an actual painting (especially an oil painting) is a three-dimensional object having irregularities such as pigments on a plane. Naturally, shadows and the like are generated on the unevenness when the painting is irradiated with light. That is, in consideration of the direction of the light source with respect to the digital photo frame 10 and the like, by adding a shadow or the like to the painting-like image displayed on the digital photo frame 10 (providing optical consistency), the painting-like image is further improved. It can be like a real painting.

  Here, when adding a shadow or the like, it is not necessary to specify an omnidirectional light source for the digital photo frame 10 as in an actual picture. This is because the light source from the back side of the painting (digital photo frame 10) has little influence on the shading formed on the uneven surface of the painting, and the light source from the front direction of the painting is also formed on the uneven surface of the painting. This is because it has little effect on shadows.

  Here, the direction with respect to the digital photo frame 10 of the light source 40 specified by the present embodiment (see FIG. 6, among the light sources that emit light, the light source that emits light that illuminates the inner wall 328 of the lens hood 32 most brightly) It shall be represented by the elevation angle α and the azimuth angle β. Here, the elevation angle α is an inclination angle of a line along the light emission direction of the light source 40 with respect to the axis 322 (line passing through the center of the circle) of the inner wall 328 (circular section) of the lens hood 32 (FIG. 6). The azimuth angle β here refers to the opening 324 with respect to a reference line (for example, a line extending upward from the center of the opening 324) when the imaging device 30 is viewed from the one end opening 324 of the lens hood 32. Of the line connecting the center of the light source and the light source 40 (see FIG. 9 and the like).

  In the present embodiment, the direction of the light source 40 in which the elevation angle α is in a range larger than 45 ° and smaller than 90 ° is specified in consideration of the above-described influence. In this embodiment, after specifying the direction of the light source 40 and the like, a process of adding a shadow or the like to the painting-like image to make it look like a more realistic painting is performed according to the specified direction. It is assumed that the slope angle (inclination angle with respect to the surface direction of the painting) due to the adhesion of the pigment in the painting does not exceed 45 °. For this reason, if the specific target range of the light source 40 is limited to the angle range, if the elevation angle α is a range smaller than 45 °, no shadow or the like occurs on the surface of the painting, and the elevation angle α is This is because if it is in a large range of 90 ° or more, light does not reach the surface of the painting, so there is no need to add a shadow or the like. The imaging device 30 and the lens hood 32 are configured so that the light source 40 located in this range can be specified. Note that the light source 40 in a direction other than the above-described elevation angle α is naturally specified depending on the assumed uneven slope angle or the like, or depending on the configuration of the imaging device 30 and the lens hood 32. May be. And in order to perform such specification, the structure (shape etc.) of the imaging device 30 and the lens hood 32 can be changed suitably.

  The imaging device 30 has an angle of view σ1 (particularly an angle of view of the elevation angle). As shown in FIG. 5A, when the inner diameter of the lens hood 32 is φ, the imaging device from the opening 324 (the surface 100a of the housing 100) on the one end side of the lens hood 32 on the inner wall 328 of the lens hood 32. The imaging device 30 and the lens hood 32 are configured such that the position where the length in the direction toward 30 is φ is positioned on a straight line that forms the angle of view σ1. Further, as shown in FIG. 5B, the maximum angle at which the imaging device 30 can capture the outside world through the opening 324 on one end side of the lens hood 32 without being blocked by the lens hood 32 is set as the outside field visual field. The angle is σ2.

  The position where the length in the direction from the opening 324 on one end side of the lens hood 32 toward the imaging device 30 is φ is positioned on a straight line constituting the field angle σ1, so that the elevation angle α is 45 °, for example. The light from the above light source falls within the region within the angle of view σ1 on the inner wall 328, and the entire region illuminated by the light source is imaged by the imaging device 30. As will be described later, since the direction of the light source is specified based on the region illuminated by the light source captured by the captured image, the direction of the light source can be specified by including the entire region in the captured image. When the elevation angle α is 45 °, for example, it is possible to limit the direction of the light source on the captured image, so that the imaging device 30 and the opening 324 can be brought close to each other by the above configuration. Thus, the area illuminated by the light source in the captured image can be enlarged, and the process of specifying the direction of the light source (for example, detection of point B described later) can be performed accurately.

  Next, FIGS. 6A and 6C show a state in which the light source 40 is arranged in the directions of the elevation angles α1 and α2 (for convenience of explanation, the azimuth angle β = 0). . FIGS. 6B and 6D show images (first images) 90 (90a, 90b) captured by the imaging device 30 in these states. In addition, an image in a circle shown in the center of FIGS. 6B and 6D is an external image 50 that is within the range of the external field viewing angle σ2 (from the imaging device 30 on one end side of the lens hood 32). (Image taken outside the lens hood 32 through the opening 324). Shown on the outside of the external image 50 is a hood image region 320 in which the inner wall 328 of the lens hood 32 is reflected. In the hood image region 320, a portion where the light emitted from the light source 40 does not reach is shown in black, and a portion where the light emitted from the light source 40 reaches (irradiated portion) is shown in white. Note that the captured images 90a and 90b shown in FIGS. 6B and 6D do not have an indirect light source, which will be described later, in order to facilitate understanding.

As shown in FIGS. 6A and 6C, a direction parallel to the axial direction of the lens hood 32 from one end of the lens hood 32 in a portion of the inner wall 328 of the lens hood 32 that is directly illuminated by the light source 40. If the maximum length to the edge of the portion directly illuminated by the light source 40 is L (L1, L2), it goes without saying that the elevation angle α (α1, α2) is tan ⁻¹ (φ / L ) That is, the light source direction specifying unit 62 obtains the maximum length L and substitutes it in this equation together with the constant φ (known value) to specify the elevation angle α. Although the details will be described later, the end point of the length L is the position farthest from the center of the opening 324 among the edges of the region R01 in the captured images 90a and 90b shown in FIGS. 6B and 6D. Correspond to the points in

  Next, a method for actually specifying the length L will be described. The length L is specified by the light source direction specifying unit 62 performing processing based on the captured image 90 captured by the imaging device 30. First, the light source direction specifying unit 62 acquires a captured image 90 from the imaging device 30, and extracts the external image 50 from the captured images 90 a and 90 b shown in FIGS. 6B and 6D for the captured image 90. The image processing target is limited to the hood image region 320. Since the range of the external image 50 in the captured image 90 is specified in advance according to the inner diameter φ of the lens hood 32, the light source direction specifying unit 62 deletes the numerical data of the corresponding part on the image data representing the captured image 90. Thus, the external image 50 can be easily deleted from the captured image 90.

  Next, as shown in FIGS. 7A and 7B, the light source direction specifying unit 62 binarizes the captured images 90a and 90b. At this time, the light source direction specifying unit 62 describes the luminance value of each pixel in the captured image (hereinafter, the brightness level detected from the pixel data is described as brightness, and the brightness level illuminating the object is described below. Specifically, the light source direction specifying unit 62 uses a threshold value for brightness values (brightness information) such as illuminance) for each pixel. The captured images 90a and 90b are binarized with white for pixels and black for pixels whose luminance value is less than the threshold. In the binarized images (monochrome images) 92a and 92b generated by the binarization process, the bright region R (R1, R2) is clearly separated from other regions. Here, the bright region R is a region corresponding to the regions R01 and R02 in the captured image 90.

  The bright region R is a set of pixels having a luminance value equal to or higher than a threshold value, and is an image region obtained by photographing a portion having brightness (illuminance) of a predetermined level or more on the inner wall 328 of the lens hood 32 in the real space. In the present embodiment, the portion having the above-described brightness is treated as a portion directly illuminated by the light from the light source 40, and the bright region R, which is an image region obtained by imaging this portion, is directly used as the light source. A portion directly illuminated by light from 40 is treated as an image area taken. The threshold value is set in advance as a numerical value that allows the bright region R to be handled as such a region.

  Next, the light source direction specifying unit 62 obtains the distance from the center point P to the edge of the bright region R for each predetermined angle. Here, the center point P is at the same position as the center of the opening 324. The light source direction specifying unit 62 uses the point of the edge of the bright region R, which is the longest distance among the obtained distances, as a point B (point B1, point B2, point B3), and uses the point P on the image as a reference. The relative coordinates of the point B are detected. Here, the light source 40 is on a straight line passing through the point B and the center point P, and is located on the opposite side to the point B side with respect to the center point P.

  Next, the light source direction specifying unit 62 specifies the length L from the detected point B. The light source direction specifying unit 62 may specify the length L by calculating the length L using an equation using a trigonometric function or the like based on the positional relationship. In consideration of the influence on accuracy due to distortion or the like, a table created in advance and recorded in the storage unit 66 in which the positional relationship and the length L are associated may be referred to. The light source direction specifying unit 62 specifies the length L by referring to the table based on the positional relationship detected above and acquiring the length L corresponding to the detected positional relationship. As this table, for example, there is an LUT (Look Up Table) in which the correspondence between the relative coordinates and the length L is mapped. For example, the light source direction specifying unit 62 refers to the LUT based on the coordinates (0, Y1) of the point B1 in the binarized image 92a shown in FIG. ) Is specified. Further, by referring to the LUT based on the coordinates (0, Y2) of the point B2 in the binarized image 92b shown in FIG. 7B, the length shown in FIG. L2 is specified. Incidentally, as shown in FIG. 7, the magnitude relationship of the Y coordinate between the point B1 and the point B2 is | Y1 | <| Y2 |, and the relationship of the length L obtained at this time is the length L1 <length. L2. The light source direction specifying unit 62 specifies the elevation angle α by substituting the obtained length L into the above formula. Further, the light source direction specifying unit 62 may specify the elevation angle α with reference to a table in which the positional relationship or the length L and the elevation angle α are associated with each other.

  Next, a method for specifying the azimuth angle β of the light source 40 will be described with reference to FIGS. 8 and 9. Here, for convenience, as shown in FIGS. 8 and 9A, the azimuth angle β is clockwise from a straight line extending vertically from the center point P when the imaging device 30 is viewed from the opposite side of the imaging direction. The angle at which the value increases. The azimuth angle β1 of the light source 40 in the state shown in FIGS. 6 and 7 is 0 ° as shown in FIG. 9A to 9C show the light source 40 arranged in the direction of the azimuth angle β2, the captured image (first image) 90c and the binarized image 92c at that time. As shown in FIG. 9B, in the captured image 90c by the imaging device 30, the irradiated portion extends to the upper right. The light source direction specifying unit 62 specifies a point B3 of relative coordinates (X3, Y3) from the binarized image 92c obtained by binarizing the captured image 90c as shown in FIG. 9C.

The light source direction specifying unit 62 specifies the azimuth angle β by any one of the following formulas (1) to (6) by specifying the relative coordinates (X3, Y3).
(1) When the coordinates of the point B are (0 <X, 0 ≦ Y) (that is, when the point B is in the first quadrant with respect to the center point P in the binarized image 92c in FIG. 9C). , Azimuth β = | tan ^-1 (Y / X) | + π / 2
(2) When the coordinates of the point B are (X = 0, 0 <Y) (that is, when the point B is at a positive position on the Y axis in the binarized image 92c in FIG. 9C). Azimuth β = π
(3) When the coordinates of the point B are (X <0, 0 <Y) (that is, when the point B is in the second quadrant with respect to the center point P in the binarized image 92c in FIG. 9C). , Azimuth β = | tan ^-1 (Y / X) | + π
(4) When the coordinates of the point B are (X = 0, Y <0) (that is, when the point B is in the negative position on the Y axis in the binarized image 92c in FIG. 9C). Azimuth β = 0
(5) When the coordinates of the point B are (X <0, Y ≦ 0) (that is, when the point B is in the third quadrant with respect to the center point P in the binarized image 92c in FIG. 9C). , Azimuth β = | tan ^-1 (Y / X) | + 3 / 2π
(6) When the coordinates of the point B are (0 <X, Y <0) (that is, when the point B is in the fourth quadrant with respect to the center point P in the binarized image 92c in FIG. 9C). , Azimuth β = | tan ^-1 (Y / X) |
Further, the light source direction specifying unit 62 may associate the relative coordinates of the point B with the center point P and the azimuth angle β, and specify the azimuth angle β with reference to a previously created LUT.

  As described above, the light source direction specifying unit 62 can specify the direction of the light source 40 by specifying the elevation angle α and the azimuth angle β of the light source 40. The light source direction specifying unit 62 performs a predetermined calculation based on the elevation angle α and the azimuth angle β, and calculates a value that represents the direction of the light source 40 by another method (such as a value in a spherical coordinate system). Thus, the direction of the light source 40 may be specified.

  Although the direction of the light source 40 has been specified as described above, in order to add a shade or the like having a predetermined shade to a painting-like image, the light source direction specifying unit 62 causes the illuminance by the light of the light source 40 in the actual environment and It is necessary to specify the illuminance due to the ambient light of the indirect light source (all light sources other than the light source 40 that emits the brightest light). And the like need to be added to the picture-like image by the optical processing unit 60. Next, referring to FIG. 10, the relative illuminance of the hood image area 320 illuminated by the light of the light source 40 and the ambient light of the indirect light source (all light sources other than the light source 40 that emits the brightest light) is specified. Will be described. FIG. 10 is a diagram in which a portion illuminated by ambient light is added to the captured image 90a of FIG.

  First, the captured image 90d shown in FIG. 10 will be described. In the captured image 90d, the region illuminated by the ambient light is shown in gray and is defined as a region R03.

  Next, two luminance detection target positions will be described. First, the detection target position of the luminance in the portion illuminated by the light of the light source 40 is the edge of the irradiated portion extending farthest from the center of the opening 324 and the center of the opening 324 in the region R01 illuminated by the light source 40. It is set to the vicinity outside the opening 324 (within a predetermined distance) on the connecting line segment (point A1 in FIGS. 6B, 6D, and 10). The reason for setting the detection target position in this way is that the light incident from the opening 324 irradiates the vicinity of the outside of the opening 324 more widely than the other portions, and the detection target position has an elevation angle α of 45 °. This is because the position is always illuminated by the light source 40 at any angle within a large range smaller than 90 °.

  Next, in the captured image 90d shown in FIG. 10, the luminance detection target position in the portion illuminated by the ambient light of the indirect light source will be described. As a point at a position illuminated by the ambient light of the indirect light source, the luminance of the point C1 that is a luminance detection target for comparison with the luminance of the point A1 is compared with the luminance of the point A1 at the position illuminated by the light source 40. Since it is a target, it is necessary that the influence of light from the light source 40 is small. For this reason, the point C1 is set on the opposite side with respect to the position of the point A1 with the opening 324 as the center. By doing so, the influence of the light source 40 is such that the azimuth angle α is in a range larger than 45 ° and smaller than 90 ° (that is, light that uniformly illuminates the periphery of the opening 324 in the inner wall 328 of the lens hood is not emitted). Few. Further, the position of the point C1 is outside the opening 324 on the line segment connecting the edge of the irradiated portion extending farthest from the center of the opening 324 and the center of the opening 324 in the region R03 illuminated by the indirect light source. Near (within a predetermined distance). The reason for setting the detection target position in this way is that the light incident from the opening 324 irradiates the vicinity near the outside of the opening 324 more widely than the other portions, and the detection target position has an elevation angle α of 0 °. This is because the position is always illuminated by an indirect light source at any angle in a range smaller than 90 °.

  Next, a specific method for specifying the illuminance will be described. The light source direction specifying unit 62 specifies a point A10 at a position corresponding to the position of the point B calculated from the binarized image 92 in the captured image 90a. (Here, the threshold value of the binarization process performed on the binarized image 92 is set to a value larger than the luminance value of the ambient light of the indirect light source and smaller than the luminance value of the light of the light source 40.) In the binarized image 92, the part illuminated by the ambient light of the indirect light source is shown in black, and the part illuminated by the light from the light source 40 is shown in white. Further, a point C10 that is on a line segment connecting the point A10 and the center of the opening 324 and is symmetrical with the point 10 with respect to the center of the opening 50 is specified. Next, the light source direction specifying part 62 is on a line segment connecting the point A10 and the center of the opening 324, and is set as a point A1 in the vicinity of the outside of the opening 324 (within a predetermined distance), and the center of the point C10 and the opening 324 Is a point C1 that is on the line segment connecting the two and near the outside of the opening 324 (within a predetermined distance). The light source direction specifying unit 62 detects the luminance of the pixel at the point A1 illuminated by the light source 40 and the luminance of the pixel at the point C1 illuminated by the indirect light source in the captured image 90, compares these luminances, Identify illuminance. The light source direction specifying unit 62 determines the density of a shadow or the like added to the painting-like image by the optical processing unit 60 according to the specified relative illuminance.

  That is, in the present embodiment, the light source direction specifying unit 62 illuminates the bright area (R01, R02) from the brightness information (luminance value) of the captured image 90, for example, based on the captured image 90 by the above processing. Based on the captured image 90, the illuminance of light is specified, the illuminance of light that illuminates the outside of the bright area directly illuminated by the light source 40 (for example, the area R03) on the inner wall 328, and the light inside and outside the specified bright area is identified. Based on the illuminance, the illuminance ratio between the direct light source 40 and the indirect light source is specified.

  In the above description, the luminance detection target position is the point A1 that is directly illuminated by the light from the light source 40 and the point C1 that is illuminated by the ambient light, but is not limited thereto. For example, in order to detect the luminance in the captured image 90 of the portion directly illuminated by the light emitted from the light source 40, it may be within the bright region R01, for example, on the line segment connecting the point A10 and the center of the opening 324. It suffices if it is on the center side of the opening 324 from the point 10. Furthermore, in order to detect the luminance in the captured image 90 of the portion illuminated by the light emitted from the indirect light source, it may be within the region R03, for example, on a line segment connecting the point A10 and the center of the opening 324. The point C10 that is point-symmetric with the point 10 with respect to the center of the opening 50 may be a target for detecting the luminance.

  In addition, for example, there may be large unevenness in the luminance values in the pixels in the vicinity of the points A1 and C1, and thus the values obtained by setting a plurality of points in a predetermined range as detection targets, respectively. More preferably, the illuminance is specified based on the averaged luminance.

  Next, a detailed example of the light source direction specifying process will be described with reference to FIG. This process is started with the operation of a digital photo frame 10 performed by a user on an operation unit (not shown) for the purpose of adding a shadow to the image.

  First, the imaging device 30 images the inner wall 328 of the lens hood 32 under the control of the light source direction specifying unit 62 (step S101). Next, the light source direction specifying unit 62 binarizes the captured image 90 (step S102). The light source direction specifying unit 62 detects the point B from the binarized image 92 (step S103). Next, the light source direction specifying unit 62 specifies the length L from the LUT based on the coordinates of the point B (step S104). Next, the light source direction specifying unit 62 specifies the elevation angle α from the specified length L and the inner diameter φ (step S105). The light source direction specifying unit 62 specifies the azimuth angle β from the relative coordinates of the point B with respect to the center point P (step S106). The light source direction specifying unit 62 specifies the direction of the light source 40 from the elevation angle α and the azimuth angle β (step S107). Next, the light source direction specifying unit 62 connects the point A10 in the captured image 90 at the position corresponding to the point B in the binarized image 92 and the center of the opening 324 among the irradiated parts illuminated by the light source 40. A point A1 is set on the line segment and in the vicinity of the outside of the opening 324 (step S108), and the brightness of the set point A1 is detected (step S109). Next, the light source direction specifying unit 62 is on a line segment connecting the point C10 that is point-symmetric with the point 10 and the center of the opening 324 with respect to the center of the opening 50. C1 is set (step S110), and the brightness of the set point C1 is detected (step S111). Next, the light source direction specifying unit 62 calculates the relative luminance of the point A1 with respect to the luminance of the point C1 (step S112). The painting-like conversion unit 52 performs a painting-like process on the original image stored in the storage unit 66 to create a painting-like image having three-dimensional shape data. The optical processing unit 60 calculates relative illuminance from the brightness of the point A1 with respect to the brightness of the point C1, and based on the specified direction of the light source 40 and the three-dimensional data of the painting-like image, the predetermined length, angle, and density. A shadow or the like is added to the picture-like image (step S113).

  In addition, as software used for adding a shadow or the like corresponding to the optical processing unit 60, there is Ray Tracing Software Shade (registered trademark). In this case, the light source direction specifying unit 62 generates (converts) the data required by the software as appropriate based on the luminance ratio, the direction of the light source, and the like, so that the shadow is added to the painting-like image by the software.

  As described above, the digital photo frame 10 according to the present embodiment includes a small number of components including the imaging device 30 and the lens hood 32, and can identify the direction of the light source 40 in the real environment with a simple process. Furthermore, it is possible to reflect (add) a shadow or the like according to the specified direction and illuminance of the light source 40 in the picture-like image.

  Further, by adding a shade or the like of the density according to the relative illuminance between the light of the light source 40 and the ambient light of the indirect light source to the picture-like image, the shade or the like of the density according to the light illuminance of the light source 40 is added to the picture-like image. It is possible to generate an image reflecting the light and dark state in the real environment rather than adding to the image.

  The optical calculation according to the present embodiment may be calculated using a programming interface such as OpenGL (registered trademark). Further, only the gloss may be added to the picture-like image instead of the shadow.

  Moreover, although the structure provided with a lens hood was demonstrated in this embodiment as what introduces the light from a light source to an imaging device, this invention is not limited to this structure. In other words, the light introducing unit only needs to be able to introduce light into the part imaged by the imaging device, and has a hole with both ends opened, and light emitted from the light source is incident on the inside of the hole from one end opening. What is necessary is just a component in which the circumference | surroundings of the opening of the said one end in the inner wall of the said hole are illuminated by light. For example, the light introducing unit may be the casing 100 to which the lens hood 32 is not attached. In this case, the hole 100b corresponds to the hole.

  In the photo frame according to the above-described embodiment, it has been described that the light source direction is specified and a shadow or the like is added to the picture-like image. However, if the image has three-dimensional shape data, the shadow or the like may be used instead of the picture-like image. The present invention can also be applied to 3D-CAD images, images representing actual photographs, and the like.

  In addition, by providing a device that can detect or set the three-dimensional shape data from the image, a shadow or the like is added to the external image excluding the hood image region from the captured image captured by the imaging device. It may be used as an original image (or an image before conversion of a painting-like image) as a target.

  Furthermore, when the light source is moving, images are continuously captured, and the direction of the light source is specified based on the captured image, thereby adding shading according to the instantaneous light source direction to the painting-like image. can do. That is, it is possible to add a shadow or the like according to the direction of the moving light source to the picture-like image in real time.

  Furthermore, in the digital photo frame according to the present embodiment, the configuration in which the imaging device and the lens hood are provided at only one location on the upper portion of the monitor 20 has been described. You may do it. In this way, 3D data can be acquired based on a plurality of images taken from a plurality of angles. Furthermore, the accuracy of specifying the light source direction can be improved.

  In the digital photo frame according to the present embodiment, the light source direction is specified in order to add a shadow or the like to a picture-like image. Therefore, the light source direction only in the surface direction of the picture is specified. The invention is not limited to this configuration. For example, when the specified light source information is used for things other than painting-like images, the light source directions on the front and back surfaces and both surfaces may be specified. In this case, it is possible to specify the light source directions in the double-side direction by providing two imaging devices oriented in the opposite directions and performing the same processing. In the present embodiment, it is assumed that the slope angle of the unevenness due to the adhesion of the pigment of the painting does not exceed 45 °, and in order to make the painting-like image more realistic, it is assumed that a shadow or the like is added. It was. For this reason, the elevation angle α is described as specifying the direction of the light source 40 in a range that is larger than 45 ° and smaller than 90 °. Do not limit.

  Further, in the present embodiment, the light source direction specifying unit has been described as acquiring the luminance value and the like of the light emitted from the light source, and specifying the direction and illuminance of the light source based on the acquired luminance value and the like. The color data of light may be detected. In this case, the color data of the light emitted from the light source can detect the irradiated part of the light on the inner wall 328 from the captured image because the inner wall 328 of the lens hood 32 in the present embodiment is white. For example, the light source direction specifying unit may detect color data for both the luminance at the point A1 and the luminance at the point C1. Based on the color data, the optical processing unit can adjust the pigment of the portion estimated to be irradiated with each light in the image to be processed. Furthermore, even when the color of the ambient light is different from the color of the light of the direct light source, the pigment of the image to be processed can be adjusted so as to more closely match the actual environment.

  Furthermore, in the digital photo frame according to the present embodiment, the specified light source (light source direction / illuminance) has been described as being used to add shading of an image, but how it is used is arbitrary. is there. For example, you may make it use the information of the direction of the identified light source for the direction control of the light-receiving surface of a solar cell.

  In the present embodiment, a digital photo frame has been described as an example, but it may be used for a digital camera, a personal computer, a notebook personal computer, a PDA (personal digital assistant), or the like.

  The program may cause the CPU to perform a light source direction specifying process in cooperation with an OS (Operation System). In this case, the OS is also recorded in the storage unit. The program is recorded on a portable storage medium (for example, CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory)) and supplied (installed) to the light source direction specifying device. May be. The program may be supplied to the light source direction specifying device via a network. In these cases, the storage unit is configured by a flash memory or the like, and the program is recorded in the flash memory.

  A flash memory or a portable storage medium on which the program is recorded becomes a computer-readable program product (a storage medium readable by a computer storing the program). The program may be a program in which at least a part of the function is realized by a dedicated circuit to operate the light source direction specifying device. That is, the light source direction specifying device may be any device that performs a light source direction specifying process as a whole, and the above program only needs to operate such a light source direction specifying device.

DESCRIPTION OF SYMBOLS 10 ... Digital photo frame (light source direction identification device), 20 ... Monitor (display unit), 30 ... Imaging device (imaging device), 32 ... Lens hood (light introduction unit), 320 Hood image area, 322... Axis, 324, 326... Aperture, 40... Light source, 50 .. external image, 52. ... Light source direction specifying part, 64 ... CPU, 66 ... Storage part (storage part), 90 (90a, 90b, 90c, 90d) ... Captured image (first image), A1. Point, B (B1, B2, B3) ... Point, C1 ... Point, L (L1, L2, L3) ... Maximum length, P ... Center point, R (R1, R2, R3) ... bright region, α (α1, α2) ... elevation angle, β (β1, β2) ... azimuth angle

Claims (7)

A light introducing portion having a hole at both ends and light emitted from a light source is incident on the inside of the hole from an opening at one end, and the incident light illuminates the periphery of the opening at the one end on the inner wall of the hole When,
An imaging device that is disposed on the opening side of the other end of the light introducing portion and that images the inner wall connected to the opening of the one end and the opening of the one end of the light introducing portion;
Based on the first image captured by the imaging device, a bright area having a brightness equal to or higher than a predetermined reference in the inner wall in the first image is identified, and the light source is based on the identified bright area. A light source direction specifying part for specifying the direction of
A light source direction identification device comprising: The light source direction specifying unit specifies a point at a position farthest from the center of the opening at the one end in the bright region, and specifies the direction of the light source based on the specified position of the point.
The light source direction specifying device according to claim 1. The light source direction specifying unit specifies the illuminance and / or color of light that illuminates the inside of the bright region based on the first image.
The light source direction identification device according to claim 1 or 2, characterized in that The light source direction specifying unit specifies the illuminance and / or color of light that illuminates outside the bright region based on the first image.
The light source direction identification device according to claim 1 or 2, characterized in that The light source direction specifying unit is
Based on the first image, the illuminance of light illuminating the bright area is identified,
Based on the first image, identify the illuminance of light that illuminates outside the bright area,
Based on the illuminance of the light inside and outside the identified bright area, the illuminance ratio between the direct light source and the indirect light source is identified.
The light source direction identification device according to claim 1 or 2, characterized in that An optical processing unit that acquires a second image and performs image processing that is specified by the light source specifying unit and adds a shadow and / or gloss to the acquired second image according to at least the direction of the light source;
The light source direction identification device according to claim 1, further comprising: A light introducing portion having a hole at both ends and light emitted from a light source is incident on the inside of the hole from an opening at one end, and the incident light illuminates the periphery of the opening at the one end on the inner wall of the hole When,
A computer that controls an apparatus including: an imaging device that is disposed on an opening side of the other end of the light introducing unit and that images the inner wall connected to the opening of the one end of the light introducing unit and the opening of the one end;
Based on the first image captured by the imaging device, a bright area having a brightness equal to or higher than a predetermined reference in the inner wall in the first image is identified, and the light source is based on the identified bright area. A light source direction specifying step for specifying the direction of
A light source direction specifying program characterized in that

Images