JP2018064250A - Information processing device, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize proper exposure correction by using data of an image captured with an electronic camera.SOLUTION: A processed image generating part 41 generates data of one or more processed images from data of a source image obtained as a result of imaging with a predetermined imaging device. A discrimination condition identification part 42 identifies one or more conditions for discriminating an imaging scene of the source image from the data of one or more processed images. A scene discrimination part 43 discriminates the imaging scene of the data of the source image on the basis of the identified one or more conditions.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

2. Description of the Related Art Conventionally, electronic devices such as a digital camera, a digital video camera, and a camera-equipped communication terminal (such as a mobile phone, a smartphone, and a tablet) include an electronic camera having a lens and a CMOS (Complementary Metal Oxide Semiconductor). Such an electronic device performs image correction processing, image analysis processing, and the like on an image signal output from an electronic camera using a semiconductor such as an ISP (Image Signal Processor) (for example, Patent Document 1). , See Patent Document 2).
The electronic device performs various corrections and performs various recognition determinations using the results of such image correction processing and image analysis processing.
For example, the electronic device generally performs an average operation on the image data in each region divided by a predetermined number of blocks for automatic exposure correction in such image correction processing, and the calculated value Is used to determine the shooting scene of the data of the image captured by the electronic camera, and the correction processing is executed according to the shooting scene.

JP 2009-177399 A JP 2002-277922 A

However, in the conventional techniques including Patent Document 1 and Patent Document 2, there are many cases where the photographing scene cannot be properly determined and appropriate exposure correction cannot be performed. This is because various shooting scenes such as the influence of sunlight, other light sources, individual subject distances and reflections exist in the image captured by the electronic camera. It is difficult for conventional techniques including Patent Document 1 and Patent Document 2 to appropriately determine a shooting scene from these various shooting scenes.
For example, when the surroundings are bright and the subject is dark, such as backlighting, it is very difficult to appropriately determine the shooting scene using the result of the above average calculation.

  Further, in another conventional automatic exposure correction method, there is a case in which a method is used in which shooting is performed after a predetermined shooting scene is set. In this case, depending on the actual shooting scene, the subject, and the situation of the peripheral light source of the subject, unintentional correction may occur when a predetermined shooting scene is used.

  The present invention has been made in view of such a situation, and an object thereof is to realize appropriate exposure correction using data of an image taken with an electronic camera.

In order to achieve the above object, an information processing apparatus of one embodiment of the present invention provides:
Processed image generation means for generating one or more processed image data from original image data obtained as a result of being imaged by a predetermined imaging device;
Discrimination condition specifying means for specifying one or more conditions for discriminating a shooting scene of the original image from the data of the one or more processed images;
Scene discriminating means for discriminating the shooting scene of the original image based on the specified one or more conditions;
It is characterized by providing.

  Each of the information processing method and program as one aspect of the present invention is each of a method and program corresponding to the above-described information processing apparatus of one aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, appropriate exposure correction | amendment is realizable using the data of the image image | photographed with the electronic camera.

It is a block diagram which shows the hardware constitutions of the information processing apparatus which concerns on one Embodiment of this invention. It is a flowchart explaining the exposure appropriate process for every scene performed by the information processing apparatus of FIG. It is a flowchart explaining the detail of the process image generation process among the exposure appropriate processes for every scene of FIG. It is a figure which shows the example of the luminance information output from ISP of FIG. It is a figure which shows the binarized image obtained as a result of performing the binarization process with respect to the luminance information of the example of FIG. FIG. 5 is a diagram illustrating an edge image obtained as a result of performing edge detection processing on the luminance information in the example of FIG. 4. FIG. 5 is a diagram illustrating an example of luminance information output from the ISP of FIG. 1 and an example different from FIG. 4. It is a figure which shows the binarized image obtained as a result of performing the binarization process with respect to the luminance information of the example of FIG. It is a figure which shows the edge image obtained as a result of performing the edge detection process with respect to the luminance information of the example of FIG. FIG. 8 is a diagram illustrating an example of luminance information output from the ISP of FIG. 1 and an example different from FIGS. 4 and 7. It is a figure which shows the binarized image obtained as a result of performing the binarization process with respect to the luminance information of the example of FIG. It is a figure which shows the edge image obtained as a result of performing the edge detection process with respect to the luminance information of the example of FIG. 3 is a flowchart illustrating details of a first determination condition scene determination process as an example of a scene determination process in the exposure appropriate process for each scene in FIG. 2. 3 is a flowchart for explaining details of a second determination condition scene determination process as another example of the scene determination process in the exposure appropriate process for each scene of FIG. 2. 2 is a flowchart for describing image recognition processing executed by the information processing apparatus of FIG. It is a figure which shows the example of the color image output from ISP of FIG. It is a figure which shows the binarized image obtained as a result of performing the binarization process with respect to the luminance information based on the data of the color image of the example of FIG. It is an example of the color image output from ISP of FIG. 1, Comprising: It is a figure which shows the example different from FIG. It is a figure which shows the binarized image obtained as a result of performing the binarization process with respect to the luminance information based on the data of the color image of the example of FIG. It is an example of the color image output from ISP of FIG. 1, Comprising: It is a figure which shows the example different from FIG.16 and FIG.18. FIG. 21 is a diagram illustrating a binarized image obtained as a result of binarization processing performed on luminance information based on color image data in the example of FIG. 20.

FIG. 1 is a block diagram showing a hardware configuration of an information processing apparatus according to an embodiment of the present invention.
The information processing apparatus 1 </ b> A includes an electronic camera 11 and a SoC (System on Chip) 12.

  The electronic camera 11 includes a lens 21 and an image sensor 22 such as a CMOS, and outputs captured image data obtained as a result of shooting to the SoC 12 as RAW data.

  The SoC 12 includes an ISP 31, an AE control unit 32, and an image recognition unit 33.

  The ISP 31 performs various image processing on the RAW data output from the electronic camera 11. Image data obtained as a result of various image processing is output to the AE control unit 32 or the image recognition unit 33. In this example, the image data provided to the AE control unit 32 is luminance information.

  The AE control unit 32 appropriately determines a shooting scene using luminance information output from the ISP 31, and performs camera control such as setting exposure based on the determination result. In the AE control unit 32, the processed image generation unit 41, the determination condition specifying unit 42, the scene determination unit 43, and the exposure setting unit 44 function in order to execute such camera control.

The processed image generation unit 41 generates processed image data based on the luminance information output from the ISP 31.
For example, the processed image generation unit 41 generates binarized image data as processed image data by performing binarization processing on the luminance information. Specifically, for example, the processed image generation unit 41 converts each pixel value in the bright area to the maximum value that can be expressed by the image data in the luminance information, while converting each pixel value in the dark area into the image information. By converting to the minimum value that can be expressed by data, binary image data is generated.
Further, for example, the processed image generation unit 41 generates edge image data as processed image data by performing edge detection processing on the luminance information. Specifically, for example, the processed image generation unit 41 detects a point where the brightness of the image becomes large as an edge. Further, for example, the processed image generation unit 41 sets the edge area as a first pixel value area (gray area in the example of FIG. 12 described later), and sets the other areas as second pixel value areas (described later). Data of an image that is a black region in the example of FIG. 12 is generated as edge image data.

  The determination condition specifying unit 42 determines the shooting scene of the original image (the captured image of the electronic camera 11 from which the processed image is generated) from the processed image data generated by the processed image generating unit 41. Is identified.

Here, in one embodiment of the present invention, a plurality of patterns are prepared in advance as shooting scenes. One or more conditions for discriminating the pattern of the original image from among a plurality of patterns of these shooting scenes are prepared in advance. Such a condition is a “condition for determining a shooting scene”. Hereinafter, the “condition for determining the shooting scene” is also referred to as “the determination condition of the shooting scene”.
As the plurality of patterns of the shooting scene, any combination of two or more arbitrary types can be adopted. In this example, the following (a) to (c) are listed as the shooting scene patterns.
(A) A scene where the upper side of the image is bright and the lower side is dark (b) A scene where the periphery of the image is bright and the center is dark (c) A scene where the periphery of the image is dark and the center is bright

In this embodiment, in order to discriminate each of these shooting scene patterns (a) to (c), various divided states of bright areas and dark areas (positions between the areas) obtained from the data of the binarized image are obtained. Relationship) and the complexity of the specific area based on the edge amount in the specific area obtained from the data of the edge image.
In other words, in the present embodiment, a condition (hereinafter referred to as “division state condition”) regarding various division states (positional relationship between the regions) of the bright region and the dark region, and the specific region based on the edge amount in the specific region. A condition relating to complexity (hereinafter referred to as “complexity condition”) is used as a “photographing scene discrimination condition”.
Here, the division state condition and the complexity condition may be used alone or in combination. For example, an example of a condition using only the division state condition (hereinafter referred to as “first determination condition”) is illustrated in FIG. Further, for example, an example of a condition (hereinafter referred to as “second determination condition”) that combines the division state condition and the complexity condition is illustrated in FIG. 14 described later.

  The scene discriminating unit 43 shoots original image data from a plurality of patterns such as the above-described patterns (a) to (c) based on the “photographing scene discrimination conditions” specified by the discrimination condition specifying unit 42. Determine the scene.

Here, when applying the division state condition, the scene determination unit 43 determines the positional relationship between the bright area and the dark area using the data of the binarized image.
That is, the scene determination unit 43 specifies the positions of the bright area and the dark area based on the binarized image data. Here, depending on the original image, there may be a plurality of bright and dark areas.
Moreover, the specific method of positional relationship is not specifically limited, For example, the following methods are employable. That is, whether the divided area is divided vertically and horizontally (other positional relationship to be compared), whether the divided area is divided into an area centered at an arbitrary position and other surrounding areas, etc. A technique of identifying various divided states necessary for shooting scene determination can be employed.

In addition, when applying the complexity condition, the scene determination unit 43 uses the edge image data to calculate the edge amount of one screen or various regions.
That is, the scene determination unit 43 can detect the respective edge amounts in the various regions by counting the number of edges in the various regions based on the edge image data.

The exposure setting unit 44 performs camera control on the electronic camera 11 so that the exposure is optimal for the photographic scene determined by the scene determination unit 43.
As described above, in the camera control (automatic exposure correction) with respect to the electronic camera 11, a highly accurate shooting scene can be used, so that optimal exposure correction can be realized.

  The processed image generation unit 41 to the exposure setting unit 44 that function in the AE control unit 32 have been described above. Hereinafter, a series of processes performed by the processed image generation unit 41 to the exposure setting unit 44 is hereinafter referred to as “exposure appropriate process for each scene”.

  The image recognition unit 33 recognizes various objects included in the color image based on the color image data output from the ISP 31. Such processing by the image recognition unit 33 is hereinafter referred to as “image recognition processing”. Details of the image recognition processing will be described later with reference to FIG.

The configuration example of the information processing apparatus 1A according to the embodiment of the present invention has been described above with reference to FIG.
Next, among the processes executed by the information processing apparatus 1A, the scene appropriate exposure process will be described with reference to FIG.

  FIG. 2 is a flow chart for explaining the exposure appropriate process for each scene executed by the information processing apparatus 1A of FIG.

  When the data of the original image taken by the electronic camera 11 is input to the ISP 31 and supplied to the AE control unit 32 as luminance information, the appropriate exposure process for each scene is started, and the following process of step S1 is executed. Is done.

  In step S <b> 1, the processed image generation unit 41 of the SoC 12 generates processed image data based on the luminance information output from the ISP 31. Such processing of the processed image generation unit 41 in step S1 is hereinafter referred to as “processed image generation processing”. Details of the processed image generation processing will be described later with reference to FIG.

  In step S <b> 2, the determination condition specifying unit 42 specifies a shooting scene determination condition from the processed image data generated in step S <b> 1.

  In step S3, the scene determination unit 43 determines the photographic scene of the original image from a plurality of patterns of the photographic scene based on the determination condition specified in step S2. Such processing of the scene determination unit 43 in step S3 is hereinafter referred to as “scene determination processing”. Details of the scene determination process will be described later with reference to FIGS.

  In step S4, the exposure setting unit 44 executes an exposure setting process according to the shooting scene determined in step S3.

  The outline of the process of appropriate exposure for each scene has been described above with reference to FIG. Further, details of each of the processed image generation processing in step S1 and the scene determination processing in step S3 of the appropriate exposure processing for each scene will be described below with reference to FIG. 3 and subsequent drawings.

  FIG. 3 is a flowchart for explaining the details of the processed image generation process in the appropriate exposure processing for each scene in FIG.

  In step S <b> 11, the processed image generation unit 41 acquires luminance information output from the ISP 31.

  In step S12, the processed image generation unit 41 generates binarized image data as processed image data by performing binarization processing on the luminance information acquired in step S11.

  In step S <b> 13, the processed image generation unit 41 specifies various division states (positional relationship between the areas) of the bright area and the dark area based on the binarized image data generated in step S <b> 12.

  In step S14, the processed image generation unit 41 performs edge detection processing on the luminance information acquired in step S11, thereby generating edge image data as processed image data.

  In step S15, the processed image generation unit 41 calculates the edge amount of one screen or various regions based on the edge image data generated in step S14.

  As a result, the processed image generation process ends. That is, the process of step S1 in FIG. 2 ends, the process proceeds to step S2, and the subsequent processes are executed.

  Here, some specific examples of the processed image data generated by the processed image generation processing will be described with reference to FIGS.

FIG. 4 shows a specific example of the luminance information Ei1 acquired in step S11 of the processed image generation process of FIG.
The luminance information Ei1 in the example of FIG. 4 is acquired by the processed image generation unit 41 in step S11 of the processed image generation process of FIG. 3 when output from the ISP 31 of FIG.

FIG. 5 is a diagram showing the binarized image Ti1 generated from the luminance information Ei1 of the example of FIG. 4 in step S12 of the processed image generation process of FIG.
That is, in step S12 of the processed image generation process of FIG. 3, the processed image generation unit 41 generates data of the binary image Ti1 by performing binarization processing on the luminance information Ei1 of the example of FIG. To do.
Specifically, the processed image generation unit 41 converts each pixel value of the bright area L1 into the maximum value that can be represented by image data in the luminance information Ei1, while converting each pixel value of the dark area B1 to Data of the binarized image Ti1 is generated by converting to a minimum value that can be expressed by image data.

  Then, in step S13 of the processed image generation process in FIG. 3, the processed image generation unit 41 determines various division states (positions between the regions) of the bright region L1 and the dark region B1 based on the data of the binarized image Ti1. As a relation), it is specified that the upper side of the image is bright and the lower side is dark.

FIG. 6 is a diagram showing the edge image Egi1 generated from the luminance information Ei1 of the example of FIG. 4 in step S14 of the processed image generation process of FIG.
That is, in step S12 of the processed image generation process of FIG. 3, the processed image generation unit 41 performs edge detection processing on the luminance information Ei1 of the example of FIG. 4 to generate data of the edge image Egi1.
Specifically, the processed image generation unit 41 detects, from the luminance information Ei1, a point (pixel) where the brightness of the image becomes large as the edge E1, and the region of the edge E1 is a region of the first pixel value (FIG. 6). The data of the edge image Egi1 is generated by setting the other area as the second pixel value area (black area in the example of FIG. 6).

  Then, in step S15 of the processed image generation process of FIG. 3, the processed image generation unit 41, based on the data of the edge image Egi1, determines the edge amount of one screen or various regions (the number of pixels constituting the region of the edge E1). (Amount based on).

As a result, the processed image generation process ends. That is, the process of step S1 in FIG. 2 ends, the process proceeds to step S2, and the subsequent processes are executed.
In this case, although the details will be described later, in the scene determination process in step S3 in FIG. 2, based on the result of the process in step S13 or step S15 described above, the photographic scene is displayed with the above “(a) upper side of the image brightly down. It is determined that the scene is a “dark scene”.

FIG. 7 shows a specific example of the luminance information Ei2 acquired in step S11 of the processed image generation process of FIG.
When the luminance information Ei2 in the example of FIG. 7 is output from the ISP 31 of FIG. 1, the processed image generation unit 41 acquires the luminance information Ei2 in step S11 of the processed image generation process of FIG.

FIG. 8 is a diagram showing the binarized image Ti2 generated from the luminance information Ei2 of the example of FIG. 7 in step S12 of the processed image generation process of FIG.
That is, in step S12 of the processed image generation process of FIG. 3, the processed image generation unit 41 generates data of the binarized image Ti2 by performing binarization processing on the luminance information Ei2 of the example of FIG. To do.
Specifically, the processed image generation unit 41 converts each pixel value of the bright area L2 to the maximum value that can be represented by image data in the luminance information Ei2, while converting each pixel value of the dark area B2 to Data of the binarized image Ti2 is generated by converting to the minimum value that can be expressed by the image data.

  Then, in step S13 of the processed image generation process of FIG. 3, the processed image generation unit 41 determines various division states (positions between the regions) of the bright region L2 and the dark region B2 based on the data of the binarized image Ti2. As a relation), it is specified that the periphery of the image is dark and the center is bright.

FIG. 9 is a diagram showing the edge image Egi2 generated from the luminance information Ei2 of the example of FIG. 7 in step S14 of the processed image generation process of FIG.
That is, in step S12 of the processed image generation process of FIG. 3, the processed image generation unit 41 performs edge detection processing on the luminance information Ei2 in the example of FIG. 7, thereby generating data of the edge image Egi2.
Specifically, the processed image generation unit 41 detects, from the luminance information Ei2, a point (pixel) where the brightness of the image becomes large as the edge E2, and the region of the edge E2 is the region of the first pixel value (FIG. 9). The edge image Egi2 is generated by setting the other area as the second pixel value area (black area in the example of FIG. 9).

  Then, in step S15 of the processed image generation process of FIG. 3, the processed image generation unit 41, based on the data of the edge image Egi2, determines the edge amount of one screen or various regions (the number of pixels constituting the region of the edge E2). (Amount based on).

As a result, the processed image generation process ends. That is, the process of step S1 in FIG. 2 ends, the process proceeds to step S2, and the subsequent processes are executed.
In this case, although the details will be described later, in the scene determination process in step S3 in FIG. 2, based on the processing results in step S13 and step S15 described above, the photographic scene is centered on the above-mentioned “(c) image with dark surroundings”. Is a bright scene ".

FIG. 10 shows a specific example of the luminance information Ei3 acquired in step S11 of the processed image generation process of FIG.
When the luminance information Ei3 in the example of FIG. 10 is output from the ISP 31 of FIG. 1, the processed image generation unit 41 acquires the luminance information Ei3 in step S11 of the processed image generation process of FIG.

FIG. 11 is a diagram illustrating the binarized image Ti3 generated from the luminance information Ei3 of the example of FIG. 10 in step S12 of the processed image generation process of FIG.
That is, in step S12 of the processed image generation process of FIG. 3, the processed image generation unit 41 generates data of the binary image Ti3 by performing binarization processing on the luminance information Ei3 of the example of FIG. To do.
Specifically, the processed image generation unit 41 converts each pixel value of the bright area L3 to the maximum value that can be represented by image data in the luminance information Ei3, while converting each pixel value of the dark area B3 to Data of the binarized image Ti3 is generated by converting to the minimum value that can be expressed by the image data.

  Then, in step S13 of the processed image generation process of FIG. 3, the processed image generation unit 41 determines various division states (positions between the regions) of the bright region L3 and the dark region B3 based on the data of the binarized image Ti3. As a relation), it is specified that the periphery of the image is bright and the center is dark.

FIG. 12 is a diagram showing the edge image Egi3 generated from the luminance information Ei3 of the example of FIG. 10 in step S14 of the processed image generation process of FIG.
That is, in step S12 of the processed image generation process of FIG. 3, the processed image generation unit 41 performs edge detection processing on the luminance information Ei3 in the example of FIG. 10 to generate data of the edge image Egi3.
Specifically, the processed image generation unit 41 detects, from the luminance information Ei3, a point (pixel) where the brightness of the image becomes large as the edge E3, and the region of the edge E3 is a region of the first pixel value (FIG. 12). The edge image Egi3 is generated by setting the other area as the second pixel value area (black area in the example of FIG. 12).

  Then, in step S15 of the processed image generation process of FIG. 3, the processed image generation unit 41, based on the data of the edge image Egi3, the edge amount of one screen or various regions (the number of pixels constituting the region of the edge E3) (Amount based on).

As a result, the processed image generation process ends. That is, the process of step S1 in FIG. 2 ends, the process proceeds to step S2, and the subsequent processes are executed.
In this case, although the details will be described later, in the scene determination process in step S3 in FIG. 2, based on the processing results in step S13 and step S15 described above, the photographic scene is centered on the above-mentioned “(b) the periphery of the image is bright. Is a “dark scene”.

The details of the processed image generation processing in step S1 of the appropriate exposure processing for each scene in FIG. 2 have been described above with reference to FIGS.
When the processed image generation process in step S1 is completed, in step S2, the conditions for determining the shooting scene are specified from the processed image data generated in step S1.
In step S3, a scene determination process is executed. That is, the photographic scene of the original image is determined from the plurality of patterns of the photographic scene based on the determination condition specified in step S2.
Therefore, details of the scene determination process will be described below.

Here, in step S2, it is assumed that one of the first discrimination condition and the second discrimination condition is specified as the shooting scene discrimination condition.
The first determination condition is a condition using only conditions relating to various divided states (positional relationship between the regions) of the bright region and the dark region, that is, the “divided state condition”.
The second determination condition is a condition in which, in addition to the “division state condition”, a condition relating to the complexity of the specific area based on the edge amount in the specific area, that is, a “complexity condition” is used in combination.

When the first determination condition is specified in step S2, the process according to the flowchart of FIG. 13 is executed as the scene determination process in step S3. Hereinafter, the scene determination process shown in FIG. 13 is referred to as a “first determination condition scene determination process”.
On the other hand, when the second determination condition is specified in step S2, the process according to the flowchart of FIG. 14 is executed as the scene determination process in step S3. Hereinafter, the scene determination process shown in FIG. 14 is referred to as a “second determination condition scene determination process”.

  Hereinafter, each of the first determination condition scene determination process in FIG. 13 and the second determination condition scene determination process in FIG. 14 will be described individually in that order.

  FIG. 13 illustrates the details of the first determination condition scene determination process executed as the scene determination process in step S3 when the first determination condition is specified in step S2 in the appropriate exposure processing for each scene in FIG. It is a flowchart to do.

In step S <b> 41, the scene determination unit 43 determines whether the division state of the bright area and the dark area obtained from the binarized image data matches the condition based on the threshold A.
If it is determined that the condition based on the threshold A is not met, NO is determined in step S41, and the process proceeds to step S43. The processing after step S43 will be described later.
On the other hand, if it is determined that the condition based on the threshold value A matches, the process proceeds to step S42.
In step S <b> 42, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 1. Thereby, the scene determination process for the first determination condition ends. That is, the process of step S3 in FIG. 2 ends, and the process proceeds to step S4.

As described above, when it is determined that the condition based on the threshold value A does not match, it is determined NO in step S41, and the process proceeds to step S43.
In step S43, the scene determination unit 43 determines whether the division state of the bright area and the dark area obtained from the binarized image data matches a condition based on the threshold value B.
If it is determined that the condition based on the threshold value B does not match, it is determined NO in step S43, and the process proceeds to step S45. The processing after step S45 will be described later.
On the other hand, if it is determined that the condition based on the threshold value B is met, the process proceeds to step S44.
In step S44, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 2. Thereby, the scene determination process for the first determination condition ends. That is, the process of step S3 in FIG. 2 ends, and the process proceeds to step S4.

As described above, when it is determined that the condition based on the threshold B is not met, it is determined as NO in Step S43, and the process proceeds to Step S45.
In step S <b> 45, the scene determination unit 43 determines whether the division state of the bright area and the dark area obtained from the binarized image data matches a condition based on the threshold value C.
If it is determined that the condition based on the threshold C is not met, NO is determined in step S45, and the process proceeds to step S47. The processing after step S47 will be described later.
On the other hand, if it is determined that the condition based on the threshold C is met, the process proceeds to step S46.
In step S <b> 46, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 3. Thereby, the scene determination process for the first determination condition ends. That is, the process of step S3 in FIG. 2 ends, and the process proceeds to step S4.

As described above, when it is determined that the condition based on the threshold C is not met, it is determined as NO in Step S45, and the process proceeds to Step S47.
In step S <b> 47, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 4.

Thereby, the scene determination process for the first determination condition ends. That is, the process of step S3 in FIG. 2 ends, the process proceeds to step S4, and the subsequent processes are executed.
In this manner, the photographic scene can be determined by each threshold based on the division state of the bright area and the dark area obtained from the binarized image data. As a result, it is possible to determine the shooting scene accurately and appropriately.

  The flow of the first discrimination condition scene discrimination process has been described above with reference to FIG. Next, the flow of the second discrimination condition scene discrimination process will be described with reference to FIG.

  FIG. 14 illustrates the details of the second determination condition scene determination process executed as the scene determination process in step S3 when the second determination condition is specified in step S2 in the appropriate exposure processing for each scene in FIG. It is a flowchart.

In step S <b> 61, the scene determination unit 43 determines whether or not the edge amount in the specific region obtained from the edge image data satisfies the condition based on the edge detection amount A.
If it is determined that the condition based on the edge detection amount A is not satisfied, NO is determined in step S61, and the process proceeds to step S64. The processing after step S64 will be described later.
On the other hand, if it is determined that the condition based on the edge detection amount A is satisfied, the process proceeds to step S62.
In step S <b> 62, the scene determination unit 43 determines whether the division state of the bright area and the dark area obtained from the binarized image data satisfies the condition based on the threshold A.
If it is determined that the condition based on the threshold A is not satisfied, it is determined as NO in Step S62, and the process proceeds to Step S64. The processing after step S64 will be described later.
On the other hand, if it is determined that the condition based on the threshold A is met, the process proceeds to step S63.
In step S63, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 1. Thereby, the scene determination process for the second determination condition ends. That is, the process of step S3 in FIG. 2 ends, and the process proceeds to step S4.

  As described above, when it is determined that the condition based on the edge detection amount A is not met or when it is determined that the condition based on the threshold A is not met, it is determined as NO in step S61 or step S62. Then, the process proceeds to step S64.

In step S <b> 64, the scene determination unit 43 determines whether or not the edge amount in the specific region obtained from the edge image data satisfies the condition based on the edge detection amount B.
If it is determined that the condition based on the edge detection amount B is not satisfied, NO is determined in step S64, and the process proceeds to step S67. The processing after step S67 will be described later.
On the other hand, if it is determined that the condition based on the edge detection amount B is satisfied, the process proceeds to step S65.
In step S <b> 65, the scene determination unit 43 determines whether the division state of the bright area and the dark area obtained from the binarized image data satisfies the condition based on the threshold B.
If it is determined that the condition based on the threshold B is not met, NO is determined in step S65, and the process proceeds to step S67. The processing after step S67 will be described later.
On the other hand, if it is determined that the condition based on the threshold value B is satisfied, the process proceeds to step S66.
In step S <b> 66, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 2. Thereby, the scene determination process for the second determination condition ends. That is, the process of step S3 in FIG. 2 ends, and the process proceeds to step S4.

  As described above, when it is determined that the condition based on the edge detection amount B is not met or when it is determined that the condition based on the threshold B is not met, it is determined as NO in step S64 or step S65. Then, the process proceeds to step S67.

In step S <b> 67, the scene determination unit 43 determines whether or not the edge amount in the specific area obtained from the edge image data satisfies the condition based on the edge detection amount C.
If it is determined that the condition based on the edge detection amount C is not satisfied, NO is determined in step S67, and the process proceeds to step S70. The processing after step S70 will be described later.
On the other hand, if it is determined that the condition based on the edge detection amount C is satisfied, the process proceeds to step S68.
In step S <b> 68, the scene determination unit 43 determines whether the division state of the bright area and the dark area obtained from the binarized image data satisfies the condition based on the threshold value C.
If it is determined that the condition based on the threshold value C is not satisfied, NO is determined in step S68, and the process proceeds to step S70. The processing after step S70 will be described later.
On the other hand, if it is determined that the condition based on the threshold value C is met, the process proceeds to step S69.
In step S <b> 69, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 3. Thereby, the scene determination process for the second determination condition ends. That is, the process of step S3 in FIG. 2 ends, and the process proceeds to step S4.

  As described above, when it is determined that the condition based on the edge detection amount C is not met or when it is determined that the condition based on the threshold C is not met, NO is determined in step S67 or step S68. Then, the process proceeds to step S70.

In step S <b> 70, the scene determination unit 43 determines whether or not the edge amount in the specific region obtained from the edge image data satisfies the condition based on the edge detection amount D.
If it is determined that the condition based on the edge detection amount D is not satisfied, NO is determined in step S70, and the process proceeds to step S73. The processing after step S73 will be described later.
On the other hand, if it is determined that the condition based on the edge detection amount D is satisfied, the process proceeds to step S71.
In step S <b> 71, the scene determination unit 43 determines whether the division state of the bright area and the dark area obtained from the binarized image data satisfies the condition based on the threshold value D.
If it is determined that the condition based on the threshold value D is not met, NO is determined in step S71, and the process proceeds to step S73. The processing after step S73 will be described later.
On the other hand, if it is determined that the condition based on the threshold value D is met, the process proceeds to step S72.
In step S <b> 72, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 4. Thereby, the scene determination process for the second determination condition ends. That is, the process of step S3 in FIG. 2 ends, and the process proceeds to step S4.

  As described above, when it is determined that the condition based on the edge detection amount D is not met or when it is determined that the condition based on the threshold D is not met, it is determined as NO in step S70 or step S71. Then, the process proceeds to step S73.

  In step S <b> 73, the scene determination unit 43 determines that the shooting scene of the original image is the shooting scene 5.

Thereby, the scene determination process for the second determination condition ends. That is, the process of step S3 in FIG. 2 ends, the process proceeds to step S4, and the subsequent processes are executed.
As described above, the shooting scene is determined based on the division state of the bright area and the dark area obtained from the binary image data and the complexity of the specific area based on the edge amount in the specific area obtained from the edge image data. be able to. As a result, it is possible to determine the shooting scene accurately and appropriately.

  Heretofore, the exposure appropriate process for each scene among the processes executed by the information processing apparatus 1A has been described with reference to FIGS. Next, image recognition processing among the processing executed by the information processing apparatus 1A will be described with reference to FIG.

  FIG. 15 is a flowchart for describing image recognition processing executed by the information processing apparatus 1A of FIG.

Here, in the image recognition processing of FIG. 15, the image recognition unit 33 converts various objects included in the color image into the following (A) to (C) based on the color image data output from the ISP 31. It is assumed that what corresponds to each pattern is recognized using each of the pattern determinations A to C.
(A) Face (B) Traffic light (C) Marking It should be noted that the object patterns that can be recognized by the image recognition processing are two or more including the above-mentioned patterns (A) to (C) by adopting various pattern recognitions. Any combination of any type can be employed.

  When the original image data photographed by the electronic camera 11 is input to the ISP 31 and supplied to the image recognition unit 33 as color image data, the image recognition process is started, and the following process of step S81 is executed. Is done.

In step S <b> 81, the image recognizing unit 33 determines whether or not the color image data output from the ISP 31 is matched with the “(A) face” pattern in the pattern determination A.
If it is determined in the pattern determination A that it does not match the “(A) face” pattern, it is determined NO in step S81, and the process proceeds to step S85. The processing after step S85 will be described later.
On the other hand, if it is determined in the pattern determination A that the pattern matches with the “(A) face” pattern, the process proceeds to step S82.
In step S <b> 82, the image recognition unit 33 performs binarization processing on the luminance information based on the color image data output from the ISP 31, thereby generating binarized image data as processed image data.
In step S83, the image recognition unit 33 determines whether the binary image data generated in step S82 has a condition necessary for color determination A.
If it is determined that the condition necessary for color determination A is not satisfied, NO is determined in step S83, and the process proceeds to step S85. The processing after step S85 will be described later.
On the other hand, when it is determined that the condition necessary for the color determination A is satisfied, the image recognition unit 33 determines that the predetermined area of the color image data output from the ISP 31 is “(A) face”. It is determined as color determination A whether or not it has the color of the pattern. If it is determined that the color of the “(A) face” pattern is not included, NO is determined in step S83, and the process proceeds to step S85. If it is determined that the pattern has the color “(A) face”, the process proceeds to step S84.
In step S <b> 84, the image recognition unit 33 recognizes that the color image data (original image data) output from the ISP 31 includes an object having the pattern “(A) face”.

As described above, if it is determined as NO in step S81 or step S83, the process proceeds to step S85.
In step S <b> 85, the image recognition unit 33 determines whether or not the color image data output from the ISP 31 is matched with the pattern “(B) traffic light” in the pattern determination B.
If it is determined in the pattern determination B that the pattern does not match the “(B) traffic light” pattern, it is determined NO in step S85, and the process proceeds to step S89. The processing after step S89 will be described later.
On the other hand, when it is determined in pattern determination B that the pattern of “(B) traffic light” is matched, the process proceeds to step S86.
In step S86, the image recognition unit 33 performs binarization processing on the luminance information based on the color image data output from the ISP 31, thereby generating binarized image data as processed image data.
In step S87, the image recognition unit 33 determines whether the binary image data generated in step S86 has a condition necessary for the color determination B.
If it is determined that the condition necessary for color determination B is not satisfied, NO is determined in step S87, and the process proceeds to step S89. The processing after step S89 will be described later.
On the other hand, when it is determined that the condition necessary for the color determination B is satisfied, the image recognition unit 33 determines that the predetermined area of the color image data output from the ISP 31 is “(B) traffic light”. It is determined as a color determination B whether or not it has the pattern color. If it is determined that the color of the “(B) traffic light” pattern is not included, NO is determined in step S87, and the process proceeds to step S89. If it is determined that the pattern has the color “(B) traffic light”, the process proceeds to step S88.
In step S <b> 88, the image recognition unit 33 recognizes that the color image data (original image data) output from the ISP 31 includes an object having a pattern of “(B) traffic light”.

As described above, if it is determined as NO in step S85 or step S87, the process proceeds to step S89.
In step S89, the image recognizing unit 33 determines whether or not the color image data output from the ISP 31 is matched with the pattern “(C) sign” in the pattern determination C.
If it is determined in the pattern determination C that the pattern does not match the “(C) sign” pattern, NO is determined in step S89, and the process proceeds to step S102. The processing after step S102 will be described later.
On the other hand, if it is determined in the pattern determination C that the pattern matches with the “(C) sign” pattern, the process proceeds to step S99.
In step S99, the image recognition unit 33 generates binarized image data as processed image data by performing binarization processing on luminance information based on the color image data output from the ISP 31.
In step S100, the image recognition unit 33 determines whether the binary image data generated in step S99 has a condition necessary for color determination C.
If it is determined that the condition necessary for the color determination C is not satisfied, NO is determined in step S100, and the process proceeds to step S102. The processing after step S102 will be described later.
On the other hand, when it is determined that the condition necessary for the color determination C is satisfied, the image recognition unit 33 determines that a predetermined area of the color image data output from the ISP 31 is “(C) indicator”. It is determined as a color determination C whether or not it has the pattern color. When it is determined that the color of the “(C) sign” pattern is not included, it is determined NO in step S100, and the process proceeds to step S102. If it is determined that the pattern has the color “(C) sign”, the process proceeds to step S101.
In step S <b> 101, the image recognition unit 33 recognizes that the color image data (original image data) output from the ISP 31 includes an object having a pattern of “(C) sign”.

As described above, when it is determined as NO in step S89 or step S100, the process proceeds to step S102.
In step S102, the image recognition unit 33 recognizes that the color image data (original image data) output from the ISP 31 includes objects of other patterns. As a result, the image recognition process ends.
Therefore, since image analysis from color information and contour extraction by binarization processing are possible, it can be applied to recognition determination. As a result, face recognition, road traffic lights, signs and the like can be recognized as images.

  Here, some specific examples of the color image recognized by the image recognition process will be described with reference to FIGS.

FIG. 16 shows a specific example of the color image Ci1 to be recognized in the image recognition process of FIG.
The data of the color image Ci1 in the example of FIG. 16 is acquired by the image recognition unit 33 when output from the ISP 31 of FIG.
The color image Ci1 is determined to match the “(A) face” pattern in the pattern determination A (YES in step S81 in FIG. 15).
Thereby, the data of the binarized image Ct1 shown in FIG. 17 is generated in step S82 of FIG.
FIG. 17 is a diagram showing the binarized image Ct1 generated from the luminance information based on the color image Ci1 in the example of FIG. 16 in step S82 of the image recognition process of FIG.
With respect to the data of the binarized image Ct1 in the example of FIG. 17, it is determined that the condition necessary for the color determination A is satisfied, and the color determination A is performed (YES in step S83). The condition necessary for the color determination A is that an area such as the area F exists in the binarized image Ct1. The color determination A refers to determination of whether or not the region F has a “(A) face” pattern color in the binarized image Ct1.
Then, it is recognized that “(A) face” is included in the color image Ci1 in the example of FIG. 16 (step S84).

FIG. 18 shows a specific example of the color image Ci2 to be recognized in the image recognition process of FIG.
The data of the color image Ci2 in the example of FIG. 18 is acquired by the image recognition unit 33 when output from the ISP 31 of FIG.
The color image Ci2 is determined to be matched with the pattern “(B) traffic light” in the pattern determination B (YES in step S85 in FIG. 15).
Thereby, the data of the binarized image Ct2 shown in FIG. 19 is generated in step S86 of FIG.
FIG. 19 is a diagram showing the binarized image Ct2 generated from the luminance information based on the color image Ci2 of the example of FIG. 18 in step S86 of the image recognition process of FIG.
For the data of the binarized image Ct2 in the example of FIG. 19, it is determined that the condition necessary for the color determination B is satisfied, and the color determination B is performed (YES in step S87). The condition necessary for the color determination B is that an area such as the area S exists in the binarized image Ct2. The color determination B refers to determination of whether or not the region S in the binarized image Ct2 has the color of the “(B) traffic light” pattern.
Then, it is recognized that “(B) traffic light” is included in the color image Ci2 in the example of FIG. 18 (step S88).

FIG. 20 shows a specific example of the color image Ci3 to be recognized in the image recognition process of FIG.
When the color image Ci3 in the example of FIG. 20 is output from the ISP 31 of FIG.
The color image Ci3 is determined to match with the pattern “(C) sign” in the pattern determination C (YES in step S89 in FIG. 15).
Thereby, the data of the binarized image Ct3 shown in FIG. 21 is generated in step S99 of FIG.
FIG. 21 is a diagram showing the binarized image Ct3 generated from the luminance information based on the color image Ci3 of the example of FIG. 20 in step S99 of the image recognition process of FIG.
For the data of the binarized image Ct3 in the example of FIG. 21, it is determined that the condition necessary for the color determination C is satisfied, and the color determination C is performed (YES in step S100). The condition necessary for the color determination C is that a region such as the region H1 and the region H2 exists in the binarized image Ct3. The color determination C refers to determining whether or not each of the region H1 and the region H2 of the binarized image Ct3 has the color of the “(C) label” pattern.
Then, it is recognized that “(C) sign” is included in the color image Ci3 in the example of FIG. 20 (step S101).

  Although one embodiment of the information processing apparatus of the present invention has been described above, the present invention is not limited to the above-described embodiment. In addition, the effects described in the present embodiment are merely a list of the most preferable effects resulting from the present invention, and the effects according to the present invention are not limited to those described in the present embodiment.

  For example, in the above-described embodiment, exposure correction is used as an application of the determination result of the photographic scene, but is not particularly limited thereto. For example, it may be used for assisting white balance (AWB: Auto White Balance) processing, noise reduction, driving of an automobile or the like.

  The hardware configuration shown in FIG. 1 is merely an example for achieving the object of the present invention, and is not particularly limited.

Further, the block diagram shown in FIG. 1 is merely an example, and is not particularly limited. That is, it is sufficient that the information processing apparatus 1A has a function capable of executing the above-described series of processes as a whole, and what block is used to realize this function is not particularly limited to the example of FIG. .
Furthermore, the location of the block is not limited to that shown in FIG.
One block may be constituted by hardware alone or in combination with software.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall apparatus configured by a plurality of devices, a plurality of means, and the like.

In summary, the information processing apparatus 1A to which the present invention is applied only needs to have the following configuration, and can take various embodiments.
That is, the information processing apparatus to which the present invention is applied (for example, the information processing apparatus 1A in FIG. 1)
Processed image generation means (for example, the processed image generation unit 41 in FIG. 1) that generates one or more processed image data from original image data obtained as a result of being imaged by a predetermined imaging device;
A determination condition specifying means (for example, a determination condition specifying unit 42 in FIG. 1) for specifying one or more conditions for determining a shooting scene of the original image from the data of the one or more processed images;
Scene discriminating means (eg, the scene discriminating unit 43 in FIG. 1) for discriminating the shooting scene of the original image based on the one or more specified conditions;
An information processing apparatus provided with
As a result, it is possible to determine the shooting scene accurately and appropriately using the data of the image shot by the electronic camera. As a result, appropriate exposure correction can be realized.

In addition, the information processing device
The processed image generation means performs binarization processing on the luminance information in the original image data to dynamically divide the original image into a bright area and a dark area, thereby converting the binarized image data. You may make it produce | generate as the data of the said processed image.
As a result, the photographic scene can be determined by each threshold based on the division state of the bright area and the dark area obtained from the binarized image data. As a result, it is possible to determine the shooting scene accurately and appropriately.

In addition, the information processing device
The processed image generation means may detect a point where the brightness of the original image becomes large, and recognize the detected point as an edge, thereby generating edge image data as the processed image data.
Thereby, a photographic scene can be determined based on the complexity of the specific area based on the edge amount in the specific area obtained from the data of the edge image. As a result, it is possible to determine the shooting scene accurately and appropriately.

In addition, the information processing device
Image recognition means (for example, FIG. 5) that executes image analysis for color information and binarization processing for luminance information in the original image data, and recognizes one or more objects included in the original image based on the execution result. 1 image recognition unit 33) may be further provided.
Thereby, image analysis from color information and contour extraction by binarization processing are possible. As a result, it can be applied to recognition determination, and face recognition, road traffic lights, signs and the like can be recognized as images.

1A ... Information processing device 11 ... Electronic camera 12 ... SoC
21 ... Lens 22 ... Imaging element 31 ... ISP
32 ... AE control unit 33 ... Image recognition unit 41 ... Processed image generation unit 42 ... Discrimination condition specifying unit 43 ... Scene discrimination unit 44 ... Exposure setting unit

Claims (6)

Processed image generation means for generating one or more processed image data from original image data obtained as a result of being imaged by a predetermined imaging device;
Discrimination condition specifying means for specifying one or more conditions for discriminating a shooting scene of the original image from the data of the one or more processed images;
Scene discriminating means for discriminating the shooting scene of the original image based on the specified one or more conditions;
An information processing apparatus comprising: The processed image generation means performs binarization processing that dynamically divides the original image into bright regions and dark regions with respect to luminance information in the original image data, thereby generating binarized image data. Is generated as data of the processed image,
The information processing apparatus according to claim 1. The processed image generation means detects a point where the brightness of the original image increases, and recognizes the detected point as an edge, thereby generating edge image data as the processed image data.
The information processing apparatus according to claim 1 or 2. An image recognition unit that executes image analysis for color information and binarization processing for luminance information in the original image data, and recognizes one or more objects included in the original image based on the execution result. Item 4. The information processing apparatus according to any one of Items 1 to 3. In an information processing method executed by information processing,
A processed image generation step for generating one or more processed image data from original image data obtained as a result of being imaged by a predetermined imaging device;
A determination condition specifying step for specifying one or more conditions for determining a shooting scene of the original image from the data of the one or more processed images;
A scene determination step of determining the shooting scene of the original image based on the specified one or more conditions;
An information processing method including: On the computer,
A processed image generation step for generating one or more processed image data from original image data obtained as a result of being imaged by a predetermined imaging device;
A determination condition specifying step for specifying one or more conditions for determining a shooting scene of the original image from the data of the one or more processed images;
A scene determination step of determining the shooting scene of the original image based on the specified one or more conditions;
A program that executes control processing including

Images