WO2013114884A1

WO2013114884A1 - Program for identifying position of subject, device for identifying position of subject, and camera

Info

Publication number: WO2013114884A1
Application number: PCT/JP2013/000530
Authority: WO
Inventors: 啓之阿部
Original assignee: 株式会社ニコン
Priority date: 2012-02-01
Filing date: 2013-01-31
Publication date: 2013-08-08

Abstract

The position of a subject in an image is identified accurately, through execution by a computer of: a segmenting procedure for segmenting a single image into a plurality of segmental images on the basis of color information or luminance information of a target image; a binarization procedure for binarization employing color information or luminance information each of the plurality of segmental images, and generating a plurality of binary images; a first evaluation value computation procedure for computing a first evaluation value to be employed for identification of a subject position within the target image, for each of the plurality of binary images; a first subject position identification procedure for identifying the subject position within the target image, on the basis of the first evaluation value; a second evaluation value computation procedure for computing a second evaluation value to be employed for re-identification of the subject position within the target image, for each of the plurality of binary images, on the basis of the subject position identified in the first subject position identification procedure; and a second subject position identification procedure for re-identifying the subject position within the target image, on the basis of the second evaluation value.

Description

Subject position specifying program, subject position specifying device, and camera

The present invention relates to an object position specifying program, an object position specifying device, and a camera.

The following imaging devices are known. This imaging apparatus performs a focus adjustment process on the AF area selected by the user (for example, Patent Document 1).

Japanese Patent Laid-Open No. 2004-205858

However, in the conventional imaging apparatus, it is difficult for the user to accurately align the AF frame with the subject, so there is a possibility that the AF area selected by the user is shifted from the actual subject position. Therefore, it is difficult to specify an accurate subject position based on the AF area.

The subject position specifying program according to the present invention causes a computer to classify a single image into a plurality of segmented images based on color information or luminance information of the target image, and to classify each of the plurality of segmented images as the color information. Alternatively, a binarization procedure for binarizing using luminance information to generate a plurality of binarized images, and for specifying a subject position in the target image for each of the plurality of binarized images A first evaluation value calculating procedure for calculating a first evaluation value to be used; a first subject position specifying procedure for specifying a subject position in the target image based on the first evaluation value; Based on the subject position specified in the subject position specifying procedure, the subject position in the target image is specified again for each of the plurality of binarized images. A second evaluation value calculation step of calculating the second evaluation values, on the basis of the second evaluation value, to execute a second subject position specifying step of specifying the object position again in the subject image.

In the second subject position specifying procedure, the subject position in the target image may be specified again based on both the second evaluation value and the first evaluation value.

Further, the first evaluation value is at least a value related to an area of a white pixel region composed of white pixels in the binarized image and a value related to a distance between the white pixel region and a predetermined reference region. And the second evaluation value is at least a value related to the area and a value related to the distance between the white pixel region and the region based on the subject position specified in the first subject position specifying procedure. It may be calculated based on this.

Another object position specifying program according to the present invention is a computer program for generating a binarized image by binarizing pixels with a predetermined threshold value, and for the first pixel exceeding the predetermined threshold value, the 2 The first determination procedure for determining the degree of clustering of the first pixels with reference to a predetermined position of the binarized image and the first determination procedure determined that the first pixel is more than the predetermined threshold. A second determination procedure for determining the degree of clumping of the first pixel with reference to a predetermined position in the clump of one pixel, and a determination procedure for determining a subject position based on the determination result of the second determination procedure And execute.

An apparatus for specifying a subject position according to the present invention includes: a division unit that divides one image into a plurality of divided images based on color information or luminance information of the target image; and the color information or luminance information for each of the plurality of divided images. A binarization unit that binarizes using the image to generate a plurality of binarized images, and a first unit used to identify a subject position in the target image for each of the plurality of binarized images. A first evaluation value calculation unit that calculates an evaluation value of the first object, a first subject position specification unit that specifies a subject position in the target image based on the first evaluation value, and the first subject position A second evaluation value used to re-specify the subject position in the target image for each of the plurality of binarized images based on the subject position specified by the specifying unit. Evaluation Comprising a calculation unit, on the basis of the second evaluation value, and a second subject position specifying unit configured to specify again the subject position in the target image.

The camera according to the present invention includes a division procedure for dividing one image into a plurality of divided images based on the color information or luminance information of the target image, and each of the plurality of divided images using the color information or luminance information. A binarization procedure for binarizing and generating a plurality of binarized images, and a first evaluation value used for specifying a subject position in the target image for each of the plurality of binarized images A first evaluation value calculation procedure to be calculated, a first subject position specification procedure for specifying a subject position in the target image based on the first evaluation value, and a specification in the first subject position specification procedure Based on the subject position, a second evaluation value calculation for calculating a second evaluation value used to re-specify the subject position in the target image for each of the plurality of binarized images. Comprising a procedure, on the basis of the second evaluation values, the execution means for executing the subject position specifying program and a second subject position specifying step of specifying the object position again in the subject image.

It is a block diagram which shows the structure of one Embodiment of a camera. It is a figure which shows the specific example of a target image. It is a flowchart figure which shows the flow of a subject position specific process. It is a figure which shows typically the binarization method of a Y plane image, a Cb plane image, and a Cr plane image. It is a figure which shows the example of a binarized image of 9 divisions. It is a figure which shows the example of the binarized image of 9 divisions in which the representative island was specified. It is a figure which shows the example of ranking based on a 1st evaluation value. It is a figure which shows the example of 1st to-be-photographed object position specification. It is a figure which shows the example of ranking based on a 2nd evaluation value. It is a figure which shows the example of 2nd to-be-photographed object position specification.

FIG. 1 is a block diagram showing a configuration of an embodiment of a camera according to the present embodiment. The camera 100 includes an operation member 101, a lens 102, an image sensor 103, a control device 104, a memory card slot 105, and a monitor 106. The operation member 101 includes various input members operated by the user, such as a power button, a release button, a zoom button, a cross key, an enter button, a play button, and a delete button.

The lens 102 is composed of a plurality of optical lenses, but is representatively represented by one lens in FIG. The image sensor 103 is an image sensor such as a CCD or a CMOS, for example, and captures a subject image formed by the lens 102. Then, an image signal obtained by imaging is output to the control device 104.

The control device 104 generates a predetermined image format, for example, JPEG format image data (hereinafter referred to as “main image data”) based on the image signal input from the image sensor 103. Further, the control device 104 generates display image data, for example, thumbnail image data, based on the generated image data. The control device 104 generates an image file that includes the generated main image data and thumbnail image data, and further includes header information, and outputs the image file to the memory card slot 105. In the present embodiment, it is assumed that both the main image data and the thumbnail image data are image data expressed in the RGB color system.

The memory card slot 105 is a slot for inserting a memory card as a storage medium, and records and records the image file output from the control device 104 on the memory card. The memory card slot 105 reads an image file stored in the memory card based on an instruction from the control device 104.

The monitor 106 is a liquid crystal monitor (rear monitor) mounted on the back surface of the camera 100, and the monitor 106 displays an image stored in a memory card, a setting menu for setting the camera 100, and the like. . Further, when the user sets the mode of the camera 100 to the shooting mode, the control device 104 outputs image data for display of images acquired from the image sensor 103 in time series to the monitor 106. As a result, a through image is displayed on the monitor 106.

The control device 104 includes a CPU, a memory, and other peripheral circuits, and controls the camera 100. Note that the memory constituting the control device 104 includes SDRAM and flash memory. The SDRAM is a volatile memory that is used as a work memory for the CPU to develop a program when the program is executed, and also as a buffer memory for temporarily recording data. The flash memory is a non-volatile memory in which data of a program executed by the control device 104, various parameters read during program execution, and the like are recorded.

In the present embodiment, the control device 104 specifies the position of the subject in the image based on the color information or luminance information of the image.

Hereinafter, processing when the subject position is specified using the image shown in FIG. 2 as the target image will be described with reference to the flowchart shown in FIG. 2A shows the target image itself, and FIG. 2B shows the main subject as an illustration for the purpose of explanation. In the following description, it is assumed that the flower portion indicated by the frame F1 in FIG. 2B is a portion photographed as a main subject by the user.

3 is executed by the control device 104 as a program that starts when input of image data from the image sensor 103 is started.

In step S101, the control device 104 converts the target image into a YCbCr format image, and displays a Y component image (Y plane image), a Cr component image (Cr plane image), and a Cb component image (Cb plane image). ) Respectively. Specifically, a target image represented in the RGB color system is represented by a luminance image composed of luminance components (Y components) in the YCbCr color space and color difference components (Cb components) using the following equations (1) to (3). , Cr component).

That is, for the target image, the control device 104 generates a luminance image composed of the Y component as a Y plane image using the following equation (1), and a color difference composed of the Cb component using the following equations (2) and (3). An image and a color difference image composed of Cr components are generated as a Cb plane image and a Cr plane image, respectively.

Y = 0.299R + 0.587G + 0.114B (1)
Cb = −0.169R−0.332G + 0.500B (2)
Cr = 0.500R−0.419G−0.081B (3)
In step S102, the control device 104 binarizes the Y plane image, the Cr plane image, and the Cb plane image generated in step S101 into nine sections.

The control device 104 examines the density values of all the pixels in the image for each of the Y plane image, the Cr plane image, and the Cb plane image generated in step S101, and calculates the average (Ave) of each density value and each density value. The standard deviation (σ) is calculated. Then, the control device 104 binarizes the Y plane image, the Cb plane image, and the Cr plane image using the average of each density value and the standard deviation of each density.

FIG. 4 is a diagram schematically showing a binarization method for a Y plane image, a Cb plane image, and a Cr plane image. As shown in FIG. 4, the control device 104 generates three binarized images, that is, nine sections, for each of the Y plane image, the Cb plane image, and the Cr plane image.

In the example of FIG. 4, three threshold values “Ave + α · σ”, “Ave + σ”, and “Ave−β · σ” are used. Ave at each threshold indicates an average of the above-described density values, and σ indicates a standard deviation of each of the above-described densities. Α and β are predetermined coefficients.

Fig. 5 shows an example of 9-level binarized images.

In step S103, the control device 104 performs a labeling process on the binarized images of nine sections generated in step S102.

First, the control device 104 extracts a set of white pixels and a set of black pixels in each binarized image as a labeling region for each of the nine binarized images generated in step S102. And the labeling area | region comprised by a white pixel is detected as an island among the extracted labeling area | regions.

In step S104, the control device 104 calculates the area of each island (white pixel region) detected by the labeling process in step S103.

Note that when calculating the area, out of the islands detected in the binarized image, an island of a certain size or larger or an island of a certain size or smaller may be excluded. For example, an island with an area ratio of 60% or more with respect to the entire area of the binarized image or an island with an area ratio with respect to the area of the entire binarized image of 1% or less may be excluded.

In step S105, the control device 104 calculates the moment of inertia of each island (white pixel region) detected by the labeling process in step S103.

The control device 104 calculates the moment of inertia around the center of the screen for the islands in the binarized image of nine sections generated in step S102. With this process, the moment of inertia around the center of the screen is calculated for each island in the binarized image. The method of calculating the moment of inertia is well known and will not be described in detail. For example, it can be calculated by the sum of the square of the pixel distance from the center of the screen × (0 or 1) density value.

In step S106, the control device 104 calculates a first evaluation value for each island based on the area of each island calculated in step S104 and the inertia moment of each island calculated in step S105.

The control device 104 calculates the first evaluation value by the following equation (4).

First evaluation value = (area of each island calculated in step S104) ^ γ ÷ (moment of inertia of each island calculated in step S105) (4)
In equation (4), γ is a predetermined coefficient.

In step S107, the control device 104 ranks the islands in the binarized image generated in step S102 based on the first evaluation value calculated in step S106.

The control device 104 compares the first evaluation value calculated in step S106 for each island specified in step S103, and specifies the island having the largest first evaluation value as a representative island. FIG. 6 shows an example of a binarized image of nine sections in which representative islands are specified.

And the control apparatus 104 ranks each island in the binarized image of 9 divisions produced | generated by step S102 based on the 1st evaluation value of the said representative island. Specifically, the ranking is performed such that the higher the first evaluation value of the representative island in each binarized image, the higher the ranking. In this case, the result of ranking the representative islands of the binarized image based on the first representative evaluation value is, for example, as shown in FIG. 7, the representative island of M (Y2), the representative island of M (Y1), M (B3) The representative island of the pass, the representative island of the M (R1) pass, the representative island of the M (R2) pass, and the representative island of the M (Y3) pass. In the example of FIG. 7, the first to sixth positions are shown.

Note that the first evaluation value calculated in step S106 increases as the area of the island increases and the inertia moment of the island decreases. For this reason, by ranking based on the first evaluation value, there are a large number of white pixels with a large area of the island and a high possibility of a subject, and the representative is a representative in the binarized image where the island is close to the center of the screen. The higher the island, the higher the ranking.

In step S108, the control device 104 specifies the first subject position based on the result of the ranking performed in step S107.

The control device 104 specifies the position of the representative island with the highest rank as the subject position in the target image based on the ranking result performed in step S107. As shown in FIG. 7, the representative island of M (Y2) is ranked first. Therefore, the control device 104 specifies the envelope frame F2 of the representative island in M (Y2) as the first subject position. FIGS. 8A and 8B show the envelope frame F2 applied to FIGS. 2A and 2B. As shown in FIGS. 8A and 8B, in the first subject position specification based on the first evaluation value, a position close to the center of the screen is likely to be prioritized, so that it is specified at a position different from the actual main subject.

In step S109, the control device 104 recalculates the moment of inertia of the representative island in the higher rank among the representative islands of the binarized images specified in step S107.

The control device 104 recalculates the moment of inertia with respect to the representative island of the higher rank among the representative islands of the binarized images specified in step S107, with the center of gravity position of each representative island as the center.

The details of the method of calculating the moment of inertia are the same as in step S105 described above. In addition, since the subject position is unknown at the time of step S105 described above, the moment of inertia is calculated assuming that the center of the screen is the subject position, whereas in step S109, the representatives ranked in step S107 are calculated. The moment of inertia is recalculated using the position of the island as the subject position.

In step S110, the control device 104 calculates a second evaluation value based on the area of each island calculated in step S104 and the inertia moment of each island calculated in step S109.

The control device 104 calculates the second evaluation value by the following equation (5).

Second evaluation value = (area of each island calculated in step S104) ^ γ ÷ (moment of inertia of each island calculated in step S109) (5)
In equation (5), γ is a predetermined coefficient.

In step S111, the control device 104 ranks the representative islands of the binarized images specified in step S107 based on the second evaluation value calculated in step S110.

The control device 104 re-ranks the representative islands of the respective binarized images specified in step S107 based on the second evaluation value of the representative island. Specifically, the ranking is performed such that the higher the second evaluation value of the representative island in each binarized image, the higher the ranking. The ranking results of the representative islands of the binarized image based on the second representative evaluation value in this case are, for example, as shown in FIG. 9, the representative island of M (R1), the representative island of M (R2), M (Y2) The representative island of Kashiwa, the representative island of M (Y1), the representative island of M (B3), and the representative island of M (Y3). In the example of FIG. 9, the first to sixth positions are shown.

Note that the second evaluation value calculated in step S110 is an evaluation value for re-evaluating the first evaluation value in consideration of the subject position specified based on the first evaluation value. For this reason, by ranking based on the second evaluation value, ranking according to the contents of the image can be performed.

In step S112, the control device 104 specifies the second subject position based on the result of the ranking performed in step S111.

The control device 104 identifies the position of the representative island with the highest rank as the subject position in the target image based on the ranking result performed in step S111. As shown in FIG. 9, the representative island of M (R1) is ranked first. Therefore, the control device 104 specifies the envelope frame F3 of the representative island in M (R1) as the second subject position. FIG. 10A and FIG. 10B show the envelope frame F3 applied to FIG. 2A and FIG. 2B. As shown in FIG. 10A and FIG. 10B, in the second subject position specification based on the second evaluation value, the subject is specified more according to the content of the image, and thus the actual main subject is specified. When the second subject position is specified, the control device 104 ends the series of processes shown in FIG.

In the above-described processing, the example in which the second subject position is specified based only on the second evaluation value is shown. However, the second subject position is specified based on both the first evaluation value and the second evaluation value. Also good. Note that the first evaluation value and the second evaluation value may be handled equivalently, or may be comprehensively evaluated by appropriately weighting.

For example, a score is given according to the ranking based on the first evaluation value, a score is given according to the ranking based on the second evaluation value, and the respective scores are added to obtain the total score of each binarized image. calculate. Then, the final subject position may be specified based on the binarized image having the highest total score.

In the case where the second subject position is specified based on both the first evaluation value and the second evaluation value, the coefficient γ described in Expression (4) and Expression (5) is used as the first object value. You may change at the time of calculation of an evaluation value and a 2nd evaluation value. For example, by setting γ = 1 when calculating the first evaluation value and setting γ = 2.5 when calculating the second evaluation value, it is possible to specify with better balance.

Furthermore, the subject position may be specified in consideration of other conditions in addition to the first evaluation value and the second evaluation value. Which evaluation value and condition the subject position should be identified may be designated by the user or may be automatically selected by the control device 104.

According to the present embodiment described above, the following operational effects can be obtained.

(1) The control device 104 divides one image into a plurality of divided images based on the color information or luminance information of the target image, and binarizes each of the plurality of divided images using the color information or information. The binarized image is generated. Then, for each of the plurality of binarized images, a first evaluation value used for specifying the subject position in the target image is calculated, and based on the calculated first evaluation value, Specify the subject position. Further, based on the specified subject position, a second evaluation value used for re-specifying the subject position in the target image is calculated for each of the plurality of binarized images, and the calculated second evaluation value is calculated. The subject position in the target image is specified again based on the value. Thereby, the subject position in the target image can be specified with high accuracy. Therefore, even when the camera user cannot catch the subject, such as when the subject moves fast or wants to take a quick shot, for example, the first subject is identified assuming that the subject is at the center of the camera screen. By specifying the second subject based on the result, the subject position can be specified with high accuracy.

(2) The control device 104 re-specifies the subject position in the target image based on both the first evaluation value and the second evaluation value. As a result, the subject position can be specified with a better balance.

(3) The control device 104 determines, as the first evaluation value, at least a value related to the area of the white pixel region composed of white pixels in the binarized image, and the distance between the white pixel region and the predetermined reference region And the second evaluation value is based on at least the value related to the area and the value related to the distance between the white pixel region and the region based on the subject position specified based on the first evaluation value. To calculate. Accordingly, the subject position can be specified with high accuracy in consideration of the area of the island, the position of the island, and the like.

<Modification>
The camera according to the above-described embodiment can be modified as follows.

(1) In the above-described embodiment, the case where the present invention is applied to a camera has been described. However, the present invention can also be applied to other devices that can read and process images, such as personal computers and portable terminals.

(2) The binarization process and the labeling process described in the above embodiment are examples, and the present invention is not limited to this example. For example, any method may be used as long as one image can be divided into a plurality of divided images based on color information or luminance information, and binarization based on color information or luminance information may be used. Any method may be used. Also, any method may be used for specifying the island by the labeling process.

In the above-described embodiment, the example in which the image data of the target image is image data expressed in the RGB color system has been shown. However, color space conversion processing or the like is appropriately performed regardless of the data format. By doing so, the present invention can be applied as well.

(3) The method of calculating each evaluation value described in the above embodiment is an example, and the present invention is not limited to this example. For example, if the evaluation value is calculated based on the value related to the area of the white pixel region composed of white pixels in the binarized image and the value related to the distance between the white pixel region and a certain region, It can be anything.

Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired. Moreover, it is good also as a structure which combined the above-mentioned embodiment and a some modification.

DESCRIPTION OF SYMBOLS 100 ... Camera, 101 ... Operation member, 102 ... Lens, 103 ... Image pick-up element, 104 ... Control device, 105 ... Memory card slot, 106 ... Monitor

Claims

On the computer,
A division procedure for dividing one image into a plurality of divided images based on color information or luminance information of the target image;
A binarization procedure for binarizing each of the plurality of segmented images using the color information or luminance information to generate a plurality of binarized images;
A first evaluation value calculation procedure for calculating a first evaluation value used for specifying a subject position in the target image for each of the plurality of binarized images;
A first subject position specifying procedure for specifying a subject position in the target image based on the first evaluation value;
Based on the subject position specified in the first subject position specifying procedure, a second evaluation value used for re-specifying the subject position in the target image for each of the plurality of binarized images. A second evaluation value calculation procedure for calculating
A program for specifying a subject position for executing a second subject position specifying procedure for specifying again a subject position in the target image based on the second evaluation value.
In the subject position specifying program according to claim 1,
In the second subject position specifying procedure, the subject position in the target image is specified again based on both the second evaluation value and the first evaluation value. Program.
In the subject position specifying program according to claim 1 or 2,
The first evaluation value is based on at least a value related to an area of a white pixel region formed of white pixels in the binarized image and a value related to a distance between the white pixel region and a predetermined reference region. Calculated,
The second evaluation value is calculated based on at least a value related to the area and a value related to a distance between the white pixel region and a region based on the subject position specified in the first subject position specifying procedure. A program for specifying a subject position.
On the computer,
A generation procedure for binarizing pixels with a predetermined threshold to generate a valued image;
A first determination procedure for determining a degree of mass of the first pixel with reference to a predetermined position of the binarized image for the first pixel exceeding the predetermined threshold;
A second determination procedure for determining the degree of clumping of the first pixel with reference to a predetermined position in the clump of the first pixel determined to be larger than the predetermined threshold by the first determination procedure. When,
A subject position specifying program for executing a determination procedure for determining a subject position based on a determination result of the second determination procedure.
A division unit for dividing one image into a plurality of divided images based on color information or luminance information of the target image;
A binarization unit that binarizes each of the plurality of segmented images using the color information or luminance information to generate a plurality of binarized images;
A first evaluation value calculation unit that calculates a first evaluation value used for specifying a subject position in the target image for each of the plurality of binarized images;
A first subject position specifying unit for specifying a subject position in the target image based on the first evaluation value;
A second evaluation value used to re-specify the subject position in the target image for each of the plurality of binarized images based on the subject position specified by the first subject position specifying unit. A second evaluation value calculation unit for calculating
A subject position specifying device comprising: a second subject position specifying unit that specifies again a subject position in the target image based on the second evaluation value.
A division procedure for dividing one image into a plurality of divided images based on color information or luminance information of the target image;
A binarization procedure for binarizing each of the plurality of segmented images using the color information or luminance information to generate a plurality of binarized images;
A first evaluation value calculation procedure for calculating a first evaluation value used for specifying a subject position in the target image for each of the plurality of binarized images;
A first subject position specifying procedure for specifying a subject position in the target image based on the first evaluation value;
Based on the subject position specified in the first subject position specifying procedure, a second evaluation value used for re-specifying the subject position in the target image for each of the plurality of binarized images. A second evaluation value calculation procedure for calculating
An execution means for executing a subject position specifying program including a second subject position specifying procedure for specifying again a subject position in the target image based on the second evaluation value. camera.