JP5158438B2 - Image generating apparatus, image generating method, and program - Google Patents

Description

  The present invention relates to an image generation apparatus, an image generation method, and a program, and more particularly to an image generation apparatus, an image generation method, and a program correction related to correction of an image in which a plurality of persons are captured.

  For example, when photographing a person against the background of a natural landscape, it is considered preferable from the composition to include the upper part (so-called bust shot) from the chest because the face appears larger.

  FIG. 7 shows a bust shot image. As shown in (a), the photographed image 100 includes persons 102 and 103 with the mountain range 101 in the background. When photographing a plurality of persons with such a bust shot composition, the persons 102 and 103 are shown. If there is a height difference X, the positions of the faces of the persons 102 and 103 shift up and down, resulting in a photo with a poor compositional balance. In such a case, it is generally necessary to give instructions such as squatting a tall person or extending a tall person to a short person, which is troublesome and cumbersome.

  Of course, the suitability of the composition in the bust shot image depends largely on the subjectivity of the photographer. Therefore, the determination of suitability of composition in (a) is of course divided, but for example, in the case of a bust shot image 104 of a large number of people with different heights as shown in (b), there is a high possibility of having a sense of incongruity. Is also true. That is, in this photographed image 104, four persons 105 to 108 are shown, but the height of one person 107 of them is extremely low, and the balance with the other persons 105, 106, 108 becomes poor. Because. In the case of such a composition, many photographers have a sense of incongruity, and for example, there is a possibility of giving an instruction such as “stretch out” to the person 107.

  As described above, the height difference problem in person photographing is particularly noticeable when a large number of people are subjects, but the present invention is not intended to limit the number of subjects. In short, it focuses on the height difference problem, which may occur at least when shooting at least two persons.

  As a conventional technique applicable to the height difference problem, for example, “electronic camera and photographing recording method” described in Patent Document 1 below is known. To express this prior art in a straightforward manner, a half image obtained by photographing one of the two persons and a half image obtained by photographing the other of the two persons are combined into a single image (the two persons are photographed). Image).

  According to this, when taking each half-image, it is possible to make the heights of the two persons uniform based on something. As a reference, for example, a part of the background or an index line in the finder can be used.

JP 2003-274263 A

  However, in the above prior art, even if it is possible to align the height of a person using a part of the background or an index line in the viewfinder, it still squats a tall person In addition, there is no change in that it is necessary to instruct a person with a short height to extend his back, and the above trouble is not solved at all.

  Therefore, an object of the present invention is to provide an image generation apparatus, an image generation method, and a program that can easily obtain a corrected image in which a composition is corrected in an image in which a plurality of persons are captured.

According to a first aspect of the present invention, there is provided a detection means for detecting a person area from an image, a first determination means for determining whether or not there are a plurality of person areas detected by the detection means, and the first determination means. If it is determined that a plurality, extend selection means for selecting the person area of the person who satisfies a predetermined condition among the plurality of human region, the human region selected by the selecting means the height in the serial image An image generation apparatus comprising generation means for generating an image corrected as described above.
Invention of claim 2, wherein said generating means, by deforming the portion other than the face of the person of the selected human region, generating a corrected image to stretch the height for the person area The image generation apparatus according to claim 1, wherein:
Invention of claim 3, wherein said generating means, by deforming the face portion of a person of the selected human region, characterized by generating the corrected image to stretch the height for the person area The image generation apparatus according to claim 1.
The invention of claim 4, wherein, when it is determined that a plurality of detected by said first determination means, determines whether the difference between the height of the person included in these human region is within a predetermined range A plurality of detected persons when the difference in the upward height is determined to be within a predetermined range by the second determination means. The image generation apparatus according to claim 1, wherein a person area of a person that satisfies a predetermined condition is selected.
According to a fifth aspect of the present invention, the person having the highest height in the image and the other person among the persons included in these person areas when it is determined that a plurality of the first determining means has detected. And a third determining means for determining whether or not the height difference is equal to or greater than a predetermined value, wherein the selecting means is determined by the third determining means to be equal to or greater than a predetermined value, 4. The image generation apparatus according to claim 1, wherein a person region of a person that satisfies a predetermined condition is selected from the plurality of detected persons.
According to a sixth aspect of the present invention, a registration unit for registering an image of a person to be corrected, and whether or not the person image detected by the registration unit is included in the person area detected by the detection unit. And a fourth determination unit that selects the person area of the registered person when it is determined that the person registered by the fourth determination unit is included. The image generating apparatus according to claim 1, wherein:
The invention according to claim 7 is characterized in that the person satisfying the predetermined condition is a person having the lowest height in the image among the plurality of detected persons in the image. An image generation apparatus according to any one of claims 1 to 4.
In the invention according to claim 8, the person who satisfies the predetermined condition is a person other than the person having the highest height in the image in the image among the plurality of detected persons. An image generating apparatus according to claim 1, wherein:
According to the ninth aspect of the present invention, the person who satisfies the predetermined condition is that a difference in an upward height by the third determination unit among the plurality of detected persons is a predetermined value or more. The image generation apparatus according to claim 5, wherein the image generation apparatus is a determined person.
According to a tenth aspect of the present invention, the person who satisfies the predetermined condition is that, among the plurality of detected persons, a difference in an upward height by the third determination means is a predetermined value or more. The image generation apparatus according to claim 5, wherein the image generation apparatus is a determined person.
According to the eleventh aspect of the present invention, a detection step of detecting a person region from an image, a determination step of determining whether or not a plurality of persons are detected by the detection step, and a plurality of persons being detected by the determination step When the determination is made, a selection step of selecting a person area of a person who satisfies a predetermined condition among the detected persons, and a corrected image in which the height of the selected person area in the image is increased is generated. An image generation method including generation means.
According to a twelfth aspect of the present invention, a computer included in the image generation apparatus includes a detection unit that detects a person region from an image, a first determination unit that determines whether a plurality of persons are detected by the detection unit, A selection unit that selects a person area of a person that satisfies a predetermined condition among the detected persons when the first determination unit determines that a plurality of persons are detected; and the selected person area Fixed image extended heights in the in the image of a program for realizing a function as a generation means for generating.

  According to the present invention, it is possible to easily obtain a corrected image in which a composition is corrected in an image showing a plurality of persons.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, taking application to a digital camera as an example. It should be noted that the specific details or examples in the following description and the illustrations of numerical values, character strings, and other symbols are only for reference in order to clarify the idea of the present invention, and the present invention may be used in whole or in part. Obviously, the idea of the invention is not limited. In addition, a well-known technique, a well-known procedure, a well-known architecture, a well-known circuit configuration, and the like (hereinafter, “well-known matter”) are not described in detail, but this is also to simplify the description. Not all or part of the matter is intentionally excluded. Such well-known matters are known to those skilled in the art at the time of filing of the present invention, and are naturally included in the following description.

  FIG. 1 is a conceptual configuration diagram of a digital camera. In this figure, a digital camera 1 includes an optical system 2 including a photographing lens 2a, a zoom lens 2b, a focus lens 2c, and the like, and a CCD, a CMOS, or the like that captures an image of a subject 3 captured through the optical system 2. An image pickup unit 4 including a two-dimensional image sensor, a distance measuring unit 5 such as a contrast AF method or a hybrid AF method for measuring a distance to the subject 3, and image processing (gamma) required for an image signal picked up by the image pickup unit 4 An image processing unit 6 for performing correction, a focus driving unit 7 for driving the focus lens 2c, a zoom driving unit 8 for driving the zoom lens 2b, and various buttons (a mode switching button 9a between a shooting operation and a reproduction operation). , The menu button 9b, the cursor key 9c, the shutter button 9d, etc. A display unit 12 including a liquid crystal display, a touch panel 13 provided on the display surface of the display unit 12, and a storage unit 14 configured by a fixed or removable mass storage device. And an external input / output unit 16 that mediates data input / output with an external device 15 such as a personal computer as necessary, a power source unit 17 including a battery, and a control unit 18.

  The control unit 18 includes a computer (hereinafter referred to as CPU) 18a, a nonvolatile memory (hereinafter referred to as ROM) 18b, a volatile memory (hereinafter referred to as RAM) 18c, and a rewritable nonvolatile memory (hereinafter referred to as PROM) 18d. A control program stored in advance in the ROM 18b or data written in advance or arbitrarily in the PROM 18d is loaded into the RAM 18c and executed by the CPU 18a. However, the present invention is not limited to this, and it is needless to say that all or part of the functions may be realized by hard logic.

  The illustrated digital camera 1 operates in the shooting mode (still image or moving image shooting mode) when the mode switching button 9a of the operation unit 9 is in the “shooting” position, and operates in the playback mode when in the “playback” position. To do.

  When the still image or moving image shooting mode is selected, an image signal periodically output from the imaging unit 4 (several tens of frames per second) is output to the display unit 12 through the image processing unit 6 and the control unit 18 to confirm the composition. Continuously displayed as a live view image. The photographer adjusts the shooting direction and the angle of view (zoom magnification) of the imaging unit 4 so that the desired composition is obtained while viewing the through image, and when the desired composition is obtained, the release operation (depressing the shutter button 9d) is performed. Operation).

  In response to the release operation, AF (automatic focus) and AE (automatic exposure) are executed, and a high-quality image signal is extracted from the imaging unit 4. At this time, in shooting in a dark place (for example, shooting a night scene), the strobe driving unit 11 causes the strobe light emitting unit 10 to emit a strobe in response to the release operation. This image signal is sent to the storage unit 14 via the image processing unit 6 and the control unit 18 and is recorded and saved in the storage unit 14 as a captured image. The captured image may be a raw image corresponding to a high-quality image signal extracted from the imaging unit 4, but the raw image has a large size and presses the storage capacity of the storage unit 14. It is desirable to record an image compressed using a general-purpose compression technique such as JPEG (Joint Photographic Experts Group) as a captured image. In the moving image shooting mode, recording is started by the first release operation, recording is ended by the second release operation, and a high-quality image signal taken out from the imaging unit 4 during that period is a predetermined moving image file format ( For example, it is recorded and saved in the storage unit 14 in MPEG format.

  On the other hand, when the playback mode is selected, the most recently captured image (still image or moving image) is read from the storage unit 14 and enlarged and displayed on the display unit 12. Alternatively, the reduced images of the captured images are read from the storage unit 14 and displayed in a list on the display unit 12, an image desired to be reproduced is selected from the list, and the original image is read from the storage unit 14 and displayed. 12 is enlarged and displayed.

  The above-described operations in the shooting mode and the playback mode are conventionally known, but in the present embodiment, the following characteristic items are included in addition thereto.

  FIG. 2 is a conceptual configuration diagram showing the characteristic items of the present embodiment. In this figure, the control unit 18 includes several block units functionally realized by a program control method, specifically, a person area detection unit 19, a person correction amount calculation unit 20, and a corrected image generation unit 21. .

  The person area detection unit 19 detects the person areas 23 and 24 from the captured image 22 (this captured image 22 may be a reproduced image read from the storage unit 14). Here, the person area refers to an area considered to be a “person” among various subjects captured in the captured image 22. For example, since a technique for recognizing a person's face from an image (so-called face recognition technique) is known, first, using this technique, a person's face among various subjects captured in a captured image 22 is detected. Recognize the face part, then recognize the part connected to the face (specifically, the part that is at the same distance from the face and in focus), and combine both of these recognition parts 23 and 24 are also possible.

  The person correction amount calculation unit 20 corrects the height of each person when there are a plurality of detected person areas 23 and 24, that is, when a plurality of persons are imprinted in the captured image 22. The amount of correction required for the operation is calculated. This correction amount will be described by taking, for example, the case where the heights of two persons with different heights are aligned, and the person with the shorter height (the person area 24 on the right side in the illustrated example) is extended to the upper side of the image. It means the amount of correction. Here, the above-mentioned “height” does not need to be the height of an actual person, but means “the height above the image” of the person in the photographed image 22. Therefore, the above “upper side of the image” means the “upper side” of the top and bottom.

  Therefore, “the amount to extend the person with the shorter height to the upper side of the image” means that the height of the person with the shorter height is increased based on the person with a higher height, thereby reducing the height difference. This is the amount of correction to do (preferably to align height).

  The corrected image generation unit 21 generates a corrected image 25 in which the height of the person is corrected by applying the above correction amount to the captured image 22. That is, the corrected image 25 also includes two person areas 26 and 27 as in the photographed image 22 before correction, but the heights of the two person areas 26 and 27 are almost the same. This is different from the photographed image 22 before correction.

  Therefore, according to this configuration, it is possible to obtain the corrected image 25 in which the height of the person is corrected. Therefore, it is possible to easily compose a plurality of person images without taking the trouble of squatting or stretching the subject. Corrections can be made.

  In the above description, “desirably having the same height” means that the correction amount in the person correction amount calculation unit 20 is optimal. That is, it means a case where the height difference becomes “zero” when the correction amount is applied and the height of a person with a short height is increased based on a person with a high height. It is ideal that the height difference is zero, but in many cases, not so rigorously. For practical purposes, it is only necessary to reduce the height difference to such an extent that it does not cause a sense of incongruity, so the above correction amount is calculated assuming that “the height difference is zero” and that there is some margin before and after that. It only has to be done.

Next, the operation of this embodiment will be described.
FIG. 3 is a diagram showing a shooting operation flow of the present embodiment. In this flow, when a full-press of the shutter (release operation) is determined while displaying a through image (step S1) (step S2), first, an imaging process is executed (step S3), and then the captured image is displayed. A person area is detected from the inside (step S4). Then, it is determined whether or not a plurality of person areas have been detected, that is, whether or not a plurality of person areas have been detected (step S5). If a plurality of person areas have not been detected (or no person area has been detected). If it is determined that the captured image does not need to be corrected and the captured image is recorded in the storage unit 14 (step S6), the flow is terminated. It is determined whether or not the detected difference in the height of the person is within a predetermined range (step S7).

  If it is not within the predetermined range, that is, if there is no height difference or there is a height difference exceeding a predetermined value, it is determined that correction is not necessary or cannot be corrected, and the captured image is recorded in the storage unit 14 (step S6). If the height difference is within a predetermined range, that is, if the height difference needs to be corrected and can be corrected, the person who increases the height (the person whose height is to be corrected In particular, a person with a short height is selected (step S8), and the correction amount of the height of the selected person is calculated, and a corrected image in which the height of the selected person is increased is generated according to the correction amount (step S9). ), The corrected image is recorded in the storage unit 14 (step S10), and the flow ends.

  FIG. 4 is a conceptual diagram of height correction. As shown in (a), in the image 28 before correction, two persons 29 and 30 are shown for convenience. The background is a natural scenery, for example, a mountain range 31 that is visible in the distance. Here, assuming that the height of the left person 29 (here, the height is the length from the lower end of the image to the top of the head) is A, and the height of the right person 30 is B, A> B. There is a relationship. That is, the height of the right person 30 is low. Therefore, in the image 28 before correction, two persons 29 and 30 having a height difference are shown, and it can be said that the face position is shifted up and down and the composition is unbalanced.

  As described above, the height is corrected for the person 30 having a shorter height. That is, as shown in (b), the image area of the person 30 is extracted, and an image of the person 30 'is generated by extending the extracted person 30's height B to A' larger than B. Therefore, the correction amount in this case corresponds to the difference between B and A ′. Here, A ′ is preferably the same as the height A of the left person 29. However, in practice, it is only necessary to reduce the difference in the height of the appearance, so it is only necessary to bring A ′ closer to A. In this (b), the person 30 'indicated by hatching is the one after generating an image in which the height is extended to A'.

  Finally, by combining the generated image of the person 30 'with the original image 28, as shown in (c), a corrected image 32 with a reduced height difference (preferably with the same height) is obtained. obtain.

  The person 33 on the left side of the corrected image 32 is the same as the person 29 of the image 28 before correction, but the person 34 on the right side is an image of a person 30 ′ generated by extending the height to A ′. It is possible to easily correct the composition of a plurality of persons without taking the trouble of squatting or stretching the back.

  In the above description, the height B of the person 30 is set to A ′ substantially corresponding to the height A of the person 29 when the height is corrected. As a method for performing the correction from B to A ′, for example, it is conceivable to simply extend the dimension of the person 30 in the vertical direction (vertical direction) from the lower end of the image to the top of the head. This simple method can be used when the amount of correction from B to A ′ is small. This is because, if the amount of correction from B to A ′ is small, even if the dimension of the person 30 in the vertical direction (vertical direction) is extended, a large change in appearance is not felt.

  FIG. 5 is a conceptual diagram of height correction that is suitable when the amount of correction from B to A ′ is large. As shown in (a), the height of the person 30 to be corrected is B, of which the height of the face 30a is C, and the body part 30b below the remaining neck is D.

  In general, the sense of incongruity associated with the deformation of the person image appears prominently in the face 30a, and the deformation of the body portion 30b is not so noticeable unless it is a considerably large deformation, and particularly when the amount of correction from B to A 'is large. As shown in (b), it is desirable to extend the dimension D of the body portion 30b in the vertical direction (vertical direction) to D ′. Here, “D ′ = A′−C”.

  In this way, since the dimension C of the face 30a does not change, even if the amount of correction from B to A ′ is large, it is possible to perform a correction that does not seem strange to the eye.

  In this example, the face dimensions before and after the correction are both set to the same C, that is, the face dimensions C are not modified at all. However, the present invention is not limited to this. An appropriate amount of the face dimension C may be corrected together with the correction. In this case, it is necessary to make the face correction amount less conspicuous by making the face correction amount smaller than the body portion correction amount. That is, the face portion of the person to be corrected may be deformed to a small extent and the portion other than the face may be greatly deformed. Even in this case, the height of the person can be increased to reduce the height difference between the persons. To what extent the face portion should be corrected, an appropriate value that does not cause discomfort associated with the deformation of the face may be determined as appropriate.

  FIG. 6 is a diagram in which a part of the shooting operation flow of FIG. 3 is changed. In this figure, the changed part is a part surrounded by a broken line. Specifically, a part for determining whether or not the difference in height of the detected person is “predetermined value” or more (step S11), and the highest height. And a portion (step S12) for selecting a person having a large difference from a high person.

  The difference in height of the detected person is greater than or equal to a predetermined value means that, for example, in photographing three or more persons, the height difference from the tallest person is greater than or equal to a predetermined value. Further, the person having the largest difference from the tallest person is a person who has been determined that the height difference from the tallest person is greater than or equal to a predetermined value in the case of photographing three or more persons (for example, FIG. 7 In this case, a plurality of persons may be used as long as the height difference from the tallest person is a predetermined value or more.

  Therefore, in this change flow, it is possible to obtain a corrected image in which the height difference of the person whose height difference from the tallest person is equal to or greater than a predetermined value is corrected to reduce the height difference (preferably the height is aligned). It can be applied to a large number of subjects.

In the above description, a corrected image is obtained by correcting the height of a person with a short height to reduce the height difference (preferably with the same height), but it is not for a person with a short height. Alternatively, it may be corrected so that the height of a certain person increases. In this case, a method may be considered in which an image of a person whose height is desired is registered in advance, and when the registered person is included in the image, the person's height is corrected. This is also effective for a person who is worried about his height.
Further, in accordance with the person with the highest height, the height of all other persons may be corrected by an appropriate amount to obtain a corrected image in which the difference in height is reduced (preferably, the height is aligned).

  Further, an image of a person area is extracted from the photographed image 22, an image obtained by extending the extracted person area is generated, and the corrected image with a corrected height is generated by combining the generated image with the original photographed image 22. However, the method for generating the corrected image is not limited to this, and any method may be used as long as the method can correct the height, such as enlarging the extracted person region and combining it with the original captured image 22. .

  Further, the “height” in the above description does not have to be the actual height of the person, but may be the “upward height” of the person in the captured image 22. In other words, in general, when shooting a person, standing people are considered, so “upward height” means “height to the heaven” and “height” means “height to the heaven”. May be "". As a result, even if there is a difference in the “height to the top” of multiple persons in the captured image 22 regardless of the actual height due to shooting at a place with a step or inclination, correct as appropriate. Is possible.

  When shooting a night scene or the like in the background, the person area may be easily detected by illuminating the person brightly by emitting a strobe light.

  In the above description, application to the digital camera 1 is taken as an example, but the present invention is not limited to this. Any imaging device capable of photographing a person may be used, and for example, a mobile phone with a camera or a portable information terminal may be used. Alternatively, an image processing apparatus other than the imaging apparatus, such as a personal computer or a printer, may be used. In particular, printers have been put into practical use that can select and print an image in a recording medium by setting a recording medium such as a memory card. If possible, it is desirable because a printer with higher commercial value can be obtained.

It is a conceptual block diagram of a digital camera. It is a notional block diagram which shows the characteristic matter of this embodiment. It is a figure which shows the imaging | photography operation | movement flow of this embodiment. It is a conceptual diagram of height correction. It is a conceptual diagram of the height correction suitable when the amount of correction is large. FIG. 4 is a diagram in which a part of the shooting operation flow of FIG. 3 is changed. It is a figure which shows a bust shot image.

Explanation of symbols

1 Digital Camera 4 Imaging Unit 14 Storage Unit 18 Control Unit 18a CPU

Claims (12)

Detecting means for detecting a person region from an image;
First determination means for determining whether or not there are a plurality of person areas detected by the detection means;
A selection unit that selects a person area of a person that satisfies a predetermined condition from the plurality of person areas when the first determination unit determines that the plurality of person areas;
Image generating apparatus according to claim further comprising: a generating means for generating a modified image as the person area selected by the selecting means extending the height in the said image. It said generating means, by deforming the portion other than the face of the person of the selected human region, to claim 1, characterized in that to generate a corrected image to stretch the height for the person area The image generating apparatus described. The said generation means produces | generates the image corrected so that the said height might be extended about the said person area by changing the face part of the person of the said selected person area. Image generation device. If it is determined that a plurality of detected by said first determination means, the difference in the height of the person included in these human region further the second determination means for determining whether it is within a predetermined range Prepared,
The selecting means is a person who satisfies a predetermined condition among the plurality of detected persons when the second determining means determines that the height difference is within a predetermined range. 4. The image generation apparatus according to claim 1, wherein an area is selected. When it is determined by the first determination means that a plurality of objects are detected, a difference in height between a person having the highest height in the image and another person among the persons included in these person regions is predetermined. A third determination means for determining whether or not the value is greater than or equal to the value;
The selection unit selects a person region of a person that satisfies a predetermined condition from the plurality of detected persons when the third determination unit determines that the value is equal to or greater than a predetermined value. The image generation apparatus according to claim 1. A registration means for registering an image of a person to be corrected;
Fourth determination means for determining whether or not the person area detected by the registration means is included in the person area detected by the detection means;
Further comprising
4. The method according to claim 1, wherein the selection unit selects a person area of the registered person when it is determined that the registered person is included by the fourth determination unit. 5. The image generating apparatus described in 1. 5. The person satisfying the predetermined condition is a person having the lowest height in the image in the image among the plurality of detected persons. An image generating apparatus according to claim 1. 2. The person satisfying the predetermined condition is a person other than the person with the highest height in the image in the image among the plurality of detected persons. 5. The image generating device according to any one of 4 above.   The person satisfying the predetermined condition is a person whose height difference is determined to be greater than or equal to a predetermined value by the third determination unit among the plurality of detected persons. The image generation apparatus according to claim 5. An imaging unit that captures an image by capturing an image
The image generation apparatus according to claim 1, further comprising an imaging unit that captures an image of the plurality of persons. A detection step of detecting a person region from the image;
A determination step for determining whether or not a plurality of persons are detected by the detection step;
A selection step of selecting a person area of a person who satisfies a predetermined condition among the detected persons when it is determined that a plurality of persons are detected by the determination step;
An image generating method comprising: generating means for generating a corrected image in which the height of the selected person region in the image is increased. In the computer provided in the image generation device,
Detecting means for detecting a person region from an image;
First determination means for determining whether or not a plurality of persons are detected by the detection means;
A selection unit that selects a person area of a person that satisfies a predetermined condition among the detected persons when it is determined that a plurality of persons are detected by the first determination unit;
A program for realizing a function as a generation unit that generates a modified image in which the height of the selected person region in the image is increased.

Images