JP2010011189A - Image processing apparatus, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus which can effectively use even an image with any defect without correction for removing the defect, and to provide an image processing program. <P>SOLUTION: The image processing apparatus 1 detects an object included in an image on the basis of an image pattern in a picture signal, and it determines the photographing condition of the detected object based on the picture signal. When the photographing condition of the determined object is defective, an image item corresponding to the defect is pasted to the specified attachment area of the image. The image item preferably has a content reflecting the defect of the personal photographing condition. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program for performing predetermined image processing on an image.

  Unlike a film camera, an imaging device such as a digital camera that captures a subject and generates digital image data can perform image processing on the captured image. For example, there is known a technique for determining the quality of an image by performing image analysis processing based on a predetermined determination criterion for an image captured by a digital camera (see, for example, Patent Document 1). In this technology, the degree of focus or brightness of the image is adopted as a criterion, and it is detected by detecting whether a problem such as defocusing, blackout, or overexposure has occurred in the subject in the image. The quality of the image is judged. According to such a technique, when a large number of prints of captured images are ordered in a lump, it is possible to order except for images judged to be defective, thereby realizing resource saving and a reduction in print cost. Can do.

JP 2001-256498 A

  In the above-described conventional technology, it is assumed that the user additionally orders an image determined to be defective, and the storage period of the image determined to be defective can be set longer than the storage period of the image determined to be good. However, in reality, it is unlikely that the user checks the image determined to be defective again, and the image determined to be defective is rarely used effectively after the determination.

  In order to solve this problem, it is conceivable to perform correction for eliminating the defect on the image determined to be defective. However, it is difficult to say that an image determined as defective is effectively used because an image desired by the user is not always obtained by such correction.

  The present invention has been made in view of the above, and provides an image processing apparatus and an image processing program that can be effectively used even if an image has a defect without correcting the defect. For the purpose.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention detects a subject included in an image based on an image pattern in an image signal corresponding to the image to be processed. And a shooting state determination unit that determines a shooting state of the subject detected by the subject detection unit based on the image signal, and a shooting state of the subject that is determined by the shooting state determination unit. And an image composition unit for pasting an image item composed of at least one of a picture, a pattern, a character, a number, and a symbol to a predetermined pasting area of the image.

  The image processing apparatus according to the present invention is characterized in that, in the above-mentioned invention, the image item has contents reflecting a defect in a shooting state of the subject.

  In the image processing apparatus according to the present invention, in the above invention, a plurality of the image items are stored, and the determination result of the shooting state determination unit corresponds to the type and the pasting area of the image item to be pasted on the image. A storage unit for storing the relationship is further provided.

  In the image processing apparatus according to the present invention, in the above invention, the subject detection unit can detect a person's face, and the shooting state determination unit determines whether or not the person is in focus. A focus determination unit, an image blur determination unit that determines whether image blur has occurred in the person image, and a light / dark determination to determine whether blackout or whiteout has occurred in the person image And a blinking determination unit for determining whether or not the person has closed his / her eyes.

  In the image processing apparatus according to the present invention as set forth in the invention described above, the photographing state determination unit further includes an age / gender determination unit that determines the age and sex of the person.

  The image processing apparatus according to the present invention is the image processing apparatus according to the above-described invention, wherein the imaging unit that captures an image of a subject within a predetermined field of view and generates an image signal, and the image composition unit based on the image captured by the imaging unit. And a display unit for displaying an image synthesized.

  In addition, an image processing program according to the present invention includes an image processing apparatus including a storage unit that stores a plurality of image items each including at least one of a picture, a pattern, characters, numbers, and symbols. A subject detection step of detecting a subject included in the image based on an image pattern in an image signal corresponding to the image; a shooting state determination step of determining a shooting state of the subject detected in the subject detection step based on the image signal When there is a problem in the shooting state of the subject determined in the shooting state determination step, an image composition step of reading out the image item corresponding to the defect from the storage unit and pasting it on a predetermined pasting area of the image, It is made to perform.

  According to the present invention, the shooting state of the subject detected based on the image pattern in the image signal of the image to be processed is determined based on the image signal. Since the image item is affixed to a predetermined affixing area of the original image, it is possible to add value to an image having a problem in the shooting state of the subject. Therefore, even an image having a defect can be effectively used without performing correction for eliminating the defect.

  The best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described below with reference to the accompanying drawings.

(Embodiment 1)
FIG. 1 is a diagram schematically showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention. An image processing apparatus 1 shown in the figure is a digital camera, and includes an imaging unit 2 that captures a predetermined visual field and generates image data, a sensor unit 3 that detects the state of the visual field of the imaging unit 2, and an imaging unit 2 An image processing unit 4 that performs image processing on the output image data, a display unit 5 that displays information including image data output by the image processing unit 4, a display control unit 6 that controls display of the display unit 5, and image data A compression / decompression unit 7 that compresses and decompresses the image, and records information such as image data on a recording medium 100 such as a memory card that is externally attached to the image processing apparatus 1, while reading information recorded on the recording medium 100 A recording medium interface 8, an input unit 9 to which an operation instruction signal of the image processing apparatus 1 is input, an audio output unit 10 that outputs audio of various information, and an auxiliary light projection unit 1 that projects auxiliary light toward an imaging region , A storage unit 13 for storing control unit 12 for controlling the operation of the image processing apparatus 1, various information related to the operation of the image processing apparatus 1.

  The imaging unit 2 receives a photographing lens 21 that collects light from a subject in a predetermined field of view, a diaphragm 22 that adjusts an incident amount of light collected by the photographing lens 21, and incident light that has passed through the diaphragm 22. The image sensor 23 such as a CCD that converts the signal into an electrical signal, the analog signal processor 24 that performs various processes including sensitivity correction and white balance on the analog signal output from the image sensor 23, and the analog signal processor 24 output the signal. An A / D conversion unit 25 that generates digital image data from the signal is included. Among these, the photographing lens 21 is an optical system including one or a plurality of lenses, and is driven by the lens driving unit 14. The diaphragm 22 is driven by the diaphragm driving unit 15.

  The image processing unit 4 detects the subject included in the image corresponding to the image signal based on the image pattern in the image signal, and the shooting state of the subject detected by the subject detection unit 41 based on the image signal. As a result of the determination by the shooting state determination unit 42 and the shooting state determination unit 42, if there is a defect in the shooting state of the subject, this problem is dealt with, and pictures, patterns (including graphics, etc.), letters, numbers, and symbols An image composition unit 43 for pasting at least one of the image items to a predetermined pasting region of the image.

  The subject detection unit 41 has a function of detecting a person's face as a subject, and also has a function of extracting positions of the detected person's face such as eyes, mouth, and nose as face feature information. The function of the subject detection unit 41 is, for example, the Viola-Jones method (P. Viola and M. Jones, “Rapid object detection using a boosted cascade of”, which performs subject detection in an image using a discrimination method based on a rectangular feature. simple features ”, Proc. of CVPR, (2001).

  The shooting state determination unit 42 determines the shooting state including the appearance of the person and facial expression detected by the subject detection unit 41 based on the image signal. Specifically, the shooting state determination unit 42 determines whether the person detected by the subject detection unit 41 is in focus, the focus determination unit 421 that determines whether or not the person is in focus, and the image blur of the person detected by the subject detection unit 41. An image blur determination unit 422 that determines whether or not an image has occurred, a light and dark determination unit 423 that determines whether or not a blackout or a whiteout has occurred in a person image detected by the subject detection unit 41, and a subject detection unit 41 includes a blinking determination unit 424 that determines whether the person detected by the person 41 is blinking.

  The focus determination unit 421 extracts a high-frequency component included in a predetermined region in the image, calculates an integrated value of the extracted high-frequency component, and then compares the integrated value with a predetermined threshold, thereby focusing on the person. Is determined (whether or not out-of-focus is present) (for details of the focus determination, refer to, for example, Patent Document 1).

  The image blur determination unit 422 determines whether or not an image blur due to the camera shake of the photographer or the subject blur has occurred based on the extraction result of the human face feature information detected by the subject detection unit 41. . When performing this determination, the image blur determination unit 422 may perform matching with an image pattern in the case where image blur has occurred.

  The brightness determination unit 423 acquires luminance data of a predetermined area in the image, calculates an average value of the acquired luminance data, and then blacks out if the average value is below a predetermined first threshold value. It is determined that the image. On the other hand, when the calculated average value exceeds a predetermined second threshold value that is higher than the first threshold value, it is determined that the image is a whiteout image (for more detailed contents of the light / dark determination, see, for example, Patent Document 1 above). reference).

  The blinking determination unit 424 detects the size of the black region in the eye region of the human face extracted by the subject detection unit 41 in the image as the opening of the eye, and then uses the detection result as a predetermined threshold value. By comparing, it is determined whether or not the person has closed his eyes (for details of the blinking determination, refer to, for example, Japanese Patent Laid-Open No. 9-147120).

  The input unit 9 includes a power button of the image processing apparatus 1, a shutter button for giving an imaging instruction, a mode switching button for switching various operation motors of the image processing apparatus 1, a control button including instructions for reproducing and editing image data, etc. Have It is also possible to realize at least a part of various buttons constituting the input unit 9 using a cross key or a touch panel.

  The control unit 12 includes a clock 121 that determines the shooting date and time, and controls the operation of the image processing apparatus 1 according to the detection result of the sensor unit 3 and the operation instruction signal input by the input unit 9. The control unit 12 is realized using an MPU or the like, and is connected to each part to be controlled via a bus line.

  The storage unit 13 includes a program storage unit 131 that stores various programs executed by the image processing apparatus 1, and a subject detection information storage unit 132 that stores information such as an image pattern necessary for the subject detection unit 41 to perform subject detection processing. The correspondence table storage unit 133 stores a correspondence table that gives a correspondence relationship between the determination result of the shooting state determination unit 42 and the type and image area of the image item to be pasted on the image according to the determination result. The shooting state determination unit 42 The image item storage unit 134 stores a plurality of image items to be attached to the processing target image according to the determination result. The program storage unit 131 stores an image processing program according to the first embodiment. The storage unit 13 is realized using a semiconductor memory such as a flash memory or a RAM that is fixedly provided inside the image processing apparatus 1.

  FIG. 2 is a flowchart showing an outline of an image processing method performed by the image processing apparatus 1 having the above configuration. In the image processing method described below, it is assumed that the reproduction mode for reproducing an image taken by the image processing apparatus 1 is set.

  First, the image processing apparatus 1 displays the reproduced image read from the recording medium 100 via the recording medium interface 8 on the display unit 5 (step S1). In step S1, an image selected by the user via the input unit 9 can be displayed.

  When a deletion instruction is input from the input unit 9 by the user who has viewed the playback image (step S2, Yes), the subject detection unit 41 detects a subject included in the image data of the playback image (step S3). On the other hand, when a deletion instruction is not input for the reproduced image displayed on the display unit 5 (No in step S2), the image processing apparatus 1 ends a series of processes.

  When the subject detection unit 41 detects a human face as a subject (step S4, Yes), the shooting state determination unit 42 indicates the shooting state of the person (whether there is blurring, image blurring, blackout, whiteout, and blinking). A determination is made for each detected person (step S5). On the other hand, when the subject detection unit 41 does not detect a person's face as a subject (step S4, No), the image processing apparatus 1 deletes the designated image (step S6), and ends a series of processing.

  When there is some trouble in the shooting state of the person determined by the shooting state determination unit 42 (step S7, Yes), the image composition unit 43 refers to the correspondence table stored in the correspondence table storage unit 133 of the storage unit 13. An image is synthesized by pasting an image item corresponding to the determination result of the shooting state determination unit 42 to a predetermined pasting area in the image (step S8). Thereafter, the display unit 5 displays the synthesized image synthesized by the image synthesizing unit 43 (step S9).

  FIG. 3 is a diagram illustrating a configuration of the correspondence table stored in the correspondence table storage unit 133. FIG. 4 is a diagram schematically illustrating a pasting area of an image item with respect to a person image. In FIG. 4, six pasting regions B to G are provided in the vicinity region of the person 301 so as to surround the pasting region A around the pasting region A corresponding to the face of the person 301. As described above, for the person detected by the subject detection unit 41, it is possible to set an image item pasting region including seven regions.

  Hereinafter, the content of the process of the image composition unit 43 according to the determination result of the shooting state determination unit 42 will be specifically described. First, a case where the in-focus determination unit 421 determines that a person in the reproduced image is out of focus (has out of focus) will be described. FIG. 5 is a diagram illustrating a display example of a reproduced image that has been determined to be “out of focus” by the focus determination unit 421. In the reproduced image 201 shown in the figure, the out-of-focus person 302 is indicated by a broken line. In this case, the image composition unit 43 attaches the three image items I-1, I-2, and I-3 to the pasting areas A, B, and F for the person 302 by referring to the correspondence table T1. In the table T1, the pasting area of the image item I-1 is described as “A *”, which means that the image item I-1 is larger than the area of the pasting area A.

  FIG. 6 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 based on the reproduced image 201. In the composite image 202 shown in the figure, the image item I-1 is a picture of a ghost outline, and the image items I-2 and I-3 are bat pictures. Among these, the image item I-1 is arranged so that the head of the ghost overlaps the head of the person 302.

  Next, a case where the image blur determination unit 422 determines that image blur has occurred in a person image will be described. FIG. 7 is a diagram illustrating a display example of a reproduced image determined as “image blurring” by the image blur determination unit 422. In the reproduced image 203 shown in the figure, the person 303 has an image blur in the vertical direction. In this case, the image composition unit 43 refers to the correspondence table T1 to attach the four image items I-4, I-5, I-6, and I-7 to the pasting areas A, B, C, and D for the person 303. A composite image is generated by pasting each of the images.

  FIG. 8 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 based on the reproduced image 203. In the composite image 204 shown in the figure, the image item I-4 is a helmet picture, the image item I-5 is a character string “GAGAGAGAGAGAGA”, and the image item I-6 is a picture of a signboard under construction. Yes, image item I-7 is a picture of construction work clothes. Among these, the image item I-4 is arranged on the head of the person 303.

  Next, a case where the light / dark determination unit 423 determines that blackout has occurred in a person's image will be described. FIG. 9 is a diagram illustrating a display example of a reproduced image determined as “blackout” by the light / dark determination unit 423. In the reproduced image 205 shown in the figure, the person 304 is blacked out. In this case, the image composition unit 43 refers to the correspondence table T1 to create a composite image by pasting the image item I-8 on the pasting area A for the person 304.

  FIG. 10 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 based on the reproduced image 205. In the composite image 206 shown in the figure, the image item I-8 is a picture of the head of the Great Buddha. Such an image item I-8 is arranged on the head of the person 304.

  Next, a case where the light / dark determination unit 423 determines that whiteout has occurred in a person's image will be described. FIG. 11 is a diagram illustrating a display example of a reproduced image that is determined to be “exposed white” by the light / dark determination unit 423. In the reproduced image 207 shown in the figure, a white-out person 305 is indicated by a one-dot chain line. In this case, the image composition unit 43 refers to the correspondence table T1 to paste the image item I-9 on the pasting area A for the person 305. However, the image item I-9 has an area larger than the area of the pasting area A (indicated as “A *” in the table T1).

  FIG. 12 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 based on the reproduced image 207. In the composite image 208 shown in the figure, the image item I-9 is a picture of a male aristocratic costume. This image item I-9 is arranged so that the position of the face of the nobility matches the face of the person 305.

  Next, a case where the blinking determination unit 424 determines that there is a blinking of a person will be described. FIG. 13 is a diagram illustrating a display example of a reproduced image that is determined as “with blinking” by the blinking determination unit 424. In the reproduced image 209 shown in the figure, the person 306 is closed. In this case, the image composition unit 43 attaches the image items I-10 and I-11 to the attachment regions A and G for the person 306 by referring to the correspondence table T1.

  FIG. 14 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 based on the reproduced image 209. In the composite image 210 shown in the figure, the image item I-10 is a picture of a nose balloon, and the image item I-11 is a character string “Zzz. Among these, the image item I-10 is arranged so as to correspond to the position of the nose of the person 306.

  As described above, the image item to be attached to the reproduction image has contents reflecting the defect in the shooting state of the person determined by the shooting state determination unit 42. In particular, the above-described image items I-1 to I-10 express the content of the defect of the original image in a humorous manner, so that the user who viewed the composite image saves or prints the composite image. There is a high possibility of doing. Therefore, even an image in which a defect has occurred can be effectively used by pasting an image item having contents reflecting the defect.

  By the way, as a result of determining the shooting state of the person by the shooting state determination unit 42, if there are a plurality of problems among blurring, image blurring, blackout, whiteout, and blinking, all image items are pasted. May be. However, in this case, there is a possibility that a plurality of image items may be affixed to one affixing area. Therefore, it is more preferable to set the priority order of image items to be affixed to each affixing area in advance.

  The display unit 5 displays an inquiry image for inquiring whether or not to save the displayed composite image when a predetermined time has elapsed since the start of displaying the composite image in step S9 described above (step S10). FIG. 15 is a diagram illustrating a display example of an inquiry image. The inquiry image 211 shown in the figure is a message “Do you want to save this image?” And “Are you sure you want to save this image?” The options “Yes” and “No” are displayed in an overlapping manner. In FIG. 15, the character “Yes” is displayed in bolder than the character “No”, and “Yes” can be selected.

  The user who has seen the inquiry image 211 inputs a selection signal for selecting whether or not to save the composite image from the input unit 9. When the user selects “Yes” via the input unit 9 (step S11, Yes), the image processing apparatus 1 displays a re-inquiry image for inquiring again whether or not to delete the reproduction image before synthesis. (Step S12). The reason for inquiring again about whether or not to delete the reproduced image is that the user who viewed the composite image may want to leave the reproduced image.

  FIG. 16 is a diagram illustrating a display example of a re-inquiry image. The re-inquiry image 212 shown in the figure is “Would you like to delete the pre-combination image?” With respect to the composite image displayed in step S9 (FIG. 16 shows the case of the composite image 202 shown in FIG. 6). "And a choice of" Yes "and" No "are displayed in an overlapping manner. In FIG. 16, the character “No” is displayed in bolder than the character “Yes”, and “No” is selectable.

  When the user who viewed the re-inquiry image 212 selects “No” via the input unit 9 and leaves the reproduced image without deleting it (step S13, No), the image processing apparatus 1 uses the compression / decompression unit 7 to display the composite image. The compressed data is written and stored in the recording medium 100 via the recording medium interface 8 (step S14). On the other hand, when the user who has seen the re-inquiry image 212 selects “Yes” via the input unit 9 and deletes the reproduced image (step S13, Yes), the image processing apparatus 1 records the composite image in the same manner as described above. While the image is written and stored in 100, the reproduction image before the synthesis image is deleted from the recording medium 100 (step S15).

  Next, a case where the user who viewed the inquiry image 211 selects “No” via the input unit 9 in step S11 (step S11, No) will be described. In this case, the image processing apparatus 1 deletes the composite image being displayed on the display unit 5 and also deletes the reproduced image from the recording medium 100 (step S16). Note that when the user selects to delete the composite image, it may be inquired again about whether or not to delete the reproduction image before the composite.

  According to the first embodiment of the present invention described above, the shooting state of the subject detected based on the image pattern in the image signal of the image to be processed is determined based on the image signal, and there is a problem with the shooting state of the subject. In some cases, since an image item corresponding to this defect is affixed to a predetermined affixing area of the original image, an added value can be generated for an image having a defect in the shooting state of the subject. Therefore, even an image having a defect can be effectively used without performing correction for eliminating the defect. As a result, waste of captured images can be reduced.

(Embodiment 2)
FIG. 17 is a diagram schematically illustrating a configuration of an imaging apparatus that is an image processing apparatus according to Embodiment 2 of the present invention. The image processing apparatus 16 shown in the figure has the same configuration as that of the image processing apparatus 1 described in the first embodiment except for the configuration of the image processing unit 17.

  The image processing unit 17 of the image processing device 16 includes a subject detection unit 41, a shooting state determination unit 44, and an image composition unit 43. Among these, the shooting state determination unit 44 includes the in-focus determination unit 421, the image blur determination unit 422, the light / dark determination unit 423, and the blinking determination unit 424, and the person's face detected by the subject detection unit 41. An age / sex determination unit 425 for determining age and sex is provided.

  The age / gender determination unit 425 collates the feature information extracted for the face of the person detected by the subject detection unit 41 with the personal face feature information previously created for a wide age group for each gender. It has a function of determining the age and sex of the person by calculating the degree of similarity between the two (see, for example, JP-A-2005-165447 for more detailed contents of age / sex determination).

  In the second embodiment, the outline of the image processing method performed when the image processing device 16 is set to the reproduction mode is the same as that of the first embodiment described above. However, in the second embodiment, when the shooting state of the person in the image is determined in step S5, the shooting state determination unit 44 is a process in which the age / sex determination unit 425 determines the age and gender of the person in the image. Do further. In the second embodiment, the correspondence table referred to by the image composition unit 43 in step S8 is different from that in the first embodiment.

  FIG. 18 is a diagram illustrating a configuration of a correspondence table stored in the correspondence table storage unit 133. The correspondence table T2 shown in the figure shows the types of image items to be affixed to the reproduction image based on the combination of the content of the defect (out-of-focus, image blurring, blackout, whiteout, blinking) and the age and sex of the person. An affixing area is given. In the correspondence table T2, the age of the person is divided into four categories: “10 years old or younger”, “11-30 years old”, “31-70 years old”, and “71 years old and older”.

  The content of the processing of the image composition unit 43 according to the determination result of the shooting state determination unit 44 will be specifically described. As is clear from the comparison between the correspondence table T1 and the correspondence table T2, when the age / gender determination unit 45 determines that the person in the image is “male under 10 years old”, the determination result of the shooting state determination unit 44 The processing of the corresponding image composition unit 43 is the same as that in the first embodiment.

  Hereinafter, a case will be described in which the age / sex determination unit 45 determines that the person in the image is “female under 10 years old”. First, a case where the in-focus determination unit 421 determines “has a blur” will be described. In this case, the image composition unit 43 attaches the image items K-1, K-2, and K-3 to the attachment regions A, B, and F for the person to be processed by referring to the table T2. In the table T2, the pasting area of the image item K-1 is described as “A *”, which means that the image item K-1 is larger than the area of the pasting area A.

  FIG. 19 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 when the focus determination unit 421 determines “defocused”. Note that the reproduced image before synthesis is the same as the reproduced image 201 shown in FIG. In the composite image 213 shown in FIG. 19, the image item K-1 is a picture of a female ghost, and the image items K-2 and K-3 are pictures of a human soul. Among these, the image item K-1 is arranged so that the head of the ghost overlaps the head of the person 307.

  Next, a case where the image blur determination unit 422 determines that “image blur exists” will be described. In this case, the image composition unit 43 refers to the table T2 to paste the image items K-4, K-5, and K-6 on the pasting areas A, E, and F for the person to be processed. In addition, the image item K-4 has an area larger than the area of the pasting area A (denoted as “A *” in the table T2) similarly to the image item K-1.

  FIG. 20 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 when the image blur determination unit 422 determines “image blur is present”. It is assumed that the reproduced image before synthesis is the same as the reproduced image 203 shown in FIG. In the composite image 214 shown in FIG. 20, the image item K-4 is a picture of a mobile phone, the image item K-5 is a picture of a musical note, and the image item K-6 is a picture schematically showing the propagation of sound waves ( Pattern). Among these, the image item K-4 is arranged so that the display of the mobile phone comes to the position of the face of the person 308.

  Next, a case where the light / dark determination unit 423 determines that “blackout is present” will be described. In this case, the image composition unit 43 refers to the table T2, and pastes the image item K-7 on the pasting area A for the person to be processed.

  FIG. 21 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 when the light / dark determination unit 423 determines “blackout is present”. It is assumed that the reproduced image before synthesis is the same as the reproduced image 205 shown in FIG. In the composite image 215 shown in FIG. 21, the image item K-7 is a picture of an angel's circle. This image item K-7 is arranged on the head of the person 309.

  Next, a case where the light / dark determination unit 423 determines that “there is whiteout” will be described. In this case, the image composition unit 43 attaches the image item K-8 to the application region A for the person to be processed by referring to the table T2. However, the image item K-8 has an area larger than the area of the pasting region A (indicated as “A *” in the table T2).

  FIG. 22 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 when the light / dark determination unit 423 determines that “there is whiteout”. It is assumed that the reproduced image before synthesis is the same as the reproduced image 207 shown in FIG. In the composite image 216 shown in FIG. 22, the image item K-8 is a picture of a female aristocratic costume. The image item K-8 is arranged so that the position of the face of the nobility's costume matches the position of the face of the person 310.

  Next, a case where the blinking determination unit 424 determines “with blinking” will be described. In this case, the image composition unit 43 refers to the table T2, and pastes the image item K-9 on the pasting area A for the person to be processed, and pastes the image item K-10 on the pasting areas B to B for the person to be processed. Affix to 6 locations in G respectively.

  FIG. 23 is a diagram illustrating a display example of a composite image generated by the image composition unit 43 when the eye blink determination unit 424 determines that “there is blinking”. Note that the reproduced image before synthesis is the same as the reproduced image 209 shown in FIG. In the composite image 217 shown in FIG. 23, the image item K-9 is a girl-manga style eye picture, and the image item K-10 is a rose flower picture. The image item K-9 is arranged so as to come to the eye position of the person 311.

  The composite images 213 to 217 when determined to be “female under 10 years old” by the determination of the age / sex determination unit 425 include image items that are completely different from the composite image determined as “male under 10 years old”. Is affixed. As described above, in the second embodiment, even if the shooting state of the person has the same problem, the combined image is completely different depending on the age and sex of the person.

  Although not shown in FIG. 18, the types of image items that differ for each gender even when the age of the person is “11 to 30 years old”, “31 to 71 years old”, or “71 years old or older”. And an affixing area, respectively.

  According to the second embodiment of the present invention described above, the shooting state of the subject detected based on the image pattern in the image signal of the image to be processed is determined based on the image signal, and there is a problem with the shooting state of the subject. In some cases, since an image item corresponding to this defect is affixed to a predetermined affixing area of the original image, an added value can be generated for an image having a defect in the shooting state of the subject. Therefore, similarly to the first embodiment described above, even an image having a defect can be used effectively without performing correction for eliminating the defect, and the waste of the captured image can be reduced.

  Further, according to the second embodiment, since the age and sex of a person are also determined when determining the shooting state of the person in the image, a composite image that reflects the personal information of the person who is the subject is generated. It becomes possible.

(Other embodiments)
Up to this point, the first and second embodiments have been described in detail as the best mode for carrying out the present invention. However, the present invention should not be limited by these embodiments. For example, in the present invention, information related to the age and sex of the user of the image processing apparatus is input and stored in the image processing apparatus in advance, and the combination of the age and sex of the user and the shooting condition of the person who is the subject is combined. Accordingly, the type of image item to be pasted on the image and the pasting area may be determined.

  Further, in the present invention, a composite image may be generated based on a combination of age / sex information of a person who is a subject, age / sex information of a user of an image processing apparatus, and a defect in a shooting state of the person.

  In the present invention, the image composition unit may generate a plurality of composite images for one image, and a user can select a desired composite image from the generated composite images. Good.

  The present invention can also be applied to subjects other than people. However, when the present invention is applied to a general subject, the functions of blinking determination and age / sex determination are unnecessary.

  The image processing apparatus according to the present invention may not have an imaging function. In this case, it is more preferable that the image processing apparatus has a function capable of transmitting and receiving image data via the Internet with an imaging apparatus such as a digital camera that captures an image to be processed.

  The image processing program according to the present invention can be widely distributed by being recorded on a computer-readable recording medium such as a CD-ROM, DVD-ROM, MO disk, or flexible disk.

  Thus, the present invention can include various embodiments and the like not described herein, and various design changes and the like can be made without departing from the technical idea specified by the claims. It is possible to apply.

It is a figure which shows typically the structure of the image processing apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the outline | summary of the image processing method which the image processing apparatus which concerns on Embodiment 1 of this invention performs. It is a figure which shows the structure of the corresponding | compatible table which the image synthetic | combination part of the image processing apparatus which concerns on Embodiment 1 of this invention refers. It is a figure which shows typically the sticking area | region of the image item with respect to the image of the person who is a subject. It is a figure which shows the example of a display of the reproduction | regeneration image when defocusing has generate | occur | produced in the image of the person who is a subject. It is a figure which shows the example of a display of a synthesized image when the blur has generate | occur | produced in the image of the person who is a subject. It is a figure which shows the example of a display of the reproduction | regeneration image when the image blurring has generate | occur | produced in the image of the person who is a subject. It is a figure which shows the example of a composite image displayed when the image blurring has generate | occur | produced in the image of the person who is a subject. It is a figure which shows the example of a display of the reproduction | regeneration image when the blackout has generate | occur | produced in the image of the person who is a subject. It is a figure which shows the example of a composite image displayed when the blackening has generate | occur | produced in the image of the person who is a to-be-photographed object. It is a figure which shows the example of a display of the reproduction | regeneration image in case the overexposure has generate | occur | produced in the image of the person who is a subject. It is a figure which shows the example of a display of a synthesized image when the overexposure has generate | occur | produced in the image of the person who is a subject. It is a figure which shows the example of a display of the reproduction | regeneration image when the person who is a subject has closed eyes. It is a figure which shows the example of a display of a synthesized image when the person who is a subject has closed eyes. It is a figure which shows the example of a display of an inquiry image. It is a figure which shows the example of a display of a re-inquiry image. It is a figure which shows typically the structure of the image processing apparatus which concerns on Embodiment 2 of this invention. It is a figure which shows the structure of the corresponding | compatible table which the image synthetic | combination part of the image processing apparatus which concerns on Embodiment 2 of this invention refers. It is a figure which shows the example of a composite image displayed when the person who is a subject is a woman of 10 years old or less, and the image of the person has defocused. It is a figure which shows the example of a composite image displayed when the person who is a subject is a woman of 10 years old or less, and the image blurring has generate | occur | produced in the image of the person. It is a figure which shows the example of a composite image displayed when the person who is a subject is a woman of 10 years old or less, and the blackout has generate | occur | produced in the image of the person. It is a figure which shows the example of a composite image displayed when the person who is a subject is a woman of 10 years old or less, and the overexposure has occurred in the image of the person. It is a figure which shows the example of a display of a synthesized image when the person who is a subject is a woman of 10 years old or less and the person has closed eyes.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,16 Image processing apparatus 2 Imaging part 3 Sensor part 4,17 Image processing part 5 Display part 6 Display control part 7 Compression / decompression part 8 Recording medium interface 9 Input part 10 Audio | voice output part 11 Auxiliary light projection part 12 Control part 13 Memory | storage Unit 14 lens driving unit 15 aperture driving unit 21 photographing lens 22 aperture 23 imaging element 24 analog signal processing unit 25 A / D conversion unit 41 subject detection unit 42, 44 photographing state determination unit 43 image composition unit 45 age / gender determination unit 100 Recording medium 121 Clock 131 Program storage unit 132 Subject detection information storage unit 133 Corresponding table storage unit 134 Image item storage unit 201, 203, 205, 207, 209 Reproduced image 202, 204, 206, 208, 210, 213-217 Composite image 211 Inquiry image 212 Re-inquiry image 3 1-31 Person 421 In-focus determination unit 422 Image blur determination unit 423 Light / dark determination unit 424 Eye blink determination unit 425 Age / gender determination unit A to G Affixed areas I-1 to I-11, K-1 to K-10 Images item

Claims (7)

A subject detection unit that detects a subject included in the image based on an image pattern in an image signal corresponding to an image to be processed;
A shooting state determination unit that determines a shooting state of the subject detected by the subject detection unit based on the image signal;
When there is a defect in the shooting state of the subject determined by the shooting state determination unit, an image item consisting of at least one of a picture, a pattern, a letter, a number, and a symbol is dealt with in response to the defect. An image composition unit to be pasted to the pasting area;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the image item has contents reflecting a defect in a shooting state of the subject.   In addition to storing a plurality of image items, the image processing apparatus further includes a storage unit that stores a correspondence relationship between a determination result of the shooting state determination unit, a type of the image item to be pasted on the image, and a pasting region. The image processing apparatus according to claim 1. The subject detection unit is capable of detecting a human face,
The shooting state determination unit
A focus determination unit that determines whether or not the person is in focus;
An image blur determination unit that determines whether image blur has occurred in the image of the person;
A light / dark determination unit that determines whether blackout or whiteout occurs in the image of the person;
A blinking determination unit for determining whether or not the person has closed eyes;
The image processing apparatus according to claim 1, further comprising: The shooting state determination unit
The image processing apparatus according to claim 4, further comprising an age / gender determination unit that determines the age and sex of the person. An imaging unit that captures an image of a subject within a predetermined visual field and generates an image signal;
A display unit that displays an image synthesized by the image synthesis unit based on an image captured by the imaging unit;
The image processing apparatus according to claim 1, further comprising: An image processing apparatus including a storage unit that stores a plurality of image items, each of which includes at least one of a picture, a pattern, letters, numbers, and symbols.
A subject detection step of detecting a subject included in the image based on an image pattern in an image signal corresponding to an image to be processed;
A shooting state determination step of determining a shooting state of the subject detected in the subject detection step based on the image signal;
When there is a problem in the shooting state of the subject determined in the shooting state determination step, an image composition step of reading the image item corresponding to the defect from the storage unit and pasting it on a predetermined pasting area of the image,
An image processing program for executing

Images