JP2006285469A - Image processor, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve such a problem that the vertical directions between an object image and date display are not matched. <P>SOLUTION: The vertical direction of image data 14a expressing an object image is determined, and when the vertical direction and the up and down direction of the image data 14a stored are not matched (the upper direction and direction of the top of the image data are not matched, or the lower direction and direction of the bottom of the image data are not matched), one of the image data 14a and date display data are rotated before the synthesis processing of the image data 14a and the date display data, so that the vertical directions of the object image and the date display can be matched. As a result, it is possible to surely output the image with date display while the vertical directions of the object image and the date display are matched without imposing any additional input operation on a user. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program that synthesize a date display at a predetermined position of a target image.

  Conventionally, when adding a date display when printing a photographed photograph image or the like with a printing apparatus, the date display is opposed to the long side of the rectangular image vertically as shown in FIGS. Normally, printing is performed at a predetermined fixed position such as a lower right position in the case of printing.

In addition, a technique is known in which a position where a date display is synthesized can be changed by a user's cursor key operation (see, for example, Patent Document 1).
Japanese Patent Laid-Open No. 2002-135560

If the date display position is fixed as in the past, the date display direction remains unchanged regardless of whether the image was shot vertically or horizontally, so the direction of the image and date display is the same. An image that does not match and feels uncomfortable may be output.
Further, in the above-mentioned document, it is necessary for the user to perform an input operation for changing the date display position, and the processing required for synthesizing the date display has been complicated. Further, even if the above change is made, the problem of the direction mismatch between the image and the date display cannot be solved.

  The present invention has been made in view of the above problems, and is an image processing apparatus capable of automatically synthesizing a date display in an appropriate direction and position on a target image without requiring a complicated input operation. An object of the present invention is to provide an image processing method and an image processing program.

In order to achieve the above object, in the image processing apparatus of the present invention, when the image data input means inputs image data representing the target image, the top and bottom determination means determines the top and bottom direction of the target image. Then, the image synthesis means synthesizes the image data and date display data for date display so that the top and bottom directions of the target image and the date display added thereto coincide with each other according to the determination of the top and bottom.
As described above, according to the present invention, for each target image having a different vertical direction such as portrait or landscape, the date display is automatically combined so that the vertical direction of the date display matches the vertical direction. When added, it is possible to generate an image with no sense of incongruity.

  As another configuration of the present invention, the top and bottom determination means may determine the top and bottom direction by reading the top and bottom information indicating the top and bottom direction of the target image recorded in advance in the header area from the header area of the image data. Good. That is, various types of information regarding the captured target image are stored in the header information of the image data, and the top and bottom information may be included therein. Therefore, the top and bottom direction of the target image can be easily determined by referring to the top and bottom information.

  The top / bottom determination unit obtains predetermined feature information representing a feature of the target image by analyzing a pixel value of the image data, and determines a top / bottom direction using a determination rule using the feature information. It is good. According to such a configuration, since the determination of the top / bottom direction is performed using the determination rule using the feature information representing the feature of the target image, even if the top / bottom information is not stored in the header area, You can automatically determine the top and bottom.

  The image synthesizing unit may read date information indicating a recording time of the target image recorded in advance in the header area from the header area of the image data, and generate the date display data according to the read date information. Good. That is, since the header information normally stores date information indicating the recording time of the target image such as the shooting date and time, the date display data is generated according to the date information, and the image data representing the target image is displayed. Is synthesized at a predetermined position.

As another configuration of the present invention, the image synthesizing unit rotates the image data by a predetermined angle in accordance with the determination of the top and bottom, and then combines the image data and the date display data, whereby the top and bottom directions of the target image and the date display are displayed. May be matched.
That is, the image composition means recognizes which of the four sides of the rectangular target image is to be directed upward based on the determination of the top and bottom, so that the side is directed upward. The image data is rotated by a predetermined angle. If the date display data is combined with the rotated image data, the top and bottom directions of the target image and the date display can be reliably matched.

  Furthermore, as another configuration of the present invention, the image synthesizing unit rotates the date display data by a predetermined angle according to the determination of the top and bottom, and synthesizes the image data and the date display data to thereby display the target image and the date display. The top and bottom directions may be matched. In this case, when recognizing the top / bottom direction of the target image, the image compositing means determines an angle for rotating the date display data in order to match the top / bottom direction of the date display, and rotates the date display data by the determined angle rotation Let That is, it is not necessary to rotate the image data side, and date display data may be used.

Further, when the image composition unit composes the image data and the date display data, the image composition unit performs a predetermined image evaluation for each region in the image data and determines a region for composing the date display according to the evaluation result. The date display data may be combined with the determined area. Even if the top and bottom direction of the target image and the date display are matched, if the date display data is combined as it is, it will be combined at a position that is not suitable for adding a date display depending on the relationship with the target image of the background. It can be considered.
Therefore, if the area for combining the date display is determined according to the image evaluation result for each area as described above, and the date display data is combined with the determined area, the position in addition to the direction is also optimal. In this state, a date display can be added on the target image.

  Up to this point, the image processing apparatus has been described as realizing the concept of the present application related to the date display composition processing. However, the idea of the present application can also be grasped as an invention of a method for realizing the image processing apparatus. That is, the image data input step of inputting image data representing the target image, the top and bottom determination step of determining the top and bottom direction of the target image, and the top and bottom of the target image and the date display added thereto according to the determination of the top and bottom It can also be grasped as an image processing method characterized by comprising an image synthesizing step for synthesizing the image data and date display data for date display so that the directions coincide. This case also has the same operations and effects as the image processing apparatuses described above.

Furthermore, the configuration of the image processing apparatus can also correspond to the invention of the program, an image data input function for inputting image data representing the target image, a top / bottom determination function for determining the top / bottom direction of the target image, In accordance with the determination of the top and bottom, the computer executes an image composition function for combining the image data and date display data for performing date display so that the top and bottom directions of the target image and the date display to be added coincide with each other. It can be grasped as an image processing program characterized by this. This case also has the same operations and effects as the image processing apparatuses described above.
Of course, it is possible to grasp an invention subordinate to the method or the program invention with the same configuration as each of the configurations of claims 2 to 7.

Embodiments of the present invention will be described in the following order.
(1) Schematic configuration of image processing apparatus (2) First embodiment (3) Second embodiment (4) Third embodiment

(1) Outline of Image Processing Apparatus FIG. 1 shows a computer 10 that is an image processing apparatus according to the present invention, a printer 20 connected thereto, and the like.
In the computer 10, the CPU 11 which is the center of the arithmetic processing controls the entire computer 10 via the system bus 10a. The bus 10a includes a ROM 12 which is a non-rewritable semiconductor memory, a RAM 13 which is a rewritable semiconductor memory, a CD-ROM drive 15, a flexible disk (FD) drive 16, various interfaces (I / F) 17a to 17e, and the like. And a hard disk (HD) 14 that is a magnetic disk is also connected via a hard disk drive.

The HD 14 stores an operating system (OS), an application program (APL), and the like, which are appropriately transferred to the RAM 13 by the CPU 11 and executed. In this embodiment, the HD 14 stores image data 14a and the like input from the outside. The image data 14a is data representing an image to be printed (target image), and the computer 10 receives the image data 14a from an input device such as the digital camera 40 via the I / F 17a (for example, USB I / F). Can be entered.
A display 18a for displaying an image corresponding to the data based on the image data is connected to the display I / F 17b. A keyboard 18b and a mouse 18c are connected to the input I / F 17c as operation input devices, and a printer I / F 17e. For example, the printer 20 is connected via a serial I / F cable.

  The printer 20 is an ink jet printer, and discharges each ink filled in four ink cartridges 28 provided corresponding to each color of CMYK (cyan, magenta, yellow, black) from the print heads 29a to 29d. Then, an image is printed by forming dots on the print medium. Of course, a printer using light cyan, light magenta, light black, dark yellow, red, violet, uncolored ink, or the like may be employed, and the number of ink types is not limited. Various printing apparatuses such as a bubble printer that generates bubbles in the ink passage and discharges the ink and a laser printer that prints a print image on a print medium using toner ink can be employed. In the printer 20, a CPU 21, a ROM 22, a RAM 23, a communication I / O 24, a control IC 25, an ASIC 26, an I / F 27, etc. are connected via a bus 20 a, and the CPU 21 controls each unit according to a program written in the ROM 22.

  Each ink cartridge 28 is mounted on a carriage that reciprocates in the main scanning direction by the carriage mechanism 27a, and a print head unit (print head assembly) 29 is mounted. Each cartridge 28 is provided with a memory chip 28a made of, for example, RAM, and each memory chip 28a is electrically connected to the control IC 25. The communication I / O 24 is connected to the printer I / F 17e of the computer 10, and the printer 20 receives raster data for each color transmitted from the computer 10 via the communication I / O 24. The ASIC 26 outputs applied voltage data corresponding to the raster data to the head driving unit 26a while transmitting / receiving a predetermined signal to / from the CPU 21. The head drive unit 26a generates an applied voltage pattern to the piezo elements built in the print heads 29a to 29d from the applied voltage data, and causes the print heads 29a to 29d to eject ink of each color in dot units. The carriage mechanism 27a and the paper feed mechanism 27b connected to the I / F 27 cause the print head unit 29 to perform main scanning, and sequentially feed print sheets while performing a page break operation as appropriate, thereby performing sub-scanning.

In the computer 10, a printer driver (PRTDRV) that controls the printer I / F 17e is incorporated in the OS, and the PRTDRV performs predetermined image processing on a target image for which a print instruction has been issued from the APL, and executes printing. The PRTDRV includes an image synthesis module, a color conversion module, a halftone (H / T) module, and a print data generation module for executing printing.
When the printing instruction is given, the PRTDRV is driven, and the processing for each pixel value of the image data 14a is performed by each module to create raster data. The created raster data is output to the printer 20 via the printer I / F 17e, and the printer 20 executes printing based on the raster data.

  More specifically, the image composition module acquires the image data 14a from the HD 14. At this time, a composition process of date display data described later is performed on the image data 14a, and a predetermined resolution conversion process is performed if necessary. The image data 14a is dot matrix data in which each element color of RGB is expressed in multiple gradations (for example, 256 gradations) to define the color of each pixel, and adopts a color system according to the sRGB standard. Yes. Of course, various data such as JPEG image data using the YCbCr color system and image data using the CMYK color system can also be used. The color conversion module is a module that converts a color system indicating the color of each pixel. The color conversion module refers to a color conversion look-up table (LUT) recorded in advance in the HD 14 and converts the image data 14a (RGB data) For each pixel, color conversion is performed to ink data expressed by gradation values for each CMYK.

The H / T module converts the ink data generated by the color conversion into H / T data that specifies ink ejection / non-ejection for each CMYK in each pixel. The H / T module can execute H / T processing by an error diffusion method, a dither method, or the like. Finally, the print data generation module generates the raster data based on the H / T data and sends it to the printer 20.
In FIG. 1, the computer 10 and the printer 20 are separate units, but the printer 20 may be configured to execute image processing executed by the computer 10 in addition to print processing.

(2) First Example Here, when image data 14a representing an arbitrary target image is input from the digital camera 40 or the like, is the image data 14a a vertically long image or a horizontally long image? Regardless, it is stored in a predetermined storage area of the HD 14 in a fixed direction. That is, regardless of the vertical direction of the target image, as shown in FIGS. 4 and 5, the two long sides of the rectangular image are up and down (the positive direction of Y of the X and Y coordinate axes in the storage area is the upward direction). Stored in the array state of the opposed pixels. When the image data 14a stored in such a direction is combined with a specific position (date display area in the figure) such as a date display indicating the shooting time, the target as shown in FIG. The vertical direction of an image may not match the vertical direction of another image.
Therefore, in this embodiment, the following image composition processing is performed.

FIG. 2 is a flowchart showing the contents of the image composition process executed by the computer 10. This processing is mainly executed by the PRTDRV image composition module.
In step S (hereinafter, description of steps is omitted) 700, the computer 10 reads the image data 14 a representing the target image from the HD 14.
In S <b> 710, the computer 10 acquires information (hereinafter, date information) indicating the time when the image data 14 a was recorded (for example, the date taken by the digital camera 40). Specifically, the date information is acquired from the header area added to the image data 14a. That is, when the image data 14a is recorded in a predetermined image file format, various information related to the image data 14a such as the shooting date and the size of the image data is recorded in the header area. Date information can be acquired by referring to the area.

  In S720, the computer 10 determines the top-and-bottom direction of the target image represented by the image data 14a. This determination can be roughly divided into a method for determining by reading the top and bottom information from the header area and a method for determining by analyzing the contents of the image data 14a.

  A case where the top and bottom information is read from the header area will be described. Some photographing devices include a so-called acceleration sensor that detects the inclination of the device at the time of photographing. When image data 14a is recorded by such a photographing device, the top and bottom direction of the target image is displayed in the header area. The specified top and bottom information is recorded together with other various information. Therefore, the computer 10 reads the top and bottom information from the header area and recognizes the top and bottom direction of the target image according to the top and bottom information.

Next, a case where the image data 14a is analyzed to determine the top / bottom direction will be described.
In this case, the computer 10 has a plurality of top and bottom determination rules for determining the top and bottom directions based on image characteristics, and performs top and bottom determination according to the rule selected from the rules.
FIG. 10 is a flowchart showing the top-and-bottom determination performed by the computer 10 by analyzing image data.

  In S100, the computer 10 selects the top / bottom determination rule based on the rule selection criterion information. The rule selection criterion information is information that specifies which of the rules is used, and such information is recorded in a predetermined storage area such as the HD 14 in advance. The number of rules specified by the rule selection criterion information may be one or plural.

  In the present embodiment, there is a rule that uses four characteristics of “person”, “sky / ground”, “straight line”, and “bar-shaped object” as the top and bottom determination rule. The rule based on “person” is a rule for analyzing an image, extracting a region that is a person as an image feature, and determining the top and bottom based on the person region. The rule based on “sky / ground” is a rule for analyzing the image, extracting a sky or ground area as an image feature, and determining the sky based on the sky / ground area. The rule based on “straight line” is a rule for analyzing the image, extracting one or more straight lines included in the image as image features, and determining the top and bottom based on the straight line. The rule based on the “bar object” is a rule for analyzing the image, extracting a bar object area such as a building as an image feature, and determining the top and bottom based on the bar object area.

In S200, S300, S400, and S500, the vertical direction of the target image and its probability are determined according to the selected vertical determination rule. Details of S200, S300, S400, and S500 will be described later.
In S600, the result of the top and bottom determination based on the selected rule is integrated and a final top and bottom determination is executed. This process will also be described later. When only one top / bottom determination rule is selected in S100, the result determined by the rule is used as it is.

Top / bottom direction determination procedure based on person area:
FIG. 11 is a flowchart showing the top and bottom determination procedure based on the person area. In S202, a person region is extracted from the target image indicated by the image data 14a. FIG. 12 shows the detailed procedure of S202.
In S262, a skin color area is extracted from the target image. At this time, first, the image data 14a is converted into an HSV (hue / saturation / luminance) color space, and an average luminance Vave of the image is calculated. Then, for each predetermined range of the average luminance Vave, an appropriate range of skin color pixel values is set and skin color pixels are extracted. Specific examples of the skin color pixel value range are as follows (when the H component range is 0 to 359 and the S and V component ranges are 0 to 255).

Skin color pixel value range when 120 ≦ Vave:
H component: 10 <H <40
S component: S> 60
V component: V> 150
However, regarding a pixel having a V component of 210 or more, regardless of the value of the S component, a pixel having an H component of 10 <H <45 is defined as a flesh color pixel. This is because it is more appropriate to ignore the S (saturation) component and make a determination only with the H (hue) component for a pixel with extremely high luminance, that is, a pixel considered to be a highlight.

Skin color pixel value range when 90 ≦ Vave <120:
H component: 0 <H <40
S component: S> 60
V component: V> 100
However, regarding a pixel having a V component of 210 or more, regardless of the value of the S component, a pixel having an H component of 10 <H <45 is defined as a flesh color pixel.

Skin color pixel value range when Vave <90:
H component: 0 <H <30 or 350 <H <360
S component: 30 <S <200
V component: V> 80
However, for a pixel having a V component of 210 or more, a pixel of 10 <H <35 is defined as a skin color pixel regardless of the value of the S component.

In this way, if the average luminance Vave (or average brightness) of the image is divided into a plurality of ranges and the range of skin color pixel values is set for each range, depending on whether the entire image is bright or dark, An appropriate skin color pixel range can be set. As a result, skin color pixels can be detected more accurately.

  A set of skin color pixels thus detected is extracted as a skin color region. At this time, only an image area gathered in a state where a certain number (for example, 10 pixels or 1% or more of the total number of pixels) or more skin color pixels are in contact may be extracted as the skin color area. If the skin color area includes facial organs (brows, eyes, nose, mouth, ears) and the like, the area including these organs may be extracted as one skin color area. . The organ of the face is also called “face element”.

FIG. 13A shows an example of the skin color area FA extracted from the target image G1. In this example, an area including a facial organ is extracted as one skin color area FA. It should be noted that any other method can be employed as the skin color pixel determination method. For example, regardless of the values of the S component and the V component, when the H component has a value within a predetermined range, it can be detected as a skin color pixel. As the color space, a color space such as a CIELab color space or a YIQ color space may be used.

  In S264, the aspect ratio R of each extracted skin color area is calculated. As shown in FIG. 13B, the aspect ratio R is a ratio of the long side length Wlong and the short side length Wshort of the rectangle including the skin color region. In S266, when the aspect ratio R is within a predetermined range (1.5 or more and less than 3 in this embodiment), the skin color area is regarded as a person's face area, and the face area (or “person area” is included in that area. ”). The processes of S264 and S266 are executed for all skin color regions in the target image (S268).

In the present embodiment, the face area is extracted as a person area, but the person area may be extracted by a method other than this. For example, it is possible to extract a wide area including some of a plurality of elements (head, torso, limbs) constituting a person as a person area.
When the extraction of the person area is thus completed, it is determined in S204 whether or not the person area exists in the target image. If the person area does not exist, it is determined in S206 that the top / bottom direction of the target image is unknown, and the process proceeds to S600 to wait for a comprehensive top / bottom direction determination process. In this case, the determination result “Unknown direction” in S206 means that the vertical direction cannot be determined based on the person area, and there is a possibility that the vertical determination using other features can be performed.

  On the other hand, when a person area exists in the target image, the top / bottom direction of the image is determined according to the top / bottom direction of the person area. Specifically, in S208, a facial organ in the person area is detected. For the facial organ extraction process, various processes such as a process using a neural network, a process using a color change or luminance change in an image, or a process using a shape in an image can be used. In this embodiment, it is assumed that eyes and mouth are detected as facial organs.

In S210 to S242, the following processing is executed depending on which of the eyes and mouth is detected as the facial organ.
If both eyes and mouth can be detected (S210), the top-to-bottom direction of the image is determined from the positional relationship of the eyes and mouth with respect to the face region (S212). For example, as shown in FIG. 13A, when both eyes and mouth are detected, the top-and-bottom direction is determined by a straight line connecting the middle position of both eyes and the mouth. That is, the direction from the mouth toward the middle position of both eyes is the celestial direction. When one eye and mouth are detected, the top and bottom direction is determined by a straight line connecting the eyes and mouth. In S214, the determination probability is determined from the detection result of the eyes and mouth. For example, if both eyes and mouth can be detected, the determination probability is determined to be 100%, and if one eye and mouth can be detected, the determination probability is set to 75%.

  When only the eyes can be detected without detecting the mouth (S220), the top-to-bottom direction of the image is determined from the position of the eyes with respect to the face region (S222). For example, the direction from the end close to the eyes to the opposite end of the face region is determined as the ground direction. In this case, since only eyes can be detected, the determination probability is set to 50% in S224. This determination probability is set to a smaller value than when both eyes and mouth are detected (S214).

  When only the mouth can be detected without detecting the eyes (S230), the top-to-bottom direction of the image is determined from the position of the mouth with respect to the face region (S232). For example, the direction from the end close to the mouth to the opposite end of the face area is determined as the top direction. Since the mouth is located closer to the edge of the face than the eyes, the determination probability is set to a slightly higher determination probability (for example, 60%) than when only the eyes can be detected.

  If both eyes and mouth cannot be detected (S240), the top-to-bottom direction is unknown and the process proceeds to S250. When the top / bottom direction is unknown, the determination probabilities for the four sides of the image are all set to 0%. In addition, the determination probability is set to 0% for the sides that are not determined to be the top-to-bottom direction (top or bottom direction) in S212, S222, and S232. Such setting of the determination probability is the same in processing using other features.

Instead of making the top-to-bottom direction unknown in S240, the side closer to the face region among the two sides intersecting the straight line along the longitudinal direction of the face region may be determined as the top direction. In this case, the determination probability is set to a lower value (for example, 30%) than when eyes or a mouth can be detected.
In S250, it is determined whether or not the processing related to all the person areas in the target image has been completed. When the processing is completed, the process proceeds to S600.

  When different determination results are obtained in a plurality of person areas, the determination results based on the person areas can be obtained by combining the determination results of the person areas. For example, the determination result having the highest determination probability among a plurality of determination results can be employed. In addition, when there are a plurality of person regions determined to have the same top-and-bottom direction, the determination probability may be modified to increase the determination probability in that direction. For example, when there are two person regions having the same first vertical direction and a determination probability of 75%, the determination probability may be corrected to 90%. At this time, if the second vertical direction determined based on another person area has a probability of 75%, the first vertical direction and the determination probability of 90% are adopted as the final determination result.

By the way, as shown in FIG. 13C, the angle θ between the sky direction Fup determined from the person area and the direction Gup parallel to one side of the image may show a considerably large value. In this case, for example, among the four side directions Gup, Gdown, Gright, and Gleft, the one closest to the top direction Fup of the face region can be selected as the top direction of the image. However, the downward direction Gdown of the image may be excluded from the candidates for the top direction. This is because it is extremely rare to shoot with the camera turned upside down.

In this way, in the top / bottom direction determination processing based on the person area, the target image is extracted by analyzing the target image, and the top / bottom direction of the image is determined from the top / bottom direction of the person area. can do. In particular, if the top / bottom direction is determined from the position of the facial organ in the person area, the top / bottom direction can be determined more accurately.
In the above description, the top-and-bottom direction is determined according to the position of the facial organ in the face area. Instead, the position of the face area and the hair portion (hair area) adjacent to the face area is determined. It is also possible to determine the top and bottom direction from the relationship. It is also possible to determine the top-and-bottom direction from the positional relationship between the skin color region (face region) and the body part (body region) adjacent to the skin color region. The hair part and the body part can be determined using, for example, shape template matching using typical shapes of these parts.

Top-to-bottom direction determination procedure based on sky and ground area:
FIG. 14 is a flowchart showing the top / bottom determination procedure based on the sky / ground area. In S302, the average luminance of the target image is obtained, and day / night determination is performed according to the average luminance. Specifically, when the average luminance of the image is equal to or higher than a predetermined threshold value (for example, 110), the image is determined to be taken at daytime. judge. The day / night determination may be performed from the shooting time information recorded in the header region attached to the image data 14a.

  FIG. 15A shows a target image G2 photographed in the daytime and includes a bright sky region. If it is determined that the image is a daytime image, the processing from S306 is executed. First, in S306, flagging is performed on pixels brighter than the average luminance of the entire image. In step S308, pixels that are flagged and are continuous are connected to each other. Hereinafter, an area where pixels are connected in this manner is referred to as a “connected area”. In S310, the color of the connected area is checked, and if the connected area has a predetermined color belonging to any one of blue, white, and gray, it is flagged again as an empty area candidate (also called “sky area candidate”). . The range of pixel values belonging to blue, white, and gray can be set as follows, for example.

Blue: 200 <H <240, S is full range, V> (1.5 times the average luminance Vave)
White: H is the entire range, S <10, V> (1.5 times the average luminance Vave)
Gray: H is the entire range, S <30, V> (1.5 times the average luminance ave)

  For example, (i) when 70% or more of pixels in a connected region are classified as a color belonging to any one of blue, white, and gray, as a criterion for determining whether or not to flag a certain region as an empty region candidate, Alternatively, (ii) some criteria such as when the average color of the connected region is classified into a color belonging to any of blue, white, and gray can be adopted. FIG. 15B shows the empty area SA (empty area candidate) extracted as described above.

  When the sky region candidate is detected in this way, in S312, the edge (side) where the sky region candidate is the widest among the four sides of the target image is determined as the top direction of the image. In this case, when the sky region candidate is in contact with a plurality of sides, the length of the sky region candidate on each side is calculated, and the side having the largest ratio of the length of the sky region candidate to the length of each side is calculated. Is determined to be the top direction of the image. In FIG. 15B, the upper side is determined to be the top direction of the target image.

  In S314, when the sky region candidate covers only one side, the determination probability is set to 100%. When the sky direction is determined based on the length of the sky region candidate on each side, the determination probability is set to 75%. Thus, for a daytime image, if the sky direction of the image is determined according to which of the four sides of the image is the longest, the sky direction can be determined fairly accurately. Is possible.

  On the other hand, if it is not determined in S304 that the image is a day image, it is determined in S320 whether the image is a night image. Specifically, for example, when the average luminance Vave of the entire image is less than 90, it can be determined that the night. Alternatively, the image may be divided into nine equal parts, and the determination may be made as night when the average luminance Vave of the central block is less than 90. If neither the daytime image nor the nighttime image is determined, it is determined in S330 that the vertical direction is unknown, and the process proceeds to S600.

FIG. 16A shows an overall dark target image G3. In this case, since the luminance is low, it is determined that the image is captured at night. White dots in the image are bright objects such as street lamps, building windows, and automobile headlights.
If it is determined in S320 that the image is a night image, the target image is equally divided into four blocks in S322, and the average luminance of each block is obtained. FIG. 16B shows a state in which the target image G3 is divided into four blocks B1 to B4. In S324, as shown in FIG. 16C, an upper area Aupper composed of two blocks B1 and B2 above the image, and a lower area Alower composed of two blocks B3 and B4 below. A luminance contrast CT1 is calculated. The luminance contrast CT1 is a ratio of the average luminance Vave (Aupper) in the upper area Aupper and the average luminance Vave (Alower) in the lower area Alower. Similarly, as shown in FIG. 16D, a left area Aleft composed of two blocks B1 and B3 on the left side of the current image direction, and two blocks B2 and B4 on the right side. The luminance contrast CT2 = Vave (Aleft) / Vave (Aright) between the right areas Aright is calculated.

  The computer 10 compares the luminance contrasts CT1 and CT2 in S324, and determines in S326 that the division direction giving a larger contrast is the vertical direction of the image. Furthermore, among the top and bottom directions, the higher luminance is determined as the ground direction. In the example in the figure, the contrast CT1 of the vertical division in FIG. 16C is larger, so it is determined that the vertical direction of the current image is the top-and-bottom direction. Moreover, since the brightness is higher in the vertical direction of the image, it is determined that the current lower direction is the ground direction. In the present specification, the “dividing direction” means a direction perpendicular to the dividing line.

  In S328, the determination probability is set to 100% when the value of the luminance contrast is sufficiently large (for example, 2 or more), and the determination probability is set to 75% in other cases. In this way, in the top / bottom direction determination process based on the sky / ground area, the sky area is extracted in the case of a daytime image, and the top / bottom direction is determined according to the position of the sky area in the image. The top and bottom direction can be determined. In the case of a night image, the vertical direction is determined using the vertical and horizontal luminance contrasts in the image, so that the vertical direction can also be correctly determined for a night image. . However, without determining whether the image is a daytime image or a night image, the sky region in the image may be extracted, and the top-and-bottom direction may be determined from the position of the sky region in the image.

  In addition, when an empty area (empty area candidate) is extracted, other areas may be recognized as the ground area. Alternatively, even when the sky area cannot be recognized, an area having a specific color or shape (for example, a vast area of brown or green) may be extracted as the ground area. In this case, it is possible to determine the top / bottom direction according to the position of the ground region in the image.

Top and bottom direction determination procedure based on a straight line:
17 and 18 are flowcharts showing the top and bottom determination procedure based on a straight line. In S402, the computer 10 detects a straight line in the target image. The straight line in the image is obtained by, for example, spatially differentiating the target image to obtain the edge amount of each pixel, and generating an image (edge image) composed of the edge amount of each pixel. It can be obtained by performing conversion. The spatial differentiation of the image can be performed using a differential filter such as a Laplacian filter.

  In S404, it is determined whether a straight line is detected. Here, the straight line means a line segment having a predetermined length (for example, 100 pixels) or more. If a straight line is not detected, it is determined in S440 that the vertical direction is unknown, and the process proceeds to S600. On the other hand, if a straight line is detected, it is determined in S406 whether the straight line is a straight line extending in the vertical direction or the entire horizontal direction of the image. When there is no straight line extending in the vertical direction or the entire horizontal direction, the process proceeds to a convergence straight line determination process (described later) shown in FIG.

On the other hand, when there is a straight line over the entire image, the computer 10 executes the processing from S410 onward. FIG. 19 shows an example of an image including a straight line (also referred to as a “penetrating straight line”) over the entire image. The target image G4 includes a horizontal line, and this horizontal line is determined as the through straight line TSG.
In S410, it is determined whether or not the longitudinal direction of the image matches the direction of the through straight line. This determination can be made based on whether or not the angle formed between the longitudinal direction of the image and the direction of the penetrating straight line is equal to or less than a predetermined value (for example, 20 degrees). If there is a long straight line that coincides with the longitudinal direction of the image, it is often considered that the straight line is a horizontal line or a horizon line. Therefore, if these two directions coincide with each other, in S412, the upper part of the current image is determined as it is (upward), the determination probability is set to 75%, and the process proceeds to S600. This is because it is extremely rare to take a picture by turning the camera upside down when taking a landscape image. On the other hand, if the two directions do not match, an empty area is detected in S420. This process is the same as that described in S310 of FIG.

If an empty area can be detected in S420, the direction in which the empty area is detected is determined to be upward (upward) in S422, the determination probability is set to 75%, and the process proceeds to S600. However, as in S312 and S314 of FIG. 14, the determination probability may be set based on the relationship between the sky region and the side (edge) of the image.
On the other hand, if an empty area cannot be detected in S420, it is determined in S430 whether a high-luminance area exists in the vicinity of the through straight line. Here, the “high luminance region” means, for example, a region where a predetermined number (for example, 20 pixels) or more of pixels having a V value in the HSV color space that is equal to or larger than a predetermined value (for example, 200). . For example, when a sunset image is taken, the sun is recognized as a high brightness area. If a high-luminance area is detected in the vicinity of the through-line, in S432, the side of the image closer to the high-luminance area than the through-line is determined as the upper side (top direction), and the determination probability is set to 75% To S600. When two or more high-luminance areas are detected, there is a high possibility that the object is due to specular reflection of the object, and it may be determined that there is no high-luminance area. If a high-luminance area cannot be detected in the vicinity of the through straight line in S430, it is determined in S440 that the top-to-bottom direction is unknown, and the process proceeds to S600.

  In the convergence straight line determination process shown in FIG. 18, first, in S450, it is determined whether or not three or more straight lines are detected. If three or more straight lines are detected, a set of three or more straight lines that converge to approximately one point (referred to as a “converging straight line group” or “disappearing line group”) is searched in S452. 20A shows the target image G5, and FIG. 20B shows a straight line group CSG that converges at the convergence point CP. The convergence point CP of the convergence straight line group CSG may have a certain allowable range.

  If three straight lines are not detected in S450, or if a convergent straight line group is not detected in S452, it is determined that the vertical direction is unknown, and the process proceeds to S600. On the other hand, when the convergence straight line group is detected in S452, the computer 10 calculates the luminance contrast on both sides of the convergence straight line group. Specifically, as shown in FIG. 20C, the average luminance in each of the two regions A1 and A2 sandwiching the convergence straight line group CSG is calculated, and the ratio of the average luminance is calculated as the luminance contrast. Then, the side of the image in which the higher luminance region exists is determined to be upward (upward direction), the determination probability is set to 60%, and the process proceeds to S600. The reason why the area with higher luminance is determined as the top direction of the image is that the higher luminance is more likely to be closer to the sky. Note that the setting probability in this case may also be set according to the relationship between the higher brightness area and the side (edge) of the image, as in S312 and S314 of FIG.

Thus, in the top / bottom direction determination process based on a straight line, when there is a penetrating straight line, the top / bottom of the image can be correctly determined according to the direction of the penetrating straight line or the positional relationship between the penetrating straight line and the sky region. Therefore, it is possible to correctly determine the top and bottom of the image when there is a through straight line such as a horizontal line or a horizon line. Further, when the convergence straight line group exists, the top and bottom of the image can be correctly determined according to the luminance on both sides of the convergence straight line group.
When the image includes a penetrating straight line, instead of the above-described determination, the direction in which the angle formed by the penetrating straight line is larger among the two directions of the vertical and horizontal directions of the image is determined as the vertical direction. You may do it. For example, when the direction of the penetrating straight line is close to the left-right direction of the image, the vertical direction of the image is the top-and-bottom direction, and it is determined that the image direction is correctly facing the top-and-bottom direction. On the other hand, when the direction of the penetrating straight line is close to the vertical direction of the image, the horizontal direction of the image is the vertical direction. In this case, the celestial direction or the ground direction may be determined using other image characteristics (for example, left and right contrast of the image).

  When the image includes a convergence line group, the top and bottom may be determined based on the direction of the convergence line group instead of the above determination. Usually, as shown in FIG. 20, the convergence point CP of the convergence line group is often directed upward in the image. Therefore, it is possible to determine the vertical direction or the horizontal direction of the image as the top-and-bottom direction based on the direction in which the convergence line group is directed. Also in this case, the celestial direction or the ground direction may be determined using other image characteristics (for example, image contrast).

Top-and-bottom direction determination procedure based on a bar-shaped object:
FIG. 21 is a flowchart showing the top / bottom determination procedure based on the bar-shaped object. A bar-like object means a long subject such as a building.
In S502, the computer 10 performs region division processing (region separation processing) using the color information of each pixel of the target image. Specifically, for example, the H component of the HSV color space is divided into a plurality of predetermined ranges in advance, and the region composed of pixels having the H component in each range is extracted to perform region division. Is possible. In this case, when the number of pixels in a certain area is extremely small (for example, 10 pixels or less), the area may be absorbed by a larger surrounding area.

It should be noted that the area division can also be performed using other methods. For example, it is also possible to obtain an edge image by spatially differentiating the target image, and to perform region division by detecting a closed region in the edge image.
In S504, a flag as a rod-shaped object is attached to an area having an aspect ratio of a predetermined value (for example, four times) or more. FIG. 22A shows a target image G6 including a bar-shaped object SOB. As shown in FIG. 22B, the aspect ratio R is the ratio of the long side length Wlong to the short side length Wshort of the rectangle including the rod-shaped object SOB.

  In S506, it is determined whether one end of the bar-shaped object overlaps the empty area. The empty area extraction processing is the same as that described in S310 of FIG. As shown in FIG. 22A, when one end of the bar-shaped object overlaps (or is in contact with) the empty area, the direction of the overlapping end is determined to be upward (top direction), and the determination probability is It is set to 75% (S508, S510). If there is a next rod-like object, the process returns from S512 to S506, and the processes after S506 are repeated. On the other hand, if both ends of the rod-shaped object do not overlap with the sky area in S506, or if there is no sky area, it is determined that the top and bottom direction is unknown, and the process proceeds to S600.

In the case of the target image G6 in FIG. 22A, since the left end of the rod-shaped object SOB overlaps the empty area, it is determined that the left side of the image is the sky direction.
Thus, when there is a bar-like object such as a building in the image, there is a high possibility that the longitudinal direction is the vertical direction. Therefore, as described above, it is possible to determine the top-and-bottom direction fairly accurately using the rod-like object.

Comprehensive top and bottom judgment procedure
FIG. 23 is a flowchart illustrating the procedure of the comprehensive top / bottom determination process. The computer 10 adds the determination probability for each end (each side) of the image using the result of the top / bottom direction determination using each feature selected in S100 (S602, S604). Specifically, the determination probabilities calculated using the features are added to the upper side, the lower side, the left side, and the right side of the image. Note that, as described above, in the determination process using each feature, when the top-to-bottom direction is unknown, all determination probabilities for the four sides are set to 0%. In addition, the determination probability is set to 0% for the sides that are not determined to be the celestial direction or the ground direction. Note that the addition of the determination probability is performed with respect to either the celestial direction or the ground direction. In this case, when there is a side determined as the ground direction, it is considered that the opposite side is determined as the top direction. In this embodiment, the determination probability is added with respect to the sky direction.

When all of the determination probabilities have been added, in S606, the end (side) having the highest determination probability is finally determined as upward (upward), and the result is output in S608.
In this way, the top and bottom direction and its determination probability are determined for each of the plurality of features, the determination probability is added for each side of the image, and the final top and bottom direction is determined using the determination probability after the addition. It is possible to determine the top and bottom direction with accuracy.
In the comprehensive top / bottom determination processing, instead of adding the determination probabilities for each side, the side indicating the highest value among the determination probabilities obtained using each feature is determined as the top / bottom direction of the image (the top or bottom direction). ) May be determined. However, if the determination probability is added, there is a possibility that the correct top-and-bottom direction can be determined more reliably.

  With the above processing, the top / bottom direction determination processing in S720 of FIG. 2 ends. Note that only one of the method of reading the top and bottom information from the header area and the method of analyzing the contents of the image data 14a may be executed, or both methods may be used in combination. When using together, the method of reading top and bottom information from the header area is executed first, and when the top and bottom information cannot be read from the header area, a method of analyzing the contents of the image data 14a may be executed.

Returning to FIG. 2, the description will be continued.
In S730, the computer 10 determines the angle and direction in which the image data 14a is rotated based on the determined vertical direction of the target image. Here, it is assumed that the image data 14a is rotated counterclockwise.
As shown in FIG. 6, among the four sides L1, L2, L3, and L4 of the target image, the rotation angle is determined to be 90 ° when the side L2 faces the top direction of the image. On the other hand, when the side L1 faces the sky, the rotation angle is determined to be 270 °. When L3 is the celestial direction, that is, when the upward direction and the celestial direction in the original arrangement state of the target image coincide with each other, the rotation angle is 0 °.

In S740, the image data 14a is rotated according to the determined rotation direction and angle. Such rotation can be realized by using a known coordinate conversion formula that specifies the coordinate value after rotation for each pixel before rotation with the coordinate value as an input value.
As a result, when the side L2 is in the top direction, the image data 14a is rotated by 90 °, and the side L2 is directed upward. When the side L1 is the top direction, the image data 14a is rotated by 270 °, and the side L1 is directed upward. Needless to say, when the side L1 is in the celestial direction, the side L1 may be rotated by 90 ° in the clockwise direction.

When the pixel array of the image data 14a is converted so that the top direction of the target image is upward as described above, in S750, the computer 10 executes a process of synthesizing date display data on the image data 14a. The date display data is image data for displaying the date indicated by the date information read from the header area, and is generated based on the date information. Of course, the date display data is generated so that the top of the date display is upward. In this embodiment, the date display data is synthesized at a specific position (for example, the lower right position) on the image data 14a after the rotation process.
As shown in FIG. 7, the image represented by the image data 14a after the date display data is combined is in a state in which the target image and the vertical direction of the date display coincide with each other. The target is H / T processing. As a result, the printer 20 outputs an image in which the target image and the vertical direction of the date display match.

FIG. 3 shows an image composition process executed by the computer 10 and shows an example different from FIG. In FIG. 3, the processes of S800 to 820 and 850 are the same as the processes of S700 to 720 and 750 of FIG. 2, but differ from FIG. 2 in that the date display data side is rotated instead of the image data 14a.
In S830, the computer 10 determines the angle and direction for rotating the date display data based on the top and bottom direction of the target image determined in S820. Here, the date display data is rotated counterclockwise.

As shown in FIG. 8, among the four sides L1, L2, L3, and L4 of the target image, when the side L1 is facing the top direction of the image, the rotation angle of the date display data is determined to be 90 °. On the other hand, when the side L2 faces the celestial direction, the rotation angle is determined to be 270 °. When L3 is the celestial direction, the rotation angle is 0 °. In FIG. 8, the top direction of date display is represented as l1, and the ground direction is represented as l2. In S840, the date display data is rotated according to the determined rotation direction and angle. That is, the date display data is rotated by 90 ° when the side L1 is in the top direction, and the date display data is rotated by 270 ° when the side L2 is in the top direction.
When the date display data rotated in this way is synthesized at a specific position on the image data 14a, the image represented by the synthesized image data 14a is in a state in which the target image and the date display have the same vertical direction as shown in FIG. Become. As a result, the printer 20 outputs an image in which the target image and the vertical direction of the date display match.

  As described above, according to the present embodiment, the vertical direction of the image data 14a representing the target image is determined, and the vertical direction and the vertical direction of the stored image data 14a match (upward and vertical). If the direction does not match, or the downward direction and the ground direction do not match), before image data 14a and date display data are combined, one of image data 14a and date display data is rotated. Match the top and bottom direction of the target image and date display. As a result, without causing the user to perform an extra input operation, it is possible to reliably output an image on which the date is displayed in a preferable manner with the target image and the vertical direction of the date display matching.

(2) Second Example When other images such as date display are combined with the target image, there is a region that is not suitable for combining the other images for each target image. Therefore, in this embodiment, the following image composition processing is performed.
FIG. 24 is a flowchart showing the contents of the image composition process executed by the computer 10. This processing is mainly executed by the PRTDRV image composition module.

The processing of S900 and 920 is the same as the processing of S700 and 710 or S800 and 810 in the first embodiment. That is, the computer 10 reads the image data 14a representing the target image from the HD 14, and obtains date information from the header area.
Next, in S940, the computer 10 performs image evaluation for each area of the target image, and sets an evaluation value α for each area based on the evaluation result. Here, the evaluation value α is a numerical value representing the appropriateness for synthesizing date display data. In the present embodiment, the larger the value, the more suitable the area for synthesis.

In this embodiment, as specific image evaluation methods, an evaluation based on the presence or absence of a person, an evaluation based on a degree of focus match, and an evaluation based on the magnitude of a luminance difference will be described.
FIG. 28 is a flowchart showing the contents of image evaluation based on the presence or absence of a person. First, in S941, the computer 10 selects one pixel area from the image data 14a. The area selected as described above has a size suitable for date display, represented by m × n pixels in the horizontal and vertical directions. In FIGS. The size of one area is shown.

  Here, there are mainly two methods for selecting a region, and either method may be adopted. First, as shown in FIG. 25, regions to be selected in advance are set at predetermined locations in the target image (for example, a total of six locations at both the left and right ends and approximately the center of the end region along the upper and lower sides of the image). There may be a method in which each region is selected in advance. The other is, as shown in FIGS. 26 and 27, within a predetermined area in the target image (for example, end areas along the upper and lower sides of the image), one pixel at a time with the size of the area as a unit. A method of sequentially selecting areas while shifting the area in the direction can be considered.

In S942, the computer 10 determines whether or not a person exists in the selected one area (selected area). The person detection can be realized by a process similar to the person area extraction process in S202 of FIG. That is, if the face area of the person is extracted from the selected area, “Yes” is determined, and the process proceeds to S943. On the other hand, if the face area of the person is not extracted, the evaluation value α3 is set for the selected area in S946.
In S943, it is determined whether there are a plurality of persons in the selected area. The number of persons is determined by the number of extracted face areas. If a plurality of human face areas are extracted from the selected area, “Yes” is determined, and an evaluation value α1 is set in S944. On the other hand, if only one face area is extracted from the selected area, the evaluation value α2 is set in S945.

  Since the size of the selection area is a very small range in the entire target image, the situation where the entire face area of a person is included in the selection area is rare. Therefore, in the determinations in S942 and 943, the face area extraction process itself is performed on a large pixel area including the selected area, the face area is extracted from the large pixel area, and a part of the face area is selected as the selected area. If it is included, it may be determined that a person exists in the selected area.

Here, the evaluation values α1 to α3 set as described above are numerical values having a relationship of α3>α2> α1. As described above, the evaluation value is an index value indicating that the larger the value is, the more suitable the area for combining the date display. A small evaluation value is given as being, and a smaller value is obtained when a plurality of persons are included. As for the evaluation value set when there are a plurality of persons, the evaluation value is set to a smaller value as the number of persons increases.
In S947, the computer 10 determines whether or not evaluation values have been set for all areas. If the processes for all areas have not been completed, the processing from S941 is repeated. On the other hand, when the process for all the areas has been completed, the image evaluation process is terminated.

Next, image evaluation based on the degree of focus match will be described. That is, since the in-focus area is considered to be an important part in the target image, the evaluation value varies depending on the degree of focus matching.
FIG. 29 is a flowchart showing the contents of image evaluation based on the degree of focus match. In S948, one area is selected as in S941 of FIG. In S949, the computer 10 totals the number of edge pixels included in the selected area. In this embodiment, how much a certain selected area is in focus is determined based on the number of edge pixels included in the selected area.

Whether or not a pixel is an edge pixel can be determined by various known methods. Here, the difference between the density (for example, luminance) of one pixel of interest and the density of surrounding pixels is a predetermined reference value. If it exceeds, the target pixel is determined to be an edge pixel.
In S950, it is determined whether or not the total number of edge pixels is greater than a predetermined threshold value Th1. The threshold value Th1 is a value corresponding to several percent of the total number of pixels (m × n) constituting the selected region. If the total number of edge pixels does not exceed the threshold value Th1, the evaluation value α6 is set for the selected area in S954. On the other hand, if the threshold Th1 is exceeded, it is determined in S951 whether or not the total number of edge pixels is less than the predetermined threshold Th2. The threshold value Th2 is a value larger than the threshold value Th1, and is a value corresponding to several percent of the total number of pixels constituting the selected region.

If the total number of edge pixels does not exceed the threshold Th2, the evaluation value α4 is set in S952, and if it exceeds the threshold Th2, the evaluation value α5 is set in S953.
Here, the evaluation values α4 to α6 have a relationship of α6> α4 and α5> α4. That is, the evaluation value is the smallest when the number of edge pixels is within the range of threshold values Th1 to Th2, and a large evaluation value is given when the number of edge pixels is outside the range of threshold values Th1 to Th2. In other words, in this embodiment, when the edge pixel ratio in the selected area is large, it is basically determined that the degree of focusing is high and is not suitable for date display composition, but the edge pixel ratio exceeds a certain value. If the value is higher, the evaluation value is exceptionally set based on the idea that it is determined to be suitable to some extent for synthesizing the date display.

As a result, there are considerable changes in the shading of the image, such as areas where clothing patterns and lawn are photographed as backgrounds, but low evaluation values are given to areas that are unlikely to be excluded from the date display composition position. It is prevented from giving. However, whether or not to make the determination in S951 is arbitrary, and if “Yes” in the branch in S950, an evaluation value smaller than α6 may be given immediately.
In S955, the computer 10 determines whether or not evaluation values have been set for all regions, and if the processing for all regions has not been completed, the processing from S948 is repeated. On the other hand, when the process for all the areas has been completed, the image evaluation process is terminated.

Next, image evaluation based on the magnitude of the luminance difference will be described. That is, when the luminance difference in the area is large, it can be predicted that some object exists in the area, and therefore the evaluation value is varied depending on the luminance difference.
FIG. 30 is a flowchart showing the contents of image evaluation based on the magnitude of the luminance difference. In S956, one area is selected as in S941 of FIG. In S957, the computer 10 calculates the variance of luminance in the selected area. That is, the RGB gradation is acquired for each pixel in the selected region, and the luminance of each pixel is calculated from the RGB gradation by a known conversion formula. And the dispersion | distribution about these brightness | luminances is calculated.

In S958, as one method for recognizing the luminance difference, the calculated variance is compared with a predetermined threshold value Th3. This threshold value Th3 is a value corresponding to several percent of the range that the luminance can take. When the variance exceeds the threshold value Th3, the evaluation value α7 is set for the selected region, while when the variance does not exceed the threshold value Th3, the evaluation value α8 is set. The evaluation values α7 and α8 have a relationship of α8> α7.
In other words, if the luminance variance is large, it can be said that the luminance variation in the selected region is large, so that the image of the selected region is not uniform and it is highly likely that some object is included. On the other hand, if the variance of luminance is small, it can be said that there is a high possibility that the selected area is a redundant image. Therefore, when the variance exceeds the threshold value Th3 as described above, a low evaluation value is given as inappropriate for the composition of date display, and conversely, when the variance does not exceed the threshold value Th3, a high evaluation value is given. give.
In S961, the computer 10 determines whether or not evaluation values have been set for all areas. If the processes for all the areas have not been completed, the processes of S956 and after are repeated. On the other hand, when the process for all the areas has been completed, the image evaluation process is terminated.

The computer 10 may execute any one of the above-described evaluation based on the presence / absence of the person, evaluation based on the degree of matching of the focus, and evaluation based on the magnitude of the luminance difference, A plurality of processes may be executed.
When the evaluation value α for each selected area is set in this way, in S970, the computer 10 selects one area to synthesize a date display based on each evaluation value α. Specifically, the computer 10 determines the area where the highest evaluation value α is set as the area to be synthesized. When a plurality of image evaluation processes are executed, the sum of evaluation values set for each image evaluation process is calculated for each selected area, and the area having the highest sum is determined as the area to be combined. do it.

In S980, the computer 10 performs processing for synthesizing date display data in the area determined in S970. The date display data is image data for displaying the date indicated by the date information read from the header area, and is generated based on the date information.
The image data 14a combined with the date display data is subsequently subjected to color conversion processing, H / T processing, and the like. As a result, the printer 20 prints an image in which the date display is combined in an area on the target image that is most suitable for combining the date display.
Note that the color component of the date display data may be determined according to the color component of the region to be the combination position. That is, in order to prevent the date display displayed in the print result from being buried in the background color and difficult to see, the color component (for example, the average value of the hue) determined in S970 is determined, and the color of the date display data The component is a color component different from the determined color component. As a result, it is possible to acquire a print result displayed in a state in which the date display can be visually recognized with certainty.

In the above description, image evaluation is performed for all selected areas and date display is combined with an area having the highest evaluation value α. However, it is not always necessary to perform image evaluation for all selected areas.
That is, the computer 10 is prepared in advance with a level of evaluation value α (hereinafter referred to as a reference evaluation value αs) that can allow the composition of date display, and when an evaluation value that exceeds the reference evaluation value αs is obtained, the remaining value is obtained. You may determine the area | region which obtained the said evaluation value exceeding the area | region as an area | region which synthesize | combines a date display, without performing image evaluation about an area | region.

In this case, each time the computer 10 sets an evaluation value for one selected region, the computer 10 compares the evaluation value with the reference evaluation value αs. When a plurality of image evaluation processes are executed, each time a plurality of image evaluation processes are executed and a total sum of evaluation values set for each selected area is calculated, the sum is calculated as a predetermined reference value. (For example, a value obtained by multiplying the reference evaluation value αs by the number of image evaluation processes to be executed). As a result, it is possible to reduce the time required for image composition processing and the calculation burden, compared to the case where image evaluation is performed for all selected regions.
Thus, if it is assumed that image evaluation is not necessarily performed for all regions, the selection order of selection regions to be subjected to image evaluation should be devised.

That is, as shown in FIG. 25, when a total of six places, that is, the left and right ends and substantially the center of the end area along the upper and lower sides of the target image can be selected areas, the date display is combined as a position among them. The selection is made in a preferred order (for example, lower right, lower left, upper right, upper left, lower center, upper center).
In addition, when performing image evaluation by grasping a selection area while shifting an area of a predetermined size by one pixel at an end area along the upper and lower sides of the target image, as shown in FIG. The selection area may be switched in the order of starting from the right end area along the area and moving to the lower left area and then moving from the right end area along the upper side to the upper left area. Further, as shown in FIG. 27, in the region along the lower side, the right end region is started and the selection region is sequentially switched so as to approach the center side one pixel at a time on the left and right sides, and then the region along the upper side. In this case, the selection area may be sequentially switched so that the right end area starts and the left and right areas are alternately shifted to the center side one pixel at a time.

  As described above, according to this embodiment, when a date display is combined with a target image, image evaluation is performed for each region, and the most suitable region for specifying a date display is specified based on the evaluation result. The date display data is synthesized in the specified area. As a result, the date display is automatically prevented from covering important parts of the image, such as where the person on the target image is shown, where the subject is in focus, or where a given object is shown. It is possible to maintain the taste of the image even when displaying the date.

  Further, image evaluation is preferentially performed from a region more suitable as a position for combining date display, and when an evaluation value having a level that is satisfactory to some extent as a region for combining date display is obtained, date display data is combined in that region. As a result, it is possible to acquire an image with the date displayed at an appropriate position that does not affect the important part of the target image in a short time and with a small calculation burden.

(4) Third Embodiment It is possible to fuse the first and second embodiments described above.
That is, it is possible to determine an optimum region for synthesizing the date display after executing a rotation process for matching the top and bottom directions of the target image and the date display. In this case, after the processes of S700 to S740 of FIG. 2 or S800 to S840 of FIG. 3 are performed, the processes of S940 to S980 of FIG. 24 are performed.

  By the processing up to S740 (S840), the arrangement direction of the image data 14a and the date display data is determined so that the top direction of the target image and the date display coincide with each other. Thereafter, the angle of the selected area with respect to the target image is maintained. However, the location is sequentially switched and image evaluation is performed for each selected region, and one region for synthesizing the date display is determined based on the evaluation value α obtained by each image evaluation.

  According to the present embodiment, the date is displayed at an appropriate location where the target image and the top-and-bottom direction coincide with each other and which does not affect the important part of the target image. That is, it is possible to acquire a print image including a date display in which both the position and the direction with respect to the target image are in an optimal state.

1 is a block diagram showing a schematic configuration of an image processing apparatus according to the present invention. 5 is a flowchart showing the contents of image composition processing according to the first embodiment. 7 is a flowchart showing another content of the image composition processing according to the first embodiment. The figure which shows the object image image | photographed horizontally. The figure which shows the target image image | photographed vertically long. Explanatory drawing which showed a mode that image data was rotated. The figure which showed the image after rotation and a synthesis | combination. Explanatory drawing which showed a mode that date display data was rotated. The figure which showed the image after rotation and a synthesis | combination. The flowchart which showed the content of the top-and-bottom direction determination process. The flowchart which shows the top-and-bottom determination procedure based on a person area. The flowchart which shows the detailed procedure of step S202. Explanatory drawing which shows the content of the top-and-bottom determination process based on a person area. The flowchart which shows the top-and-bottom determination procedure based on the sky / ground area. Explanatory drawing which shows the content of the top-and-bottom determination process based on the sky and ground area regarding a daytime image. Explanatory drawing which shows the content of the top-and-bottom determination process based on the sky and the ground area regarding a night image. The flowchart which shows the top-and-bottom determination procedure based on a straight line. The flowchart which shows the procedure of a convergence straight line determination. Explanatory drawing which shows the example of the image containing the straight line over the whole image. Explanatory drawing which shows the example of the image containing a convergence straight line group. The flowchart which shows the top-and-bottom determination procedure based on a rod-shaped object. Explanatory drawing which shows the content of the top-and-bottom determination process based on a rod-shaped object. The flowchart which shows the procedure of a general top-and-bottom determination process. The flowchart which shows the content of the image composition process concerning a 2nd Example. The figure which showed an example of the selection order of a selection area | region. The figure which showed an example of the selection order of a selection area | region. The figure which showed an example of the selection order of a selection area | region. The flowchart which showed the image evaluation process based on the presence or absence of a person. The flowchart which showed the image evaluation process based on the matching degree of focus. The flowchart which showed the image evaluation process based on the magnitude | size of a brightness | luminance difference.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Computer, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Hard disk (HD), 14a ... Image data, 20 ... Printer, 40 ... Digital camera

Claims (9)

Image data input means for inputting image data representing the target image;
A top / bottom determination means for determining the top / bottom direction of the target image;
In accordance with the determination of the top and bottom, image combining means for combining the image data and date display data for performing date display so that the top and bottom directions of the target image and the date display added to the target image match,
An image processing apparatus comprising: The top-and-bottom direction determination unit determines the top-and-bottom direction by reading the top-and-bottom information indicating the top and bottom direction of the target image recorded in advance in the header area from the header area of the image data. Image processing apparatus. The top / bottom determination means acquires predetermined feature information representing a feature of the target image by analyzing a pixel value of the image data, and determines a top / bottom direction using a determination rule using the feature information. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized. The image synthesizing means reads out date information indicating the recording time of the target image recorded in advance in the header area from the header area of the image data, and generates the date display data according to the read date information. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized. The image synthesizing unit rotates the image data by a predetermined angle according to the determination of the top and bottom, and synthesizes the image data and the date display data to match the top and bottom directions of the target image and the date display. The image processing apparatus according to claim 1. The image synthesizing unit rotates the date display data by a predetermined angle in accordance with the determination of the top and bottom to synthesize the image data and the date display data, thereby matching the top and bottom directions of the target image and the date display. The image processing apparatus according to any one of claims 1 to 4. The image synthesizing means, when synthesizing the image data and the date display data, performs a predetermined image evaluation for each area in the image data and determines an area for synthesizing the date display according to the evaluation result. The image processing apparatus according to claim 1, wherein date display data is synthesized with the determined area. An image data input step for inputting image data representing the target image;
A top and bottom determination step for determining the top and bottom direction of the target image,
In accordance with the determination of the top and bottom, the image combining step of combining the image data and the date display data for performing date display so that the top and bottom directions of the target image and the date display added to the target image coincide with each other. An image processing method. An image data input function for inputting image data representing the target image;
A top / bottom determination function for determining the top / bottom direction of the target image;
In accordance with the determination of the top and bottom, the computer executes an image composition function for combining the image data and date display data for performing date display so that the top and bottom directions of the target image and the date display to be added coincide with each other. An image processing program characterized by that.