JP2007243551A - Image processing unit and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing unit for enabling a user who has little in expertise of image processings to process an image photographed in various environments appropriately and easily according to the various environments, and to provide an imaging apparatus that becomes especially useful, in combination with the image processing unit. <P>SOLUTION: The image processing unit (computer 10) performs prescribed image processings on an image photographed by a digital camera 20. The image processing unit comprises an information acquisition means (realized by an information acquisition section 11, an arithmetic unit 13, or the like here) for acquiring various kinds of photographic environment information, including a photographed image, position information in photography, photographic direction, and photographing time; an image processing table T in which prescribed image processing is prescribed, separate from the photographic environment information; and an image processing means (realized by the arithmetic unit 13, appropriate software, or the like here) for referring to the table T to execute the prescribed image processings, corresponding to the content of the photography environment information related to the photographed image to the photographed image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that performs predetermined image processing on an image captured by an imaging apparatus such as a digital camera, and an imaging apparatus that is particularly useful when used in combination with the image processing apparatus.

  In recent years, there has been a demand for an image processing apparatus capable of easily realizing ideal image correction even for a person who lacks expertise in image processing. Therefore, conventionally, image correction software that automatically corrects an image according to the user's preference, for example, an image processing apparatus equipped with a retouch application has been put into practical use (see, for example, Patent Document 1).

For example, when a person is photographed with the sunset in the background (back), the subject is likely to be darker than the appropriate value, that is, a so-called backlight image compared to the background. Therefore, in such a case, it is desirable to optimize the image by performing appropriate correction. Therefore, conventionally, when an object (face) is detected using an appropriate face recognition means, when the detected object is darker than an appropriate value compared to the surrounding image, the image is a backlight image. A method (software) or the like that performs determination and executes image processing (image correction) prepared in advance is used.
JP 2002-344989 A

  However, in such an image processing method using the face recognition means, if the brightness of the subject (face) is extremely low, even if the subject (face) is present in the image, the face cannot be recognized. There is also a concern that mistakes occur at a certain frequency and appropriate correction is not performed even though the image is a backlight image that should be corrected. Usually, many of these face recognition means detect faces by using the position of the skin color area on the image as a clue. Therefore, especially when a person is photographed in a reddish lighting environment (for example, sunset). , It becomes difficult to detect the skin color region containing a large amount of red component, and the recognition error is more likely to occur.

  The present invention has been made in view of such circumstances, and even a user who lacks expertise in image processing can easily perform appropriate image processing on images taken in various environments according to those environments. It is an object of the present invention to provide an image processing apparatus that can be used, and an imaging apparatus that is particularly useful when used in combination with the image processing apparatus.

  In order to achieve such an object, according to the first aspect of the present invention, as an image processing apparatus that performs predetermined image processing on an image captured by an imaging apparatus, an image captured by the imaging apparatus, and the image Information acquisition means for acquiring position information at the time of shooting, information on the shooting direction when the image was shot, and shooting time when the image was shot; and the information acquisition means A predetermined process according to the content of each piece of information acquired includes a table defined as image processing for the image, and a predetermined processing corresponding to the content of each piece of information acquired by the information acquisition unit with reference to the table. And image processing means for executing image processing (for example, image processing / correction) on the image.

  As described above, by acquiring position information (shooting location), shooting direction, shooting time, etc. at the time of shooting, it becomes possible to more accurately grasp the environment (shooting environment) at the time of shooting. Appropriate image processing can be performed on an image shot in a different environment according to the environment. In addition, by associating predetermined processing (for example, optimal image processing for each environment) in advance in the above table according to the contents of each information, even a user who lacks expertise in image processing can be in a shooting environment. Accordingly, optimal image processing can be easily performed.

  In the second aspect of the present invention, the position information includes latitude, longitude, and altitude, and the information related to the shooting direction indicates the vertical orientation of the imaging device at the time of shooting. It is effective to include information for specifying the degree of inclination and the direction in which the imaging apparatus is facing at the time of shooting.

  According to such a configuration, it is possible to specify the shooting position (where the image was taken on the earth, including high and low altitudes) in a three-dimensional manner, and the shooting direction is also suitable for the sky. It is possible to accurately grasp the direction in which the imaging lens was photographed, including whether it is facing the ground or the like.

  In addition, regarding the image processing apparatus according to the first or second aspect, as in the invention according to the third aspect, in addition to the information, the information acquisition unit further acquires temperature, humidity, and atmospheric pressure. It is effective to include these temperature, humidity, and atmospheric pressure in the information used for defining the processing in the table. By doing so, it is possible to easily specify the shooting environment (for example, indoor / outdoor determination, season specification, altitude, etc.) with high accuracy.

  By the way, in general, image correction software (for example, a retouch application) has a certain standard (substantially common standard widely applied to all captured images), and whether or not the target image satisfies this standard. Based on the above, it is determined whether or not to execute a predetermined image correction. When the image does not satisfy this standard, a predetermined image correction is performed to optimize the image.

  However, in the conventional general software, for example, as represented by the recognition error of the face recognition unit described above, an incorrect correction is performed or a correction to be performed is performed depending on the shooting environment (for example, the type of illumination). It is in fact that concerns remain that it will not be done.

  In view of such circumstances, in the invention according to claim 4, in the table, the content of each information acquired by the information acquisition unit is taken into account as a constant reference for determining the quality of the image. In addition, the predetermined image processing is associated with each criterion, and the image processing means refers to the table based on whether the image satisfies the criterion defined in the table. When the quality of the image is determined and it is determined that the standard is not satisfied, the image is corrected by executing predetermined image processing on the table associated with the standard. . With such a configuration, the above-described position information (shooting location) at the time of shooting (shooting location), shooting direction, shooting time, and other information (shooting environment information) are also used for image quality determination (determination of whether or not to correct). Is reflected, and image correction is appropriately performed with higher accuracy.

  Specifically, for example, according to the invention described in claim 5, whether or not the image is backlit based on the content of each information acquired by the information acquisition means as a reference for determining the quality of the image. It is effective to have a configuration including a reference for determining (image quality). With such a configuration, for example, the position of the sun is ascertained from the shooting time (season or time of day), and the imaging lens is used during shooting based on the position information (shooting location) and shooting direction at the time of shooting. Regarding the above-mentioned determination of whether or not it is backlit by determining (estimating) whether it is facing the sun and whether the shooting location is indoors or outdoors, and taking this into consideration along with the color information of the target image This can be performed more accurately and accurately. In addition, if the season and weather are grasped from the photographing time, position information, and photographing direction, and these are taken into account, it is possible to further improve the backlight determination accuracy. Also, many backlight images were taken in the evening in the evening (and sunset) (and sunset images are prone to recognition errors by the face recognition means), so the above shooting time (season and day) It is also effective to determine whether or not it is such an image (an image of sunset) based on the time zone in FIG.

  For example, according to the invention described in claim 6, as a reference for determining the quality of the image, based on the contents of each information acquired by the information acquisition means, the color temperature and contrast of the image, and It is also effective to have a configuration including a reference for determining whether or not at least one of the color balances is appropriate (image quality). With such a configuration, for example, from the shooting time and position information (shooting location) and the shooting direction, by grasping whether the shooting location is indoors or outdoors, the season, weather, etc., this is used as color information of the target image. In addition, it is possible to accurately and more accurately determine whether or not the color temperature, contrast, and color balance of the image are appropriate. That is, for example, if the shooting location is outdoors and the weather is “cloudy” or “rainy”, the color temperature is high, the color balance is biased to blue, and the contrast tends to be low. Make corrections. These factors (color temperature, color balance, and contrast) are also affected by the season (for example, winter / summer / spring / autumn) and the time (for example, day / night). It is desirable to determine the type of correction (for example, the type of filter) and the degree in consideration of the time.

  Moreover, regarding the image processing apparatus according to any one of claims 1 to 6, according to the invention according to claim 7, in the table, contents of each information acquired by the information acquisition unit Are classified for each predetermined scene (scene), and the predetermined image processing is associated with each classified scene, and the image processing means is configured to acquire each piece of information acquired by the information acquisition means. Based on the content of the image, it is determined which of the scenes the image corresponds to, and the image is corrected by executing predetermined image processing on the table associated with the determined scene. It is effective to adopt a configuration to perform.

  In this way, by assigning several scenes in advance on the table and associating the optimal image processing for each scene, the image can be corrected appropriately and accurately through the scene determination. become. In addition, for the scene discrimination at this time, various information such as the position information (shooting location) at the time of shooting, the shooting direction, the shooting time, and the like can be accurately determined. Thus, appropriate image correction can be executed with high accuracy for each of these scenes.

  For the scenes defined in the table, for example, according to the invention described in claim 8, at least the type of the image corresponds to any of the three categories: subject only, landscape only, subject and landscape. It is effective to classify by. With such a configuration, an optimum image for each scene in consideration of conditions that are particularly important in determining the type of image processing including image correction, that is, the presence or absence of the subject (for example, a person) Processes can be associated. The subject (the object that is the center of the image) is not limited to a person, but may be a building such as a building or a house.

  According to the invention described in claim 9, the scene defined in the table includes the season, the weather, the time zone in one day (for example, morning / day / evening / night), and the shooting location (for example, Even if it is classified by at least one of indoor / mountain / sea, etc., it is effective. These are parameters that are particularly important in determining the type and degree of image processing (image correction), and can be easily obtained from the contents of each information acquired by the information acquisition means. .

  On the other hand, in the invention according to claim 10, as an image processing device that performs predetermined image processing on an image captured by the imaging device, an image captured by the imaging device, and when the image is captured Position information, information related to the shooting direction when the image was shot, and shooting time when the image was shot, and information for specifying the angle of view of the imaging device. Information acquisition means; map acquisition means for acquiring a map around the shooting position with reference to the position information; a map acquired by the map acquisition means; contents of each information acquired by the information acquisition means; Based on the angle of view of the imaging device, all or a part of the object shown in the image taken by the imaging device (for example, only the one shown at the edge of the image) An object identification means for identifying which of the objects registered in the map corresponds to the object, and a predetermined number of all or a part of the objects identified by the object identification means. Trimming execution means for executing trimming on an image photographed by the imaging device using the object as an index.

  In the invention according to claim 11, as an image processing device that performs predetermined image processing on an image captured by the imaging device, an image captured by the imaging device, and when the image is captured Information for acquiring position information, information related to the shooting direction when the image was shot, and information for specifying the shooting time when the image was shot and the angle of view of the imaging device Acquisition means; map acquisition means for acquiring a map around the shooting position with reference to the position information; a map acquired by the map acquisition means; contents of each information acquired by the information acquisition means; and Based on the angle of view of the image pickup device, a predetermined object of all or a part of the object shown in the image taken by the image pickup device (for example, only the image shown at the edge of the image). Object identification means for identifying which of the objects registered in the map corresponds to the object, and all or a part of the objects identified by the object identification means (for example, images) Whether the object is a landmark object (for example, a landmark such as Tokyo Tower or Mount Fuji) in which only a part of the object in the map is registered. A landmark determination unit that determines whether or not, and a trimming execution unit that performs trimming on an image photographed by the imaging apparatus using the landmark object recognized by the landmark determination unit as an index; To do.

  With such a configuration, it is possible to acquire a map around the shooting position by the map acquisition unit, and based on the acquired map, the trimming execution unit can acquire an object or landmark in the image. Trimming (setting an outline) using an object as an index can be executed. For this reason, an image in which unnecessary portions are accurately cut off can be easily obtained. As for the landmark object in the map, an existing map in which the landmark object is registered in advance may be used, and the user may register a part (or all) of the map separately. It may be.

  In the image processing apparatus according to claim 11, as in the invention according to claim 12, a plurality of landmark objects are registered in the map acquired by the map acquisition unit, and the land When a plurality of landmark objects are recognized by the mark determination means, it is effective that the trimming execution means can select which landmark object is used as the trimming index. This makes it possible to perform trimming according to the user's preference.

  Further, according to the invention of claim 13, if a plurality of landmark objects are registered with priorities in the map acquired by the map acquisition unit, the landmark determination unit When a plurality of landmark objects are recognized, it is effective that the trimming execution unit executes the trimming using the landmark object having the highest priority as an index. By doing so, accurate trimming is automatically executed based on the priority order, and as a result, smooth image processing that saves the user's effort as much as possible is suitably realized.

  According to the fourteenth aspect of the present invention, as an image processing device that performs predetermined image processing on an image photographed by the imaging device, an image photographed by the imaging device, and when the image is photographed Information for acquiring position information, information related to the shooting direction when the image was shot, and information for specifying the shooting time when the image was shot and the angle of view of the imaging device Acquisition means; map acquisition means for acquiring a map around the shooting position with reference to the position information; a map acquired by the map acquisition means; contents of each information acquired by the information acquisition means; and Based on the angle of view of the imaging device, for all or some of the predetermined objects in the image captured by the imaging device, the object Object identifying means for identifying which of the objects registered in the object, and for all or a part of the objects identified by the object identifying means, the object Attention point determination means for determining whether or not an object has a point of interest set as a part of an object in the map (for example, a shooting point such as a signboard or emblem to which special attention should be paid for a building); Trimming execution means for performing trimming on an image photographed by the imaging device using at least one of the objects as an index when there is an object for which the attention point is determined to be set by the attention point determination means; It is set as the structure provided.

  In such a configuration, an attention point that is a particularly notable part of a predetermined object (for example, a shooting point such as a signboard or an emblem that should be particularly noted for a building) is registered in the map acquired by the map acquisition unit. Then, the trimming is performed by the trimming execution means using this point of interest as an index (for example, trimming with the point of interest as a boundary or trimming so that the point of interest is centered). That is, according to such a configuration, even a user who lacks expertise in image processing can easily obtain a preferable image only by setting a point of interest in advance (or using a pre-registered map). (Automatically) can be created.

  In the image processing device according to any one of claims 9 to 14, as in the invention according to claim 15, whether the person who is the subject appears in the image captured by the imaging device. A subject presence / absence determination unit for determining whether or not the subject person is reflected by the subject presence / absence determination unit. It is effective to adopt a configuration for performing trimming.

  Normally, when a person who is the subject (that is, a person with a small background is excluded), this person (subject) is also left in the image together with the previous object, landmark object, or point of interest. It is desirable to trim. In this regard, according to the configuration of the fifteenth aspect, since the trimming is executed using the person as the subject as an index, the person as the subject is erroneously cut off as an unnecessary part. This is precisely prevented. As a specific configuration, for example, when the invention described in claim 11 is applied together and a person as a subject is included in the screen, the landmark object is the center (main index). In addition, a configuration is possible in which trimming is performed so that the subject is not completely visible (sub-index).

  Further, in the invention according to claim 16, in the image processing device according to any one of claims 1 to 15, a predetermined map (for example, a world map) based on the position information acquired by the information acquisition means. 1 or a plurality of images are arranged on the map by respectively arranging the photographed images (images acquired by the information acquisition means) at locations corresponding to the shooting positions on a map of each country including Japan. To create a new image. According to such a configuration, for example, even when many images are taken at various places during a trip, a new image is created so that the shooting position for each shot image can be seen at a glance.

  On the other hand, in the invention described in claim 17, a position information detecting means (for example, GPS (Global Positioning System) for detecting position information at the time of shooting is used as an image pickup apparatus that picks up a shot image and stores it in an appropriate storage device. And a shooting time detecting means (for example, a radio clock) for detecting the shooting time, and a shooting direction detecting means (for example, a tilt sensor including a magnetic sensor) for detecting information related to the shooting direction. At the time of shooting, the position information and the shooting time detection unit and the shooting direction detection unit respectively detect the position information and the shooting time and the shooting direction information stored in the storage device. According to such an image pickup apparatus, position information (shooting location), shooting direction, shooting time, and the like at the time of shooting can be easily obtained. That is, such an imaging apparatus is particularly useful when used in combination with the image processing apparatus according to any one of claims 1 to 15.

  Further, in addition to each of the above means, if the apparatus further comprises means (for example, a predetermined sensor) for detecting temperature (air temperature), humidity and pressure (atmospheric pressure), for example, the image processing apparatus according to claim 3 or the like A particularly useful imaging device is realized in combination with the above.

  Hereinafter, an embodiment in which an image processing apparatus and an imaging apparatus according to the present invention are embodied will be described with reference to FIGS.

First, an outline (schematic configuration) of an image processing system according to these apparatuses will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of the system.
As shown in FIG. 1, this system mainly includes a monitor 18 used as a display device of a computer (here, a computer 10), an ink-jet color printer 17, and outputs of these image output devices. And a computer 10 (image processing apparatus) that performs predetermined image processing.

  The digital camera 20 (imaging device) has an appropriate storage medium and records (stores) a still image (or moving image) captured through a predetermined photoelectric conversion element as digital data. On the other hand, the server (database) 30 holds various maps (for example, world maps, maps of countries including Japan, etc.), weather / climate information of each location, and the like. By permitting only some users (or all users), various types of information such as the map are provided to the users as data (digital data). The input device 16 is composed of, for example, a mouse, a keyboard, and the like, and the user's intention can be appropriately reflected in the operation of the computer 10 by the user operating the input device 16. ing.

  More specifically, the computer 10 (image processing apparatus) includes a storage device 12 including, for example, a hard disk and a ROM (read only memory). The storage device 12 includes, for example, various drivers (software) including a driver for the input device 16 including a printer driver and a monitor driver, as well as an OS (operating system) and application software, and various types of drivers. A plurality of types of image processing tables T (details will be described later) and the like in which predetermined image processing is defined separately from the shooting environment information are stored.

  In addition, the computer 10 is provided with an information acquisition unit 11 including an appropriate interface and the like, and a captured image stored in the storage medium of the camera 20 and various information related to the image (specifically, details). Further, various information such as a map provided by the server 30 is also taken into the storage device 12 via the information acquisition unit 11.

  Further, in addition to the storage device 12 and the information acquisition unit 11, the computer 10 includes an arithmetic device 13 including, for example, a CPU (basic processing device) and a RAM (random access memory), and a printer driving circuit 14 including an appropriate circuit. And the monitor drive circuit 15, and the printer 17 and the monitor 18 can be controlled through these components. In short, like the general computer (electronic control device), the computer 10 basically executes various controls based on the instructions (arithmetic processing) of the arithmetic device 13. The reading of data via the acquisition unit 11 is also executed based on the instruction of the arithmetic device 13. That is, in this embodiment, information acquisition means for acquiring each information acquired (captured) by the camera 20 by the information acquisition unit 11, the storage device 12, the arithmetic unit 13, and the like, and imaging Map acquisition means for acquiring a map around the position is realized.

  FIG. 2 is a block diagram showing the program configuration of the printer driver. The printer driver is executed based on the arithmetic processing of the arithmetic device 13, for example.

  As shown in FIG. 2, this printer driver basically performs image correction or trimming (for example, various filters) on an image to be output by the printer 17 (image for printing). An image correction processing unit P1 that performs image processing such as cutting an unnecessary part of an image to determine an outline) and the image data subjected to the image correction and image processing are output by the printer 17 A printer image creation unit P2 for converting the image data into (printable) image data.

  That is, for example, printing application software is executed under the management of the OS, targeting an image (image stored in the storage device 12) taken by the camera 20 as an object (image for printing). On this software, for example, when a printing instruction is given from the user to the arithmetic device 13 through the input device 16 (for example, the user clicks a “print button” on the screen of the monitor 18 with a mouse), the printer driver The desired image correction and image processing, conversion into image data that can be output (printed) by the printer 17, and the like are performed on the printing image.

  In this embodiment, the face recognition means described above is prepared as a part of the image correction processing unit P1. Incidentally, as such face recognition means, for example, a skin color region is extracted, and a corresponding region (for example, shape information such as the center of gravity, locus length, area, etc. regarding the face) is extracted from this extracted skin color region ( What detects a face area) can be used.

FIG. 3 is a block diagram showing the configuration of the digital camera 20 in more detail.
As shown in FIG. 3, the camera 20 is roughly composed of an image pickup device 21 (photoelectric conversion device) such as a CCD sensor or a CMOS sensor, an appropriate display device (display DP), and a memory card, for example. And a control unit 22 and a signal processing circuit (an imaging signal processing circuit 23 and an image compression circuit 24) such as a processor and an ASIC.

  Further, in this camera 20 (imaging device), for example, a position information detection unit (position information detection unit) 25a using a GPS (Global Positioning System), a shooting time detection unit (shooting time detection unit) such as a radio clock, etc. ) 25b, and a direction sensor for specifying the direction in which the camera 20 (specifically, its lens) is directed using, for example, a magnetic sensor, or the vertical direction of the camera 20 using, for example, an acceleration sensor An imaging direction detection unit (imaging direction detection means) 25c configured by an inclination sensor or the like that detects the degree of inclination, and a temperature detection unit 25d and a pressure detection unit that detect temperature (air temperature), pressure (atmospheric pressure), and humidity, respectively. 25e and various sensors as the humidity detection unit 25f are further provided. Then, by these control units, under the control of the control unit 22, position information at the time of shooting (specifically, latitude, longitude, altitude, etc.), shooting time (specifically, time of year, month, date, day, etc.) , Information related to the shooting direction (specifically, the degree of inclination of the up and down direction of the camera 20 at the time of shooting, and information for specifying the direction the camera 20 was facing at the time of shooting), and the temperature (air temperature) at the time of shooting, By reading photographing environment information such as pressure (atmospheric pressure), humidity, and the like, these pieces of information are combined with the image information (captured image) (and the information is associated with each image), for example, Exif (digital It is stored in the storage medium MC as a file in the format of a standard camera recording format). In addition to the above various shooting environment information, the storage medium MC also includes shooting conditions for each image such as exposure time, flash, exposure mode, white balance, lens focal length, and the like. In the state associated with the (photographed image), for example, it is stored as an Exif format file and can be read out as necessary.

  That is, at the time of photographing by the camera 20, image information (image photographed by the camera 20) is acquired (captured) through appropriate photoelectric conversion by the imaging element 21, and each of the detection units 25a to 25a. The imaging environment information listed above is respectively acquired by 25f, and all of the information (image information and various imaging environment information) is stored in the storage medium MC. Under the control of the control unit 22, appropriate image processing (imaging signal processing and image compression processing) is performed on the image information (captured image) by the imaging signal processing circuit 23 and the image compression circuit 24. After being applied, the image information (photographed image) subjected to the image processing is further stored (or the image information is updated) in the storage medium MC.

  In addition, since the camera 20 includes an operation unit OP (for example, a button or a lever) together with the display DP, the operation unit OP is operated while viewing the display (for example, a liquid crystal monitor) DP (at this time, the display DP). The screen display is controlled by the control unit 22 in accordance with the operation (operation or state) of the operation unit OP), so that the user stores his / her preferences in the storage medium MC as necessary. It can be reflected in various information (image information and various photographing environment information).

  Next, referring also to FIGS. 4 to 6, an example of image processing executed by the system, more specifically, the arithmetic device 13 (FIG. 1) as the image processing means and the image correction processing unit P1 ( An example of image correction performed on the image taken by the camera 20 under the cooperation of FIG. FIG. 4 is a flowchart showing the main processing procedure of the image processing, and FIGS. 5 and 6 are tables showing an example of the image processing table T (FIG. 1) used for the image processing.

  As shown in FIG. 4, in this series of processing, first, in step S11, an image to be subjected to the image processing is acquired. Specifically, based on a command from the arithmetic unit 13 (FIG. 1), the captured image stored in the storage medium MC (FIG. 3) of the camera 20 and the image are stored via the information acquisition unit 11. Various related information (the shooting environment information and the shooting conditions listed above) is taken into the storage device 12 and stored.

  In step S12, the shooting environment information of the image acquired in step S11 is confirmed. In step S13, the quality of the image (captured image) is determined based on the shooting environment information. .

  Specifically, this image quality determination is based on the image processing table T (FIG. 1) stored in the storage device 12 based on the determination items defined in the table T (for example, backlight or color temperature quality determination). Etc.). Incidentally, although this determination item can be set in advance (automatic determination), the user may select it every time it is determined.

FIG. 5 is a chart showing an example of a table used for such image quality determination.
As shown in FIG. 5, in this table, predetermined image processing (A to D) is associated with each fixed criterion (quality criterion for each determination item) for determining image quality. Yes. In addition, here, the contents of each piece of information (imaging environment information) acquired by the information acquisition unit 11 (information acquisition means) is taken into account as the reference (good / bad determination reference).

  That is, the image quality determination in step S13 is performed based on whether or not the image to be processed satisfies the criteria (quality criteria) defined in such a table.

  Specifically, for example, when determining whether or not the light is backlit, for example, the position of the sun is grasped from the shooting time (season or time zone in one day), and position information (shooting location) at the time of shooting and Based on the shooting direction, it is determined (estimated) whether or not the imaging lens faces the sun during shooting and whether the shooting location is indoor or outdoor. Then, these are combined with the color information of the target image (for example, using a face recognition unit) to determine whether the light is backlit. That is, for example, when the recognition of the face recognition means is insufficient (not sure), it is determined that it is not backlit, for example, indoors based on the content of the shooting environment information, or is taken outdoors and at the time of shooting. When the lens faces the sun, it is determined that the light is backlit. On the other hand, when the recognition by the face recognition unit is sufficient (reliable), the subject is recognized by the face recognition unit as usual, and it is determined that the backlight is backlit when the brightness of the subject is darker than the background.

  Note that the determination mode shown here is merely an example, and any determination mode can be adopted depending on the application or the like. For example, the content of the shooting environment information is more important, and when it is estimated that the information is indoors, it is determined that the backlight is back light regardless of the degree of recognition of the face recognition unit. Also good. Further, from the shooting time, position information, and shooting direction, the season and weather (for example, acquired using the information provided by the server 30 (FIG. 1)) are also grasped and taken into account. As a result, it is possible to further increase the accuracy of determining backlight. Also, many backlight images were taken in the evening in the evening (and sunset) (and sunset images are prone to recognition errors by the face recognition means), so the above shooting time (season and day) It is also effective to determine whether or not this image (sunset image) is based on the image of the shooting environment information (for example, only when the image is a sunset image). Etc.)

  In addition, regarding the determination of whether the shooting location is indoor or outdoor, if the shooting time, temperature (air temperature), pressure (atmospheric pressure), humidity, etc. are also taken into account in the position information at the time of shooting, the accuracy is further improved. (For example, if the season is winter and the temperature is high (including the time zone), it is determined indoors). However, a configuration that makes it clear whether the shooting location is indoor or outdoor from the position information at the time of shooting (for example, the position information includes latitude, longitude, altitude, etc. as highly accurate detection values and If it is clear that it is indoor or outdoor, or if the positional information includes information on whether it is indoor or outdoor in advance), only the positional information at the time of shooting is sufficient.

  On the other hand, when determining whether the color temperature, contrast, and color balance of an image are appropriate, for example, whether the shooting location is indoor or outdoor based on the shooting time and position information (shooting location) and the shooting direction. And the season, the weather (for example, acquired using the information provided by the server 30 (FIG. 1)), and the like. Then, these are combined with the color information of the target image to determine whether the color temperature, contrast, and color balance of the image are appropriate. That is, for example, if the shooting location is outdoors and the weather is “cloudy” or “rainy”, the color temperature is high, the color balance is biased to blue, and the contrast tends to be low. When an acceptable range (good / bad judgment criterion) for determining whether or not the color balance / contrast is acceptable is strict, and when the image value does not satisfy the allowable range, it is determined that the image is not appropriate. On the other hand, if the shooting location is indoors or the weather is “sunny”, the pass / fail judgment is made based on the normal allowable range.

  Note that this determination mode is also merely an example, and any determination mode can be adopted depending on the application or the like. For example, on the contrary, when the weather is “clear”, the allowable range (good / bad determination criterion) may be relaxed (widened). Also, if the shooting location is indoors, the weather does not affect the image quality. Therefore, if it is determined whether the shooting location is indoors or outdoors, the determination of the weather can be omitted.

  If it is determined in step S13 that the above criteria (good / bad determination criteria) are not satisfied, in a subsequent step S14, a predetermined image on the table (FIG. 5) associated with the criteria (or determination items) is obtained. The processing (any one of the above-described A to D) is executed, and accordingly, the image is appropriately corrected. On the other hand, when it is determined in step S13 that the above-described criterion (good / bad determination criterion) is satisfied, the process proceeds to the next step S15 without executing the image correction.

  Note that the image processing (A to D) on the table (FIG. 5) may include variables (parameters). That is, variables are incorporated in advance in the image processing (A to D) in the table (FIG. 5), and based on the contents of each information (imaging environment information) acquired by the information acquisition unit 11 (information acquisition means). The processing contents of these image processes (A to D) may be partially changed. For example, in the case of image processing (BD), the degree of deterioration of each of the above elements (color temperature, color balance, contrast) when the shooting location is outdoors and the weather is “cloudy” or “rainy” is input as a variable. If correction processing is performed to compensate for this degree of deterioration, more appropriate image correction can be performed. These factors (color temperature, color balance, and contrast) are also affected by the season (for example, winter / summer / spring / autumn) and the time (for example, day / night). It is desirable to determine the type of correction (for example, the type of filter) and the degree so that the time is also taken into account (input to the image processing (BD) as a variable).

  In addition, a plurality of processes are prepared for each criterion (or determination item) (that is, a plurality of predetermined image processes can be performed on a photographed image), and the user can execute, for example, on application software. One of them may be selected.

  Further, in the subsequent step S15, a scene (scene) is determined for the image (captured image) based on the contents of each information (capture environment information) acquired by the information acquisition unit 11 (information acquisition means). This scene discrimination is also executed based on the image processing table T (FIG. 1) stored in the storage device 12.

FIG. 6 is a chart showing an example of a table used for such scene discrimination.
As shown in FIG. 6, in this table, the contents of each piece of information (imaging environment information) acquired by the information acquisition unit 11 (information acquisition unit) is classified for each predetermined scene (scene). Yes. Specifically, in this table, the type of the image to be processed corresponds to one of the three categories of “subject only”, “landscape only”, “subject and landscape”, as well as the season (for example, Spring / summer / autumn / winter), weather (for example, sunny / cloudy / rainy), time zone in one day (for example, 0: 0 to 3:00 /.../ 21:00 to 24:00), temperature (for example, ˜4) ° C /.../ 28 ° C ~), pressure (eg high pressure / normal / low pressure), humidity (eg high humidity / normal / low humidity), shooting location (eg indoor / mountain / sea / city / ...), image color (eg Image parameters such as white tone / black tone / red tone / ...) are prepared to classify various scenes (scenes), and various scenes (scenes) are defined by combinations of these image parameters. It has become so. For example, if the image is “subject only” and the season is “spring” (other image parameters are optional), for example, the season is “spring” and the shooting location is “outdoor (not indoors)”. When the color of the image is “white tone” (other image parameters are arbitrary), another scene or the like is assumed. In addition, the determination of the shooting location, for example, whether or not the shooting location is “mountain” is not only the positional information (altitude) at the time of shooting, but also temperature, humidity, and atmospheric pressure (change depending on altitude (altitude)) Further, it can be performed based on the season and the color of the image (for example, the color of the mountain corresponding to the season is registered in advance).

  That is, the determination of the scene in step S15 is performed by the contents of each piece of information (imaging environment information) acquired by the information acquisition unit 11 (information acquisition unit) (that is, the season, weather, time zone in day, temperature ( Temperature), pressure (atmospheric pressure), humidity, shooting location, etc.) and image information (that is, the type of image or color of the image, etc.), which scene is set in advance It is done in the form of judgment (discrimination).

  In the subsequent step S16, the image is corrected by executing predetermined image processing on the table (FIG. 5) associated with the determined scene (scene).

  Specifically, for example, when the season is “spring” and the color of the image is “white”, the scene of “sakura”, for example, the season is “autumn”, and the color of the image is “red” In the case of “autumn leaves”, for example, when the season is “winter” and the color of the image is “white”, it is “snow”, the weather is “clear”, and “temperature is low” By determining (discriminating) a “shade” scene, etc., optimal image processing preset for each of these scenes (scenes) is executed. For example, in the case of a “Cherry Blossom” scene, the entire image is corrected to pink (pink) to improve the appearance, or “Mountain” or “Sea” scenes (where the shooting location is “Mountain” or “Sea”). ), Image processing such as correction is performed so that the ridgeline or horizontal line of the mountain is at an appropriate position (for example, the horizontal line is at a position of “7: 3”). Further, when executing such image processing, the scene is further subdivided (for example, “cherry blossom”) according to other image parameters (for example, weather, time zone in a day, temperature (air temperature), pressure (atmospheric pressure), humidity, etc.). (Evening “Evening” and “Rain” scenes, etc.), it is possible to perform optimal image correction for each of these various types of scenes. It becomes like this.

  On the other hand, for example, if position information (photographing location) is used in combination with the above-described face recognition means, it is possible to determine the presence or absence of a subject accurately and more accurately. For example, when it is determined that an image is taken in a reddish lighting environment by grasping the lighting environment from the position information (photographing place) (for example, lighting information is registered in advance for each place), the face By setting the threshold value of the recognition means to a value stricter than usual, it is possible to determine the presence / absence of the subject accurately and more accurately.

In this embodiment, image correction is performed on an arbitrary captured image in this way.
Next, referring also to FIGS. 7 to 9, an example of the image processing executed by the system, more specifically, the arithmetic unit 13 (FIG. 1) as the object identification means, the landmark determination means, and the trimming execution means. ) And the image correction processing unit P1 (FIG. 2) and the like, an example of image processing (trimming) performed on the image photographed by the camera 20 will be described. 7 is a flowchart showing the main processing procedure of the image processing, FIGS. 8A and 8B are schematic views showing one aspect of the image processing (trimming), and FIG. It is a schematic diagram which shows an example of this map.

  As shown in FIG. 7, in this series of processing, first, in step S21, an image to be subjected to the image processing is acquired. Specifically, based on a command from the arithmetic unit 13 (FIG. 1), the captured image stored in the storage medium MC (FIG. 3) of the camera 20 and the image are stored via the information acquisition unit 11. Various related information (the shooting environment information listed above and other shooting conditions including the focal length of the lens) is taken into the storage device 12 and stored.

  In step S22, the shooting environment information of the image acquired in step S21 is confirmed. In step S23, the shooting position is determined based on the shooting environment information (specifically, position information at the time of shooting). Get a map of the surrounding area. Specifically, a map corresponding to the vicinity of the shooting position is downloaded as data from the server 30 via the information acquisition unit 11 based on a command from the arithmetic unit 13 (FIG. 1). In the map acquired here, it is assumed that a special object, that is, a landmark object (for example, a landmark such as Tokyo Tower or Mt. Fuji) is registered (landmark registration) for only some objects. However, after the map is acquired, the landmark object may be deleted or added by the user.

  Subsequently, in the subsequent step S24, the photographed image is obtained based on the photographing environment information acquired in step S21 and the focal length of the lens (information for specifying the angle of view of the camera 20) and the map acquired in step S23. For all the objects shown in (except for the subject), it is identified which of these objects corresponds to each of the objects registered in the map.

  For example, when the photographed image is a photographed image as illustrated in FIG. 8A and the map around the photographed position is a map as illustrated in FIG. Are used to detect the objects OB1 to OB4 and the object OB5 that is the subject, and collate the image with the map (including the angle of view of the camera 20), thereby removing the subject OB1 from the positional relationship. ... OB4 is identified as one of the objects OB1 to OB4 and OB6 to OB8 registered in the map, respectively (matching of symbols (OB1 to OB4) in FIG. 8A and FIG. 9) Corresponds to the result of this identification). When the object is detected, the presence / absence of a person as a subject is also confirmed by, for example, a face recognition unit (subject presence / absence determination unit). The line R in FIG. 9 indicates the angle of view of the camera 20. Here, for convenience of explanation, the camera 20 and the object OB5 (subject) are displayed on the map. However, the camera 20 and the object OB5 that is the subject are not actually shown on the map. .

  Although the object is identified here except for the subject, the object is also identified based on the map for the subject in order to improve the recognition accuracy of the recognition means (for example, face recognition means) related to the subject. The possibility of misidentification may be verified. Further, such a method may be adopted when performing face recognition in the above-described image correction (FIG. 4).

  Depending on the application, it may be sufficient to identify only some objects. For example, in the case of assuming trimming only in the horizontal direction, for example, only those appearing at both ends (horizontal ends) of the image (for example, two objects OB1 and OB4 in FIG. 8A). , Identification is enough.

  Also, for objects that are not shown on the map (for example, clouds and sun), when using a recognition means that recognizes them as objects, the objects on the map can be compared with the map. Identification that does not correspond to any of them will be performed.

  Next, in step S25, it is determined whether each of the objects OB1 to OB4 identified in step S24 is a landmark object in which only some of the objects in the map are registered. In subsequent step S26, trimming conditions are set based on the determination result in step S25 and the determination result of the presence / absence of the subject (obtained in step S24), and further in step S27, trimming is performed under this condition. Execute.

  For example, in the map illustrated in FIG. 9, when the object OB1 is a landmark object (an object registered as a landmark), the object OB1 and the object OB5 (subject) are used as indices (vertical / horizontal boundaries). Is set with reference to the objects OB1 and OB5), and for example, the captured image is trimmed with a frame (outline) T1 as indicated by a one-dot chain line in FIG. 8A. FIG. 8B shows an image after trimming. Here, the trimming condition is set (for example, using a predetermined table or the like) so that the landmark object is invisible (sub index) with the object OB5 (person) as the subject as the center (main index). Setting).

  Further, this trimming mode (trimming condition) is merely an example, and any trimming mode can be employed depending on the application. For example, the trimming condition may be set so that the subject is invisible (sub index) with the landmark object as the center (main index).

  Further, for example, when a plurality of landmark objects are recognized in step S25, which landmark object can be selected as an index for trimming (for example, on the application software, the user can select the input device). If it is configured such that it is selected through 16 (FIG. 1), trimming can be performed according to the user's preference.

  For example, when a plurality of landmark objects are registered with priority in the map, when the plurality of landmark objects are recognized in step S25, the landmark having the highest priority is registered. It is also effective to perform trimming using an object as an index. By doing so, accurate trimming is automatically executed based on the priority order, and as a result, smooth image processing that saves the user's effort as much as possible is suitably realized.

  In addition, for example, when the landmark object is not included in the screen, the subject is used as an index, or when the person who is the subject is not included in the screen, the landmark object is used as an index. In addition, when neither a landmark object nor a person as a subject is included in the screen, a predetermined object (for example, the object identified in step S24, which is most visible on the edge of the screen). Various trimming modes are conceivable, such as using an index or the like as an index or not performing trimming.

In this embodiment, image processing (trimming) is performed on an arbitrary captured image in this way.
The final image, which is completed by performing image processing such as image correction (FIG. 4) and image processing (FIG. 7) as necessary, is appropriately laid out after printing and is used for printing. The image is subjected to conversion to a bitmap image (dot image), quantization processing, CMYK conversion (conversion from RGB data to CMYK data) and the like through the printer image creation unit P2 (FIG. 2). Thus, the image data can be output (printed) by the printer 17 and then transferred to the printer 17. Then, printing is performed on a predetermined printing paper through the printer 17.

  In this embodiment, the predetermined layout provided by the server 30 (FIG. 1), for example, based on the shooting environment information (particularly position information) acquired through the information acquisition unit 11 or the like when the image is laid out. By arranging the images (captured images) at locations corresponding to the shooting positions on the map, a new image in which one or more images are arranged on the map is created. That is, in this embodiment, the newly created image is output (printed) by the printer 17.

  Specifically, as shown in FIG. 10, for example, a predetermined map of Japan is downloaded from the server 30 and a plurality of captured images PCT captured at different locations correspond to the capturing positions on the map. By arranging each of them, a new image is created so that the shooting position for each shot image can be seen at a glance, as shown in FIG. However, this is only an example, and the layout can be performed in an arbitrary manner depending on the application. For example, a map used for the layout of the captured image may be a world map or a map of a country other than Japan.

  As described above, according to the image processing apparatus and the imaging apparatus according to this embodiment, the following excellent effects can be obtained.

  (1) As an image processing apparatus (computer 10) that performs predetermined image processing on an image captured by the image capturing apparatus (digital camera 20), together with the captured image, the positional information at the time of capturing, the capturing direction, and the capturing time Predetermined image processing is defined separately from the information acquisition means (in this case, realized by the information acquisition unit 11, the storage device 12, the arithmetic unit 13, and the like) for acquiring various types of shooting environment information. Referring to this image processing table T and this table T, predetermined image processing (image correction and image processing) corresponding to the contents of the shooting environment information relating to the shot image is executed on the shot image. The image processing means (here, realized by the arithmetic device 13 (FIG. 1), the image correction processing unit P1 (FIG. 2), etc.) is provided. As a result, it becomes possible to more accurately grasp the environment (photographing environment) at the time of photographing, and as a result, appropriate image processing can be performed on images photographed in various environments according to those environments. become able to.

  (2) Furthermore, since the optimum image processing for each shooting environment is associated in advance in the table T, it is easy for the user who lacks expertise in image processing to perform the optimum image processing according to the shooting environment. To be able to do that.

  (3) The shooting environment information includes latitude, longitude, altitude, further, the degree of inclination of the camera 20 in the vertical direction at the time of shooting, and information for specifying the direction in which the camera 20 was pointed at the time of shooting. The configuration. This makes it possible to specify the shooting position (where the image was shot on the earth, including high and low altitudes) in a three-dimensional manner, and the shooting direction is also toward the sky or the ground. It is possible to accurately grasp the direction in which the imaging lens was taken, including whether the image is taken.

  (4) Furthermore, as the configuration in which the shooting environment information includes temperature, humidity, and atmospheric pressure, the information used for defining the processing in the image processing table T includes these temperature, humidity, and atmospheric pressure. (See FIG. 6). By doing so, it is possible to easily specify the shooting environment (for example, indoor / outdoor determination, season specification, altitude, etc.) with high accuracy.

  (5) In the image processing table T, the content of the shooting environment information is added as a fixed standard for determining the quality of the shot image, and predetermined image processing (correction processing) is performed for each of these standards. The image quality is determined based on whether or not the captured image satisfies the standard defined in the table T. When it is determined that the image does not satisfy the standard, the image is associated with the standard. The image (captured image) is corrected by executing predetermined image processing on the table T. As a result, the above-described position information (shooting location) at the time of shooting (shooting location), shooting direction, shooting time, and other information (shooting environment information) are also reflected in the image quality determination (determination of whether or not to correct). As a result, the image is corrected appropriately with higher accuracy.

  (6) In addition, as a reference for determining the quality of the image in the image processing table T, a criterion for determining whether or not the image is backlit (quality of the image) based on the contents of the shooting environment information is provided. The configuration. This makes it possible to accurately and more accurately determine whether or not the backlight is backlit.

  (7) Further, a criterion for determining whether the color temperature, contrast, and color balance of the captured image are good based on the contents of the shooting environment information is also provided as a reference for determining the quality of the image in the table T. The configuration. This makes it possible to accurately and more accurately determine whether or not the color temperature, contrast, and color balance of the image are appropriate.

  (8) In the image processing table T, the contents of the shooting environment information are classified for each predetermined scene (scene), and the optimum image processing for each scene is associated with each classified scene. Based on the content of the shooting environment information, it is determined which of the scenes the captured image to be processed corresponds to, and a predetermined value on the table T associated with the determined scene. The image is corrected by executing the image processing. By doing so, the correction of the image is appropriately and accurately performed. In addition, the scene information at this time can be accurately discriminated from a variety of scenes by using the information such as the position information (shooting location) at the time of shooting, the shooting direction, and the shooting time. As a result, appropriate image correction is executed with high accuracy for each of these scenes.

  (9) Further, in classifying the scenes defined in the image processing table T, the image type is classified according to one of the three categories of the subject only, the landscape only, the subject and the landscape. did. This makes it possible to associate optimum image processing for each scene, taking into account the presence or absence of a subject that is particularly important in determining the type of image processing including image correction. .

  (10) The scenes defined in the image processing table T are classified according to the season, weather, time zone in one day, and shooting location. These are also particularly important parameters for determining the type and degree of image processing (image correction). By adopting such a configuration, these important parameters are taken into account, and the optimum image for each scene. Processes can be associated.

  (11) Further, map acquisition means for acquiring a map around the shooting position with reference to the shooting environment information (especially position information) (in this case, realized by the information acquisition unit 11, the storage device 12, the arithmetic device 13, etc.) And the map acquired by the map acquisition means, the content of the shooting environment information, and the angle of view of the camera 20 (for example, objects OB1 to OB5 in FIG. 8A) of the shot image. For all or a part of predetermined objects, it is identified which of the objects (for example, objects OB1 to OB4 and OB6 to OB8 in FIG. 9) registered in the map corresponds to the object. Object identification means, and all of the objects identified by the object identification means, Alternatively, for some predetermined objects, landmark determination means for determining whether the object is a landmark object in which only some of the objects in the map are registered, and the landmark determination means With respect to the captured image, using the landmark object (for example, the object OB1 in FIG. 9) certified by the above as an index, or all or a part of the objects identified by the object identification means as an index. (In this case, the object identification unit and the trimming execution unit include the arithmetic unit 13 (FIG. 1), the image correction processing unit P1 (FIG. 2), etc.) Is realized). As a result, trimming (outline setting) using an object or landmark object in the image as an index can be executed, and as a result, an image (an image having a preferable composition) in which unnecessary portions are accurately cut off can be obtained. It can be easily obtained.

  (12) Further, when a plurality of landmark objects are recognized in step S25, it is possible to select which landmark object is to be used as a trimming index. It is also possible to perform trimming.

  (13) Further, when a plurality of landmark objects are registered with priority in the map, when a plurality of landmark objects are recognized in step S25, the land with the highest priority is registered. If trimming is performed using the mark object as an index, accurate trimming is automatically performed based on the priority order, and smooth image processing with minimal user effort is achieved. Will be.

  (14) Subject presence / absence determination means (face recognition means) for determining whether or not a person as a subject appears in the photographed image is further provided, and it is determined by the subject presence / absence determination means that a person as a subject is reflected. In this case, trimming is performed using the person who is the subject as an index. By doing so, it is possible to suitably prevent the person as the subject from being accidentally cut off as an unnecessary part.

  (15) At the time of image layout, based on the shooting environment information (particularly position information) acquired through the information acquisition unit 11 or the like, for example, the shooting position on the predetermined map provided by the server 30 (FIG. 1) A new image in which one or a plurality of images are arranged on a map is created by arranging the images (photographed images) in corresponding places. Thereby, for example, even when many images are taken at various places during a trip, a new image is created so that the shooting position for each shot image can be seen at a glance.

  (16) On the other hand, as an imaging device (digital camera 20) that captures captured images and stores them in an appropriate storage device (storage medium MC), a position information detection unit 25a that detects position information at the time of shooting, and a shooting time An imaging time detection unit 25b to detect, an imaging direction detection unit 25c to detect information related to the imaging direction, a temperature detection unit 25d to detect temperature (air temperature), humidity, and pressure (atmospheric pressure), a pressure detection unit 25e, and humidity detection 25f, and at the time of shooting, each piece of information detected by each unit is stored in the storage device (storage medium MC). By using such an imaging apparatus in combination with the image processing apparatus (computer 10), this can be realized more easily and accurately in the above-described image processing system (FIG. 1).

  The embodiment described above may be modified as follows.

  A means (attention point determination means) for determining whether or not the object identified by the object identification means is an object for which a point of interest has been set, for all or a part of the predetermined objects. When there is an object for which the point of interest is determined to be set by the point of interest determination means, means for performing trimming on the image photographed by the digital camera 20 (trimming) using at least one of them as an index (Execution means). These means can be realized by, for example, the arithmetic device 13 (FIG. 1), the image correction processing unit P1 (FIG. 2), and the like. The attention point is set as a part of an object in the map, for example, a shooting point such as a signboard or emblem that should be particularly noted for a building.

  Hereinafter, referring to FIG. 11, the captured image is an image as illustrated in FIG. 11A, and the emblem EM in the figure is an attention point of the object OB <b> 4 (particular attention should be given to the map registered in the map). (Part)) will be described. 11A corresponds to the previous FIG. 8A, the same elements as those in FIG. 8A are denoted by the same reference numerals, For convenience, the explanation is omitted.

  That is, in this case, for example, using the emblem EM (attention point) as an index, for example, so that the attention point is at the center (that is, with a frame (contour) T2 as indicated by a one-dot chain line in FIG. 11A), Trimming of the captured image is executed. By doing so, even a user who lacks expertise in image processing can easily obtain an image with a composition centered on the point of interest. FIG. 11B shows an image after trimming.

  In the above embodiment, image correction (steps S13 and S14) based on the quality determination of the captured image is performed, but this is not essential and can be omitted as appropriate. Conversely, image correction based on scene determination (steps S15 and S16) may be omitted, and only image correction based on the quality determination of the captured image may be performed.

  In the above embodiment, the information acquisition unit is realized by the information acquisition unit 11, the storage device 12, the arithmetic device 13, and the like. That is, a system is assumed in which the image processing apparatus (computer 10) exchanges data (data communication) with the camera 20 through an appropriate communication line or the like (or wireless communication). However, this information acquisition means is not limited to such a configuration, and an arbitrary one can be adopted. For example, the image processing apparatus (computer 10) is provided with a slot that allows an appropriate storage medium (for example, the storage medium MC) to be inserted / removed (attached / detached), and the captured image is taken from the camera 20 via the storage medium. Alternatively, it may be configured to receive (acquire) various shooting environment information. In short, any configuration that can acquire necessary information is sufficient.

  The above various shooting environment information can be omitted or added as appropriate according to the application. In short, it is sufficient if you have the information you need to achieve your goals. That is, as long as the usage is limited (or limited), the amount of information of each piece of information can be reduced. Specifically, for example, when the sky or the ground is not photographed (the camera 20 is used only for photographing in the horizontal direction using a predetermined tripod or the like), the up and down direction at the time of photographing with the camera 20 is used. Since there is no need to use this, the information relating to the shooting direction need only include information for specifying the direction in which the camera 20 is facing at the time of shooting.

  In each of the above embodiments, the image correction processing unit P1 is prepared as part of the driver software (printer driver) for the printer 17, but the image correction processing unit P1 is provided in any manner and usage. It is. For example, if the image correction processing unit P1 is provided as part of the driver software of the monitor 18, image correction or the like can be performed on the screen output of the monitor 18. Further, the image correction processing unit P1 may be stored in, for example, the storage device 12 as a unit that performs image correction or image processing on an arbitrary image independently of the driver software. Furthermore, these patterns may be arbitrarily combined.

  -About the said digital camera 20, this can also be set as the structure further provided with the means to determine whether an imaging | photography place is indoor or outdoor. For example, when a GPS or the like that does not receive radio waves indoors is used as the position information detection unit 25a, it is possible to determine that the position information cannot be detected indoors.

  -The detection aspect of each detection part in the said digital camera 20 is not limited to the said aspect, Arbitrary aspects can be employ | adopted according to a use etc. For example, the detection of the shooting position by the position information detection unit 25a may be estimated from the preceding and following shooting points. As the position information detection unit 25a, in addition to the GPS, for example, a device using base stations in various places installed in the PHS, a device using a wireless LAN, or the like can be used.

  Arbitrary information including the result of determining whether the shooting location is indoor or outdoor is stored in the storage medium MC as, for example, an Exif format file in a state associated with the shot image. Good. For example, it is conceivable that the user himself / herself records arbitrary shooting conditions (for example, the presence or absence of a subject) at the time of shooting.

  As described above, there are a wide variety of configurations that the digital camera 20 can take, and the configuration of the computer 10 can also take various configurations corresponding to the configuration of the camera 20. That is, the configuration of the computer 10 can be changed as appropriate according to the configuration of the camera 20.

  Further, the image processing apparatus (computer 10) may be mounted on an arbitrary imaging apparatus (for example, the digital camera 20 or a mobile phone with a camera) (for example, integrated in the imaging apparatus main body). .

The block diagram which shows typically the outline | summary (schematic structure) of the image processing system containing the image processing apparatus and imaging device which concern on the embodiment about the image processing apparatus and imaging device which concern on this invention. FIG. 2 is a block diagram showing a program configuration of a printer driver used in the system. FIG. 2 is a block diagram schematically showing a schematic configuration of the imaging apparatus according to the embodiment. The flowchart which shows the main process sequence about an example of the image process (image correction) performed by the said system. The chart which shows an example of the table used for the quality determination of an image. The chart which shows an example of the table used for scene discrimination | determination. The flowchart which shows the main process sequence about an example of the image process (trimming) performed by the said system. (A) And (b) is a schematic diagram which shows the one aspect | mode of trimming. The schematic diagram which shows an example of the map around an imaging | photography position. The schematic diagram which shows an example of the map used for the layout of an image. (A) And (b) is a schematic diagram which shows the modification (another aspect) of trimming.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Computer, 11 ... Information acquisition part, 12 ... Memory | storage device, 13 ... Arithmetic unit, 14 ... Printer drive circuit, 15 ... Monitor drive circuit, 16 ... Input device, 17 ... Color printer, 18 ... Monitor, 20 ... Digital camera , 21 ... imaging device, 22 ... control unit, 23 ... imaging signal processing circuit, 24 ... image compression circuit, 25a ... position information detection unit, 25b ... shooting time detection unit, 25c ... shooting direction detection unit, 25d ... temperature detection unit , 25e ... pressure detection unit, 25f ... humidity detection unit, 30 ... server (database), DP ... display, EM ... emblem, MC ... storage medium, OP ... operation unit, OB1 to OB8 ... object, P1 ... image correction processing unit , P2 ... printer image creation unit, PCT ... taken image, T ... image processing table.

Claims (17)

In an image processing apparatus that performs predetermined image processing on an image captured by an imaging apparatus,
Acquires an image captured by the imaging device, position information when the image is captured, information on a capturing direction when the image is captured, and a capture time when the image is captured. Information acquisition means for
A table in which a predetermined process for each content acquired by the information acquisition unit is defined as an image process for the image;
An image processing unit that refers to the table and that can execute one or more predetermined image processing corresponding to the content of each piece of information acquired by the information acquisition unit on the image;
An image processing apparatus comprising: The position information includes latitude, longitude, and altitude, and the information relating to the shooting direction is for specifying the degree of inclination of the up-and-down direction of the imaging device at the time of shooting and the direction in which the imaging device was turned at the time of shooting. The image processing apparatus according to claim 1, comprising: 2. The information acquisition unit further acquires temperature, humidity, and atmospheric pressure in addition to the information, and the information used for defining the processing in the table includes the temperature, humidity, and atmospheric pressure. Or the image processing apparatus of 2. In the table, the content of each piece of information acquired by the information acquisition unit is added as a constant reference for determining the quality of the image, and the predetermined image processing is associated with each reference. And
The image processing means determines whether the image is acceptable or not based on whether or not the image satisfies the standard defined in the table with reference to the table and determines that the image does not satisfy the standard. The image processing apparatus according to claim 1, wherein the image is corrected by executing predetermined image processing on the table associated with the reference. The image processing apparatus according to claim 4, further comprising: a reference for determining whether the image is backlit based on the content of each piece of information acquired by the information acquisition unit as a reference for determining whether the image is good or bad. . As a criterion for determining the quality of the image, it is determined whether at least one of the color temperature, the contrast, and the color balance of the image is appropriate based on the content of each information acquired by the information acquisition unit The image processing apparatus according to claim 4, further comprising a reference for performing the operation. In the table, the contents of each information acquired by the information acquisition means are classified for each predetermined scene, and the predetermined image processing is associated with each classified scene,
The image processing means determines which of the scenes the image corresponds to based on the content of each information acquired by the information acquisition means, and on the table associated with the determined scene. The image processing apparatus according to claim 1, wherein the image is corrected by executing the predetermined image processing. The image processing apparatus according to claim 7, wherein the scenes defined in the table are classified according to at least one of the three categories of the image type: the subject only, the landscape only, the subject and the landscape. . The image processing apparatus according to claim 7 or 8, wherein the scenes defined in the table are classified by at least one of a season, a weather, a time zone in a day, and a shooting location. In an image processing apparatus that performs predetermined image processing on an image captured by an imaging apparatus,
The image taken by the imaging device, the position information when the image was taken, the information about the shooting direction when the image was taken, the shooting time when the image was taken, and the Information acquisition means for acquiring information for specifying the angle of view of the imaging device;
Map acquisition means for acquiring a map around the shooting position with reference to the position information;
Based on the map acquired by the map acquisition means, the contents of each information acquired by the information acquisition means, and the angle of view of the imaging device, all of the objects shown in the image taken by the imaging device, Alternatively, for some predetermined objects, object identification means for identifying which of the objects registered in the map the object corresponds to,
Trimming executing means for performing trimming on an image photographed by the imaging device, using a predetermined object as an index for all or a part of the objects identified by the object identifying means;
An image processing apparatus comprising: In an image processing apparatus that performs predetermined image processing on an image captured by an imaging apparatus,
The image taken by the imaging device, the position information when the image was taken, the information about the shooting direction when the image was taken, the shooting time when the image was taken, and the Information acquisition means for acquiring information for specifying the angle of view of the imaging device;
Map acquisition means for acquiring a map around the shooting position with reference to the position information;
Based on the map acquired by the map acquisition means, the contents of each information acquired by the information acquisition means, and the angle of view of the imaging device, all of the objects shown in the image taken by the imaging device, Alternatively, for some predetermined objects, object identification means for identifying which of the objects registered in the map the object corresponds to,
For all or some of the objects identified by the object identification means, it is determined whether or not the object is a landmark object in which only some objects in the map are registered. Landmark determination means;
Trimming execution means for performing trimming on an image photographed by the imaging device, using the landmark object recognized by the landmark determination means as an index;
An image processing apparatus comprising: A plurality of landmark objects are registered in the map acquired by the map acquisition means, and the trimming execution means determines which landmark object when the landmark determination means recognizes the plurality of landmark objects. The image processing apparatus according to claim 11, wherein an object can be selected as an index for trimming. In the map acquired by the map acquisition unit, a plurality of landmark objects are registered with priorities, and the trimming execution unit recognizes the plurality of landmark objects by the landmark determination unit. The image processing apparatus according to claim 11, wherein the trimming is performed using the landmark object having the highest priority as an index. In an image processing apparatus that performs predetermined image processing on an image captured by an imaging apparatus,
The image taken by the imaging device, the position information when the image was taken, the information about the shooting direction when the image was taken, the shooting time when the image was taken, and the Information acquisition means for acquiring information for specifying the angle of view of the imaging device;
Map acquisition means for acquiring a map around the shooting position with reference to the position information;
Based on the map acquired by the map acquisition means, the contents of each information acquired by the information acquisition means, and the angle of view of the imaging device, all of the objects shown in the image taken by the imaging device, Alternatively, for some predetermined objects, object identification means for identifying which of the objects registered in the map the object corresponds to,
Whether all or a part of the objects identified by the object identification means is an object with a point of interest set as a part of the object in the map. Attention point determination means for determining
Trimming execution means for performing trimming on an image photographed by the imaging device, using at least one of the objects as an index when there is an object for which the attention point is determined by the attention point determination means;
An image processing apparatus comprising: A subject presence / absence determining unit that determines whether or not a person as a subject is reflected in an image captured by the imaging device;
15. When the subject presence / absence determination unit determines that a person as a subject is reflected, the trimming execution unit executes the trimming using the person as a subject as an index as well. An image processing apparatus according to 1. A new image in which one or a plurality of images are arranged on the map by arranging the photographed images at positions corresponding to the photographing positions on a predetermined map based on the position information acquired by the information acquisition means. The image processing apparatus according to any one of claims 1 to 15. In an imaging device that captures a captured image and stores it in an appropriate storage device,
A position information detecting means for detecting position information at the time of shooting; a shooting time detecting means for detecting the shooting time; and a shooting direction detecting means for detecting information related to the shooting direction;
At the time of shooting, the position information, the shooting time detection unit, and the shooting direction detection unit, which are detected by the position information detection unit, the shooting time detection unit, and the shooting direction detection unit, respectively, are stored in the storage device. Imaging device.

Images