JP5625499B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and a program, and more particularly to a technique that enables a visually effective art conversion process.

  In recent years, imaging devices such as digital cameras and camera-equipped mobile phones have become widespread. An image of a subject imaged by such an imaging device is stored in the form of image data that can be easily processed.

  For example, in Patent Document 1, Patent Document 2, and the like, image data of a captured image is subjected to image processing, and image data of a painting-like image expressed by each pixel group resembling a painter's one-touch stroke is generated. An image processing method is disclosed. Image processing that converts image data in this way to generate art-like image data is hereinafter referred to as “art conversion”.

  In addition, the human face is recognized from the image data of the captured image, and the face area is processed with a lower gradation correction than the other area. An electronic camera to perform is disclosed in Patent Document 3.

Japanese Patent Laid-Open No. 10-11569 JP 2006-31688 A JP 2007-282119 A

  Image processing for art-converting image data of a captured image is effective in generating a unique image such as a picture drawn by a painter. However, if a strong conversion is performed when performing art conversion, the image may be crushed, and it may be difficult to understand what was captured. For example, when art conversion is performed on image data of an image obtained by photographing a famous building as a part of the background as a memory, there is a concern that the famous building as a memory cannot be identified. On the other hand, there is a demand for a user who prefers strong art conversion or a user who wants to perform art conversion only on a specific building.

  If the object whose identifiability is to be maintained is a human, for example, it is conceivable to recognize a human face as in the technique disclosed in Patent Document 3 and perform art conversion by differentiating only the face area. However, when a subject other than a human being is to be maintained as a target to be identified, specifically, a subject unique to each region, for example, a famous building or a landscape, the method disclosed in Patent Document 3 Then there is a limit.

  Therefore, when performing art conversion on image data, the present invention maintains the identifiability of a specific subject even if it is a specific subject such as a famous building or a unique subject for each region. An image processing apparatus capable of visually effective image processing, which can perform image processing by differentiating the area of the specific subject and the area excluding the specific subject, and The purpose is to provide a program.

The invention according to claim 1 is an image processing apparatus that performs image conversion processing on image data including information on presence / absence of a subject of interest and information on distance and depth, based on the information on the distance and depth included in the image data. When the target subject recognition unit for recognizing the target subject and starting the image conversion process, the information on the presence or absence of the target subject in the image data is confirmed, and if it is determined that the target subject is included, In the image data, the image data portion corresponding to the target subject recognized by the target subject recognition unit is different from the image data portion corresponding to the other portion of the image excluding the target subject. An image that performs image conversion processing on the entire image data when the image data is subjected to image conversion processing and it is determined that the subject of interest is not included. An image processing apparatus including a conversion processing unit is provided.

According to a fourth aspect of the present invention, a computer of an image processing apparatus that performs image conversion processing on image data including information on presence / absence of a subject of interest and information on distance and depth is based on the information on the distance and depth included in the image data. In addition, the information processing unit functions as a target subject recognition unit for recognizing the target subject, and when starting the image conversion process, the information on the presence / absence of the target subject in the image data is confirmed to determine that the target subject is included. In this case, in the image data, the image data portion corresponding to the target subject recognized by the target subject recognition unit is different from the image data portion corresponding to the other part of the image excluding the target subject. It said image data is the image conversion processing, if it is determined that the information does not include an object of interest, the entire image data There is provided an image processing program to function as the image conversion processing means for image tone conversion processing.

  According to the present invention, visually effective art conversion can be performed.

It is a block diagram which shows the structure of the hardware of the stereo digital camera which concerns on 1st Embodiment. It is a functional block diagram which shows the functional structure of the data processing part of the stereo digital camera of FIG. It is a figure which shows an example of the attention object specific table memorize | stored in the memory of the data processing part of FIG. It is a figure which shows an example of the image conversion process content table memorize | stored in the memory of the data processing part of FIG. It is a figure which shows an example of the image converted by the art conversion process which the data processing part of FIG. 2 performs. It is a flowchart explaining a flow by the art conversion process which the data processing part of FIG. 2 performs. It is a figure which shows an example of the image converted by the art conversion process which the data processing part of FIG. 2 performs. It is a figure which shows an example of the image converted by the art conversion process which the data processing part of FIG. 2 performs. It is a block diagram which shows the structure of the hardware of the digital camera which concerns on 2nd Embodiment. It is a functional block diagram which shows the functional structure of the data processing part of the digital camera of FIG. It is a figure which shows an example of the pattern matching table memorize | stored in the memory of the data processing part of FIG. 10 is a flowchart for explaining the flow by art conversion processing executed by the data processing unit of FIG. 9.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the following description, a digital camera is used as an example of the image processing apparatus. However, this is merely an example and does not limit the present invention.

[First Embodiment]
An embodiment of the present invention will be described with reference to FIGS.

[Digital camera configuration]
A stereo digital camera 1 according to a first embodiment of the image processing apparatus of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing an internal hardware configuration of the stereo digital camera 1. FIG. 2 is a functional block diagram showing a functional configuration of data processing unit 12 data of the stereo digital camera 1. 3 and 4 are diagrams showing examples of the subject-of-interest specifying table 112A and the image conversion processing content table 113A stored in the memory 34 of the stereo digital camera 1. FIG.

  As shown in FIG. 1, the stereo digital camera 1 includes a photographing unit 11, a data processing unit 12, and a user interface unit 13.

  The imaging unit 11 is configured to acquire image data of a three-dimensional image. Therefore, the photographing unit 11 includes left and right optical lens units 21 and two image sensors 22 corresponding to the left and right optical lens units, respectively.

  Each of the left and right optical lens units 21 has a front lens (not shown) that collects light in order to photograph the subject, and a focus lens that focuses the subject on the light receiving surface of each image sensor 22 (not shown). And a zoom lens (not shown) that freely changes the focal length within a certain range. Each of the left and right optical lens units 21 is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

  Each of the two image sensors 22 includes a photoelectric conversion element, an AFE (Analog Front End), and the like. The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element. A subject image is incident on the photoelectric conversion element from the optical lens unit 21 through a shutter unit (not shown). Therefore, the photoelectric conversion element photoelectrically converts (photographs) the subject image at regular intervals, accumulates image signals, and sequentially supplies the accumulated image signals to the AFE as analog signals. The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing and output as an output signal of the image sensor 22. Hereinafter, the digital signal of the image signal is referred to as “image data”. In this way, image data is output from the image sensor 22 and supplied to the data processing unit 12.

  The data processing unit 12 includes a CPU (Central Processing Unit) 31, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a memory 34, a display control unit 35, and an image processing unit 36. Prepare.

  The user interface unit 13 includes a display unit 41 configured by a display or the like provided in the case of the stereo digital camera 1, a position information acquisition unit 42 that acquires information about the current position of the stereo digital camera 1 and the subject, and a user An operation unit 43 that receives the instruction operation, a communication unit 44 that controls communication with an external device, and a drive 45 that reads / writes data from / to the removable storage medium 51 that stores image data. The position information acquisition unit 42 receives GPS signals from a plurality of GPS (Global Positioning System) satellites, and calculates a latitude and longitude indicating the current position of the stereo digital camera 1 based on the GPS signals. (Not shown) and various sensors such as a distance measuring sensor for measuring the distance between the subject of interest and the stereo digital camera 1, and an orientation sensor for acquiring the shooting direction (optical axis direction) of the stereo digital camera 1. . Further, the distance between the subject of interest at the time of shooting and the stereo digital camera 1 is, for example, when the distance measuring sensor is configured to include an infrared sensor, the time until the reflected wave returns by irradiating infrared rays or ultrasonic waves, The distance can be detected by the irradiation angle. Alternatively, when the stereo digital camera 1 is, for example, a passive system, it can be acquired from the lens position at the time of contrast autofocus. . In this case, the distance measuring sensor is not essential.

The CPU 31 executes various processes according to programs recorded in the ROM 32. The stereo digital camera 1 according to the present embodiment includes an art conversion mode for executing an art conversion process on a captured image or an image input from the removable storage medium 51, which will be described later, in addition to the normal shooting mode. The ROM 32 executes various functions including a normal shooting mode and various processes such as a process in an art conversion mode including an art conversion process to be described later, and various functions including a display control unit 35, an image processing unit 36, and a user interface unit 13. Is stored. The CPU 31 performs processing according to a program stored in the ROM 32, thereby including various functions including the display control unit 35, the image processing unit 36, and the user interface unit 13, a normal shooting mode, and an art conversion process to be described later. Various processing in the art conversion mode is realized.
The RAM 33 appropriately stores data necessary for the CPU 31 to execute various processes.

  The memory 34 includes a DRAM (Dynamic Random Access Memory), a ROM (Read Only Memory), and the like. In the DRAM, the image data output from the image sensor 22 is temporarily stored. The ROM stores image data, parameters, various flag values, threshold values, and the like necessary for various image processes, including regional information, art conversion process contents, and the like. The area information stored in the ROM includes a target object specifying table 112A that stores a target object and position information where the target object exists, for example, a latitude and longitude range and a region, which will be described in detail later, and a map. Contains information. Further, the memory 34 stores an image conversion processing content table 113A in which image conversion processing content and settable parameters are stored in association with each other, and image display image data (hereinafter referred to as “display image data”). A display memory area for saving and reading is also included.

  The display control unit 35 reads display image data stored in the display memory area of the memory 34, and displays an image expressed by the display image data (hereinafter referred to as “display image”) of the user interface unit 13. Execute the control to be displayed on the screen. For example, the display control unit 35 causes the display unit 41 to display the display image by generating an RGB signal based on the display image data and supplying the RGB signal to the display unit 41. The RGB signals generated by the display control unit 35 can be output to an external device (not shown) by the communication unit 44 of the user interface unit 13. As a result, the display image can be displayed even by an external device such as a television receiver, a personal computer, or a projector.

  The image processing unit 36 is configured by a DSP (Digital Signal Processor) or the like, and executes various image processing such as white balance correction processing and γ correction processing on the image data stored in the memory 34. In the present embodiment, at least a part of various image processes executed by the imaging target recognition unit 101, the target distance analysis unit 102, the filter unit 103, the parameter change unit 104, and the art conversion unit 105, which will be described later with reference to FIG. It is executed by the unit 36 and the remaining part is executed by the CPU 31. In other words, in the present embodiment, an imaging target recognition unit 101, a target distance analysis unit 102, a filter unit 103, a parameter change unit 104, and an art conversion unit 105, which will be described later, are stored in the ROM 32 and hardware such as a CPU 31 and an image processing unit 36. The program (software) is combined.

  The removable storage medium 51 stores the image data generated by the data processing unit 12. The removable storage medium 51 is realized by a memory card, for example, and constitutes a storage means.

  Although not shown, the operation unit 43 includes a shutter key, a power button, a zoom key, a mode switching key, and the like, and generates an operation signal based on each operation and sends it to the data processing unit 12. For example, when the mode switching key is operated by the operator and the art conversion mode is designated, an operation signal is sent to the data processing unit 12, and the CPU 31 switches the mode to the designated art conversion mode. Similarly, when the mode switch key is operated by the operator and the normal shooting mode is designated, the CPU 31 switches the mode to the designated normal shooting mode. In addition to this, various operation signals are sent to the data processing unit 12 as the operator operates the operation unit 43, and the CPU 31 performs processing according to the operation signals.

  Next, the functional configuration of the data processing unit 12 will be described with reference to FIG.

  The data processing unit 12 includes a memory 34, a display control unit 35, a shooting target recognition unit 101, a target distance analysis unit 102, a filter unit 103, a parameter change unit 104, and an art conversion unit 105. Yes. In the memory 34 of the data processing unit 12, an original frame storage unit 111, a photographing object information storage unit 112, an image conversion processing content storage unit 113, and a processing frame storage unit 114 are configured. . The shooting target information storage unit 112 stores a target subject specifying table 112A that stores a target subject and position information where the target subject exists, for example, a latitude and longitude range and a region in association with each other. The image conversion processing content storage unit 113 stores an image conversion processing content table 113A in which the content of the image conversion processing and the parameters that can be set are stored in association with each other.

  FIG. 3 shows an example of the subject-of-interest specifying table 112A stored in the photographic object information storage unit 112. As shown in FIG. 3, the target subject specifying table 112A associates the target subject with the latitude and longitude ranges where the target subject exists.

  “Attention subject” refers to an object that the user wants to maintain identifiability even when performing art conversion, or an art conversion that differentiates the area of the object from the area excluding the object Specifically, it is preferable to use a subject unique to each region, such as a famous building or a landscape. In the attention subject specifying table 112A, a highly-known building, landscape, etc. are stored as attention subjects in advance together with position information. It should be noted that the user operates the operation unit 43 to separately register the desired building, landscape, etc. as the subject of interest in association with the position information, separately as the subject of interest. Also good. Alternatively, the target subject stored in the target subject specifying table 112 </ b> A may be invalidated by operating the operation unit 43 by the user during shooting. In this case, for example, a flag for setting validity / invalidity for each subject of interest may be provided in the subject-of-interest identification table 112A, and the flag validity / invalidity setting may be changed according to the operation of the operation unit 43 by the user.

  FIG. 4 shows an example of the image conversion processing content table 113 </ b> A stored in the image conversion processing content storage unit 113. As shown in FIG. 4, in the image conversion processing content table 113A, although details are omitted, the content of the image conversion processing to be executed, the settable parameters, and the validity / invalidity thereof are stored in association with each other.

  When the operator presses the shutter key, the photographing unit 11 photographs a subject included in the angle of view, and outputs image data of a three-dimensional image (hereinafter referred to as “frame image”) obtained as a result. The image capturing unit 11 also includes distance information from the stereo digital camera 1 to the subject, depth information of the subject, and information on the position and orientation of the stereo digital camera 1 when necessary, in the image data of the captured image. Give. Information on the distance from the stereo digital camera 1 to the subject at the time of shooting, and further on the depth of the subject is referred to as “information on the distance and depth of the subject of interest”.

  Specifically, the imaging unit 11 controls the position information acquisition unit 42 to calculate the latitude and longitude indicating the current position of the stereo digital camera 1 based on the GPS signal from the GPS satellites and the map information. get. Note that the position information acquisition unit 42 can improve the accuracy of the calculation by using error information of the GPS signal from the GPS satellite, position information of the wireless base station, and the like regarding the calculation of latitude and longitude. Furthermore, the photographing unit 11 is obtained from the position and direction of the stereo digital camera 1, the distance between the subject and the stereo digital camera 1, and the left and right image sensors 22 using various sensors of the position information acquisition unit 42. Subject depth information based on the image information is also acquired and added to the image data as three-dimensional information. Such addition of three-dimensional information is also disclosed in, for example, Japanese Patent Application Laid-Open No. 2005-37379, and any known method can be used, and thus detailed description thereof is omitted.

  In the present embodiment, for example, whether or not the subject of interest is photographed, that is, whether or not the subject of interest is present, is also added to the image data of the photographed image. Information regarding the presence or absence of the subject of interest may be recorded on the image data by the user himself / herself operating the operation unit 43 during shooting. Alternatively, it is determined whether or not the target subject is included in the image data of the image captured by the CPU 31 with reference to the target subject specifying table 112A stored in the memory 34, and the image data is determined based on the determination. Information on the presence or absence of the subject of interest may be recorded.

  The original frame storage unit 111 stores the image data of the frame image output from the photographing unit 11 as image data of the original frame image.

  The original frame storage unit 111 is not only the frame image output from the imaging unit 11 but also the image data of the frame image read from the removable storage medium 51 or the image data of the frame image acquired from the communication unit 44. Is also stored as image data of the original frame image.

  The shooting target recognition unit 101 determines whether or not the subject of interest has been shot based on information recorded in the image data of the frame image stored in the original frame storage unit 111.

  The target distance analysis unit 102 is based on the information about the distance and depth of the target subject embedded in the image data with respect to the target subject of the frame image for which the target subject is determined to be captured by the capturing target recognition unit 101. To identify the subject. Specifically, the target distance analysis unit 102 determines a subject having the same depth information of the subject and distance information at the time of shooting from the camera that has photographed the subject of interest to the subject of interest, and uses the subject having the same distance information as the subject of interest. Identify. When the image data includes information regarding the position and orientation when the stereo digital camera 1 is captured, the target distance analysis unit 102 captures the distance from the camera that captured the target subject to the target subject, the stereo digital Based on the position and orientation at the time of shooting by the camera 1, the position information where the target subject exists is specified, and the target subject specifying table 112 A that associates the target subject with the position information where the target subject exists is referred to and stored in the image data. The subject of interest may be specified. The target distance analysis unit 102 constitutes a target subject recognition unit. When the target distance analysis unit 102 refers to the target subject identification table 112A, the target distance analysis unit 102 and the shooting target information storage unit 112 constitute a target subject recognition unit.

  The filter unit 103 filters the pixels of the image data in the region corresponding to the target subject specified by the target distance analysis unit 102.

  When the art conversion unit 105 performs art conversion, the parameter change unit 104 sets parameters for image data in a region corresponding to the subject of interest so that the shape of the image is not so distorted from the original image.

  The art conversion unit 105 performs a normal art conversion process on the image data in the region excluding the region filtered by the filter unit 103. On the other hand, the art conversion unit 105 performs an art conversion process on the image data in the region filtered by the filter unit 103 in accordance with the parameters set by the parameter change unit 104. The filter unit 103, the parameter change unit 104, and the art conversion unit 105 constitute an image conversion processing unit.

  In the present embodiment, the art conversion unit 105 executes a process for changing the tone of the image data. Here, the painting style is defined based on the impression that can be felt from the image. “Japanese painting style”, “Western painting style”, “Watercolor painting style”, “Ink painting style”, “Pen painting style” ", Van Gogh", "Pastel", etc.

  As a method for changing the image data to a specific image tone, for example, techniques disclosed in Japanese Patent Laid-Open Nos. 10-011569 and 2006-031688 may be used. Specifically, pixels having high correlation are collected from pixels constituting image data on the basis of color or the like to form a group. Subsequently, for each group of pixels, for the same group of pixels, the color is replaced with the representative color of the group. A group of pixels in which the colors are replaced with the representative colors in this way form each brush stroke. According to such image processing, by changing the correlation of the pixels collected as a group and the shape (length (distance), flatness) of the group composed of pixels of the same color (representative color) The formed brush can be changed, and as a result, the tone of the image represented by the image data can be changed to various painting tone. In the present embodiment, the optimum combination of parameters such as the degree of correlation used in each processing, the length representing the shape of the group, and the flatness is set for each image tone. It is stored in the processing content table 113A. The art conversion unit 105 changes to various image styles based on the parameter information stored in the image conversion processing content table 113A of the memory 34.

  Changing each of the image data of the plurality of moving objects to different brushes is, for example, adjusting the shape of a group of pixels that make up the brush strokes, for example, the flatness of the group, when changing the image tone. Can be realized.

  FIG. 5 shows an example of a group of pixels representing a brush generated in this way. For example, as shown in FIGS. 5A and 5A, when the flatness of a group constituted by pixels of the same color representing the touch is increased, the touch becomes thick, and as a result, the touch of the image is displayed roughly. . As shown in FIGS. 5 (1) and 5 (c), when the flatness of the group composed of pixels of the same color is reduced, a thin brushstroke can be created, and as a result, the touch of the image is displayed finely. The groups shown in FIGS. 5 (1) and 5 (b) represent standard brush strokes.

  Here, in the example of the image conversion processing content table 113A in FIG. 4, among the image data of the plurality of moving objects, the touch of the image data of the region of the subject of interest being filtered is reduced (the touch is made finer), The parameters are set so that the parameter for thickening the touch of the image data in the region not filtered, that is, the region excluding the subject of interest (making the touch rough) is effective, and the image conversion process is performed. For this reason, as shown in FIGS. 5 (2) and 5 (a), when changing the image tone of the image data of the normal subject 73A, the flatness of the generated group is set high. On the other hand, for the image data constituting the subject 73E, the flatness of the generated group is set lower than when processing the image data constituting the normal subject 73A. As a result, the image data of the normal subject 73A is processed into an image with a tone of a thick brush, and the image data of the subject 73E is processed into an image of a tone with a thin brush. As a result, for the subject of interest 73E, it can be avoided that the image is crushed and it becomes difficult to understand what was photographed, and the identifiability can be maintained.

In addition to this, the art conversion process can also be performed using commercially available software such as Photoshop (registered trademark), which is software of Adobe Systems. Examples of parameters for changing the image tone include, for example, contrast, brightness, color density, hue, sharpness, and if necessary, noise reduction filter effect settings, color temperature, and γ correction γ value. Any one or combination of these parameters can be specified. However, the present invention is not limited to this, and any parameter may be included as long as it is appropriate for expressing the above-described image tone.
In the present embodiment, it is assumed that an optimal combination of the above-described parameters is preset and stored in association with the image conversion processing content table 113A for each of various image tones.

  In this manner, image data of a frame image (hereinafter referred to as “art converted image”) obtained by performing art conversion processing on the image data of the original frame image by the art conversion unit 105 is processed frame. Stored in the storage unit 114.

  The display control unit 35 reads the image data stored in the processing frame storage unit 114 and executes control for causing the display unit 41 to display an image expressed by the image data, that is, an art converted image.

  Note that the image data of the art converted image stored in the processing frame storage unit 114 may be stored in the removable storage medium 51.

[Flow of art conversion process]
When the user switches to the art conversion mode in accordance with the operation of the mode switching key of the operation unit 43, the CPU 31 controls various functions including the data processing unit 12 according to the art conversion processing program stored in the ROM 32. Hereinafter, art conversion processing executed mainly by the data processing unit 12 in the art conversion mode will be described.

  With reference to FIG. 6, the flow of the art conversion process which is one function of the stereo digital camera 1 will be described. The processing shown below starts when the processing shown below is switched to the art conversion mode in accordance with the operation of the mode switching key by the user. The image data to be processed includes information on the presence / absence of the subject of interest, distance information from the stereo digital camera 1 to the subject of interest at the time of shooting, depth information of the subject, and, if necessary, stereo digital at the time of shooting. It is assumed that the position information of the camera 1 is added.

  First, the CPU 31 reads image data of one image from the image data stored in the removable storage medium 51 in accordance with the art conversion processing program stored in the ROM 32, and stores it in the original frame storage unit 111 as image data of the original frame image. (Step S11).

  Subsequently, the photographing target recognition unit 101 determines whether or not the subject of interest has been shot based on information recorded in the image data of the frame image stored in the original frame storage unit 111 (step S12). If it is determined in step S12 that the subject of interest has not been photographed, the process proceeds to step S21, where the art conversion unit 105 performs art conversion processing on the image data. On the other hand, if it is determined in step S12 that the subject of interest has been shot, the process proceeds to step S13.

  In step S <b> 13, the target distance analysis unit 102 performs the processing from the camera embedded in the image data to the target subject with respect to the target subject of the frame image in which the target subject is determined to be captured by the capturing target recognition unit 101. The distance information R is acquired.

  Subsequently, the target distance analysis unit 102 executes distance information analysis processing to analyze the distance information of the image data (step S14). Specifically, the target distance analysis unit 102 captures a subject embedded in the image data with respect to the subject of the frame image in which it is determined that the subject of interest has been shot by the shooting target recognition unit 101. Whether the distance information R from the subject to the subject of interest and the depth information of the subject are equal is determined, and the subject determined to be equal is identified as the subject of interest. At this time, when the position information of the subject is known, the position information where the target subject exists may be determined with reference to the target subject specifying table 112A, and the subject may be specified as the target subject (step S15).

  The filter unit 103 performs a filtering process on the pixels of the image data in the area corresponding to the target subject identified by the target distance analysis unit 102 (step S16).

  The art conversion unit 105 performs a normal art conversion process on the image data in the region excluding the region filtered by the filter unit 103 (step S17).

  The parameter changing unit 104 performs parameter conversion (step S18). Specifically, the parameter changing unit 104 changes the parameters so that the image of the region corresponding to the subject of interest is not too distorted compared to the original image shape by the art conversion by the art converting unit 105. Do. For example, in the example of FIG. 5, the parameter changing unit 104 determines the region of the subject of interest that has not been filtered by the filter unit 103 for the image data of the region of the subject of interest 73E that has been filtered by the filter unit 103. The brush parameters are changed so that the flatness of the generated group is lower than that in the case of processing the excluded area, for example, the normal subject 73A.

  The art conversion unit 105 performs an art conversion process on the image data of the region filtered by the filter unit 103 according to the parameters set by the parameter change unit 104 (step S19).

  In this way, the image data of the art converted image obtained by subjecting the image data of the original frame image to the image tone changing process by the art converting unit 105 is stored in the processing frame storage unit 114.

The display control unit 35 reads the image data of the art converted image stored in the processing frame storage unit 114 and controls the display unit 41 to display the art converted image on the display unit 41.
As a result, the art conversion process ends.

[Example of art conversion process]
With reference to FIG. 7 and FIG. 8, the image tone change accompanying the art conversion process will be specifically described.

  With reference to FIG. 7, an example in which image data is art-converted while maintaining the identifiability of the region of interest is described. FIG. 7A is a diagram showing an original frame image 210 before art conversion, and FIG. 7B is a diagram showing an art conversion image 210A after art change. Art conversion is performed around the area excluding the subject of interest.

  In FIG. 7A, the original frame image 200 includes a human subject 211 as a foreground subject and a triumphal arch 212 as a background subject.

For example, in FIG. 7A, it is assumed that the triumphal arch 212 is recognized as the target subject by the target distance analysis unit 102 as the target subject region.
In this case, the original frame image 200 is filtered by the filter unit 103 on the image data of the subject area of interest. The image data excluding the region of interest of interest is processed by the art conversion unit 105 into a tone-like image composed of a thick touch, while the image data of the region of interest of the subject filtered is composed of a thin stroke. The image data is subjected to art conversion processing (FIG. 7 (2)). Specifically, as shown in FIG. 7 (2), the human subject 211A that is not the subject of interest is strongly art-converted, but the triumphal arch 212A that is the region of the subject of interest is a clear image that remains discriminable. It becomes.

  In this manner, the image data excluding the region of the subject of interest that is not filtered by the art conversion processing is processed into an image having a tone of a thick brush, while the region of the subject of interest that has been filtered is processed. The image data is processed into an image with a fine brush stroke. For this reason, in the image displayed on the display unit 41, the region of the subject of interest is prevented from being crushed and it becomes difficult to understand what was photographed, and the identifiability can be maintained. Therefore, even if the subject of interest is a subject unique to each region, for example, a famous building or landscape, the identifiability of the subject of interest can be maintained, or the region of the specific subject Art conversion can be performed by differentiating the area excluding a specific subject. Therefore, visually effective art conversion can be realized.

[Example of other art conversion process]
In the present embodiment, a case has been described in which strong art conversion processing is performed on a region excluding the subject of interest while maintaining the identifiability of the region of the subject of interest, but the present invention is not limited to this. For example, it is possible to execute strong art conversion for the region of the subject of interest while maintaining the identifiability only for the region excluding the subject of interest.

  For example, the user changes the setting of the image conversion processing content table 113A (see FIG. 4) stored in the image conversion processing content storage unit 113 by the operation of the operation unit 43 to enable art conversion. The parameter setting may be changed so that the parameter “Roughen” the item “Touch” and “Fine” the item “Touch the non-filtering area” are effective.

  FIG. 8 shows an example in which art conversion is performed by setting the region excluding the subject of interest in such a manner that the possibility of identification is higher than the region of the subject of interest. FIG. 8 (1) is a diagram showing an original frame image 400 before art conversion, and FIG. 8 (2) is a diagram showing an art converted image 400A after image change. Art conversion is performed mainly on the subject of interest.

  In FIG. 8A, the original frame image 400 includes a person subject 411 as a foreground subject and a triumphal arch 412 as a background subject.

Also in FIG. 8A, it is assumed that the triumphal arch 412 is recognized as the target subject area by the target distance analysis unit 102 as the target subject region.
In the original frame image 400, the filter unit 103 filters the image data of the target subject area, and the art conversion unit 105 processes the image data excluding the target subject area into an image with a thin tone. On the other hand, the image data is subjected to art conversion processing so that the image data of the region of the subject of interest that has undergone the filtering process has a thick brush stroke (FIG. 8 (2)). Specifically, as shown in FIG. 8 (2), the triumphal arch 412B that is the region of the subject of interest is strongly art-converted, but the human subject 411A that is not the subject of interest remains clear enough to be discernable. It becomes.

  As described above, the image data excluding the region of the subject of interest that has not been filtered is processed into an image having a thin touch, while the image data of the region of the subject of interest that has been filtered is thick. The image data is processed into an art-tone image. In the image displayed on the display unit 41, it is possible to avoid obscuring the image of the region excluding the subject of interest and obscure what was captured, and maintain the identifiability. In this way, by changing the setting of the image conversion processing content table 113A and changing the content of the art conversion for the filtered region, the art conversion is performed to the extent that the art conversion is strongly performed or the discriminability remains. The area to be subjected to can be set as a target subject or can be set as a region excluding the target subject. For this reason, even if the subject of interest is a subject unique to each region, for example, a famous building or landscape, the strength of art conversion can be freely changed between the subject of interest and the region excluding the subject of interest. . That is, art conversion can be performed by differentiating the subject of interest and the area excluding the subject of interest. Therefore, visually effective art conversion can be realized.

[Other image conversion processing examples]
Furthermore, although the example in which the art conversion process is executed by differentiating the region of the subject of interest from the region excluding the subject of interest has been described, the present invention is not limited to this. Instead of art conversion, it can also be applied to image conversion processing such as white balance processing, color processing, gradation correction processing, and outline emphasis. Also, regarding the parameters to be set, “to make the touch fine” and “to make rough”, but this is only an example, and various stages other than “rough” and “fine” may be provided. Other settings may be provided. As a result, more detailed art conversion can be realized.

  Specifically, after executing the processing from steps S11 to S16, S17, and S18 described with reference to FIG. 6, in S17, S19, and S21, instead of the art conversion processing, white balance processing, color Image processing such as processing, tone correction processing, and contour emphasis may be executed so that parameters are different between the filtered area and the unfiltered area, and many parameter items can be set in detail. It may be provided. When the user operates the operation unit 43, the setting may be made in the image conversion processing content table 113A stored in the image conversion processing content storage unit 113. This makes it possible to differentiate between the region of the subject of interest and the region excluding the subject of interest and perform more detailed image conversion.

  In this embodiment, a stereo digital camera is used as the digital camera. However, the present invention is not limited to a stereo digital camera. The present invention can also be applied to a two-dimensional image digital camera. This will be described in the next embodiment.

[Second Embodiment]
Next, a digital camera according to a second embodiment of the present invention will be described with reference to FIGS. Although the first embodiment has been described using a stereo digital camera, the present embodiment is different in that a two-dimensional image shooting digital camera is used.
Specifically, as illustrated in FIG. 9, the digital camera 1B according to the present embodiment includes one optical lens unit 21B instead of the imaging unit 11 configured to acquire three-dimensional image data, An imaging unit 11B having one image sensor 22B and configured to acquire two-dimensional image data is provided. Furthermore, the digital camera 1B according to the present embodiment replaces the shooting target recognition unit 101, the shooting target information storage unit 112 that stores the target subject specification table 112A, and the target distance analysis unit 102 with a target subject image table 115B. A stored two-dimensional subject information storage unit 115 and a pattern matching unit 116 are provided. In addition to these points, the digital camera 1B according to the present embodiment is different from the stereo digital camera 1 according to the first embodiment in that it processes two-dimensional image data. Since these are well known in the art, the same numbers are assigned to components having substantially the same functions, and detailed description thereof is omitted.

  Hereinafter, the description will be focused on differences from the stereo digital camera 1 according to the first embodiment.

  The two-dimensional subject information storage unit 115 stores a target subject image table 115B that stores image data of a template image of a target subject. The two-dimensional subject information storage unit 115 constitutes a target subject storage unit. The pattern matching unit 116 includes image data of the frame image stored in the original frame storage unit 111 and image data of the template image stored in the target subject image table 115B stored in the two-dimensional subject information storage unit 115. In comparison, it is determined whether or not the subject included in the image of the frame image stored in the original frame storage unit 111 is the subject of interest, and the region of the subject corresponding to the subject of interest is specified. The pattern matching unit 116 constitutes a target subject recognition unit.

  FIG. 11 shows an example of the noted subject image table 115B. As illustrated in FIG. 11, the target subject image table 115B associates image data of a template image of the target subject with information for identifying the target subject for efficient search processing executed by the pattern matching unit 116. I remember it. For example, together with the image data of the subject of interest, the name of the subject of interest, information on the area where the subject of interest exists, and a flag for setting validity / invalidity for each subject of interest can be stored in association with each other. .

[Flow of art conversion process]
The flow of art conversion processing will be described with reference to FIG. The processing shown below starts when the mode is switched to the art conversion mode in accordance with the operation of the mode switching key by the user.

  First, the CPU 31 reads image data of one image from the image data stored in the removable storage medium 51 in accordance with the art conversion processing program stored in the ROM 32, and stores it in the original frame storage unit 111 as image data of the original frame image. (Step S11).

  The pattern matching unit 116 compares the image data of the frame image stored in the original frame storage unit 111 with the image data of the template image stored in the target subject image table 115B stored in the two-dimensional subject information storage unit 115. Pattern matching is performed (step S22), and it is determined whether or not the subject of interest has been identified (step S23). If the subject of interest has been identified, the process proceeds to step S16. If the subject of interest has not been noted, step S21 is performed. Proceed to

  The filter unit 103 performs a filtering process on the pixels of the image data in the area corresponding to the target subject identified by the target distance analysis unit 102 (step S16).

  The art conversion unit 105 performs a normal art conversion process on the image data in the region excluding the region filtered by the filter unit 103 (step S17).

  When the art conversion unit 105 performs art conversion on the image data of the region corresponding to the subject of interest, the parameter changing unit 104 sets parameters so that the image is not so distorted from the original image ( Step S18).

  The art conversion unit 105 performs an art conversion process on the image data of the region filtered by the filter unit 103 according to the parameters set by the parameter change unit 104 (step S19).

  In this way, the image data of the art converted image obtained by subjecting the image data of the original frame image to the image tone changing process by the art converting unit 105 is stored in the processing frame storage unit 114.

The display control unit 35 reads the image data of the art converted image stored in the processing frame storage unit 114 and controls the display unit 41 to display the art converted image on the display unit 41.
As a result, the art conversion process ends.

  In the present embodiment, components having the same functions as those in the first embodiment are denoted by the same reference numerals and detailed description thereof is omitted, but only the art conversion unit 105 of the present embodiment is briefly described. To do.

  Also in this embodiment, the art conversion unit 105 can perform the same processing as in the first embodiment. Although already described in the first embodiment, the details are omitted. In this embodiment, the art conversion unit 105 also performs “Japanese style”, “Western style”, “Watercolor style”, Performs various tone conversions, such as “ink ink”, “pen”, “Gogh”, and “pastel”.

  As for the method of changing the image data to a specific image tone, as described in the first embodiment, pixels having high correlation are collected from pixels constituting the image data on the basis of color or the like to form a group. Subsequently, for each group of pixels, for the same group of pixels, the color is replaced with the representative color of the group. A group of pixels in which the colors are replaced with the representative colors in this way form each brush stroke. According to such image processing, by changing the correlation of the pixels collected as a group and the shape (length (distance), flatness) of the group composed of pixels of the same color (representative color) The formed brush can be changed, and as a result, the tone of the image represented by the image data can be changed to various painting tone. In the present embodiment, the optimum combination of parameters such as the degree of correlation used in each processing, the length representing the shape of the group, and the flatness is set for each image tone. It is stored in the processing content table 113A. The art conversion unit 105 changes to various image styles based on the parameter information stored in the image conversion processing content table 113A of the memory 34.

  Changing each of the image data of the plurality of moving objects to different touches is, for example, as described with reference to FIG. 5, in changing the tone of a group of pixels that constitute the touches. You may implement | achieve by adjusting the shape, for example, the flatness of a group.

  In addition to this, the art conversion process can also be performed using commercially available software such as Photoshop (registered trademark), which is software of Adobe Systems. Examples of parameters for changing the image tone include, for example, contrast, brightness, color density, hue, sharpness, and if necessary, noise reduction filter effect settings, color temperature, and γ correction γ value. Any one or combination of these parameters can be specified. However, the present invention is not limited to this, and any parameter may be included as long as it is appropriate for expressing the above-described image tone.

  Furthermore, as described with reference to FIG. 7, not only the case where the strong art conversion process is performed on the area excluding the subject of interest while maintaining the identifiability of the region of the subject of interest. For example, as described with reference to FIG. 8, it is also possible to execute strong art conversion for the region of the subject of interest while maintaining the identifiability only for the region excluding the subject of interest. For this reason, even if the subject of interest is a subject unique to each region, for example, a famous building or a landscape, the identifiability of the subject of interest can be maintained. Alternatively, the strength of art conversion can be freely changed between the subject of interest and the region excluding the subject of interest. That is, art conversion can be performed by differentiating the subject of interest and the area excluding the subject of interest. Therefore, visually effective art conversion can be realized.

In the present embodiment, it is assumed that an optimal combination of the above-described parameters is preset and stored in association with the image conversion processing content table 113A for each of various image tones.
Furthermore, although the example in which the art conversion process is executed by differentiating the region of the subject of interest from the region excluding the subject of interest has been described, the present invention is not limited to this. Instead of art conversion, it can also be applied to image conversion processing such as white balance processing, color processing, gradation correction processing, and outline emphasis. Also, regarding the parameters to be set, “to make the touch fine” and “to make rough”, but this is only an example, and various stages other than “rough” and “fine” may be provided. Other settings may be provided.

  Specifically, after executing the processing up to steps S11, S22, S23, S17, and S18 described with reference to FIG. 12, in S17, S19, and S21, instead of the art conversion processing, white balance processing is performed. Image processing such as color processing, gradation correction processing, and contour emphasis may be executed so that parameters are different between the filtered region and the unfiltered region. At this time, a large number of parameter items may be provided so that details can be set. When the user operates the operation unit 43, the setting may be made in the image conversion processing content table 113A stored in the image conversion processing content storage unit 113. This makes it possible to differentiate between the region of the subject of interest and the region excluding the subject of interest and perform more detailed image conversion.

  As described above, in this embodiment, even in a normal digital camera 1B for capturing a two-dimensional image, the image data of the region of the subject of interest is filtered, and the image data and the filter processing of the region subjected to the filtering process are performed. Image processing conversion can be performed by differentiating image data that has not been made. For example, as in the case of using the stereo digital camera 1 according to the first embodiment, in the image displayed on the display unit 41, the image of the subject of interest is crushed, and it is difficult to understand what was captured. Can be maintained, and the identifiability can be maintained. Therefore, visually effective art conversion and image processing can be performed. Alternatively, the strength of art conversion can be freely changed between the subject of interest and the region excluding the subject of interest. Therefore, visually effective art conversion and image processing can be realized.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and modifications, improvements, and the like within the scope that can achieve the object of the present invention are included in the present invention.

  In the above-described embodiments, the image processing apparatus to which the present invention is applied has been described as an example configured as a digital camera including a stereo digital camera. However, if the necessary original image data can be acquired from the communication unit 44 or the removable storage medium 51, the photographing unit 11 is not essential. Therefore, the present invention is not particularly limited to a digital camera, and can be applied to electronic devices in general. Specifically, for example, the present invention relates to information processing for reproducing and displaying a captured image supplied from a photo frame, a projector, a television receiver, a video camera, a portable navigation device, a portable game machine, or a stereo digital camera. The present invention can be applied to a device having a function of processing and displaying an image such as a device.

  The series of processes described above can be executed by hardware or can be executed by software.

  When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  Although not shown, the recording medium including such a program is not only constituted by a removable medium distributed separately from the apparatus main body in order to provide a program to the user, but also in a state of being incorporated in the apparatus main body in advance. It consists of a recording medium provided to the user. The removable medium is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preinstalled in the apparatus main body is configured by, for example, the ROM 32 in FIG.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the order, but is not necessarily performed in chronological order, either in parallel or individually. The process to be executed is also included.

DESCRIPTION OF SYMBOLS 1, 1B ... Digital camera, 11 ... Image pick-up part, 12, 12B ... Data processing part (image processing apparatus), 13 ... User interface part, 31 ... CPU, 32 ... ROM (Program), 33 ... RAM, 34 ... Memory, 35 ... Display control unit, 36 ... Image processing unit, 41 ... Display unit (display means), 42 ... Acquisition of position information , 43 .. operation unit, 44 .. communication unit, 45... Drive, 51... Removable storage medium, 111... Original frame storage unit (image data storage unit), 112. Object information storage unit (target subject image storage unit), 113... Image conversion processing content storage unit, 114... Processing frame storage unit, 115... Two-dimensional subject information storage unit (target subject image storage unit) 115 ... pattern match Grayed section (target object recognition unit), 103 ... filter section, 104 ... parameter changing section, 105 ... Art converting unit (the image conversion processing unit)

Claims (4)

An image processing apparatus that performs image conversion processing on image data including information on presence / absence of a subject of interest and information on distance and depth,
A target subject recognition unit that recognizes the target subject based on information about the distance and depth included in the image data;
When starting the image conversion process, the information on the presence / absence of the subject of interest in the image data is confirmed, and if it is determined that the subject of interest is included, it is recognized by the subject of interest recognition unit in the image data. In addition, the image data is image-converted so that the image data portion corresponding to the target subject is different from the image data portion corresponding to other portions of the image excluding the target subject, and the target subject is included. If it is determined that the image data is not included, an image conversion processing unit that performs a tone conversion process on the entire image data ,
Image processing device. The image conversion processing unit
The image data is subjected to image conversion processing so that the region of the target subject recognized by the target subject recognition unit in the image data is clearer than other regions of the image excluding the target subject.
The image processing apparatus according to claim 1. The image conversion processing unit includes: a filter unit that performs a filtering process on an image data portion corresponding to the subject of interest;
A parameter changing unit for changing parameters for performing art conversion on the image data portion that has been filtered,
The image processing apparatus according to claim 1. A computer of an image processing apparatus that performs image conversion processing on image data including information on presence / absence of a subject of interest and information on distance and depth ,
Based on the information on the distance and depth included in the image data, it functions as an attention subject recognition means for recognizing the attention subject ,
When starting the image conversion process, the information on the presence or absence of the subject of interest in the image data is confirmed, and if it is determined that the subject of interest is included, it is recognized by the subject of interest recognition means in the image data. The image data is image-converted so that the image data portion corresponding to the subject of interest is different from the image data portion corresponding to other portions of the image excluding the subject of interest, and the subject of interest is not included. An image processing program for functioning as image conversion processing means for performing image tone conversion processing on the entire image data when it is determined .

Images