JP2006133941A - Image processing device, image processing method, image processing program, and portable terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device, etc. having a shape extraction part for extracting the shape of an object in an image, which is capable of extracting an inner part (inner side) or an outer part (outer side) by the predetermined range for a border of the object as the contour information of the object. <P>SOLUTION: An outline extraction part extracts a rough contour of an object in an original image. Based on the rough contour, the shape extraction part extracts a detailed contour. An outline extraction condition determination part determines conditions for setting a template size, etc. and contents of necessary image processing. An outline calculation part calculates a rough contour by the setting conditions. A shape extraction condition determination part establishes conditions necessary for extracting detailed contour, and a shape calculation part extracts the detailed contour. At this time, setting conditions including the template size, detailed extracting conditions, etc. are determined so that the extraction result falls within the predetermined error range. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention is similar to a contour extraction technique in an image. When extracting objects such as people, objects, animals, and buildings in the image, it is possible to ensure the accuracy of object outline extraction, so that the system application that uses the extraction results can obtain the boundary according to the purpose. And greatly improve the quality of various applications.

  As a conventional method for obtaining contour information from a still image (particularly face information), Patent Document 1 is cited as a system for creating a portrait.

  Further, Patent Document 2 is cited as a system for extracting a subject.

  Further, Patent Document 3 is cited as a system for extracting facial feature values.

Further, Non-Patent Document 1 and Non-Patent Document 2 can be cited as surveys of existing technology for detecting the position of a face.
JP 2000-31248 A JP 2001-197289 A JP-A-8-77334 M.M. -H. Yang, D.D. J. et al. Kriegman and N.M. Ahuja, “Detecting faces in images: a survey”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24, no. 1,2002, 34-58. E. Hjerm, A .; W. M.M. Smes and B.M. Kee Low, “Face Detection: a survey”, Computer Vision and Image Understanding, Vol. 83, no. 3,2001, 236-274

  Here, in any case, it is said that the extraction of the feature amount of the face or object, particularly the contour, is more stable than before, but the extraction result is incorrect at the stage of using the extracted result. The effect on the situation was not considered, and the approach was to “extract as accurately as possible”.

  When an actual usage scene is considered, the influence due to the mistake in the contour extraction result is not always negligible. The problem is particularly serious in applications using human faces.

  For example, considering the example of extracting the contour of a stationary face (face in a still image) and moving the face by morphing using the contour, it is out of the question if the contour is greatly off, and it is extracted almost correctly Even in the case where it is made, if the background outside the face is included in the contour line, the quality at the time of execution looks very inferior.

  In other words, no problem will occur if the contour can be extracted without any deviation of one dot when extracting the contour, but in practice, a certain amount of error is unavoidable.

  That is, the conventional extraction technique is insufficiently examined for errors, and as a result, there is a problem in that quality degradation is caused when an application using the extraction is considered.

  A conventional example in which an object is extracted from the image example (A) in FIG. 2 is schematically shown as an object extraction example (B). From the viewpoint of the application that uses the extraction result, such a result may lead to a decrease in the quality of the application itself. Conventionally, however, the condition is specified in terms of “how much error is allowed” for object extraction. There was nothing to do, and error absorption was left to the application to use the results.

  In order to solve the above-described conventional problems, an image processing apparatus according to the present invention includes a shape extraction unit that extracts the shape of an object in an image, and the shape extraction unit has a predetermined amount with respect to the boundary of the object. The inside (inside) or the outside (outside) can be extracted as the contour information of the object in the range of.

  With this configuration, when the image processing apparatus extracts the contour of the object, the extraction accuracy for the contour of the object can be predicted in advance, and a system that absorbs the accuracy error can be constructed. Since it is guaranteed that the inside or outside of the object is always extracted with an error within a predetermined amount (for example, within 10 pixels), an improvement in the quality of the application can be expected.

  In the image processing apparatus according to the present invention, the object may be a human face or head. In addition, a predetermined amount range may be determined by a predetermined calculation formula using the size of the target object, and a plurality of target objects may be provided.

  With these configurations, further improvement in extraction accuracy can be expected, prediction of extraction accuracy can be improved, and further improvement in application quality can be expected.

  The image processing apparatus according to the present invention may further include an object deforming unit for deforming the object based on the extracted outline. The object deforming unit can also deform the object using morphing or affine transformation.

  With these configurations, it is possible to provide object information as further normalized information, and it is possible to display the extracted object as it is operating, so that improvement in application quality can be expected.

  The image processing apparatus according to the present invention may include a plurality of objects to be extracted.

  With these configurations, it is possible to further reduce the time and effort of extraction, improve user convenience, and increase the speed of system processing.

  According to the image processing apparatus of the present invention, it is ensured that the boundary line of the extracted object is inside or outside the object. As a result, it is possible to acquire a boundary according to the purpose of the application, and the quality of various applications is greatly improved.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 schematically shows a block configuration of the image processing apparatus according to the present embodiment.

  First, an original image to be processed is input from the image input unit 101. Further, when there are requirements for extraction by an external application, system, or user, they may be input through the extraction condition input unit 104.

  From the data input from the image input unit 101 and the extraction condition input unit 104, the rough extraction unit 110 extracts a rough outline (outline outline) of the object in the original image.

  Based on the outline extracted by the outline extraction unit 110, the shape extraction unit 120 extracts the detailed outline (detail outline) of the object in the original image.

  Whether or not the extracted detailed contour is normal is determined by the extraction result determination unit 106, and if normal, the extraction result is output via the extraction result output unit 108.

  In addition, even if it is determined that it is not normal, the extraction result may be output as it is through the extraction result output unit 108, but the extraction condition input unit 104, the rough extraction condition determination unit 111 described later, and the shape extraction condition Information can be fed back and extracted again so that the determination unit 121 can improve the extraction accuracy.

  Next, the internal operations of the general extraction unit 110 and the shape extraction unit 120 will be described.

  In the rough extraction unit 110, a rough extraction condition determination unit 111 that determines rough extraction conditions determines a setting condition such as a template size necessary for a given extraction requirement, and image processing contents necessary for the setting condition.

  The approximate extraction image processing unit 112 performs image processing for extracting an approximate contour based on the conditions determined by the approximate extraction condition determination unit 111.

  The approximate calculation unit 114 calculates the approximate contour of the image processed by the approximate extraction image processing unit 112 under the setting conditions determined by the approximate extraction condition determination unit 111. At this time, data such as a template necessary for the outline extraction can be stored in the outline extraction setting DB unit 113, and can be loaded and used by the outline calculation unit 114 as necessary.

  As a rough extraction method of the rough calculation unit 114, a method of extracting an approximate position of an object by template matching or a neural network can be used. Of course, the method is not limited to these, and any existing method can be used.

  Furthermore, the approximate calculation unit 114 may have a plurality of extraction methods, and the approximate extraction condition determination unit 111 selects and determines an appropriate method as the approximate extraction method used by the approximate calculation unit 114 each time according to the required conditions. Also good. For example, when there is a process with a relatively large amount of computation and a process with a relatively small amount of computation, it can be selected based on speed priority or accuracy priority.

  As an example of the operation of the rough calculation unit 114, FIG. 9 schematically illustrates a case where an object whose contour is to be extracted using template matching is a person (face). In this example, the face of a person like the image example (A) is targeted. An image processing example (B) is a mosaic of this image example (A) with a predetermined size. Here, the size and granularity of the detection target template influence the accuracy of detection accuracy (particularly position). In addition, converting (mosaicing) an image input with a granularity corresponding to the size is a general template matching method. The size (granularity) to be mosaicked here is determined by the approximate extraction condition determining unit 111. You may decide with.

  For example, the granularity of the mosaic can be determined according to the required error range.

  For example, when the required error accuracy is “plus or minus n pixels”, the detection position accuracy when performing template matching is set as the target by setting the mosaic size and template size to n × n pixels or less. It can fit within “plus or minus n pixels”. The point at this time is that the approximate position (not the shape) of the object can be detected within the required error using the existing method. In other words, the point is that the shape of the object cannot be extracted within the required error, which is a problem to be solved by the present invention.

  FIG. 3 schematically shows the required error accuracy. FIG. 3A shows an example of the correct contour when the face is the target. FIG. 3B shows an example in which an extraction limit is set inside a certain range with respect to the correct contour example 302 with reference to the height and width of the object. Here, a horizontal line at the eye height is set as the object width estimation line 304, and a vertical line connecting the nose and the eyebrows is set as the object height estimation line 306. With reference to the vertical and horizontal lines, an outline in which the outline is reduced in a predetermined range (for example, 5%) in the internal direction is schematically shown as an extraction limit area line 308. Similarly, FIG. 4 shows an example in which a limit line is set outside the object.

  Similar to FIG. 3, the extraction limit area line 402 schematically shows a region expanded by a predetermined range (for example, 5%) in the external direction of the object.

  Users and systems set such detailed contour extraction limits or request output of extraction results within such extraction limits to recognize and authenticate output results, classification, applications, entertainment applications, It can be used under stable conditions by FA or the like. FIG. 5 shows an example in which a face is assumed as an object as an application example. In FIG. 5, the face inclination in the original image is indicated by a face inclination line A (502). It is the figure which showed typically the example which added the process which "tilts" a face after extracting the outline of the face. The previous face inclination line A (502) is transformed to become the face inclination line B (504). Here, the application requests the extraction processing within the predetermined error range for the contour extraction processing, or the contour extraction processing guarantees the output of the extraction processing result within the predetermined error range, to other backgrounds etc. (Or an error range can be predicted in advance, so that pre / post processing such as blurring the display with a filter or the like can be performed only in a range where the influence is expected).

  In practice, the shape of the target object is often unknown in advance. Although it may be difficult to set the extraction limit for the correct contour as shown in FIG. 3 and FIG. 4, in this case, the limit value may be set for the horizontal width and vertical width of the general contour. it can.

  In addition, by combining with the method of keeping the detection position deviation by changing the template matching size or template as described above within the predetermined range, more flexible response to user request (error range request) is also possible It becomes possible.

  FIG. 6B shows an example in which a detailed contour extraction region is set for a general contour. When the outline contour example 602 is extracted as the outline, the outline maximum horizontal width 604 and the outline maximum vertical width 606 are calculated from the longest vertical width and width of the outline outline example 602, respectively. At the same time, it is possible to set the extraction area by enlarging or reducing it by a predetermined range as in the examples of FIGS. At this time, if a template match is used as a base for the outline extraction, the extraction accuracy can be further improved by combining this predetermined search range with the shift amount of the detection position by the template match. This is because the detection accuracy by the template is associated with the setting of the extraction region of the detailed contour. For example, if each mosaic in FIG. 9B is a small block of 8 × 8 pixels, pixels at positions (0, 0) to (7, 7) form one block of the upper left mosaic. Here, it is assumed that a mosaic image is separately generated for a block with pixels at positions (j, k) to (j + 7, k + 7) by j and k where j = 0 to 7 and k = 0 to 7. A plurality of mosaic images that are slightly shifted vertically and horizontally are prepared. When template matching is performed on each of the plurality of mosaic images, an output different from the case where (0, 0) to (7, 7) are set to one block can be obtained. This deviation distribution is regarded as the existence probability of the detailed contour.For example, when the deviation amount is small in the horizontal direction and the deviation amount is large in the vertical direction, the extraction area of the detailed area is limited to the horizontal direction. Thus, extraction with higher accuracy is desired. Of course, this is a straightforward example, and it goes without saying that setting the extraction region of the detailed contour is not limited to this example.

  By the way, with regard to face detection, in actuality, an example of prior determination / addition determination of the correctness of the detection position by template matching using skin color information, or a determination machine such as BOOSTING (in some cases, a plurality of determination results) Efforts are being made to reduce the failure of the recognition stage by using features specific to the object, such as an example of detecting the position with high accuracy by using it. Specifically, for example, Non-Patent Document 1 and Non-Patent Document 2 describe a technical survey. In addition, the design of a judgment formula such as BOOSTING is detailed in, for example, the following documents (Anil k. Jain, Robert PW Duin, Jiangchang Mao, “Statistical Pattern Recognition: A Review”, Vol. 22, No. 22). pp. 4-37, 2000.).

  However, it should be noted again that none of the techniques referred to in any of the above-mentioned documents mentions the accuracy of contour detection.

  Now, in accordance with the approximate position and size of the object thus extracted, the approximate calculation unit 114 forms an approximate contour. For example, it is formed like the outline contour example 602 in the outline outline example (A) of FIG. Here, an arbitrary mathematical means can be used as the method of forming the outline, but it may be formed as a curve, for example, as in the outline outline example 602, and the outline of the outline extraction example (C) in FIG. A shape matching the template (in this case, an octagon) may be used as in the region 908.

  Also, there may be a plurality of candidates in the image, such as outline candidate 1 (702), outline candidate 2 (704), outline candidate 3 (706), outline candidate 4 (708) in FIG. A plurality of outline contour candidates are detected and may be selected and selected by any method. For example, as a selection method, there is a method of allowing a user to select via a system (via a user interface) (no drawing). As a selection method, a general method such as a determination method based on the previous skin color information or a determination method based on BOOSTING can be used.

  Furthermore, as a case where there are a plurality of candidates, there may be a case where a plurality of objects exist as shown in FIG. In this example, there are five approximate contour candidates: approximate contour candidate 1 (802), approximate contour candidate 2 (804), approximate contour candidate 3 (806), approximate contour candidate 4 (808), and approximate contour candidate 5 (810). It is displayed as. Similar to the example of FIG. 7, this may be selected and selected by an arbitrary method. Moreover, you may designate the target person who wants to extract.

  Next, the internal operation of the shape extraction unit 120 will be described.

  The shape extraction condition determination unit 121 sets conditions necessary for extracting a detailed contour. If the detailed contour extraction area shown in FIGS. 3, 4 and 6 is set in advance, or if the information of the target object is known in advance, information specific to them is set. When the detailed contour is extracted by the subsequent shape calculation unit 124, the shape extraction condition determination unit 121 also performs various settings depending on the method used there. At the same time, the shape extraction image processing unit 122 can perform image processing based on the determination by the shape extraction condition determination unit 121 so that the detailed contour can be easily extracted by the shape calculation unit 124. Examples of image processing in the shape extraction image processing unit 122 include noise removal, dynamic range correction / normalization, edge emphasis and extraction, morphology conversion, binarization, and the like. Can be used.

  As a detailed contour extraction method in the shape calculation unit 124, a detection method such as pattern matching or a neural network is used only within a detailed contour extraction region, or a dynamic contour extraction method typified by Snakes is used. You can also.

  Snakes is famous as a flexible contour extraction method. For example, when Snakes is used as a detailed extraction method of the shape calculation unit 124, energy definition of the contour (object) (distance energy, luminance energy, smoothness energy, etc.), weighting of each energy, determination of calculation range of each energy, etc. Is required.

  These can be determined by the shape extraction condition determination unit 121.

  For example, if the extraction target is known as a face, elliptic energy (which can be said to be a smooth energy deformation) can be added as an energy definition. Since the face is often elliptical, if the elliptical energy is defined as a function that reduces the energy when the face is close to an ellipse, it is possible to extract a contour closer to the shape of the face.

  Here, Snakes is a method based on energy convergence, so if modeling for an object is inaccurate, if it converges correctly to the object or falls into a local solution, the computation diverges and the calculation does not end Cases can occur.

  As a technique for solving these problems, a technique is known in which the start state (initial contour) of Snakes is set around the object by limiting the object, and both stability and high-speed convergence are achieved. For example, the method described in “Ultrasonic left ventricular volume measurement method using dynamic contour model” (Yamauchi et al., June 2002, Proceedings of the IEICE, ME and Bio Cybernetics) is there.

  As disclosed in this example, determination of extraction conditions (determination of detailed contour extraction area, etc.) and image processing for shape calculation (noise removal, dynamic range correction / normalization, edge enhancement and extraction, morphology conversion) (Binarization etc.) can be determined in relation to each other.

  For example, if the energy function is determined as a function that slowly converges near the edge as an extraction condition, noise removal (morphological processing, etc.) is performed to remove isolated points so that image processing can easily converge near the edge. After that, edge enhancement is performed and the extracted region is blurred by a Gaussian filter or the like, so that the energy function has a local value other than the vicinity of the edge.

  Of course, this is an example of the internal configuration of the shape extraction unit 120, and it goes without saying that any combination of image processing and extraction methods may be used depending on the attributes of the object and the state of the image.

  In addition, in the contour extraction process in the shape calculation unit 124, when various conversion processes (for example, frequency conversion and morphology conversion) are performed, a conversion table (coefficient lookup table, morphology operator, etc.) is shaped. It can be stored in the extraction setting DB unit 123 and read out as necessary.

  Of course, it is needless to say that the shape calculation unit 124 can store or write / read arbitrary data necessary for the contour extraction process other than this.

  For example, it may function as a learning database that writes past extraction results and reflects them in future extractions.

  Moreover, the setting example of the extraction area is schematically shown in FIGS. The outer search range boundary 1201 and the inner search range boundary 1203 having the width n and the width m respectively correspond to the approximate region 1202 in FIG. The width n and the width m are determined according to the required error. FIG. 13 shows this more specifically. The search range is set to fall within a predetermined error range (assuming plus or minus 15% here) from the approximate contour (outline region 1302) extracted as shown in the approximate contour example 602 in FIG. 6A. is doing. A space between the outer search range boundary 1301 and the inner search range boundary 1303 corresponds to the search range. An example of extracting a detailed contour within the search range is an extracted contour 1304.

  Next, the operation of the extraction result determination unit 106 will be described.

  A part for determining whether or not the extracted detailed contour is normal is the extraction result determination unit 106. A specific example will be described with reference to schematic diagrams of FIGS.

  FIG. 10A is an example of an original image. Here, it is assumed that a detailed contour of the face area is to be obtained.

  Assume that the required error for output is about plus or minus 15% of the face width (the width of the object to be extracted).

  FIG. 10B shows a schematic region 1004 as an example of the outline extraction. FIG. 10C shows a schematic region 1006 as another schematic region extraction example.

  In addition, FIG. 14A and FIG. 14B are schematically shown as detailed region extraction examples in each case.

  When the outline such as the outline area 1004 is extracted, the extraction outline 1404 is extracted as in the normal extraction example of FIG. In addition, when an approximate contour such as the approximate region 1006 is extracted, an extracted contour 1406 is extracted as an example of abnormal extraction as illustrated in FIG.

  Here, it is the determination operation of the extraction result determination unit 106 in this example. In this example, since the extraction target is known to be a face (or an object close to a face-like ellipse), the extracted detailed contour The extracted contour 1404 and the extracted contour 1406 can be determined by measuring the circularity (ellipticity). Arbitrary methods may be used to calculate the degree of circularity. For example, Hasegawa et al. “Basic Techniques of Image Processing”, Technical Review, 1986, etc.

  At this time, for example, by providing a predetermined threshold for the circularity, the extracted contour 1404 and the extracted contour 1406 can be distinguished. In addition, it may be determined whether the curve is a convex curve or a concave curve. Of course, the means for determining abnormality / normality is not limited to this, and it is possible to use an existing method such as extracting features of the extracted contour and classifying the feature values using SVM, BOOSTING, etc. it can.

  Similarly, FIG. 11 shows a case where an image other than a face is input when a face is an extraction target. FIG. 11A shows an example of an original image. When this image is mosaicked in the same manner as in FIG. 9B and template matching is performed, the distribution of shading can be the same as that in FIG. 9B. An example is shown in FIG. When the schematic area is extracted in the example of FIG. 11, it is expected that the schematic area 1104 is obtained. On the other hand, when the detailed contour is extracted, for example, the result is as shown in FIG.

  If a contour is extracted as in the case of extracting a face, a result like an extracted contour 1504 is obtained. In this case, it is regarded as abnormal as in the example of FIG.

  The following cases can also be considered for the entire system.

  For example, conditions (arbitrary restrictions / allowable items such as the number, position, size, etc.) relating to extraction can be input from the user or the system via the extraction condition input unit 104.

  If there is no external extraction condition designation, an arbitrary default value may be used in this embodiment. Further, the extraction condition input unit 104 is not an essential component in the present embodiment, and can be arbitrarily installed. Needless to say, even when the extraction condition input unit 104 is not provided, the extraction condition can be set and operated in a fixed manner, or the extraction condition can be changed dynamically according to the input image.

  In the present embodiment, the extraction result determination unit 106 is not an essential component and can be arbitrarily installed. It goes without saying that the detailed contour obtained through the outline extraction unit 110 and the shape extraction unit 120 can be output through the extraction result output unit 108 even without the extraction result determination unit 106.

  In the present embodiment, an arbitrary default value may be substituted for the rough extraction setting DB unit 113 and the shape extraction setting DB unit 123. In addition, when the extraction target is determined or when the approximate condition is known even when it is unknown, the approximate extraction condition determination unit 111 and the shape extraction condition determination unit 121 are omitted or replaced with arbitrary default processing. May be.

  Similarly, when it is not necessary to make the process variable, such as when the target is specified for the outline extraction image processing unit 112 and the shape extraction image processing unit 122, an arbitrary default process may be used. Moreover, you may abbreviate | omit as needed.

  The approximate calculation unit 114 and the shape calculation unit 124 can feed forward the process to the extraction result output unit 108 according to the processing result. For example, when it can be determined that each calculation unit can extract with sufficient accuracy, or when it can be determined that extraction is not possible, feed-forwarding can be performed with information to that effect.

  The extraction result determination unit 106 can also feed back to any part in FIG. 1, such as the shape extraction condition determination unit 121, the approximate extraction condition determination unit 111, and the extraction condition input unit 104. For example, when the extraction result determination unit 106 determines that the accuracy is insufficient, information can be fed back to each determination unit so that the accuracy is further increased.

  For example, in the case of a face, when the shape obtained as a result of extraction does not look like a face, the shape extraction condition determination unit can change the detailed contour calculation conditions closer to the face shape. Information can also be fed back to 121. Arbitrary information, features, and information on the contour, such as the fact that it is a concave curve so that the energy definition or weighting that is difficult to be concaved or weighting can be reset can be reset when the contour is a concave curve. It is possible to feed back the analysis result obtained by combining.

  Also, if it is determined that there is no face at the position of the extraction result, especially if it is extracted at the wrong position and there is a possibility that there is a face at another position in the image, rough extraction The approximate position can be corrected by feeding back to the condition determining unit 111.

  In addition, the extraction conditions themselves can be changed by combining them.

  In this case, the extraction condition input unit 104 is fed back.

  Also, feedback information can be disclosed to the user, and processing can be performed on the feedback information via the extraction condition input unit 104 while reflecting selections and changes from the user.

  The processing procedure described in the present invention can be realized as a program for causing the CPU to execute the processing procedure. In this case, the program may be stored in a recording medium and executed from a storage device (such as a hard disk) of the device that stores the contents, or may be directly executed from the recording medium. The recording medium here refers to a recording medium such as a semiconductor memory such as a ROM, a RAM, or a flash memory, a magnetic disk memory such as a flexible disk or a hard disk, an optical disk such as a CD-ROM, DVD, or BD, or a memory card.

  The present invention may also be executed on a portable terminal. Various applications using images are assumed. However, as a characteristic point of the portable terminal, there is a problem that the calculation performance and the storage capacity are poor compared to a personal computer or the like. In other words, it is very hard for each application installed in the portable terminal to absorb the blur of the contour extraction result both in terms of hardware specifications (CPU performance and memory capacity) and in terms of software specifications (the program size cannot be increased). It is difficult, and it can be said that the problem of the prior art appears remarkably in the portable terminal. It is obvious that this point is solved by the present invention.

  In addition, in a portable terminal, the user interface is poor compared to a personal computer or the like. If the contour extraction result is distorted, it is foreseen that the correction is very difficult. In the present invention, since the extraction accuracy can be predicted in advance, the correction interface can be simplified.

  Furthermore, as a specific application example in a portable terminal, the outline of a necessary object is extracted from input image data and deformed by morphing or the like (such as cutting out a face to make it appear to move) It can also be used to realize processing / functions such as moving the position of the extracted object (generation of a composite photo, conversion to chroma key, etc.), processing such as pan / zoom (slide show function, etc.).

  With this configuration, when the image processing apparatus extracts the contour of the object, the extraction accuracy for the contour of the object can be predicted in advance, and a system that absorbs the accuracy error can be constructed. It can be guaranteed that the inside or outside of the object is always extracted with a predetermined amount of error (for example, within 10 pixels).

  Furthermore, according to the present invention, when extracting an object such as a person, an object, an animal, or a building from an image, the accuracy of extracting the contour of the object is guaranteed, and the system application using the extraction result has a boundary according to the purpose. It becomes possible to acquire the quality of various applications.

The figure which showed typically the block configuration of the image processing apparatus in this Embodiment A diagram showing an example of original image and an example of object extraction by a conventional method The figure which shows the example which sets the inner side of the example of correct outline extraction and an extraction area The figure which shows the example which sets the outside of an extraction area The figure which shows the example processed using the extracted outline Figure showing an example of outline extraction and setting examples such as width and height Diagram showing examples of outline contour candidates Diagram showing examples of outline contour candidates Diagram showing the schematic extraction mechanism A schematic diagram showing an example of rough extraction, especially when wrong extraction is performed. A schematic diagram showing an example of rough extraction, especially when incorrect extraction is performed The figure which showed typically the example which sets the search range of detailed extraction based on the roughly extracted area The figure which applied the search range of detailed extraction to the image example schematically Schematic diagram of detailed extraction example Schematic diagram of detailed extraction example

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image input part 104 Extraction condition input part 106 Extraction result determination part 108 Extraction result output part 110 Outline extraction part 111 Outline extraction condition determination part 112 Outline extraction image processing part 113 Outline extraction setting DB part 114 Outline calculation part 120 Shape extraction part DESCRIPTION OF SYMBOLS 121 Shape extraction condition determination part 122 Shape extraction image processing part 123 Shape extraction setting DB part 124 Shape calculation part 202 Object 302 Correct outline example 304 Object width estimation line 306 Object height estimation line example 308 Extraction limit area line 402 Extraction limit Area line 602 Outline outline example 604 Outline maximum horizontal width 606 Outline maximum vertical width 702 Outline outline candidate 1
704 Outline outline candidate 2
706 Outline contour candidate 3
708 Outline contour candidate 4
802 Outline outline candidate 1
804 Outline contour candidate 2
806 Outline outline candidate 3
808 Outline contour candidate 4
810 Outline contour candidate 5
904 Divided region 905 High concentration region 906 Medium concentration region 908 Approximate region 1004 Approximate region 1006 Approximate region 1104 Approximate region 1201 Outer search range boundary 1202 Approximate region 1203 Inner search range boundary 1301 Outer search range boundary 1302 Approximate region 1303 Inner search range boundary 1304 Extracted contour 1404 Extracted contour 1406 Extracted contour 1504 Extracted contour

Claims (32)

An image processing apparatus,
An outline extraction unit for extracting an outline of an object in an image;
A shape extraction unit for extracting the shape of the object from which the outline extraction unit has extracted an outline;
The shape extraction unit is configured with respect to the boundary of the object.
An image processing apparatus that extracts inside (inside) or outside (outside) as outline information of the object by a predetermined amount based on the outline information of the object from which the outline is extracted by the outline extraction unit. The image processing apparatus according to claim 1, wherein the object is a human face or head. The image processing apparatus according to claim 1, wherein the range of the predetermined amount is determined by a predetermined calculation formula using the size of the object. The object is a human face or head;
The image processing apparatus according to claim 1, wherein the range of the predetermined amount is determined by a predetermined calculation formula using the size of the face or the size of the head. An image processing apparatus according to any one of claims 1 to 4, wherein
An image processing apparatus comprising a plurality of the objects. An image processing apparatus according to any one of claims 1 to 5,
Based on the extracted shape of the contour, an extraction result determination unit that determines whether the contour has correctly extracted the object;
An extraction result output unit for outputting the extraction result;
An image processing apparatus that outputs the contour information via the extraction result output unit when the extraction result determination unit determines that the extraction is correctly performed. The image processing apparatus according to claim 6,
When the extraction result determination unit determines that the object cannot be correctly extracted based on the extracted shape of the outline, the outline extraction unit or the shape extraction unit feeds back the outline extraction unit. The image processing apparatus is characterized in that the processing unit or the shape extraction unit changes the processing conditions at the time of each extraction based on the fed back information and performs the extraction process again. The image processing apparatus according to any one of claims 1 to 7,
The approximate extraction unit includes an approximate extraction condition determination unit, an approximate extraction image processing unit, and an approximate calculation unit,
The approximate calculation unit performs an approximate extraction process according to a predetermined condition,
The schematic extracted image processing unit includes:
A predetermined process is performed on the image used for the rough extraction process in the rough calculation unit, and the rough extraction condition determination unit is
Set the predetermined condition necessary for the rough extraction process in the rough calculation unit,
An image processing apparatus configured to set the predetermined processing necessary for image processing in the schematic image processing unit. An image processing apparatus according to any one of claims 1 to 8,
The shape extraction unit includes a shape extraction condition determination unit, a shape extraction image processing unit, and a shape calculation unit,
The shape calculation unit performs an object extraction process according to a predetermined condition,
The shape extraction image processing unit performs a predetermined process on an image used for an object extraction process in the shape calculation unit,
The shape extraction condition determination unit sets the predetermined condition necessary for the object extraction process in the shape calculation unit, and sets the predetermined process necessary for the image processing in the shape image processing unit. An image processing apparatus. The image processing apparatus according to claim 8,
The approximate calculation unit calculates an approximate position and shape based on template matching,
The outline extraction image processing unit performs at least one of segmentation, mosaicing, filtering, noise removal, or gradation changing processing of a region to be matched,
The shape extraction condition deciding unit is the size of the area for matching, the size of the template, the size to be mosaicked, the type of filtering, the number of stages, the filter size, the filter coefficient, or the noise removal An image processing apparatus that determines at least one of a method and a noise removal size or a number of gradations after the gradation changing process is changed. The image processing apparatus according to claim 8,
The approximate calculation unit calculates an approximate position and shape based on a neural network,
The outline extraction image processing unit performs at least one of segmentation of an area to be input to a neural network, or mosaic processing, filter processing, noise removal, or gradation change processing,
The shape extraction condition determining unit is configured to input a region size to be input to the neural network, a size to be mosaicked, or at least one of a filtering process type, the number of stages, a filter size, a filter coefficient, a noise removal method, and a noise removal size. An image processing apparatus that determines at least one of at least one or the number of gradations after gradation change processing. The image processing apparatus according to claim 9,
The shape calculation unit extracts a contour based on a contour extraction unit that performs a dynamic operation by repetitive convergence calculation,
The shape extraction image processing unit performs at least one of noise removal, filter processing, morphology conversion processing, or gradation change processing,
The shape extraction condition determination unit includes at least one of a type of the control term of the convergence calculation, a weight of the control term of the convergence calculation, a maximum number of iterations of the convergence calculation, a noise removal method, and a noise removal size. Or the size to be mosaicked, or at least one of the filter processing type, number of stages, filter size, filter coefficient, or the type / size of the morphological conversion processing operator, or An image processing apparatus that determines at least one of the number of gradations. An image processing apparatus according to any one of claims 1 to 9, wherein
It has an extraction condition input part,
The predetermined amount range is input via the extraction condition input unit,
When the range of the predetermined amount is input via the extraction condition input unit, the input value is preferentially used as the range of the predetermined amount,
When the range of the predetermined amount is not input via the extraction condition input unit, the shape extraction unit uses the range of the predetermined amount determined based on the outline information of the object at a later stage. An image processing apparatus for extracting a shape of an object. The image processing apparatus according to claim 10 or 11,
It has an extraction condition input part,
The predetermined amount range is input via the extraction condition input unit,
When a plurality of the predetermined amount ranges are set, the input value is preferentially used as the predetermined amount range,
The shape extraction condition determination unit determines at least one of the determination of the size, the determination of the type, the determination of the coefficient, the determination of the number of steps, or the determination of the number of gradations within the predetermined amount range. An image processing apparatus that is determined according to An outline extraction step for extracting an outline of an object in the image;
The outline extraction step includes a shape extraction step of extracting the shape of the object from which the outline is extracted;
The shape extraction step is performed on the boundary of the object.
An image processing method, wherein the outline extraction step extracts the inside (inside) or the outside (outside) as outline information of the object by a predetermined amount based on the outline information of the object from which the outline is extracted. An outline extraction step for extracting an outline of an object in the image;
The outline extraction step includes a shape extraction step of extracting the shape of the object from which the outline is extracted;
The outline extraction step extracts the inside (inside) or the outside (outside) as the outline information of the object by a predetermined amount based on the outline information of the object from which the outline is extracted,
An image processing method characterized in that the object is in particular a human face or head. An outline extraction step for extracting an outline of an object in the image;
The outline extraction step includes a shape extraction step of extracting the shape of the object from which the outline is extracted;
The outline extraction step extracts the inside (inside) or the outside (outside) as the outline information of the object by a predetermined amount based on the outline information of the object from which the outline is extracted,
A range of the predetermined amount is determined by a predetermined calculation formula using a size of the object. An outline extraction step for extracting an outline of an object in the image;
The outline extraction step includes a shape extraction step of extracting the shape of the object from which the outline is extracted;
The outline extraction step extracts the inside (inside) or the outside (outside) as the outline information of the object by a predetermined amount based on the outline information of the object from which the outline is extracted,
The object is in particular a human face or head;
An image processing method, wherein the range of the predetermined amount is determined by a predetermined calculation formula using the size of the face or the size of the head. The image processing method according to any one of claims 15 to 18, comprising:
An image processing method, comprising: a plurality of the objects. The image processing method according to any one of claims 15 to 19, wherein
An image processing method comprising: an object deformation step for deforming the object based on the extracted contour. The image processing method according to claim 20, wherein
The image processing method characterized in that the object deformation step deforms the object using morphing or affine transformation. The image processing method according to any one of claims 15 to 18, comprising:
A quantity specification step,
The amount specifying step can specify a range of the predetermined amount,
Furthermore, it has an extraction impossible notification step,
The extraction impossible notification step includes:
When the shape extraction step fails to extract contour information within the predetermined amount range specified in the amount specifying step,
An image processing method characterized by performing a predetermined notification outside. A program for causing a computer to execute all the steps included in claim 15. A program for causing a computer to execute all the steps included in claim 16. A program for causing a computer to execute all the steps included in claim 17. A program for causing a computer to execute all the steps included in claim 18. A program for causing a computer to execute all the steps included in claim 19. A program for causing a computer to execute all the steps included in claim 20. A program for causing a computer to execute all the steps included in claim 21. A program for causing a computer to execute all the steps included in Claim 22. A portable terminal having an image display device,
An outline extraction step for extracting an outline of an object in the image;
The outline extraction step includes a shape extraction step of extracting the shape of the object from which the outline is extracted;
The shape extraction step is performed on the boundary of the object.
The outline extraction step has an image processing step of extracting inside (inside) or outside (outside) as outline information of the object by a predetermined amount based on the outline information of the object from which the outline is extracted. Portable terminal. A portable terminal having an image display device,
An outline extraction step for extracting an outline of an object in the image;
The outline extraction step includes a shape extraction step of extracting the shape of the object from which the outline is extracted;
The shape extraction step is performed on the boundary of the object.
The outline extraction step extracts the inside (inside) or the outside (outside) as the outline information of the object by a predetermined amount based on the outline information of the object from which the outline is extracted,
The object is in particular a human face or head;
A portable terminal characterized in that the range of the predetermined amount is determined by a predetermined calculation formula using the size of the face or the size of the head.

Images