JP6031961B2 - Contour detection device, contour detection method, and contour detection program - Google Patents

Description

  The present invention relates to a contour detection device, a contour detection method, and a contour detection program.

  A technique for detecting the contour of a specific object from an image such as a photograph by image processing is known. Such a technique is used, for example, for detecting the outline of a person and erasing or replacing a background image outside the outline.

  In addition, as a technique for accurately detecting the contour of a desired object such as a person, the user can designate a region for detecting a contour using an input device such as a mouse, and the contour detection range is designated or There is a technique for limiting to a nearby region. For example, there is a method of searching for a pixel having a high edge detection intensity toward the end point between the start point and the end point specified by the user operation. There is also a method for searching for a contour point by scanning a pixel in a width direction intersecting with the center line in a belt-like region designated by a user operation.

JP 2000-209430 A JP-A-7-92650 JP 2000-339483 A

  In the method of allowing the user to designate the contour detection region, the contour detection accuracy is improved by setting the detection region as much as possible along the contour shape. For example, in a portion where the contour line is sharply bent like the ear region when the human face is viewed from the front, the detection region is set along the contour curve, thereby detecting the contour detection accuracy. Will improve. However, there is a problem that the operation for setting the detection region so as to follow the shape of the contour is very troublesome and the work efficiency is low.

  In one aspect, an object of the present invention is to provide a contour detection device, a contour detection method, and a contour detection program capable of efficiently performing highly accurate contour detection.

  In one proposal, the following contour detection apparatus is provided. The contour detection apparatus includes an area setting unit, an input reception unit, and a search unit. The area setting means sets a specific area on the input image. The input receiving unit sets a search area having a certain width or more on the input image that is extended from the start area to the end area in response to an input operation. The search means determines the angle in the extension direction of the search region at the position of the target pixel set from the search region, and the search range in which an edge search range for searching for an edge is associated with each angle in the extension direction Based on the information, an edge search range corresponding to the determined angle in the extending direction is determined, and a pixel that is highly likely to be an edge from the pixels located within the determined edge search range among the surrounding pixels of the set target pixel And the edge search process for setting the next target pixel is repeated until the target pixel reaches the end region from the start region. In addition, when the target pixel is included in the specific region, the search unit extends the edge search range determined for the target pixel, and from the pixels located in the extended edge search range among the surrounding pixels of the target pixel. Then, a pixel that is highly likely to be an edge is set as the next pixel of interest.

  Further, in one proposal, there are provided a contour detection method in which processing similar to that in the above contour detection device is executed, and a contour detection program for causing a computer to execute processing similar to that in the above contour detection device.

  According to one aspect, highly accurate contour detection can be performed efficiently.

It is a figure which shows the structural example and processing example of the outline detection apparatus which concern on 1st Embodiment. It is a figure which shows the hardware structural example of the outline detection apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the structural example of the processing function with which an outline detection apparatus is provided. It is a figure which shows the example of the state transition at the time of trace operation execution. It is a figure for demonstrating the calculation method of a trace direction angle. It is a figure which shows the example of the information registered into trace area information. It is a figure which shows the example of the classification | category of a trace direction angle. It is a figure shown about the example of the relationship between an angle range and an edge search range. It is a figure which shows the example of calculation of edge strength. It is a figure for demonstrating the problem by said outline detection process. It is a figure which shows the example of a detection of an ear area | region. It is a figure which shows the example of a setting of a trace area | region. It is a figure which shows the example of a setting of the search range in an ear | edge area | region. It is a flowchart which shows the example of the process sequence of the outline detection apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the structural example of the processing function of the outline detection apparatus which concerns on 3rd Embodiment. It is a figure which shows an example of the outline of an ear | edge. It is a figure which shows the example of a setting of the search range in an ear area | region. It is a flowchart which shows the example of the process sequence of the outline detection apparatus which concerns on 3rd Embodiment. It is a flowchart which shows the example of the process sequence of the outline detection apparatus which concerns on 3rd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example and a processing example of the contour detection apparatus according to the first embodiment. The contour detection device 1 in FIG. 1 is a device that executes processing for detecting a contour line of an object from an input image, and includes a region setting unit 2, an input reception unit, and a search unit 4. Further, the search means 4 can refer to the search range information 5.

  Each process of the area setting unit 2, the input receiving unit 3, and the search unit 4 is realized by, for example, a processor included in the contour detection device 1 executing a predetermined program. Alternatively, at least a part of the processing of the area setting unit 2, the input reception unit 3, and the search unit 4 may be realized by a dedicated circuit. The search range information 5 is stored in, for example, a nonvolatile storage device included in the contour detection device 1.

  The area setting unit 2 sets a specific area on the input image. FIG. 1 shows a rectangular specific area 10 as an example, but the method of setting the specific area is not limited to this example. For example, the specific region may be a region defined only in the horizontal direction (x-axis direction) range on the input image or a region defined only in the vertical direction (y-axis direction).

  For example, the specific region is set so as to include a portion where the outline of the object is sharply bent. As a specific example, a region including a human ear can be set as a specific region. When detecting the outline of a person with the face facing forward, the outline may be sharply bent at the upper and lower ends of the ear.

  The specific area may be set by a user operation input, or may be automatically set by the contour detection device 1. As an example of the latter, a method of setting a region including an ear as a specific region based on a human face detection result is conceivable.

  The input receiving unit 3 sets a search area for searching for an edge on the input image in accordance with a user input operation. The search area is an area having a certain width or more that extends from the start area toward the end area. FIG. 1 shows an example of the search area 20 that is gently curved from the start area 21 toward the end area 22.

  It is desirable that the search area can be set by a simple operation as much as possible. As an example of the search area setting method, a method in which a search area having a certain width is set along the movement locus when the user performs an operation of continuously moving the designated position is considered. In this case, the designated position can be, for example, the position of the mouse cursor or the touch position on the touch panel. As a similar example, a method in which when the user performs an operation of moving an icon image having a certain size, a region where the icon image has moved can be set as a search region.

  In the set search region, the search unit 4 sets a pixel of interest starting from one of the pixels in the start region, and selects a pixel that is likely to be an edge from the surrounding pixels of the pixel of interest. The edge search process set to is repeated until the target pixel reaches the end region. Thereby, the set pixel group of interest is detected as a contour line.

  More specifically, the search means 4 performs the following processing. The search means 4 determines the angle in the extension direction of the search region at the set position of the target pixel. This angle in the extending direction can be defined as, for example, the inclination of the tangent at the intersection of the vertical line drawn from the pixel of interest to the center line of the search area and the center line.

  The search means 4 refers to the search range information 5. The search range information 5 is associated with an edge search range for searching for an edge for each angle in the extension direction of the search region. Based on the search range information 5, the search unit 4 determines a search range angle corresponding to the extension direction angle determined for the pixel of interest. Then, the search unit 4 determines a pixel that is highly likely to be an edge from pixels located within the edge search range determined based on the search range information 5 among the surrounding pixels of the pixel of interest, and selects the pixel. Set to the next pixel of interest. The search means 4 repeats such edge search processing until the target pixel reaches the end region from the start region.

  For example, when the pixel 31 in FIG. 1 is set as the target pixel, the search unit 4 determines the angle of the stretching direction A1 for the pixel 31 and, based on the search range information 5, the edge corresponding to the angle of the stretching direction A1. The search range B1 is determined. And the search means 4 determines the pixel 32 with high possibility of being an edge from the pixels located in the edge search range B1 among the surrounding pixels of the pixel 31, and makes the pixel 32 the next pixel of interest. Set.

  Here, the size of the inner angle of the edge search range is basically a substantially constant value regardless of the angle in the stretching direction. The edge search range is, for example, limited to a range in which a pixel of interest that can be set next (that is, a pixel that is a determination target of an edge) is directed from the current pixel of interest toward the extension direction of the search region. Is set as follows. As a result, the contour line extending along the search area, that is, the contour line intended by the user can be accurately detected.

  In addition, when the pixel of interest is included in the specific region set by the region setting unit 2, the search unit 4 extends the edge search range determined for the pixel of interest. Then, the search means 4 determines a pixel having a high possibility of being an edge from pixels located within the extended edge search range from the surrounding pixels of the target pixel, and sets it as the next target pixel.

  For example, the pixel 41 in FIG. 1 is included in the set specific area 10. When this pixel 41 is set as the pixel of interest, the search means 4 determines the angle of the extending direction A2 for the pixel 41 and, based on the search range information 5, determines the edge search range B2 corresponding to the angle of the extending direction A2. Determine. Further, the search means 4 extends the determined edge search range B2 to a new edge search range B3. Then, a pixel 42 that is highly likely to be an edge is determined from the pixels located within the expanded edge search range B3 among the surrounding pixels of the pixel 41, and the pixel 42 is set as the next pixel of interest. .

  By extending the edge search range in this way, the search means 4 can accurately detect the contour line even when the contour line of the object is sharply bent. That is, in the specific area 10, even if the direction of the contour line of the object is greatly deviated from the extending direction of the search area, the contour line can be accurately detected. For this reason, the user does not set the search region so that the contour line accurately follows the shape of the contour line, but also sets the search region so as to follow the rough contour shape. It is only necessary to set this, and the user's work efficiency is improved. Therefore, highly accurate contour detection can be performed efficiently.

[Second Embodiment]
Next, an example of a contour detection apparatus for detecting a human contour will be described. FIG. 2 is a diagram illustrating a hardware configuration example of the contour detection apparatus according to the second embodiment. The contour detection apparatus 100 is realized as a computer as shown in FIG. 2, for example.

  The entire contour detection apparatus 100 is controlled by a processor 101. A RAM (Random Access Memory) 102 and a plurality of peripheral devices are connected to the processor 101 via a bus 109. The processor 101 may be a multiprocessor. The processor 101 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a programmable logic device (PLD). The processor 101 may be a combination of two or more elements among CPU, MPU, DSP, ASIC, and PLD.

  The RAM 102 is used as a main storage device of the contour detection device 100. The RAM 102 temporarily stores at least part of an OS (Operating System) program and application programs to be executed by the processor 101. The RAM 102 stores various data necessary for processing by the processor 101.

  Examples of peripheral devices connected to the bus 109 include an HDD (Hard Disk Drive) 103, a graphic processing device 104, an input interface 105, an optical drive device 106, a device connection interface 107, and a network interface 108.

  The HDD 103 magnetically writes and reads data to and from the built-in disk. The HDD 103 is used as an auxiliary storage device of the contour detection apparatus 100. The HDD 103 stores an OS program, application programs, and various data. As the auxiliary storage device, other types of nonvolatile storage devices such as SSD (Solid State Drive) can be used.

  A monitor 104 a is connected to the graphic processing device 104. The graphic processing device 104 displays an image on the screen of the monitor 104a in accordance with an instruction from the processor 101. Examples of the monitor 104a include a display device using a CRT (Cathode Ray Tube) and a liquid crystal display device.

  A keyboard 105 a and a mouse 105 b are connected to the input interface 105. The input interface 105 transmits signals sent from the keyboard 105a and the mouse 105b to the processor 101. Note that the mouse 105b is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, and a trackball.

  The optical drive device 106 reads data recorded on the optical disc 106a using laser light or the like. The optical disk 106a is a portable recording medium on which data is recorded so that it can be read by reflection of light. The optical disk 106a includes a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (ReWritable), and the like.

  The device connection interface 107 is a communication interface for connecting peripheral devices to the contour detection apparatus 100. For example, a memory device 107 a and a memory reader / writer 107 b can be connected to the device connection interface 107. The memory device 107a is a recording medium equipped with a communication function with the device connection interface 107. The memory reader / writer 107b is a device that writes data to the memory card 107c or reads data from the memory card 107c. The memory card 107c is a card type recording medium.

  The network interface 108 is connected to the network 108a. The network interface 108 transmits / receives data to / from other computers or communication devices via the network 108a.

  With the hardware configuration described above, the processing functions of the second embodiment can be realized. Note that the contour detection apparatus 1 shown in the first embodiment can also be realized by the same hardware as the contour detection apparatus 100 shown in FIG.

  FIG. 3 is a block diagram illustrating a configuration example of processing functions included in the contour detection apparatus. The contour detection apparatus 100 sequentially searches for a pixel having a strong edge strength with respect to the extending direction of the search region in a search region set on the input image according to the input operation of the user and having a certain width or more. As a result, the outline of the person on the input image is detected. In particular, the contour detection apparatus 100 is capable of detecting a contour line near the head of a person whose face is facing the front with high accuracy. The search area is set as an area where the icon image has moved when the user performs an operation of moving an icon image of a predetermined size by a drag operation. Furthermore, in the region including the ears extracted by face detection, the contour detection apparatus 100 changes the search processing procedure for pixels with high edge strength from the other procedures, thereby contour lines in the region including the ears. Improve detection accuracy.

  The contour detection apparatus 100 includes an image reading unit 111, a face detection unit 112, a trace input unit 113, and a contour detection unit 114. Each process of the image reading unit 111, the face detection unit 112, the trace input unit 113, and the contour detection unit 114 is realized by the processor 101 executing a predetermined program. However, at least a part of these processes may be realized by a dedicated circuit such as a circuit in the graphic processing device 104, for example.

  The storage device of the contour detection apparatus 100 stores ear region information 121, trace region information 122, search range information 123, and contour information 124. Among these pieces of information, the search range information 123 is stored in advance in a nonvolatile storage device such as the HDD 103. On the other hand, the ear region information 121, the trace region information 122, and the contour information 124 are temporarily stored in at least a volatile storage device such as the RAM 102.

  The image reading unit 111 reads an input image showing a person into the RAM 102 from a nonvolatile storage device connected to the contour detection device 100 such as the HDD 103 or an information processing device on the network 108a. In the present embodiment, it is assumed that at least the head of a person whose face is facing the front is reflected in the input image.

  The face detection unit 112 detects a human face from the input image, and extracts a region including an ear (ear region) from the face detection result. Information indicating the ear region is stored in the RAM 102 as ear region information 121.

  When the user performs a “trace operation” in which an icon image of a predetermined size is moved by a drag operation, the trace input unit 113 sets the area where the icon image has moved as a search area. Hereinafter, an area where the icon image has moved is referred to as a “trace area”. Information indicating the trace area is stored in the RAM 102 as the trace area information 122.

  The trace input unit 113 calculates a “trace direction angle” for each pixel in the set trace area, and registers the calculated trace direction angle in the trace area information 122 in association with each pixel. The trace direction angle is a reference angle for determining a travelable direction for edge search for each pixel. Details of the trace direction angle will be described later.

The search range information 123 stores an edge search range indicating a range in which an edge search is performed for each trace direction angle in association with each other.
The contour detection unit 114 detects a contour line by sequentially tracing pixels with high edge strength from the start region to the end region in the trace region. In such edge search processing, the contour detection unit 114 refers to the ear region information 121, the trace region information 122, and the search range information 123. The contour detection unit 114 outputs the detected contour information as contour information 124.

  In addition, when the target pixel to be processed is included in the ear region indicated by the ear region information 121, the contour detection unit 114 determines the range of the search progress direction starting from the target pixel (the determination target of edge strength). The range of pixels to be expanded) is expanded from the range in other regions. Thus, the contour line can be accurately detected even in a region near the ear where the contour line is sharply bent.

Next, a basic contour detection processing procedure of the contour detection apparatus 100 will be described. The basic contour detection process includes the following processing steps.
(1) Set the trace area according to the trace operation.

(2) The trace direction angle is calculated for each pixel in the trace area.
(3) Any pixel in the start area of the trace area is set as the target pixel, and the following (3-1) to (3-3) are repeated until the target pixel reaches the end area of the trace area.

(3-1) The edge search range corresponding to the trace direction angle for the target pixel is determined based on the search range information 123.
(3-2) The edge strength of the pixels included in the edge search range among the surrounding pixels of the target pixel is calculated.

(3-3) The pixel with the strongest edge strength is set as the next pixel of interest.
The calculation of the trace direction angle is not performed for all the pixels in the trace area at the time (2), but may be performed for the set target pixel at the time (3-1).

Hereinafter, the processes (1) to (3) will be described with reference to FIGS.
First, FIG. 4 is a diagram illustrating an example of state transition at the time of executing a trace operation. The trace operation and the trace area will be described with reference to FIG.

  In this embodiment, as an example, it is assumed that the user performs a trace operation for designating a trace area using a mouse. Then, it is assumed that the trace area 210 is set by the user moving the rectangular icon image 201 by a drag operation.

  In FIG. 4, a trace area 210 is an area surrounded by a broken line. In addition, a dotted line arrow in FIG. 4 indicates a trajectory in which the center point of the icon image 201 has moved, and this trajectory is hereinafter referred to as a “trace area center line”.

  When the user presses the mouse button with the mouse cursor placed at a desired position on the input image 200 as in state 1 in FIG. 4, an icon image 201 is displayed at the position of the mouse cursor. The icon image 201 may be displayed at the position of the mouse cursor before the mouse button is pressed.

  When the user moves the designated position on the screen by moving the mouse while holding down the mouse button, the icon image 201 is also moved along with the movement. At this time, as in state 2 in FIG. 4, the area where the icon image 201 has moved is set as the trace area 210 and the set trace area 210 is displayed in an identifiable state on the screen. Thereafter, when the user finishes pressing the mouse button at a desired position, the setting of the trace area 210 is completed as shown in state 3 of FIG.

  When detecting the outline of the human head, the user may move the icon image 201 so that the outline of the human head on the input image 200 is always included in the icon image 201.

  When the touch panel is used, for example, the user touches a desired position on the input image, moves the contact position while keeping the finger in contact, and then releases the finger at the desired position. , The trace area 210 can be specified.

  In the example of FIG. 4, the area where the icon image 201 has been moved is set as the trace area 210. However, for example, there is an area having a certain width centered on the locus of the mouse cursor or the locus of the finger contact position. The trace area 210 may be set.

Further, the shape of the icon image 201 is not limited to a rectangle as shown in FIG.
By the way, as will be described later, a start area and an end area of the trace area 210 are set in order to determine the start point and end point of the edge search. As the start region, a plurality of pixels adjacent to the end on the start point side of the trace region 210 are set, and as the end region, a plurality of pixels adjacent to the end on the end point side of the trace region 210 are set.

  When the rectangular icon image 201 is used as shown in FIG. 4, for example, as the start area, one or two sides facing the end of the area where the icon image 201 is first arranged in the trace area 210 are used. Pixel is set. When the icon image 201 moves from the initial position in the left-right direction (x-axis direction) or the up-down direction (y-axis direction), a pixel on one side facing the end is set as the start area. On the other hand, when the icon image 201 is moved obliquely from the initial position, pixels on two sides facing the end are set as the start area. FIG. 4 shows the latter example, and a pixel adjacent to the sides 211 and 212 is set as a start region.

  The same applies to the end region. Among the regions where the icon image 201 is finally arranged in the trace region 210, pixels on one or two sides facing the end are set. When the icon image 201 moves in the horizontal direction (x-axis direction) or the vertical direction (y-axis direction) with respect to the final position, a pixel on one side facing the end is set as the end region. On the other hand, when the icon image 201 moves obliquely with respect to the final position, pixels on two sides facing the end are set as the end region. FIG. 4 shows the latter example, and a pixel adjacent to the sides 213 and 214 is set as an end region.

  FIG. 5 is a diagram for explaining a method of calculating the trace direction angle. In FIG. 5, as in FIG. 4, the area surrounded by the broken line indicates the trace area 210, and the dotted arrow indicates the center line of the trace area 210.

  When the trace area 210 is set as shown in FIG. 4, the trace input unit 113 calculates the trace direction angle for each pixel in the trace area 210 and registers it in the trace area information 122. The trace direction angle indicates an angle in the extending direction of the trace region 210 at the position of each pixel. The trace direction angle is a reference angle for determining which of the surrounding pixels is a search target when focusing on the corresponding pixel.

  An example of a specific calculation method will be described using the pixel 221 in FIG. 5 as an example. The trace direction angle θ1 is set as an angle with respect to a predetermined reference direction (vertical direction in the present embodiment) on the screen. The trace direction angle θ1 for the pixel 221 can be calculated as the slope θ2 of the tangent line 223 of the center line at the intersection 222 between the perpendicular line and the center line drawn from the pixel 221 with respect to the center line.

  The slope θ2 of the tangent 223 may be approximately calculated by a simpler calculation method for the purpose of reducing the calculation load and improving the processing speed. For example, the angle of a straight line connecting the coordinates of the intersection point 222 and the coordinates on the center line that is separated from the end point side by a predetermined distance from the intersection point 222 may be calculated as an approximate value of the slope θ2 of the tangent line 223.

  FIG. 6 is a diagram illustrating an example of information registered in the trace area information. When the trace area 210 is set by the user's operation input, the trace input unit 113 creates a record for each pixel included in the trace area 210 in the trace area information 122. Each record is provided with columns of, for example, coordinates, trace direction angle, start area flag, and end area flag.

  The coordinates of each pixel are registered in the coordinate column. In the column of the trace direction angle, the trace direction angle for each pixel described in FIG. 5 is registered. The start area flag is a flag indicating whether or not the corresponding pixel belongs to the start area described with reference to FIG. 4. If the start area flag belongs to the start area, “1” is registered, and if not, “0” is registered. The The end region flag is a flag indicating whether or not the corresponding pixel belongs to the end region described with reference to FIG. 4. If the end region flag belongs to the end region, “1” is registered, and if not, “0” is registered. The It should be noted that for the pixel belonging to the end region, no further edge search is performed when the pixel is set as the pixel of interest, and therefore the trace direction angle need not be registered for that pixel.

  FIG. 7 is a diagram illustrating an example of classification of trace direction angles. As described above, the search range information 123 stores an edge search range indicating a range in which edge search is performed for each trace direction angle in association with each other. In the present embodiment, the trace direction angle is classified into eight angle ranges D0 to D7 shown in FIG. 7, and an edge search range is associated with each classified angle range.

  Here, the reason why the trace direction angle is classified into the eight angle ranges D0 to D7 is related to the presence of eight pixels around the pixel of interest in the trace region. The angle ranges D0 to D7 may be set so as to have substantially equal angles with respect to the upper side, upper right side, right side, lower right side, lower side, lower left side, left side, and upper left side of the target pixel, respectively. .

  In the example of FIG. 7, the upper side of the display screen is the angle reference direction, the range of 338 ° to less than 360 ° and the range of 0 ° to less than 24 ° is the angle range D0, and the range of 24 ° to less than 68 ° is the angle range D1. The range from 68 ° to less than 114 ° is the angle range D2, the range from 114 ° to less than 158 ° is the angle range D3, the range from 158 ° to less than 204 ° is the angle range D4, and the range is from 204 ° to less than 248 °. Is set to an angle range D5, a range of 248 ° to less than 294 ° is set to an angle range D6, and a range of 294 ° to less than 338 ° is set to an angle range D7.

  FIG. 8 is a diagram illustrating an example of the relationship between the angle range and the edge search range. In FIG. 8, with respect to the surrounding pixels of the pixel of interest, N0, N1, N2, N3, N4, N5, N6, N0, N1, N2, N3, N4, N5, N6 in the order of upper side, upper right side, right side, lower right side, lower side, lower left side, left side, upper left side. The reference numeral N7 is attached.

  The “search target pixel” illustrated in FIG. 8 indicates a pixel that is a target of edge search, more specifically, a pixel that is a target of calculation of edge strength. The edge search range indicates a range in which the search target pixel is included among the surrounding pixels of the target pixel. The edge search range by the contour detection unit 114 is limited to the vicinity range of the direction indicated by the trace direction angle calculated for the pixel of interest. In other words, the pixels that can be set as the next target pixel are limited to the pixels included in the edge search range among the peripheral pixels of the current target pixel. Thereby, the edge located in the direction largely deviated from the extending direction of the trace area is not detected, and the contour line intended by the user is easily detected.

  In the present embodiment, as an example, as shown in FIG. 8, the edge search range is set for each angle range shown in FIG. When the trace direction angle is included in the angle range D0, the edge search range is set in the vicinity of the upward direction of the pixel of interest, and the pixels N7, N0, and N1 are search target pixels. When the trace direction angle is included in the angle range D1, the edge search range is set near the upper right direction of the pixel of interest, and the pixels N0, N1, and N2 are search target pixels. When the trace direction angle is included in the angle range D2, the edge search range is set in the vicinity of the target pixel in the right direction, and the pixels N1, N2, and N3 are search target pixels. When the trace direction angle is included in the angle range D3, the edge search range is set near the lower right direction of the pixel of interest, and the pixels N2, N3, and N4 are search target pixels.

  When the trace direction angle is included in the angle range D4, the edge search range is set near the lower direction of the pixel of interest, and the pixels N3, N4, and N5 are search target pixels. When the trace direction angle is included in the angle range D5, the edge search range is set near the lower left direction of the pixel of interest, and the pixels N4, N5, and N6 are search target pixels. When the trace direction angle is included in the angle range D6, the edge search range is set in the vicinity of the target pixel in the left direction, and the pixels N5, N6, and N7 are search target pixels. When the trace direction angle is included in the angle range D7, the edge search range is set in the vicinity of the upper left direction of the target pixel, and the pixels N6, N7, and N0 are search target pixels.

In the example of FIG. 8, the search target pixel is set in a range of one pixel around the target pixel, but the search target pixel may be set in a range of a plurality of pixels around the target pixel.
In the search range information 123, information indicating the position of the search target pixel (for example, N0 to N7 shown in FIG. 8) is registered in advance for each angle range. When the target pixel is set, the contour detection unit 114 determines the search target pixel corresponding to the trace angle range for the target pixel based on the search range information 123. The contour detection unit 114 calculates the edge strength for each of the determined search target pixels, and sets the pixel having the strongest edge strength as the next pixel of interest.

  FIG. 9 is a diagram illustrating an example of calculating the edge strength. The edge strength is an index indicating the high possibility that the target pixel is an edge. FIG. 9 shows an example in which the edge intensity of the search target pixel is calculated using the Sobel filter.

  In the method using the Sobel filter, filter coefficients are prepared for each of the x direction and the y direction. Using each filter coefficient, the x component and the y component of the edge strength are calculated. Then, the edge strength is calculated by combining the x component and the y component.

  In the filter calculation in each of the x and y directions, filter coefficients are set for a total of nine pixels including the search target pixel and its surrounding pixels. In the example of FIG. 9, the filter calculation in the x direction for the search target pixel is “brightness value of upper left pixel × (−1) + brightness value of upper pixel × 0 + brightness value of upper right pixel × 1 + brightness value of left pixel”. X (-2) + brightness value of search target pixel × 0 + brightness value of right pixel × 2 + brightness value of lower left pixel × (−1) + brightness value of lower pixel × 0 + brightness value of lower right pixel × 1 The x-component of the edge strength is thereby calculated.

  In the example of FIG. 9, the y-direction filter calculation for the search target pixel is “brightness value of upper left pixel × (−1) + brightness value of upper pixel × (−2) + brightness value of upper right side pixel × (-1) + Luminance value of left pixel × 0 + Luminance value of search target pixel × 0 + Brightness value of right pixel × 0 + Luminance value of lower left pixel × 1 + Luminance value of lower pixel × 2 + Luminance value of lower right pixel This is performed by the formula “× 1”, and thereby the y component of the edge strength is calculated.

Then, the edge strength E is calculated by taking the square root of the value obtained by adding the square value of the x component and the square value of the y component.
Next, FIG. 10 is a diagram for explaining a problem caused by the contour detection process.

  According to the contour detection procedure described with reference to FIGS. 4 to 9 described above, an edge located in a direction greatly deviating from the extending direction of the trace region is not detected, and a contour line intended by the user is easily detected. On the other hand, in the region where the contour line bends sharply, there is a possibility that the portion where the contour line is bent is not included in the search range, and as a result, the contour line may not be detected accurately.

  As shown in FIG. 10, when it is desired to detect a contour line near the head of a person with the face facing forward, the contour line may be sharply bent in the ear region. For example, when performing an edge search from the upper side to the lower side of the right side of the head, the outline may be sharply bent to the left side as indicated by an arrow 301 at the base of the upper side of the ear.

  In order to accurately detect the sharply curved contour line, the user may set the trace area as accurately as possible along the shape of the contour line. In the example of FIG. 10, the contour of the ear portion can be detected by the user operating the icon image 201 to move along the arrow 301. However, such operations are time consuming and work efficiency is low. In particular, when contour detection is performed from each of a large number of input images, there is a problem that it takes a lot of work and time.

  Therefore, the contour detection apparatus 100 according to the present embodiment extracts an area including the ear from the input image, and extends the search range from the other areas when the target pixel is located in the area including the ear. . As a result, even when the contour line is sharply bent, the portion where the contour line is bent is included in the search range.

  FIG. 11 is a diagram illustrating an example of detection of an ear region. In the present embodiment, the face detection unit 112 detects the face region R0 from the input image, and sets a region including the ear (ear region R1) based on the detected position of the face region R0. The ear region R1 is desirably set so that its boundary is located outside the outer edge of the ear.

  In the present embodiment, as an example, the ear region R1 is set as a coordinate range in the vertical direction (y-axis direction) of the screen. In this case, the coordinates of the upper end and the lower end of the ear region R1 can be obtained, for example, as division points obtained by dividing the vertical range of the detected face region R0 at a predetermined ratio.

  FIG. 12 is a diagram illustrating an example of setting a trace area. When setting the trace area 210, for example, the size of the icon image 201 is set to be larger than at least the horizontal range of the ear. As shown in FIG. 12, the user moves the icon image 201 roughly along the outline of the head. At this time, it is not necessary to change the moving direction of the icon image 201 along the shape of the ear, and the icon image 201 may be moved so that the horizontal range of the ear is included in the icon image 201. With such a simple operation, the contour line can be accurately detected even for the ear portion.

  FIG. 13 is a diagram illustrating an example of setting a search range in the ear region. As shown in FIG. 8, in the region R2 other than the ear region R1, only three pixels among the surrounding pixels of the pixel of interest are set as search target pixels. On the other hand, in the ear region R1, for example, the setting range of the search target pixel is expanded by one pixel each counterclockwise and clockwise.

  In FIG. 13, an edge search is performed from the upper side to the lower side in the right region of the head, and an example of setting the search range when the trace direction angle is included in the angle range D4 (see FIGS. 7 and 8). Indicates. When the target pixel is located in the region R2 outside the ear region R1, among the surrounding pixels of the target pixel, three pixels N3 to N5 are set as search target pixels. On the other hand, when the target pixel enters the ear region R1, among the surrounding pixels of the target pixel, five pixels N2 to N6 are set as search target pixels. As described above, when the target pixel is located in the ear region R1, the search range is expanded, so that the outline of the ear can be accurately detected.

Next, FIG. 14 is a flowchart illustrating an example of a processing procedure of the contour detection apparatus according to the second embodiment.
[Step S <b> 11] The image reading unit 111 reads image data of an input image into the RAM 102.

  [Step S12] The face detection unit 112 performs face detection from the input image, and detects an ear region based on the face detection position. The face detection unit 112 stores information indicating the detected ear region in the RAM 102 as ear region information 121.

[Step S13] The trace input unit 113 accepts a trace operation by the user.
[Step S14] The trace input unit 113 sets a trace area based on the trace operation. The trace input unit 113 creates a record corresponding to each pixel in the trace area in the trace area information 122 and registers the coordinates of each pixel. Further, the trace input unit 113 calculates the trace direction angle for each pixel and registers it in the trace area information 122. Further, the trace input unit 113 discriminates the start area and the end area in the trace area, and sets the start area flag and the end area flag of the trace area information 122 based on the discrimination result.

[Step S15] The contour detection unit 114 calculates the edge strength of each pixel in the start region in the trace region, and sets the pixel having the highest edge strength as the first pixel of interest.
[Step S16] The contour detection unit 114 reads the trace direction angle associated with the current pixel of interest from the trace area information 122. The contour detection unit 114 refers to the search range information 123 and determines the position of the search target pixel corresponding to the angle range to which the read trace direction angle belongs.

  [Step S <b> 17] The contour detection unit 114 determines whether the current pixel of interest is included in the ear region based on the ear region information 121. When the pixel of interest is included in the ear region, the process of step S18 is executed, and when it is not included, the process of step S19 is executed.

  [Step S18] The contour detection unit 114 expands the edge search range. For example, in addition to the search target pixel determined in step S <b> 16, the contour detection unit 114 further sets each pixel in the clockwise direction and the counterclockwise direction among the surrounding pixels as a search target pixel.

  [Step S19] The contour detection unit 114 calculates the edge strength for each of the search target pixels, sets the search target pixel having the highest edge strength as a new target pixel, and updates the current target pixel.

  [Step S20] The contour detection unit 114 determines whether or not the updated current pixel of interest is included in the end region. If the target pixel is not included in the end region, the process of step S16 is executed, and the edge search process is continued. On the other hand, when the target pixel is included in the end region, the target pixel becomes the end of the contour line. In this case, the contour detection unit 114 outputs the information of the contour line by registering the positions of all the target pixels set by repeating steps S16 to S19 in the contour information 124.

  According to the processing in FIG. 14 described above, the contour line intended by the user is accurately detected. In particular, the contour line can be accurately detected even in the ear region where the contour line may be bent sharply. Then, the contour line can be accurately detected without faithfully moving the icon image along the curved contour line. Therefore, the contour line can be efficiently detected by a simple operation.

  In the process of FIG. 14, the trace direction angles of all the pixels in the trace area are calculated in advance in step S15 before the edge search process is started. However, as another example, the trace direction angle of the pixel of interest may be calculated at the time of step S16 without calculating the trace direction angle at the time of step S15.

  In the process of FIG. 14, the edge strength is calculated at the time of step S15 and step S19. However, as another example, after setting the trace area in step S14 and before executing the process of step S15, the edge strengths of all the pixels in the trace area are calculated in advance, similarly to the trace direction angle, It may be registered in the trace area information 122. In this case, in step S15 and step S19, the edge intensity of the corresponding pixel is read from the trace area information 122.

  Further, either or both of the trace direction angle and the edge intensity for all the pixels on the input image may be calculated and temporarily stored in the storage device before the trace operation is accepted in step S13. Good.

[Third Embodiment]
FIG. 15 is a block diagram illustrating a configuration example of processing functions of the contour detection apparatus according to the third embodiment. In FIG. 15, the same components as those in FIG. 3 are denoted by the same reference numerals, and description thereof is omitted here. The contour detection device 100a according to the third embodiment is obtained by modifying the contour detection device 100 according to the second embodiment as follows.

  In the contour detection apparatus 100a, the ear region information 121a holds information indicating the region including the ears, similarly to the ear region information 121 of the second embodiment, but the ear region information 121a indicates the region including the ears. Information indicating a divided area divided into two halves is held. The face detection unit 112a detects a face from the input image, discriminates the upper and lower divided regions of the ear region based on the face detection result, and obtains position information indicating the divided regions as ear region information. 121a is registered. Further, the face detection unit 112a registers the face position information 125 in the RAM 102 based on the face detection result. The face position information 125 is information for determining whether the target pixel is located on the right side or the left side of the face. For example, coordinates indicating the center of the face with respect to the horizontal direction (y direction) are registered.

  The basic edge search process performed by the contour detection unit 114a is the same as that of the contour detection unit 114 of the first embodiment, but the contour detection unit 114a determines whether the pixel of interest is above or below the ear region. The direction in which the edge search range is expanded is changed depending on whether it is located in the divided area.

  FIG. 16 is a diagram illustrating an example of an outline of an ear. As described above, when it is desired to detect a contour line near the head of a person whose face is facing the front, the contour line may be sharply bent in the ear region. For example, as shown in FIG. 16, when performing an edge search from the upper side to the lower side of the right side of the head, the outline may be sharply bent to the left side as indicated by an arrow 302 at the base of the upper side of the ear. . On the other hand, when the edge search is performed from the upper side to the lower side of the outline of the ear, the outline may be sharply bent rightward as indicated by an arrow 303 on the lower side of the ear.

  In view of the above characteristics, in the region including the upper half of the ear among the ear region, the direction of extending the search range of the edge can be limited to the counterclockwise direction, and in the region including the lower half of the ear, The direction in which the edge search range is expanded can be limited to the clockwise direction. As a result, the processing speed can be improved by reducing the number of pixels for calculating the edge strength while maintaining the detection accuracy of the ear outline at the same accuracy as in the second embodiment.

  FIG. 17 is a diagram illustrating a setting example of the search range in the ear region. In the example of FIG. 17, an upper region R11 corresponding to the upper half of the ear region and a lower region R12 corresponding to the lower half are set. The upper region R11 and the lower region R12 can be set by dividing the ear region determined in the second embodiment into two in the vertical direction.

  In the example of FIG. 17, the edge search is performed from the upper side to the lower side in the right region of the head, and the search range when the trace direction angle is included in the angle range D4 (see FIGS. 7 and 8). A setting example is shown. When the target pixel is located in the region R2 outside the ear region, three pixels N3 to N5 among the surrounding pixels of the target pixel are set as search target pixels.

  On the other hand, when the target pixel enters the upper ear region R11, the search range is extended by one pixel in the counterclockwise direction, and four pixels N2 to N5 are set as search target pixels. When the target pixel enters the lower region R12 of the ear, the search range is expanded by one pixel in the clockwise direction, and four pixels N3 to N6 are set as search target pixels.

  The direction in which the search range is extended is that the side near the ear that is currently located on the face is searched (that is, whether the target pixel is located on the left or right side of the center of the face), and whether the edge search is up or down It depends on what you are going to. A detailed setting method is shown in the flowchart of FIG.

  18 and 19 are flowcharts illustrating an example of a processing procedure of the contour detection apparatus according to the third embodiment. In FIG. 18 and FIG. 19, processing steps in which processing similar to that in FIG. 14 is executed are denoted by the same reference numerals and description thereof is omitted.

After step S11 shown in FIG. 14 is executed, the next step S12a is executed.
[Step S12a] The face detection unit 112a detects a face from the input image, and detects the center of the face and the upper and lower areas of the ear based on the face detection position. The face detection unit 112a stores the horizontal coordinate (x coordinate) indicating the center of the face in the RAM 102 as face position information 125, and stores information indicating the upper and lower areas of the ear in the RAM 102 as ear area information 121a. save. In the ear region information 121a, for example, y-coordinates indicating vertical boundaries for each of the upper region and the lower region are set.

  After step S12a is executed, steps S13 and S14 shown in FIG. 14 are executed, and steps S15 and S16 are further executed by the contour detection unit 114a. Thereafter, the processing after the next step S31 is executed.

  [Step S31] The contour detection unit 114a determines, based on the ear region information 121a, whether the current pixel of interest is included in the ear region (that is, whether it is included in either the upper region or the lower region of the ear). . When the pixel of interest is included in the ear region, the process of step S32 is executed, and when it is not included, the process of step S19 is executed.

  [Step S32] The contour detection unit 114a determines whether the current edge search direction is upward or downward. Whether the direction of the edge search is upward or downward can be determined based on, for example, the sign of the difference between the y coordinate of the current pixel of interest and the y coordinate of the pixel of interest set immediately before. If it is determined that the edge search direction is upward, the process of step S33 is executed. If it is determined downward, the process of step S36 is executed.

  [Step S33] The contour detection unit 114a determines, based on the ear region information 121a, whether the current pixel of interest is included in the upper region or the lower region of the ear. If it is included in the upper area, the process of step S34 is executed. If it is included in the lower area, the process of step S35 is executed.

  [Step S34] The contour detection unit 114a determines whether the target pixel is located on the right side or the left side of the face based on the face position information 125. When located on the right side, the process of step S40 is executed, and when located on the left side, the process of step S39 is executed.

  [Step S35] Based on the face position information 125, the contour detection unit 114a determines whether the target pixel is located on the right side or the left side of the face. When located on the right side, the process of step S39 is executed, and when located on the left side, the process of step S40 is executed.

  [Step S36] The contour detection unit 114a determines, based on the ear region information 121a, whether the current pixel of interest is included in the upper region or the lower region of the ear. If it is included in the upper region, the process of step S37 is executed. If it is included in the lower region, the process of step S38 is executed.

  [Step S37] The contour detection unit 114a determines whether the target pixel is located on the right side or the left side of the face based on the face position information 125. When located on the right side, the process of step S39 is executed, and when located on the left side, the process of step S40 is executed.

  [Step S38] The contour detection unit 114a determines whether the target pixel is located on the right side or the left side of the face based on the face position information 125. When located on the right side, the process of step S40 is executed, and when located on the left side, the process of step S39 is executed.

  [Step S39] The contour detection unit 114a expands the edge search range in the counterclockwise direction. For example, in addition to the search target pixel determined in step S <b> 16, the contour detection unit 114 a further sets one pixel in the counterclockwise direction among the surrounding pixels as the search target pixel.

  [Step S40] The contour detection unit 114a expands the edge search range clockwise. For example, in addition to the search target pixel determined in step S <b> 16, the contour detection unit 114 a sets one more clockwise pixel among the surrounding pixels as the search target pixel.

After step S39 or step S40 is executed, steps S19 and S20 are executed by the contour detection unit 114a.
According to the third embodiment described above, when the target pixel is included in the ear region, the extended range of the edge search range is narrower than that of the second embodiment, and the edge strength is calculated. The number of target pixels decreases. Thereby, the processing speed of outline detection can be improved.

  Note that, in the third embodiment, compared to the second embodiment, the extended range to the right or left of the edge search range when the target pixel is included in the ear region may be widened. . For example, in step S18 in FIG. 14, the search range is expanded by one pixel in each of the counterclockwise direction and the clockwise direction, whereas in steps S39 and S40 in FIG. 19, two pixels are set in the counterclockwise direction or the clockwise direction. And the search range may be extended. As a result, even when the contour is bent more greatly, the contour can be accurately detected. Further, the number of pixels expanded in the clockwise direction or the counterclockwise direction in the third embodiment from the number of pixels expanded in the clockwise direction or the counterclockwise direction in the second embodiment (one pixel in the above example). By setting (2 pixels in the above example) to within 2 times, the number of pixels to be calculated for edge does not increase, and the processing speed can be suppressed to the same level or lower than that of the second embodiment. .

  The processing functions of the devices (contour detection devices 1, 100, 100a) described in the above embodiments can be realized by a computer. In that case, a program describing the processing contents of the functions that each device should have is provided, and the processing functions are realized on the computer by executing the program on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic storage device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Examples of the magnetic storage device include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape. Examples of the optical disc include a DVD, a DVD-RAM, a CD-ROM, and a CD-R / RW. Magneto-optical recording media include MO (Magneto-Optical disk).

  When distributing the program, for example, a portable recording medium such as a DVD or a CD-ROM in which the program is recorded is sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. In addition, each time a program is transferred from a server computer connected via a network, the computer can sequentially execute processing according to the received program.

DESCRIPTION OF SYMBOLS 1 Contour detection apparatus 2 Area | region setting means 3 Input reception means 4 Search means 5 Search range information 10 Specific area 20 Search area 21 Start area 22 End area 31, 32, 41, 42 Pixel A1, A2 Stretch direction B1-B3 Edge search range

Claims (8)

Area setting means for setting a specific area on the input image;
In response to an input operation, an input receiving unit that sets a search area having a certain width or more extended from the start area toward the end area on the input image;
An edge search range for searching for an edge is associated with each angle in the extension direction by determining an angle in the extension direction of the search area at the position of the target pixel set from the search area , and each edge search Based on search range information set so that the range includes the extension direction of the search region, an edge search range corresponding to the determined angle of the extension direction is determined, and the surrounding pixels of the set target pixel Edge search processing for determining a pixel that is highly likely to be an edge from the pixels located within the determined edge search range and setting the pixel as the next target pixel is performed from the start area to the end area. Search means to repeat until it reaches,
Have
When the pixel of interest is included in the specific region, the search means expands the edge search range determined for the pixel of interest, and is located within the expanded edge search range among the surrounding pixels of the pixel of interest. From a pixel, a pixel that is likely to be an edge is set as the next pixel of interest.
A contour detection apparatus characterized by the above.   The contour detection apparatus according to claim 1, wherein the specific region is a region including a human ear. The region setting means, said detecting a human face from the input image, edge detection apparatus according to claim 2, wherein the setting a region including the ear on the basis of the detection result of the face. It said area setting means sets a first region and a second region obtained by dividing the area including the ear in the vertical direction of the ear on the input image,
The search means uses the edge search range determined for the target pixel as a reference based on the extension direction determined for the target pixel, depending on whether the target pixel is included in the first region or the second region. As selectively expand against the right or left side,
The contour detection apparatus according to claim 2 or 3,   The input receiving means receives an input operation for continuously moving a designated position on the input image, and sets an area having a certain width or more along a locus of the designated position as the search area. The contour detection apparatus according to any one of claims 1 to 4.   The input accepting means accepts an input operation for continuously moving an icon image having a certain size on the input image, and sets an area where the icon image has moved to the search area. Item 5. The contour detection device according to any one of items 1 to 4. Contour detection device
Set a specific area on the input image,
In response to an input operation, a search area having a certain width or more extended from the start area to the end area is set on the input image,
An edge search range for searching for an edge is associated with each angle in the extension direction by determining an angle in the extension direction of the search area at the position of the target pixel set from the search area , and each edge search Based on search range information set so that the range includes the extension direction of the search region, an edge search range corresponding to the determined angle of the extension direction is determined, and the surrounding pixels of the set target pixel Edge search processing for determining a pixel that is highly likely to be an edge from the pixels located within the determined edge search range and setting the pixel as the next target pixel is performed from the start area to the end area. Repeat until it reaches,
Execute the process,
In the edge search process, when the target pixel is included in the specific region, the edge search range determined for the target pixel is expanded, and the position within the expanded edge search range among the surrounding pixels of the target pixel A pixel that is likely to be an edge is set as the next pixel of interest from
A contour detection method characterized by the above. On the computer,
Set a specific area on the input image,
In response to an input operation, a search area having a certain width or more extended from the start area to the end area is set on the input image,
An edge search range for searching for an edge is associated with each angle in the extension direction by determining an angle in the extension direction of the search area at the position of the target pixel set from the search area , and each edge search Based on search range information set so that the range includes the extension direction of the search region, an edge search range corresponding to the determined angle of the extension direction is determined, and the surrounding pixels of the set target pixel Edge search processing for determining a pixel that is highly likely to be an edge from the pixels located within the determined edge search range and setting the pixel as the next target pixel is performed from the start area to the end area. Repeat until it reaches,
Let the process run,
In the edge search process, when the target pixel is included in the specific region, the edge search range determined for the target pixel is expanded, and the position within the expanded edge search range among the surrounding pixels of the target pixel A pixel that is likely to be an edge is set as the next pixel of interest from
A contour detection program.

Images