JP2013029930A - Image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device that can achieve desired segmentation processing even by rough coordinate designation, so that the usability of processing operations can be improved.SOLUTION: An image processing device: displays image information to be processed; accepts at least one designation of a point within the displayed image information; forms a region having a predetermined shape including the point within the image information for each accepted point; and executes segmentation processing with information on the formed region as an initial region of the foreground, thereby separating the image information into the foreground and the background.

Description

  The present invention relates to an image processing apparatus, and more particularly to improvement in convenience of processing operation.

  In recent years, mobile terminals such as so-called smartphones that can perform the same processing as a general computer have been widely used. Many of such smartphones have a relatively small screen and can be used to specify rough points on an image, such as using a touch panel.

  On the other hand, a technique called image segmentation is known as a technique for extracting a target region from given image information. As a method of processing such segmentation, there are various methods such as graph cuts and region expansion methods. For example, Patent Document 1 discloses an example in which segmentation processing is used for object recognition.

JP 2010-027025 A

  In such a conventional segmentation technique, for example, when a graph cut is used, it is assumed that a shape surrounding an area (foreground) to be extracted is designated in advance as an initial area. However, as described above, when this is mounted on a smartphone, it may be difficult to perform segmentation processing because detailed area designation is difficult.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus that can achieve desired segmentation processing and improve the convenience of processing operation even by rough coordinate designation. To do.

  The present invention for solving the problems of the conventional example is an image processing apparatus, means for displaying image information to be processed, means for receiving at least one designation of a point in the displayed image information, For each of the received points, a region forming means for forming a region of a predetermined shape including the point in the image information, and performing segmentation processing using the information of the formed region as an initial region of the foreground, the image information Are separated into foreground and background, and output means for outputting the image portion of the foreground obtained by the separation.

  Here, the area forming means may set the size of the predetermined shape area to be in contact with an adjacent predetermined shape area or to be in contact with a shape surrounding the image information.

  The region forming means further forms a small region within the formed region having a predetermined shape and including the designated point included in the region, and the separating unit includes the small region. The separation process may be performed assuming that all of the pixels are included in the foreground.

  Furthermore, in the image processing apparatus according to an aspect of the present invention, the separation unit further performs a segmentation process using an area outside the predetermined shape formed by the region forming unit as an initial region of the background. Is.

  According to another aspect of the present invention, there is provided a program for displaying, on a computer, means for displaying image information to be processed, means for receiving at least one point designation in the displayed image information, and each received point. A region forming means for forming a region of a predetermined shape including the point in the image information, and performing segmentation processing using the information on the formed region as an initial region of the foreground, and converting the image information into the foreground and background. And separating means for separating the foreground and output means for outputting the image portion of the foreground obtained by the separation.

  In this way, by forming an initial region based on the designation of points and performing segmentation processing, desired segmentation processing can be achieved even by rough coordinate designation, and convenience of processing operations can be improved.

It is a block diagram showing the example of a structure of the image processing apparatus which concerns on embodiment of this invention. It is a functional block diagram showing the example of the image processing apparatus which concerns on embodiment of this invention. It is explanatory drawing showing the example of the list | wrist of the process target coordinate information which the image processing apparatus which concerns on embodiment of this invention produces | generates. It is explanatory drawing showing the example of a screen which the image processing apparatus which concerns on embodiment of this invention displays. It is explanatory drawing showing the example of a process of the area | region production | generation of the image processing apparatus which concerns on embodiment of this invention. It is explanatory drawing showing the example of the area | region which the image processing apparatus which concerns on embodiment of this invention produces | generates. It is a flowchart figure showing the operation example of the image processing device concerning an embodiment of the invention.

  Embodiments of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, the image processing apparatus 1 according to the embodiment of the present invention includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, and an image acquisition unit 15. It is configured.

  Here, the control unit 11 is a program control device such as a CPU (Central Processing Unit), and operates according to a program stored in the storage unit 12. In the present embodiment, the control unit 11 acquires image information to be processed and causes the display unit 14 to display and output the image information. In addition, the control unit 11 accepts at least one designation of a point in the displayed image information, and forms each region of a predetermined shape including the accepted point in the image information. Then, the control unit 11 performs segmentation processing using the formed region information as the initial region of the foreground, separates the image information into the foreground and the background, and outputs the foreground image portion obtained by the separation. Details of specific processing by the control unit 11 will be described in detail later.

  The storage unit 12 is a memory device or the like and holds a program executed by the control unit 11. This program may be provided by being stored in a computer-readable recording medium such as a DVD-ROM (Digital Versatile Disc Read Only Memory) and copied to the storage unit 12. The storage unit 12 also operates as a work memory for the control unit 11.

  The operation unit 13 is a transparent touch panel arranged on the display unit 14, for example. The operation unit 13 generates information indicating a position instructed by touching the user and outputs the information to the control unit 11. The display unit 14 is a liquid crystal display panel or the like, and displays and outputs image information in accordance with an instruction input from the control unit 11.

  The image acquisition unit 15 is at least one of an imaging unit that acquires image information by imaging, such as a camera, or a communication unit that acquires image information by communication with an external device such as a network interface or a USB (Universal Serial Bus) interface. including. The image acquisition unit 15 outputs the acquired image information to the control unit 11.

  Next, the operation of the control unit 11 of the present embodiment will be described. The control unit 11 according to the present embodiment stores the image information acquired and output by the image acquisition unit 15 in the storage unit 12. In addition, the control unit 11 is functionally operated with the configuration illustrated in FIG. 2 by executing a program stored in the storage unit 12. That is, the control unit 11 is functionally configured to have an image information acquisition unit 21, an image display control unit 22, a designated point receiving unit 23, an area forming unit 24, a separation unit 25, an output unit 26, an information And a processing unit 27.

  The image information acquisition unit 21 reads one of the image information stored in the storage unit 12 in accordance with an instruction from the user. Further, the image information acquisition unit 21 may instruct the image acquisition unit 15 to acquire and acquire image information in accordance with an instruction from the user. The image display control unit 22 causes the display unit 14 to display and output the image information acquired by the image information acquisition unit 21.

  The designated point receiving unit 23 indicates a position (designation of a point in the image information) indicated by the user operating the operation unit 13 on the image information displayed and output by the display unit 14 by the image display control unit 22. Accept at least one. When the operation unit 13 is a touch panel laminated on the display unit 14, the operation unit 13 displays pixel position information (coordinate information) on the laminated display unit 14 corresponding to the position touched by the user. Output.

  Each time the designated point receiving unit 23 receives input of coordinate information designated by the user from the operation unit 13, the designated point receiving unit 23 accumulates and stores the coordinate information P as a processing target coordinate information list in the storage unit 12 (FIG. 3). . In addition, the designated point receiving unit 23 receives information indicating that the user has long-pressed one point on the operation unit 13 (an operation for keeping the point on the touch panel to remain for a predetermined time or longer) from the operation unit 13. You may accept and do the following:

That is, when the long-pressed coordinate information S is input, among the coordinate information P1, P2,... Pn included in the processing target coordinate information list,
| S-Pi | <T
(Where | S−Pi | is the distance between the coordinates represented by the coordinate information S and the coordinates represented by Pi, i = 1, 2,... N, T is a predetermined threshold value) If there is such a Pi (if there are a plurality of coordinates information Pi, | S-Pi | is the smallest Pi), a graphic of a predetermined shape surrounding the Pi is displayed on the display unit 14. In addition, an icon (for example, an X mark icon) that accepts a deletion instruction is displayed in the vicinity of the figure. This display is illustrated in FIG. 4, for example.

  When the user drags the figure with the predetermined shape on the display unit 14, the coordinate information Pi is changed according to the drag. Further, when the user touches a deletion instruction icon arranged near the figure, the coordinate information on the processing target coordinate information list is added or changed, such as deleting the coordinate information Pi from the processing target coordinate information list. You may perform the process to perform.

  The area forming unit 24 refers to the coordinate information included in the processing target coordinate information list, and for each point received by the user's instruction operation, forms an area of a predetermined shape including the point in the image information. Specifically, the region forming unit 24 sequentially selects the coordinate information included in the processing target coordinate information list as the attention coordinate Pg.

The region forming unit 24 makes a distance | Pi−Pg |
Is selected from the coordinate information Pi (i = 1, 2,... N, where i ≠ g) included in the processing target coordinate information list. Then, the region forming unit 24 calculates half of the distance | Pj−Pg | to the other nearest point as information rg that defines the size of the region having a predetermined shape including the point Pg. In other words,
rg = | Pi-Pg | / 2
(FIG. 5).

  The area forming unit 24 relates to the coordinate information Pi (i = 1, 2,... N) included in the processing target coordinate information list, and information ri (which defines the size of an area of a predetermined shape including each point. When i = 1, 2,..., n) is generated, this information ri is stored in the storage unit 12 in association with information specifying the corresponding coordinate information. Then, the region forming unit 24 generates a circular region (i = 1, 2,..., N) having a radius ri around the coordinates represented by the coordinate information Pi (FIG. 6). By doing so, the size of the region of the predetermined shape becomes a size in contact with the adjacent (closest) region of the predetermined shape.

  Further, the region forming unit 24 sets the size of the region in contact with the shape surrounding the image information if n = 1, that is, if there is only one coordinate information Pi included in the processing target coordinate information list. That is, a circular area inscribed in the rectangle circumscribing the image information (maximum size in contact with the frame) may be generated with the coordinates represented by the coordinate information Pi as the center.

  The separation unit 25 performs segmentation processing using the formed region information as an initial region of the foreground, and separates the image information into the foreground and the background. For this segmentation process, a known process such as a graph cut can be adopted, and detailed description thereof is omitted here. Specifically, GrabCut automatic segmentation, which is an implementation example of graph cut, may be employed. This grab cut is described in detail in, for example, C. Rother, V. Kolmogorov, and A. Blake, GrabCut: Interactive foreground extraction using iterated graph cuts, ACM Trans. Graph., Vol. 23, pp. 309-314, 2004. Have been described.

  The output unit 26 outputs the separated foreground image portion of the image information. In the present embodiment, the foreground image portion is separated for each coordinate information Pi. The output unit 26 outputs an image portion serving as a foreground for each coordinate information Pi. The information processing unit 27 executes predetermined information processing using the image portion output by the output unit 26. An example of processing by the information processing unit 27 will be described later.

  The present embodiment has the above configuration and operates as described below. When the user operates the image processing apparatus according to the present embodiment to instruct to acquire and display image information, the image processing apparatus 1 according to the present embodiment is displayed on the display unit 14 as illustrated in FIG. The image information acquired by the instruction is displayed, and an operation for designating a point is awaited (S1).

  When the user taps one point in the image portion to be extracted on the displayed image information, the image processing apparatus 1 acquires and records coordinate information on the image information corresponding to the tapped position (S2). ). The image processing apparatus 1 repeats the process S2 until the user performs an operation indicating that the designation has been completed, and accumulates and records the coordinate information of at least one designated point.

  When the user performs an operation indicating that the point has been specified, the image processing apparatus 1 refers to the coordinate information included in the processing target coordinate information list, and for each point received by the user's instruction operation, A region having a predetermined shape including the image is formed in the image information (S3). Here, it is assumed that the predetermined shape is, for example, a circle. In addition, each region has a size not larger than the size in contact with each other without overlapping.

  The image processing apparatus 1 executes a segmentation process using a program such as a grab cut with the formed area as an initial area representing the foreground (S4). Then, out of the image information obtained by the segmentation process, a partial image serving as the foreground is extracted as a process target (S5), and a predetermined process is executed (S6).

  Further, the control unit 11 of the image processing apparatus 1 according to the present embodiment may perform the following processing as the processing of the region forming unit 24 and the separation unit 25. That is, the area forming unit 24 may further form a small area including the coordinates represented by the coordinate information Pi in a circular area determined for the coordinate information Pi (referred to as an outer peripheral area for distinction). . Specifically, the small area has a shape similar to the determined shape of the outer peripheral area and has a predetermined size. In addition, when the small area determined by the predetermined size protrudes from the determined outer peripheral area, the area where the outer peripheral area and the small area overlap (to be determined by their logical product) is reduced. This is an area.

  In another example, this small region is set to be a shape similar to the shape of the determined outer peripheral region, and a region that is smaller than the size of the outer peripheral region by a predetermined ratio or size. Also good.

  Then, the separation unit 25 of the control unit 11 performs the segmentation process on the assumption that all the pixels in the small area are pixels constituting the foreground. Specifically, when using grab cut, after performing automatic segmentation with the outer region as the initial region, this small region is designated as the foreground by the user (C. Rother, V. Kolmogorov, and A. Blake, GrabCut : Interactive foreground extraction using iterated graph cuts, ACM Trans. Graph., Vol. 23, pp. 309-314, 2004, αn = 1 (foreground brush) .

  Further, in the image processing apparatus 1 according to the present embodiment, as a process of the separation unit 25, a segmentation process is performed as a background initial area for a part of the image information that is not included in any outer peripheral area (a part outside the area). May be performed. Specifically, when grab cut is used, after performing automatic segmentation with the outer region as the initial region, this small region is designated as the foreground by the user (C. Rother, V. Kolmogorov, and A. Blake, GrabCut: Interactive foreground extraction using iterated graph cuts, ACM Trans. Graph., Vol. 23, pp. 309-314, 2004, assuming that αn = 0 (background brush)) To do.

Further, in the description so far, the region forming unit 24 determines the half of the distance | Pj−Pg | from the target coordinate information Pg to another nearest point (the coordinate point represented by the coordinate information Pj), that is, The size of the area is defined as a point through which the outer periphery of the area having a predetermined shape including the point Pg passes through the midpoint of each point. In other words,
rg = | Pj-Pg | / 2
I was trying.

  However, the present embodiment is not limited to this. For example, the area forming unit 24 superimposes the graphic image representing the formed area on the image information to be processed, and displays it on the display unit 14, and performs the area adjustment according to an instruction to accept the adjustment from the user. Also good. This adjustment can be performed as follows, for example. That is, each time the user taps one point in the area, the size of the tapped area is enlarged.

  Therefore, when the region forming unit 24 performs the process of accepting the adjustment, information Ti (i = 1, 2, 2) indicating the number of taps for each coordinate information Pi (i = 1, 2,..., N) in advance. ..., n) are stored in the storage unit 12 in association with each other. Since the number of taps Ti is initially 1 each, it is “1”.

  As an example, when an area is formed in the process S3, a graphic image representing the formed area is superimposed on the image information to be processed and displayed on the display unit 14 to accept adjustment of the area from the user. I do.

Here, when the coordinate information of the point designated for adjustment by the user is input, the region forming unit 24 refers to the information of the formed region and specifies the region including the point represented by the coordinate information. That is, the coordinate information of the point designated by the user for adjustment is Pa, the coordinate information Pi (i = 1, 2,..., N) included in the processing target coordinate information list, and the size of the corresponding area. And (here circle radius ri)
| Pa−Pi | ≦ ri
The coordinate information Pi is searched. If there is such coordinate information Pi, the region forming unit 24 sets the region corresponding to the coordinate information Pi as the designated region, and increments the number of taps Ti associated with the coordinate information Pi by “1”.

Then, the area forming unit 24 uses the coordinate information Pi and the distance | Pj−Pi | to the other nearest point to determine the size r i of the designated area related to the designated coordinate information Pi.
r′i = (Ti / (Ti + Tj)) × | Pj−Pi |
Update to Tj is the number of taps associated with the coordinate information Pj. Further, the area forming unit 24 sets the size rj of the area related to the coordinate information Pj with respect to the coordinate information Pj nearest to the coordinate information Pi by r′j = (Tj / (Ti + Tj)) × | Pj−Pi |
And

At this time, the region forming unit 24
| Pi-Pk | <r'i + rk
Or | Pj−Pk | <r′j + rk
It is checked whether there is coordinate information Pk and radius rk. That is, as a result of enlarging the size of the area, it is checked whether or not it overlaps with the area related to other coordinate information. When the coordinate information Pi is found such that | Pi−Pk | <r′i + rk (to be overlapped) Pk and rk, the region forming unit 24
r'i = | Pi-Pk | -rk
And When the coordinate information Pj is found such that | Pj−Pk | <r′j + rk (which overlaps), the region forming unit 24
r'j = | Pj-Pk | -rk
And

  When the user inputs an instruction to end the adjustment of the area (an instruction to determine the area), the process proceeds to step S4 and the process is continued.

  Next, an example of processing performed by the control unit 11 in the image processing apparatus 1 according to the present embodiment for the foreground image portion extracted from the image information as a processing target will be described. According to an example of the present embodiment, the image information acquired by the image processing device 1 is, for example, image information obtained by capturing food and drinks stacked on each of a plurality of dishes.

  And a user taps the part of the food and drink of each dish from the image which imaged the displayed food and drink. The image processing apparatus 1 defines an area including the tapped position, and extracts an image portion including food and drink by segmentation processing using the area as an initial area of the foreground.

  The image processing apparatus 1 according to the present embodiment uses a standard one for a feature amount calculated based on a pixel value, such as a histogram of pixel values of an extracted image portion, and a dish identified by the feature amount. For each of the extracted image parts, with reference to a database (component value database) stored in association with nutritional component values (for example, calories and salt content) in the case of a defined amount (for example, for one person), And the nutritional component value of the predetermined quantity is acquired. In addition, such a database can be created from information such as “Food Standard Composition Table” of the Ministry of Education, Culture, Sports, Science and Technology.

  Next, the image processing apparatus 1 corrects the acquired nutrient component value based on the number and size of the extracted foreground image portions. Specifically, if the average number of the foreground image portions extracted by one person in an ordinary meal is N dishes and the number of extracted foreground image portions does not exceed N, the size of the extracted foreground image portion is larger. Then, an average value rave is calculated (for example, it may be a radius ri of a circle circumscribing the extracted image portion). Then, taking the average value as the size for one person, the size of each extracted image portion of the foreground ri / rave is multiplied by the nutrient value obtained from the component value database for the corresponding image portion, and accumulated.

  Also, if the number of extracted foreground image parts exceeds this N, until the number of N or less image parts is obtained, n pieces of extracted foreground image parts in descending order of size or smaller At least one of m pieces is removed in order from the one by repeatedly increasing n and m. As a result, if N or less image portions are obtained, an average value rave of these sizes is calculated. Then, taking the average value as the size for one person, the size of each extracted image portion of the foreground ri / rave is multiplied by the nutrient value obtained from the component value database for the corresponding image portion, and accumulated. Then, the image processing apparatus 1 displays the accumulation result on the display unit 14 and presents it to the user.

[Individual size]
When the area forming unit 24 accepts a user's adjustment instruction for one of the formed areas, in the above description, the size of the area to be adjusted is enlarged according to the instruction. At the same time, the size of the adjacent area has been reduced so as to be in contact with the area to be adjusted without overlapping. However, the present embodiment is not limited to this.

  The size of each region may be individual, the region that is subject to adjustment by the user may be enlarged or reduced according to an instruction, and the size of the adjacent region may not be adjusted. .

That is, when the coordinate information Pa of the point designated for adjustment by the user is input, the region forming unit 24 refers to the information of the formed region and specifies the region including the point represented by the coordinate information. . That is, the coordinate information of the point designated by the user for adjustment is Pa, the coordinate information Pi (i = 1, 2,..., N) included in the processing target coordinate information list, and the size of the corresponding area. And (here circle radius ri)
| Pa−Pi | ≦ ri
The coordinate information Pi is searched. If there is such coordinate information Pi, the area forming unit 24 sets the area corresponding to the coordinate information Pi as the designated area, and sets the number of taps Ti associated with the coordinate information Pi of the designated area to “1”. "Is incremented.

Then, the area forming unit 24 determines the size r i of the designated area related to the coordinate information Pi.
ri = ri + .DELTA.r.times.Ti
Ri is updated (the equal sign in this case means that the value on the right side is assigned to the variable on the left side). Here, Δr is set to a predetermined value as an increment for one tap.

  In this case, when the user inputs an instruction to end the adjustment of the region, the region forming unit 24 uses two different pieces of coordinate information Pi (i = 1, 2,..., N), Pj (j = 1, 2, .., N), it is checked whether there is a combination of i and j such that | Pi−Pj | <ri + rj (whether there is a region overlapping each other).

  Then, the region forming unit 24 outputs information indicating the presence or absence of overlapping regions to the separating unit 25. When the information indicating that there is no overlapping area is input, the separation unit 25 performs segmentation processing using the information on the formed area as the initial area of the foreground, and outputs image information as described above. , Separate into foreground and background.

  On the other hand, when information indicating that there is an overlapping area is input, the separation unit 25 uses each of the areas related to the individual coordinate information Pi (i = 1, 2,..., N) as the initial area of the foreground. Perform segmentation processing individually. Then, the logical sum of the foreground parts obtained individually as a result of the individual segmentation processing is used as the foreground part of the image information, and the part other than the foreground part is used as the background.

  As in this example, the separation unit 25 separates the foreground and the background by executing segmentation processing at least for regions that do not overlap each other and individually for overlapping regions.

[Modification of processing to change foreground size]
In addition, when the region forming unit 24 accepts the adjustment of the region and the regions may be separated, the region forming unit 24 determines in advance regardless of the nearest coordinate information with respect to the input coordinate information Pi. The size rdefault may be used as an initial value, and the size of the area corresponding to the coordinate information Pi may be used. In other words,
ri = rdefault
And

  In this case, the user may designate the coordinate information regarding the initial region, and then repeat the tap as appropriate, and repeat the tap until the size of the region becomes a desired one.

  Further, in the description so far, the example has been described in which the region forming unit 24 accepts the adjustment process, and the region is enlarged according to the number of taps. However, the adjustment process enables both the enlargement and reduction of the region. Also good.

  In this example, the area forming unit 24 performs a process of accepting both the enlargement / reduction adjustments, and therefore the weight value Wi representing the enlargement / reduction ratio for each coordinate information Pi (i = 1, 2,..., N) in advance. (I = 1, 2,..., N) are associated and stored in the storage unit 12. The weight value Wi is initially set to a predetermined initial value X, for example, “0”.

  In addition, a predetermined size rdefault is set as an initial value of the size of the region corresponding to the coordinate information Pi regardless of the nearest coordinate information with respect to the input coordinate information Pi.

  As an example, when the region forming unit 24 forms a region in the process S3, a graphic image representing the formed region is superimposed on the image information to be processed and displayed on the display unit 14, and each region is displayed. An icon to be tapped when instructing enlargement of an area (for example, an icon with a “+” sign) and an icon to be tapped when instructing reduction of an area (for example, “−” ”Symbol icon). Then, processing for accepting the adjustment of the area from the user is performed. These icons for enlarging or reducing instructions only need to be able to specify which region is related, and the display position is not particularly limited to the above.

  In another example, you can specify the area to be adjusted and perform operations to expand or narrow it with two fingers ("pinch out" and "pinch in" operations on Apple's iPhone). Instructing enlargement / reduction may be performed.

  Here, for adjustment, the icon for enlargement / reduction associated with the graphic image in one of the areas is tapped, or “pinch out” or “pinch in” in which one of the areas is specified. When the above operation is performed, the region forming unit 24 refers to the information of the formed region, and specifies the specified region Pi (the region related to the icon that is tapped when the icon is tapped).

  Then, the area forming unit 24 adjusts the weight value Wi associated with the specified coordinate information Pi. That is, when an icon for instructing expansion of an area is tapped, for example, the weight value Wi is increased by the number of taps, and when an icon for instructing reduction of the area is tapped, for example, the weight value Wi is decreased by the number of taps. Let

  Also, in the case of operations such as “pinch out” and “pinch in”, the proximity amount (the distance between the initial two points and the distance between the two points) depends on the amount at which the points touched by two fingers are close or separated in these operations. The absolute value of the difference between the two points after the operation) is multiplied by a predetermined coefficient, and only the result of the multiplication is subtracted from the current weight value Wi.

  When a pinch-out operation (an operation in which the points touched by two fingers are separated) is performed, the separation amount (the difference between the initial distance between the two points and the distance between the two points after the operation) (Absolute value) is multiplied by a predetermined coefficient, and the current weight value Wi is increased by the result of the multiplication.

The area forming unit 24 uses the weight value Wi adjusted in this way to calculate the size r i of the specified area related to the coordinate information Pi specified earlier.
ri = rdefault + .DELTA.r.times. (Wi-X)
Ri is updated (the equal sign in this case means that the value on the right side is assigned to the variable on the left side). Here, Δr is a predetermined value as an enlargement / reduction amount per increment “1” of the weight value W, rdefault is an initial value of the size of the region, and X is an initial value of the weight value.

  Further, when the user inputs an instruction to end the adjustment of the area, the area forming unit 24 receives two different pieces of coordinate information Pi (i = 1, 2,..., N), Pj (j = 1, 2,. For all combinations of n), it is checked whether there is a combination of i and j such that | Pi−Pj | <ri + rj (whether there is an overlapping area).

  Then, the region forming unit 24 outputs information indicating the presence or absence of overlapping regions to the separating unit 25. When the information indicating that there is no overlapping region is input, the separation unit 25 that performs the processing from step S4 onward performs segmentation processing using the information on the formed region as the initial region of the foreground, as described above. To separate the image information into foreground and background.

  On the other hand, when information indicating that there is an overlapping area is input, the separation unit 25 uses each of the areas related to the individual coordinate information Pi (i = 1, 2,..., N) as the initial area of the foreground. Perform segmentation processing individually. Then, the logical sum of the foreground parts obtained individually as a result of the individual segmentation processing is used as the foreground part of the image information, and the part other than the foreground part is used as the background.

  As in this example, the separation unit 25 separates the foreground and the background by executing segmentation processing at least for regions that do not overlap each other and individually for overlapping regions.

[Example of changing the default foreground size according to the image contents]
Further, when adjusting the enlargement or reduction of the area, the area forming unit 24 adjusts the size after adjustment according to the change in the distance between the two points designated in the number of taps, the pinch-out operation, or the like. Instead of determining the size of the subsequent area, the size of the adjusted area may be determined according to a predetermined rule.

As an example, when the area forming unit 24 uses the weight value Wi, when an enlargement operation is performed on the area corresponding to the coordinate information Pi, Wi is incremented by “1”.
r′i = rdefault + Δr × (Wi−X)
And the statistic of the pixel value of the part surrounded by the area of the tentative size in the image information to be processed is compared with the statistic of the pixel value of the part outside the area. The temporary size may be adjusted while incrementing Wi by “1” until the comparison result matches a predetermined condition. Then, when the condition is met, the increment of Wi is stopped, and the temporary size r'i at that time is set as a new size of the area corresponding to the coordinate information Pi. In other words,
ri = r'i
And Even when the number of taps Ti is used, the same processing may be performed as Ti instead of Wi here. In this case, the tap count Ti is not the actual tap count but has the same meaning as the weight value.

Similarly, when a reduction operation is performed on the area corresponding to the coordinate information Pi, Wi is decremented by “1”,
r′i = rdefault + Δr × (Wi−X)
And the statistic of the pixel value of the part surrounded by the area of the tentative size in the image information to be processed is compared with the statistic of the pixel value of the part outside the area. Until the comparison result matches a predetermined condition, Wi may be decremented by “1” and the provisional size may be adjusted. When the condition is met, the decrement of Wi is stopped, and the provisional size r'i at that time is set as a new size of the area corresponding to the coordinate information Pi. In other words,
ri = r'i
And

  In these cases, the statistic of the pixel value may be, for example, an average of pixel values or a mode value in a histogram. The condition relating to the comparison result is, for example, that the absolute value of the average difference between the pixel values inside and outside the region exceeds a predetermined threshold value, or the absolute value of the difference between the mode values inside and outside the region is predetermined. It may be set as a condition such as exceeding the threshold value.

  Further, when incrementing or decrementing the weight value Wi, if the provisional magnitude r'i falls below a predetermined lower limit value, the decrement is stopped at that point, and the provisional magnitude r'i at that time is It may be set as a new size of the area corresponding to the coordinate information Pi, or when the area of the temporary size r′i corresponding to the coordinate information Pi is in contact with the side of the image information, the increment is stopped. The provisional size r′i at that time may be set as a new size of the area corresponding to the coordinate information Pi.

  According to this Embodiment, the image part of each food / beverage can be specified by simple operation, for example from the image information of food / beverage.

  DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 11 Control part, 12 Storage part, 13 Operation part, 14 Display part, 15 Image acquisition part, 21 Image information acquisition part, 22 Image display control part, 23 Designated point reception part, 24 Area formation part, 25 Separation unit, 26 output unit, 27 information processing unit.

Claims (5)

Means for displaying image information to be processed;
Means for accepting at least one designation of a point in the displayed image information;
For each received point, a region forming means for forming a region of a predetermined shape including the point in the image information;
Separation means for separating the image information into a foreground and a background by performing segmentation processing using the information of the formed area as an initial area of a foreground,
Output means for outputting the image portion of the foreground obtained by the separation,
An image processing apparatus. The image processing apparatus according to claim 1,
The region forming means determines the size of the region having a predetermined shape,
An image processing apparatus having a size that touches an adjacent area of a predetermined shape or a size that touches a shape surrounding image information. The image processing apparatus according to claim 1 or 2,
The region forming means further forms a small region within the formed region of the predetermined shape and including the specified point included in the region,
The image processing apparatus that performs the separation process on the assumption that all pixels in the small region are included in the foreground. The image processing apparatus according to any one of claims 1 to 3,
The image processing apparatus that performs segmentation processing using the part outside the region of the predetermined shape formed by the region forming unit as the initial region of the background. Computer
Means for displaying image information to be processed;
Means for accepting at least one designation of a point in the displayed image information;
For each received point, a region forming means for forming a region of a predetermined shape including the point in the image information;
Separation means for separating the image information into a foreground and a background by performing segmentation processing using the information of the formed area as an initial area of a foreground,
Output means for outputting the image portion of the foreground obtained by the separation,
Program to function as.

Images