KR101718085B1 - Apparatus and Method of Enhancing Visual Flavor of Food Image - Google Patents
Apparatus and Method of Enhancing Visual Flavor of Food Image Download PDFInfo
- Publication number
- KR101718085B1 KR101718085B1 KR1020150127809A KR20150127809A KR101718085B1 KR 101718085 B1 KR101718085 B1 KR 101718085B1 KR 1020150127809 A KR1020150127809 A KR 1020150127809A KR 20150127809 A KR20150127809 A KR 20150127809A KR 101718085 B1 KR101718085 B1 KR 101718085B1
- Authority
- KR
- South Korea
- Prior art keywords
- food
- input image
- image
- signal information
- image signal
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/643—Hue control means, e.g. flesh tone control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/68—Circuits for processing colour signals for controlling the amplitude of colour signals, e.g. automatic chroma control circuits
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a video signal processing technique, and more particularly, to an apparatus and method for enhancing a visual taste of a food image by processing a video image signal.
To this end, a visual flavor emphasizing method of a food image according to the present invention includes a food object recognition step of analyzing an input image and determining whether a food object is included in the input image, analyzing the input image, And an image correction step of correcting the input image to increase the visual flavor of the input image according to the food color model according to the image signal information of the input image and the evaluated illumination condition, .
Description
The present invention relates to a video signal processing technique, and more particularly, to an apparatus and method for enhancing a visual taste of a food image by processing a video image signal.
Food photographs and images including food are used in food advertising such as home shopping, menu display of restaurants, and sales packaging of food products at a mart. Food photographs and images are widely used as means for intuitively grasping information about foods sold by consumers and for inducing them to purchase the food.
Therefore, it is necessary for the consumer to make the food look better to buy the food, and such work is usually done through post-editorial work after photographing the food. In other words, the work for emphasizing the flavor feeling on the food photographs has conventionally been performed passively by people.
However, such manual editing takes a long time, and there is a limit in that a considerable amount of money including labor costs is put into use. Also, according to subjective judgment of the editor, the quality of the result is not uniformized in the process of performing the editing, and the flavor feeling of the food image may not be emphasized depending on the ability of the editor.
In addition, although existing techniques for extracting specific information by signal processing of food images exist, these technologies do not disclose a configuration for emphasizing the visual flavor of foods included in food images.
US Published Patent US 2013/0335418 A1 (Dec. 19, 2013)
A problem to be solved by the present invention is to overcome the limitations of the conventional passive image editing method, to automatically emphasize the visual flavor of the food on the food included in the input image, and to maximize emphasis effect based on the learned data And to provide a device for the same.
According to one aspect of the present invention, there is provided a visual flavor enhancing method for a food image, comprising: a food object recognizing step of analyzing an input image to determine whether a food object is included in the input image; An illumination condition evaluating step of analyzing the input image to evaluate an illumination condition of the input image; And an image correction step of correcting the input image to increase a visual flavor of the input image according to the food color model according to the image signal information of the input image and the evaluated illumination condition, .
The food object recognition step may further include a food color model determination step of determining the food color model according to the image signal information of the input image.
Here, the food color model may be a color model including predetermined information on the color or brightness of the image signal of the food object.
Here, the food object recognition step may include extracting a shape feature according to a shape of an object included in the input image and a color feature according to a color included in the input image in the input image and using the extracted shape feature and color feature And determine whether the food object is included in the input image.
The food object recognition step may further include a food recognition step of determining whether the food object is included in the input image.
Wherein the food recognizing step comprises: extracting the shape feature and the color feature by calculating an image signal value of pixels included in the input image from the input image; A high-dimensional feature extraction step of extracting a high-dimensional feature by encoding the extracted feature and color features; And classifying whether the food object is included in the input image using a classifier previously learned based on the extracted high dimensional feature.
The food object recognition step may further include a food category classification step of classifying a food category of the food object included in the input image if it is determined that the food object is included in the input image.
Wherein the food category classification step includes a low dimensional feature extraction step of calculating the image signal values of pixels included in the input image from the input image and extracting the shape feature and the color feature; A high-dimensional feature extraction step of extracting a high-dimensional feature by encoding the extracted feature and color features; And a classifying step of classifying the food object included in the input image into a predetermined plurality of the food categories using a classifier previously learned based on the extracted high dimensional feature.
Here, the low-dimensional feature extraction step extracts SIFT (Scale Invariant Feature Transform) -based features from the input image with the feature feature, and the high-dimensional feature extraction step converts the feature feature and the color feature into a Fisher's Vector And the classification step is classified using the classifier based on a hierarchical support vector machine.
The food color model determination step may include analyzing global image signal information and local image signal information of the input image and determining the food color model of the input image according to the analysis result.
Wherein the step of determining the food color model comprises: extracting the global image signal information according to a video signal of the entire input image; Dividing the input image into image blocks of a predetermined size and extracting the regional image signal information according to the image signals of the divided image blocks; And a color model classifying step of determining, based on the feature vector including the extracted global image signal information and the local image signal information, which of the food color models the predetermined image corresponds to, using the classifier learned in advance .
The extracting of the global image signal information may include extracting a histogram according to a size of an image signal value of pixels included in the input image as the global image signal information and extracting the regional image signal information, And extracts the histogram according to the size of the image signal value of the pixels included in the image block as the regional image signal information.
Wherein the color model classifying step determines whether the input image corresponds to the predetermined food color model using the classifier based on a Gaussian mixture model or a support vector machine.
Wherein the food object recognition step classifies a food category of the food object included in the input image if the food image is included in the input image, And the food color model is determined using the classifier based on the feature vector further including the classified food category.
The lighting situation evaluating step may include analyzing global image signal information and local image signal information of the input image and determining the lighting situation of the input image according to the analysis result.
The lighting situation evaluation step may include: extracting the global image signal information according to a video signal of the entire input image; Dividing the input image into image blocks of a predetermined size and extracting the regional image signal information according to the image signals of the divided image blocks; And a lighting condition classifying step of classifying the input image to which of the predetermined lighting conditions correspond, using a classifier learned in advance based on the feature vector including the extracted global image signal information and the regional image signal information can do.
The extracting of the global image signal information may include extracting a histogram according to brightness of pixels included in the input image as the global image signal information and extracting the regional image signal information may include And extracts each histogram corresponding to the color, saturation, and brightness of the pixels as the regional video signal information.
Wherein the image correction step corrects the image signal values of the pixels included in the input image according to the food color model and the predetermined setting according to the illumination condition.
Wherein the image correction step performs at least one of tone matching correction, gamma correction, and color correction according to the predetermined setting to correct an image signal value of pixels included in the input image.
In order to solve the above problem, another type of the present invention can be a computer program stored in a medium for executing the visual flavor emphasizing method of the food image.
According to another aspect of the present invention, there is provided an apparatus for enhancing visual flavor of a food image, comprising: a food object recognition unit for analyzing an input image and determining whether a food object is included in the input image; An illumination condition evaluating unit for analyzing the input image and evaluating an illumination condition of the input image; And an image correction unit for correcting the input image to increase the visual flavor of the input image according to the food color model according to the image signal information of the input image and the evaluated illumination condition.
Here, the food object recognizing unit extracts a shape feature according to a shape of an object included in the input image and a color feature according to a color included in the input image in the input image, and uses the extracted shape feature and color feature A food recognition unit for determining whether the food object is included in the input image; A food category classifying unit for classifying a food category of the food object included in the input image if it is determined that the food object is included in the input image; And determining the food color model according to the image signal information of the input image, analyzing the global image signal information and the regional image signal information of the input image, and analyzing the global image signal information and the regional image signal information of the input image, And a food color model determining section that determines a food color model.
Wherein the illumination condition evaluating unit analyzes the global image signal information and the regional image signal information of the input image and determines the illumination condition of the input image according to the analysis result, And the image signal values of the pixels included in the input image are corrected according to a predetermined setting according to illumination conditions.
The apparatus and method for enhancing a visual image of a food image according to the present invention have an effect of automatically correcting a food image to visually emphasize a flavor feeling. According to the apparatus and method for emphasizing the visual image of a food image according to the present invention, a user who uses a food image can easily acquire a food image emphasizing a flavor by automatically correcting a general food image to have a more flavorful feeling .
FIG. 1 is a flowchart illustrating a visual flavor enhancing method of a food image according to an embodiment of the present invention.
2 is a detailed flowchart of the food object recognition step.
3 is a detailed flowchart of the food recognition step.
4 is a detailed flowchart of the food category classification step.
5 is a detailed flowchart of a food recognition step and a food category classification step according to an embodiment of the present invention.
6 is a detailed flowchart of the food color model determination step.
7 is a detailed flowchart of the lighting situation evaluation step.
8 is a block diagram of a visual flavor emphasizing device for food images according to another embodiment of the present invention.
9 is a detailed block diagram of the food object recognition unit.
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.
Food photographs and images are widely used as a means to intuitively grasp information about the food sold by the consumer and to induce them to purchase the food. It is therefore necessary to make the food look more appetizing so that the consumer can purchase the food. This work is usually done through post-production work of the editor after photographing the food. In other words, the work for emphasizing the flavor feeling on the food photographs has conventionally been performed passively by people. However, such manual editing takes a long time, and there is a limit in that a considerable amount of money including labor costs is put into use. Also, according to subjective judgment of the editor, the quality of the result is not uniformized in the process of performing the editing, and the flavor feeling of the food image may not be emphasized depending on the ability of the editor. In addition, although existing techniques for extracting specific information by signal processing of food images exist, these technologies do not disclose a configuration for emphasizing the visual flavor of foods included in food images.
Accordingly, the present invention overcomes the limitations of the conventional passive image editing method, emphasizes the visual flavor of the food automatically on the food included in the input image, and maximizes the emphasis effect based on the learned data We propose a device for that.
FIG. 1 is a flowchart illustrating a visual flavor enhancing method of a food image according to an embodiment of the present invention.
The method for enhancing visual flavor of a food image according to the present invention may include a food object recognition step (S100), a lighting condition evaluation step (S200), and an image correction step (S300).
Meanwhile, the device for enhancing a visual image of a food image according to another embodiment of the present invention may include a food
Here, the food
Here, the visual flavor enhancing device for a food image according to the present invention can be implemented by each independent component of all the components. Or a visual flavor enhancing device for a food image according to the present invention is a computer program having a program module that performs a part or all of functions in combination with one or a plurality of hardware by selectively or partially combining all or a part of each component . In addition, the visual flavor enhancing device for a food image according to the present invention may be implemented as a software program and operated on a processor or a signal processing module, or may be embodied in hardware to be included in a chip or an element. In addition, the device for enhancing the visual image of a food image according to the present invention may be included in a hardware or software module. For example, the device for enhancing the visual image of a food image according to the present invention may be a device or a computer or an embedded system connected to a camera device to receive an input image and output a corrected image. In addition, the device for emphasizing the visual image of the food image according to the present invention can be performed in each of the above devices.
Hereinafter, a method for enhancing visual flavor of a food image according to the present invention will be described in detail.
As described above, the method for enhancing a visual image of a food image according to the present invention may include a food object recognition step (S100), a lighting condition evaluation step (S200), and an image correction step (S300).
In the food object recognition step S100, the food object recognition unit analyzes the input image and determines whether or not the food object is included in the input image.
In the illumination condition evaluation step S200, the illumination condition evaluation unit analyzes the input image and evaluates the illumination condition of the input image.
In the image correction step S300, the image correction unit corrects the input image to increase the visual flavor of the input image according to the food color model according to the image signal information of the input image and the evaluated illumination condition.
First, the food object recognition step S100 will be described in more detail.
In the food object recognition step S100, the food object recognition unit analyzes the input image and determines whether or not the food object is included in the input image.
Here, the food object recognition unit may receive the input image from a camera or an image acquisition apparatus connected to the outside. Here, the input image may be one image frame or a moving image composed of a plurality of image frames. Here, the food object is an object representing food consumed by a person among the objects included in the input image. For example, the food object may be an object representing various kinds of food such as rice, soup, meat, fish, fruits, and the like. In the food object recognition step S100, whether or not such a food object is included in the input image is first recognized.
Herein, the food object recognition step S100 extracts a shape feature according to a shape of an object included in the input image and a color feature according to a color included in the input image in the input image, To determine whether the food object is included in the input image.
Here, the shape feature is a feature indicating information according to the shape or appearance of the object included in the input image. For example, the shape feature may be a feature indicating a boundary or an edge component of an object, and may be a feature indicating a texture component of the object. For this purpose, it is possible to extract an edge from the input image and to calculate a gradient or a gradient to extract a shape feature, perform local filtering on the input image, extract shape features according to the result, It is also possible to extract shape features by performing a distance operation or other operation between them. Here, various types of features used in image recognition can be used to express the shape and appearance of objects included in the image.
The color feature may be information according to the color included in the input image, and may be a feature according to global or regional color distribution in various color spaces. For example, the color feature may be a histogram according to the color of the pixels included in the input image, and may be a color histogram obtained in some areas if necessary. Here, the color feature may use various types of features that have been used in image recognition to represent the color distribution of pixels included in the image.
Here, the food object recognition step S100 may include a food recognition step S110 and may further include a food category classification step S120 or a food color model determination step S130 as needed.
2 is a detailed flowchart of the food object recognition step S100.
The food recognition step S110 determines whether the food image is included in the input image.
The food category classification step S120 classifies the food category of the food object included in the input image if it is determined that the food object is included in the input image.
Here, the food category may be a category for classifying the food object according to a predetermined type. For example, the food category can be meat, fish, cereal, etc., depending on the material of the food object, and more specifically, beef, poultry, pork, etc. for meat. Alternatively, the food category may be set as a stew, a roast, a steaming or the like according to the cooking method of the food object. It goes without saying that the food category may be set to include predetermined classes according to the setting of the user in addition to the above-described examples.
The food color model determination step (S130) determines the food color model according to the image signal information of the input image.
Here, the food color model may be a color model including predetermined information on the color or brightness of the image signal of the food object. Here, the food color model may include predetermined rule information for limiting the range of the image signal value or converting the image signal value into a specific range, and may include information about a function or table for converting the image signal value to a specific value It is possible.
Hereinafter, the food recognition step (S110) will be described in more detail.
3 is a detailed flowchart of the food recognition step S110.
Here, the food recognition step S110 may include a low dimensional feature extraction step S111, a high dimensional feature extraction step S112, and a classification step S113. In FIG. 3, the classification step S113 is represented as a first classification step S113 in order to distinguish it from the classification step S123 of the food category classification step S120.
The low-dimensional feature extraction step S111 may extract the morphological feature and the hue feature by calculating an image signal value of pixels included in the input image from the input image.
Here, the low dimensional feature extraction step (S111) preferably extracts features based on SIFT (Scale Invariant Feature Transform) from the input image with the feature feature. Here, it is preferable to extract the Dense SIFT feature from the input image. For example, the Dense SIFT is described by Andrea Vedaldi and Brian Fulkerson in " Vlfeat: an open and portable library of computer vision algorithms, Proceeding MM '10 Proceedings of the international conference on Multimedia " api / dsift.html "can be used.
If necessary, the low-dimensional feature extraction step S111 may also extract and use edge features, corner features, LoG (Laplacian of Gaussian), and DoG (Difference of Gaussian). Also, various existing feature description methods including SULF (Speed Up Robust Features) and HOG (Histogram of Oriented Gradients) may be used.
The high dimensional feature extraction step (S112) can extract the high dimensional features by encoding the shape features and color features extracted as the low dimensional feature as described above. Here, the high dimensional feature represents a feature to be generated using low dimensional features such as the shape feature or the color feature. Here, the high-dimensional feature extraction step (S112) can extract the high-dimensional features to generate and utilize more discriminating upper-level features by clustering, integrating or integrating low-dimensional features having a high degree of variation for each object or object in the image have. For this purpose, the high dimensional feature extraction step (S112) may obtain the high dimensional feature by Fisher's Vector encoding of the morphological feature and the color feature. Fischer vector coding is described, for example, in " F. Perronnin and C. Dance. Fisher kenrels on visual vocabularies for image categorization. In Proc. CVPR, 2006. " Or by using the method proposed by Florent Perronnin, Jorge Schnech, and Thomas Mensink, Improving the fisher kernel for large-scale image classification, In Proc. ECCV, 2010. "
Next, the classification step S113 can classify whether the food object is included in the input image using a classifier previously learned based on the extracted high dimensional feature.
Here, the classifier can be learned based on the low-dimensional and high-dimensional features extracted as above for a plurality of learning images that know information about whether or not the food object is included in advance, and the parameters of the classification function used by the classifier in the learning process are set .
Preferably, the classification step S113 may be categorized using the classifier based on a hierarchical support vector machine. That is, whether the food object is included in the input image. For example, here a multi-layer support vector machine can be implemented using the method proposed in "Schwenker, Friedhelm." Hierarchical support vector machines for multi-class pattern recognition.
Hereinafter, the food category classification step S120 will be described in more detail.
4 is a detailed flowchart of the food category classification step S120.
Here, the food category classification step S120 may include a low dimensional feature extraction step S121, a high dimensional feature extraction step S122, and a classification step S123. In FIG. 4, the classification step S123 is indicated as a second classification step S123 in order to distinguish it from the classification step S113 of the food recognition step S110.
Here, the food category classification step S120 may extract the low-dimensional features in the same manner as the food recognition step S110, and perform classification using the previously learned classifiers after extracting the high-dimensional features. Here, in the classification step S123, as in the classification step S113, the category of the food is categorized instead of only the food image. Hereinafter, the low dimensional feature extraction step S121 and the high dimensional feature extraction step S122 may operate in the same manner as the low dimensional feature extraction step S111 and the high dimensional feature extraction step S112 in the food recognition step S110 Briefly, If necessary, the feature extracted in the low dimensional feature extraction step S111 and the high dimensional feature extraction step S112 in the food recognition step S110 may be used as described below.
The low dimensional feature extraction step S121 extracts the shape feature and the color feature by calculating an image signal value of pixels included in the input image from the input image. Here, the low-dimensional feature extraction step S121 may extract features based on SIFT (Scale Invariant Feature Transform) from the input image with the feature feature.
In the high-dimensional feature extraction step S122, the extracted feature features and color features are encoded to extract high-dimensional features. Here, the high dimensional feature extraction step S122 may obtain the high dimensional feature by Fisher's Vector encoding the shape feature and the color feature.
The classifying step S123 classifies the food objects included in the input image into a plurality of predetermined food categories by using a classifier previously learned based on the extracted high dimensional features. The classification step S123 may be classified using the classifier based on a hierarchical support vector machine.
If necessary, classification may be performed in the classification step S123 using the shape features and color features extracted in the food recognition step S110 and the high dimensional features extracted therefrom in the food category classification step S120. 5 is a detailed flowchart of the food recognition step and the food category classification step according to an embodiment of the present invention.
Hereinafter, the food color model determination step (S130) will be described in more detail.
The food color model determination step (S130) analyzes the global image signal information of the input image and the regional image signal information, and determines the food color model of the input image according to the analysis result.
Here, the food color model may be a color model including predetermined information on the color or brightness of the image signal of the food object. Here, the food color model may include predetermined rule information for limiting the range of the image signal value or converting the image signal value into a specific range, and may include information about a function or table for converting the image signal value to a specific value It is possible.
Here, the food color model determination step S130 may include a global image signal information extraction step S131, a regional image signal information extraction step S132, and a color model classification step S133.
6 is a detailed flowchart of the food color model determination step.
The global video signal information extraction step S131 extracts the global video signal information according to the video signal of the entire input video.
Here, the global image signal information extraction step S131 may extract the histogram according to the size of the image signal value of the pixels included in the input image as the global image signal information. Here, the global image signal information extraction step S131 may extract the histogram according to the brightness of the pixels included in the input image as the global image signal information.
The local image signal information extracting step S132 divides the input image into image blocks having a predetermined size and extracts the regional image signal information according to the image signals of the divided image blocks. For example, the image block may have a size of 4 x 4. Also, the local video signal information may be extracted for a part of the image blocks included in the input image as needed.
The local image signal information extraction step (S132) may extract the histogram according to the size of the image signal value of the pixels included in the image block as the regional image signal information. Here, the regional image signal information extraction step (S132) may extract each histogram according to the hue, saturation, and brightness of the pixels included in the image block as the regional image signal information.
The color model classifying step S133 may include classifying the input image into the predetermined food color model using a classifier previously learned based on the feature vector including the extracted global image signal information and the regional image signal information .
That is, in the color model classification step S133, a histogram according to the brightness of the pixels included in the input image and a histogram according to the image signal value acquired for each of the image blocks, for example, a histogram Can be extracted as a feature vector. The feature vector extracted as described above may be input to a classifier previously learned to determine the food color model corresponding to the input image.
Here, the color model classification step S133 may determine which of the food color models the input image corresponds to, using the classifier based on the Gaussian mixture model or the support vector machine.
In this case, when it is determined that the food object is included in the input image in the food object recognition step (S100) and the food category of the food object included in the input image is classified, the color model classification step (S133) The food color model may be determined using the classifier based on the feature vector further including the food category classified in the food object recognition step S100.
Next, the illumination condition evaluation step (S200) will be described in more detail.
In the illumination condition evaluation step S200, the illumination condition evaluation unit analyzes the input image and evaluates the illumination condition of the input image. Here, the illumination condition may be preset to a certain type of information about the situation of the illumination of the input image. For example, the lighting conditions may be set as a shaded lighting condition, a dark lighting condition, a bright lighting condition, or may be set as an incandescent lighting condition, a natural lighting condition, a fluorescent lighting condition, or the like.
More specifically, the illumination condition evaluation step (S200) may analyze the global image signal information and the regional image signal information of the input image and determine the illumination state of the input image according to the analysis result. Here, the illumination condition evaluation step S200 can extract the global and local image signal information in the same manner as in the above-described food color model determination step S130, and generate and classify the feature vectors including the global and local image signal information have. Therefore, the operation detailed in the food color model determination step (S130) can be similarly performed in the illumination condition evaluation step (S200). However, it is needless to say that the illumination condition evaluation step (S200) and the food color model determination step (S130) are different in that the objects to be classified are different from each other in illumination condition and food color model.
7 is a detailed flowchart of the illumination condition evaluation step (S200).
The illumination condition evaluation step S200 may include a global image signal information extraction step S210, a regional image signal information extraction step S220, and a lighting condition classification step S230.
The global video signal information extraction step (S210) extracts the global video signal information according to the video signal of the entire input video.
Here, the global image signal information extraction step S210 may extract the histogram according to the size of the image signal values of the pixels included in the input image as the global image signal information. At this time, the histogram according to the brightness of the pixels included in the input image can be extracted as the global image signal information.
The local image signal information extracting step S220 divides the input image into image blocks having a predetermined size and extracts the regional image signal information according to the image signals of the divided image blocks.
Here, the local image signal information extraction step S220 may extract the histogram according to the size of the image signal value of the pixels included in the image block as the regional image signal information. At this time, each histogram corresponding to color, saturation, and brightness of pixels included in the image block can be extracted as the regional image signal information.
The illumination condition classifying step S230 classifies which of the predetermined illumination conditions the input image corresponds to using the previously learned classifier based on the feature vector including the extracted global image signal information and the regional image signal information do.
Here, the illumination condition classifying step S230 can classify the input image to which the illumination condition corresponds to the predetermined illumination condition, using the classifier based on the Gaussian mixture model or the support vector machine.
Next, the image correction step S300 will be described in more detail.
In the image correction step S300, the image correction unit corrects the input image to increase the visual flavor of the input image according to the food color model according to the image signal information of the input image and the evaluated illumination condition.
The image correction step S300 may correct the image signal values of the pixels included in the input image according to the food color model and the predetermined setting according to the illumination condition. Here, the image correction step S300 may perform at least one of tone matching correction, gamma correction, and color balance correction according to the predetermined setting, The image signal values of the pixels can be corrected. For example, the color correction can convert the color signal value in an image by performing affine transformation on a pixel basis.
Yet another embodiment of the present invention can be a computer program stored on a medium for implementing a visual flavor enhancing method of a food image according to the present invention described in detail with reference to FIGS. 1 to 7 above.
8 is a block diagram of a visual flavor emphasizing device for food images according to another embodiment of the present invention.
The apparatus for enhancing a visual image of a food image according to another embodiment of the present invention can operate in the same manner as the method of enhancing a visual image of a food image according to the present invention described in detail with reference to FIGS. 1 to 7 have. The overlapping portions will be omitted and briefly described.
The apparatus for enhancing a visual image of a food image according to another embodiment of the present invention may include a food
The food
The illumination condition evaluation unit 200 analyzes the input image and evaluates the illumination condition of the input image.
Here, the illumination condition evaluation unit 200 may analyze the global image signal information and the regional image signal information of the input image, and may determine the illumination state of the input image according to the analysis result.
The image correcting unit 300 corrects the input image to increase the visual flavor of the input image according to the food color model according to the image signal information of the input image and the evaluated illumination condition.
Here, the image correction unit 300 may correct the image signal values of the pixels included in the input image according to the food color model and the predetermined setting according to the illumination condition.
9 is a detailed block diagram of the food
The food
The
The food
The food color
It is to be understood that the present invention is not limited to these embodiments, and all elements constituting the embodiment of the present invention described above are described as being combined or operated in one operation. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them.
In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. In addition, such a computer program may be stored in a computer readable medium such as a USB memory, a CD disk, a flash memory, etc., and read and executed by a computer to implement an embodiment of the present invention. As the recording medium of the computer program, a magnetic recording medium, an optical recording medium, or the like can be included.
Furthermore, all terms including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined in the Detailed Description. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.
It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.
S100: Food object recognition step
S110: food recognition step
S120: food category classification step
S130: food color model determination step
S200: Lighting condition evaluation step
S210: Global video signal information extraction step
S220: Step of extracting regional video signal information
S230: Illumination condition classification step
S300: image correction step
100: food object recognition unit
200: lighting condition evaluation unit
300: image correction unit
110: Food recognition section
120: food category classification section
130: Food Color Model Decision Unit
Claims (21)
An illumination condition evaluating step of analyzing the input image to evaluate an illumination condition of the input image; And
An image correction step of correcting the input image to increase a visual flavor of the input image according to a food color model according to the image signal information of the input image and the evaluated illumination condition;
Wherein the illumination condition evaluation step includes the steps of:
Extracting global video signal information corresponding to a video signal of the entire input video;
Dividing the input image into image blocks of a predetermined size and extracting regional image signal information according to the image signals of the divided image blocks; And
And a lighting condition classifying step of classifying the input image into which of the predetermined lighting conditions is preliminarily performed using a classifier learned in advance based on a feature vector including the extracted global image signal information and the regional image signal information Wherein the visual flavor emphasizing method comprises the steps of:
Wherein the food object recognition step further includes a food color model determination step of determining the food color model according to the image signal information of the input image,
Wherein the food color model is a color model that includes predetermined information about the color or brightness of the image signal of the food object.
The food object recognition step may include extracting a shape feature according to a shape of an object included in the input image and a color feature according to a color included in the input image in the input image and using the extracted shape feature and color feature, And determining whether the food object is included in the input image.
Wherein the food object recognition step further includes a food recognition step of determining whether the food object is included in the input image,
Wherein the food recognition step comprises:
A low dimensional feature extraction step of extracting the shape feature and the color feature by calculating an image signal value of pixels included in the input image from the input image;
A high-dimensional feature extraction step of extracting a high-dimensional feature by encoding the extracted feature and color features; And
And classifying whether the food object is included in the input image using a classifier previously learned based on the extracted high dimensional feature.
Wherein the food object recognition step further includes a food category classification step of classifying a food category of the food object included in the input image if it is determined that the food object is included in the input image, How to emphasize visual flavor.
A low dimensional feature extraction step of extracting the shape feature and the color feature by calculating an image signal value of pixels included in the input image from the input image;
A high-dimensional feature extraction step of extracting a high-dimensional feature by encoding the extracted feature and color features; And
And classifying whether the food object included in the input image corresponds to a predetermined plurality of the food categories using a classifier previously learned based on the extracted high dimensional features. A method of emphasizing the visual flavor of images.
The low-dimensional feature extraction step extracts SIFT (Scale Invariant Feature Transform) -based features from the input image with the feature feature,
Wherein the high dimensional feature extraction step comprises Fisher's Vector encoding of the morphological feature and the color feature to obtain the high dimensional feature,
Wherein the classifying step is performed using the classifier based on a hierarchical support vector machine.
Wherein the food color model determining step analyzes the global image signal information of the input image and the regional image signal information and determines the food color model of the input image according to the analysis result. Emphasis method.
Extracting the global image signal information according to a video signal of the entire input image;
Dividing the input image into image blocks of a predetermined size and extracting the regional image signal information according to the image signals of the divided image blocks; And
And a color model classifying step of determining, based on the feature vector including the extracted global image signal information and the regional image signal information, whether the input image corresponds to the predetermined food color model using a previously learned classifier Wherein the visual flavor emphasizing method comprises the steps of:
Wherein the extracting of the global image signal information comprises: extracting a histogram according to an image signal value of pixels included in the input image as the global image signal information;
Wherein the extracting of the regional image signal information comprises extracting a histogram according to a size of an image signal value of pixels included in the image block as the regional image signal information.
Wherein the color model classifying step determines whether the input image corresponds to the predetermined food color model using the classifier based on a Gaussian mixture model or a support vector machine. Way.
Wherein the food object recognition step classifies a food category of the food object included in the input image if it is determined that the food object is included in the input image,
Wherein the food color model determination step determines the food color model using the classifier based on the feature vector further including the food category classified in the food object recognition step .
Wherein the extracting of the global image signal information includes extracting a histogram according to brightness of pixels included in the input image as the global image signal information,
Wherein the extracting of the regional image signal information comprises extracting each histogram according to color, saturation, and brightness of the pixels included in the image block as the regional image signal information.
Wherein the image correcting step corrects the image signal values of the pixels included in the input image according to the food color model and a predetermined setting according to the illumination condition
Wherein the image correction step corrects the image signal values of the pixels included in the input image by performing at least one of tone matching correction, gamma correction, and color correction according to the predetermined setting. How to emphasize flavor.
An illumination condition evaluating unit for analyzing the input image and evaluating an illumination condition of the input image; And
An image correcting unit for correcting the input image to increase a visual flavor of the input image according to a food color model according to image signal information of the input image and the evaluated illumination condition;
, Wherein the illumination condition evaluating unit
Extracting global video signal information according to a video signal of the entire input video,
Divides the input image into image blocks of a predetermined size, extracts regional image signal information according to the image signals of the divided image blocks,
Wherein the classifying unit classifies the input image into a predetermined illumination state using a classifier learned in advance based on a feature vector including the extracted global image signal information and the regional image signal information. Visual flavor emphasizing device.
Extracting a shape feature according to a shape of an object included in the input image and a color feature according to a color included in the input image in the input image, and using the extracted shape feature and color feature, A food recognition unit for determining whether an object is included;
A food category classifying unit for classifying a food category of the food object included in the input image if it is determined that the food object is included in the input image; And
The method comprising the steps of: determining the food color model according to the image signal information of the input image, analyzing the global image signal information and the regional image signal information of the input image, And a food color model determining unit for determining a color model.
Wherein the illumination condition evaluating unit analyzes the global image signal information and the local image signal information of the input image and determines the illumination condition of the input image according to the analysis result,
Wherein the image correction unit corrects an image signal value of pixels included in the input image according to the food color model and a predetermined setting according to the illumination condition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020150127809A KR101718085B1 (en) | 2015-09-09 | 2015-09-09 | Apparatus and Method of Enhancing Visual Flavor of Food Image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020150127809A KR101718085B1 (en) | 2015-09-09 | 2015-09-09 | Apparatus and Method of Enhancing Visual Flavor of Food Image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| KR20170030334A KR20170030334A (en) | 2017-03-17 |
| KR101718085B1 true KR101718085B1 (en) | 2017-03-20 |
Family
ID=58502224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020150127809A KR101718085B1 (en) | 2015-09-09 | 2015-09-09 | Apparatus and Method of Enhancing Visual Flavor of Food Image |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR101718085B1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI705412B (en) * | 2019-03-22 | 2020-09-21 | 台灣奈米碳素股份有限公司 | System for evaluating flavor of food based on its releasing gas |
| CN115798685B (en) * | 2022-12-18 | 2024-05-24 | 广西康久生物科技有限公司 | Depression diet management method based on food image segmentation |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004192129A (en) * | 2002-12-09 | 2004-07-08 | Fuji Photo Film Co Ltd | Method and device for extracting facial area |
| JP2014155167A (en) * | 2013-02-13 | 2014-08-25 | Morpho Inc | Image processing apparatus, method, and program, and recording medium |
| JP2014211748A (en) * | 2013-04-18 | 2014-11-13 | ソニー株式会社 | Information processing apparatus and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH11185034A (en) * | 1997-12-24 | 1999-07-09 | Casio Comput Co Ltd | Image data correction device and recording medium recording image data correction processing program |
| US9165398B2 (en) | 2011-02-25 | 2015-10-20 | Lg Electronics Inc. | Analysis of food items captured in digital images |
-
2015
- 2015-09-09 KR KR1020150127809A patent/KR101718085B1/en active IP Right Grant
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004192129A (en) * | 2002-12-09 | 2004-07-08 | Fuji Photo Film Co Ltd | Method and device for extracting facial area |
| JP2014155167A (en) * | 2013-02-13 | 2014-08-25 | Morpho Inc | Image processing apparatus, method, and program, and recording medium |
| JP2014211748A (en) * | 2013-04-18 | 2014-11-13 | ソニー株式会社 | Information processing apparatus and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| KR20170030334A (en) | 2017-03-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108460389B (en) | Type prediction method and device for identifying object in image and electronic equipment | |
| JP6719457B2 (en) | Method and system for extracting main subject of image | |
| AU685904B2 (en) | Produce recognition system | |
| da Costa Barbon et al. | Development of a flexible computer vision system for marbling classification | |
| TWI672638B (en) | Image processing method, non-transitory computer readable medium and image processing system | |
| US20110274314A1 (en) | Real-time clothing recognition in surveillance videos | |
| Huang et al. | Real-time classification of green coffee beans by using a convolutional neural network | |
| TWI525555B (en) | Image processing apparatus and processing method thereof | |
| Morgand et al. | Generic and real-time detection of specular reflections in images | |
| JP6529686B2 (en) | IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND PROGRAM | |
| WO2013043680A1 (en) | A classifier for use in generating a diffuse image | |
| KR20140096595A (en) | Method for image classification | |
| KR101718085B1 (en) | Apparatus and Method of Enhancing Visual Flavor of Food Image | |
| Bergman et al. | Perceptual segmentation: Combining image segmentation with object tagging | |
| Yusuf et al. | Human face detection using skin color segmentation and watershed algorithm | |
| CN113139557A (en) | Feature extraction method based on two-dimensional multivariate empirical mode decomposition | |
| JP2010186246A (en) | Image processing apparatus, method, and program | |
| CN111435427A (en) | Method and device for identifying rice and cooking appliance | |
| CN111368767A (en) | Method and device for identifying home material color tone and electronic equipment | |
| Zakiyyah et al. | Characterization and Classification of Citrus reticulata var. Keprok Batu 55 Using Image Processing and Artificial Intelligence | |
| Jaiswal et al. | Saliency based automatic image cropping using support vector machine classifier | |
| Yang et al. | Text detection in video images using adaptive edge detection and stroke width verification | |
| Chua et al. | Contour detection from deep patch-level boundary prediction | |
| Kompella et al. | Collective-reward based approach for detection of semi-transparent objects in single images | |
| Vincent et al. | Shadow detection: A review of various approaches to enhance image quality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| E701 | Decision to grant or registration of patent right | ||
| GRNT | Written decision to grant |