CN111696151A - Method and device for identifying volume of food material in oven and computer readable storage medium - Google Patents
Method and device for identifying volume of food material in oven and computer readable storage medium Download PDFInfo
- Publication number
- CN111696151A CN111696151A CN201910198846.3A CN201910198846A CN111696151A CN 111696151 A CN111696151 A CN 111696151A CN 201910198846 A CN201910198846 A CN 201910198846A CN 111696151 A CN111696151 A CN 111696151A
- Authority
- CN
- China
- Prior art keywords
- image
- food material
- current food
- dimensional image
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30128—Food products
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for identifying food volume in an oven and a computer readable storage medium, and belongs to the technical field of intelligent terminals. The method comprises the following steps: acquiring a current food material image borne by a mold in an oven through image acquisition equipment; performing transformation processing on the current food material image through a Hough transformation algorithm to generate a size image of the mold; performing segmentation processing on the current food material image through an example-level segmentation algorithm to generate a pixel area image of the current food material; obtaining a food material correction two-dimensional image according to a stored image distortion correction algorithm of the image acquisition equipment; determining three-dimensional image information of the mold and three-dimensional image information of the food material according to the corresponding relation between the two-dimensional image and the three-dimensional image information; and determining the volume of the current food material according to the three-dimensional image information of the mold and the three-dimensional image information of the food material. According to the method for identifying the volume of the food material in the oven, the volume of the food material can be identified without a weight measuring device, so that the intelligent level of the oven is improved.
Description
Technical Field
The invention relates to the technical field of intelligent terminals, in particular to a method and a device for identifying the volume of food materials in an oven and a computer-readable storage medium.
Background
The identification of the volume of food material in an oven, which is one of kitchen appliances that are frequently used, is a key issue to improve the possibility of one-touch baking.
At present, the intelligent oven does not recognize the volume of food materials, so that parameters of the food materials to be baked need to be manually selected, when an operator estimates the volume of the food materials incorrectly, the situation that the food is not baked or is scorched can occur, and therefore the intelligence level of the oven is still to be improved.
Disclosure of Invention
The embodiment of the invention provides a method and a device for identifying the volume of food materials in an oven and a computer-readable storage medium. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.
According to a first aspect of embodiments of the present invention, there is provided a method of food volume identification in an oven, comprising:
acquiring a current food material image of a current food material loaded by a mould in an oven through image acquisition equipment;
performing transformation processing on the current food material image through a Hough transformation algorithm to generate a size image of the mold;
performing segmentation processing on the current food material image through an example-level segmentation algorithm to generate a pixel area image of the current food material;
according to a stored image distortion correction algorithm of the image acquisition equipment, obtaining a mould correction two-dimensional image corresponding to the size image and a food material correction two-dimensional image corresponding to the pixel area image;
determining the three-dimensional image information of the mold corresponding to the two-dimensional image for mold correction and the three-dimensional image information of the food material corresponding to the two-dimensional image for food material correction according to the corresponding relation between the two-dimensional image and the three-dimensional image information;
and determining the volume of the current food material according to the three-dimensional image information of the mold and the three-dimensional image information of the food material.
In an embodiment of the present invention, a hough transform algorithm is used to transform a current food material image to generate a size image of a mold, including:
converting the current food material image into a gray level image, and processing through an edge detection operator to obtain a binary edge image;
carrying out Hough transform processing on the two-dimensional edge image;
screening one or more candidate peak values with peak values larger than a preset threshold value through a peak value detection function;
and generating a size image of the mold on a preset image according to the image characteristics corresponding to the one or more candidate peak values respectively.
In an embodiment of the present invention, a current food material image is segmented by an example-level segmentation algorithm to generate a pixel area image of a current food material, including:
inputting the current food material image subjected to preprocessing operation into a trained convolutional neural network to obtain a corresponding characteristic image;
determining a plurality of first candidate interested areas corresponding to each pixel point in the characteristic image;
performing binary classification and image bounding box regression processing on each first candidate region of interest input area proposal network to obtain a second candidate region of interest corresponding to each pixel point;
and generating a pixel area image of the current food material image according to the classification of the second candidate region of interest and the regression processing of the image bounding box.
In an embodiment of the present invention, before acquiring the current food material image of the current food material in the oven, the method further includes:
acquiring reference position information of at least three selected reference points on a reference image of the image acquisition equipment;
acquiring pixel position information of each first pixel point corresponding to each selected reference point on a known information shooting image of the image acquisition equipment;
and obtaining each coefficient in an image distortion correction calibration formula of the image acquisition equipment according to the reference position information and the pixel position information, and storing a corresponding image distortion correction algorithm.
In an embodiment of the present invention, the method further includes:
and according to the volume of the current food material, carrying out roasting prompt of the current food material.
According to a second aspect of the embodiments of the present invention, there is provided an apparatus for identifying a volume of food material in an oven, including:
the acquiring unit is used for acquiring a current food material image of a current food material loaded by a mould in the oven through image acquisition equipment;
the transformation unit is used for carrying out transformation processing on the current food material image through a Hough transformation algorithm to generate a size image of the mold;
the segmentation unit is used for segmenting the current food material image through an example-level segmentation algorithm to generate a pixel area image of the current food material;
the correction unit is used for correcting the two-dimensional image of the mould corresponding to the size image and the two-dimensional image of the food material corresponding to the pixel area image according to the stored image distortion correction algorithm of the image acquisition equipment;
the conversion unit is used for determining the three-dimensional image information of the die corresponding to the two-dimensional image for die correction and the three-dimensional image information of the food material corresponding to the two-dimensional image for food material correction according to the corresponding relation between the two-dimensional image and the three-dimensional image information;
and the volume identification unit is used for determining the volume of the current food material according to the three-dimensional image information of the die and the three-dimensional image information of the food material.
In an embodiment of the present invention, the transformation unit is specifically configured to:
converting the current food material image into a gray level image, and processing through an edge detection operator to obtain a binary edge image;
carrying out Hough transform processing on the two-dimensional edge image;
screening one or more candidate peak values with peak values larger than a preset threshold value through a peak value detection function;
and generating a mould size image on the preset image according to the image characteristics corresponding to the one or more candidate peak values respectively.
In an embodiment of the present invention, the segmentation unit is specifically configured to:
inputting the current food material image subjected to preprocessing operation into a trained convolutional neural network to obtain a corresponding characteristic image;
determining a plurality of first candidate interested areas corresponding to each pixel point in the characteristic image;
performing binary classification and image bounding box regression processing on each first candidate region of interest input area proposal network to obtain a second candidate region of interest corresponding to each pixel point;
and generating a pixel area image of the current food material image according to the classification of the second candidate region of interest and the regression processing of the image bounding box.
In an embodiment of the present invention, the apparatus further includes a calibration unit, configured to:
acquiring reference position information of at least three selected reference points on a reference image of the image acquisition equipment;
acquiring pixel position information of each first pixel point corresponding to each selected reference point on a known information shooting image of the image acquisition equipment;
and obtaining each coefficient in an image distortion correction calibration formula of the image acquisition equipment according to the reference position information and the pixel position information, and storing a corresponding image distortion correction algorithm.
In an embodiment of the present invention, the apparatus further includes a prompt unit, configured to:
and according to the volume of the current food material, carrying out roasting prompt of the current food material.
According to a third aspect of embodiments of the present invention, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the above-described method.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, the food material is subjected to image acquisition, the mold size image is obtained through a Hough transform algorithm, the two-dimensional image information of the food material can be determined through target segmentation and image recognition, the corresponding image conversion is carried out, the three-dimensional image information of the food material can be obtained, finally, the volume of the food material is recognized according to the length, width and thickness information carried in the three-dimensional image information, namely, the volume of the food material can be recognized through image processing, the volume of the food material can be obtained without a pressure sensor or a weight measuring device, the hardware cost is saved, the volume of the food material is recognized, data support is provided for intelligent baking of an oven, and the intelligent level of the oven can be further improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a flow chart illustrating a method of food volume identification in an oven according to an exemplary embodiment;
FIG. 2 is a block diagram illustrating a food volume identification arrangement in an oven according to an exemplary embodiment;
FIG. 3 is a block diagram illustrating a food volume identification arrangement in an oven according to yet another exemplary embodiment;
FIG. 4 is a pictorial illustration of a fiducial showing an image capture device according to an exemplary embodiment.
Detailed Description
The following description and the drawings sufficiently illustrate specific embodiments of the invention to enable those skilled in the art to practice them. The examples merely typify possible variations. Individual components and functions are optional unless explicitly required, and the sequence of operations may vary. Portions and features of some embodiments may be included in or substituted for those of others. The scope of embodiments of the invention encompasses the full ambit of the claims, as well as all available equivalents of the claims. Embodiments may be referred to herein, individually or collectively, by the term "invention" merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed. Herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed. The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the structures, products and the like disclosed by the embodiments, the description is relatively simple because the structures, the products and the like correspond to the parts disclosed by the embodiments, and the relevant parts can be just described by referring to the method part.
In the embodiment of the invention, the oven can acquire the images of the food materials, the size images of the mold are obtained through the Hough transform algorithm, the two-dimensional image information of the food materials can be determined through target segmentation and image recognition, the corresponding image conversion is carried out, the three-dimensional image information of the food materials can be obtained, finally, the volume of the food materials is recognized according to the length, width and thickness information carried in the three-dimensional image information, namely, the volume of the food materials can be recognized through image processing, the volume of the food materials can be obtained without a pressure sensor or a weight measuring device, the hardware cost is saved, in addition, the volume of the food materials is recognized, data support is provided for intelligent baking of the oven, and the intelligent level of the oven can be further improved.
Fig. 1 is a flow chart illustrating a method of food volume identification in an oven according to an exemplary embodiment. As shown in fig. 1, the process of identifying the volume of food material in the oven includes:
s1, acquiring a current food material image of the current food material loaded by a mold in the oven through image acquisition equipment;
optionally, the oven is configured with an image acquisition device, so that the current food material image of the current food material in the oven can be acquired through the image acquisition device, and therefore the oven can identify the food material according to the current food material image.
Here, the image capturing device is not specifically limited, and may be a camera installed in the oven cavity, and the camera acquires a current food material image of a current food material in the oven.
Optionally, before acquiring the current food material image of the current food material in the oven, the method further includes: acquiring reference position information of at least three selected reference points on a reference image of the image acquisition equipment; acquiring pixel position information of each first pixel point corresponding to each selected reference point on a known information shooting image of the image acquisition equipment; and obtaining each coefficient in an image distortion correction calibration formula of the image acquisition equipment according to the reference position information and the pixel position information, and storing a corresponding image distortion correction algorithm.
S2, performing transformation processing on the current food material image through a Hough transformation algorithm to generate a size image of the mold;
in the embodiment of the present invention, a hough transform may be specifically adopted to process the current food material image, where the hough transform is an image processing method through feature detection. Feature detection refers to using a computer to extract image information and determine whether each image point belongs to an image feature, and the result of feature detection is to divide the image points into different subsets, which often belong to isolated points, connected curves or continuous regions.
Optionally, the specific process may further include inputting the current food material image obtained by the image acquisition device into the transformation unit 220, and performing transformation processing on the current food material image through a hough transformation algorithm to generate a size image of the mold, including: converting the current food material image into a gray level image, and processing through an edge detection operator to obtain a binary edge image; then, carrying out Hough transform processing on the two-dimensional edge image; further screening one or more candidate peak values with peak values larger than a preset threshold value through a peak value detection function; and finally, generating a size image of the mold on a preset image according to the image characteristics corresponding to the one or more candidate peak values respectively.
S3, performing segmentation processing on the current food material image through an example-level segmentation algorithm to generate a pixel area image of the current food material;
optionally, the present invention may specifically adopt an example-level segmentation algorithm to perform segmentation processing on the current food material image acquired by the image acquisition device.
In the embodiment of the invention, the food material image can be specifically segmented by adopting an example-level segmentation algorithm to generate the pixel area image of the current food material. The example-level segmentation means that the terminal can automatically frame different examples from an image by using a target detection method and then mark the different example areas pixel by using a semantic segmentation method. Whereas semantic segmentation does not distinguish different instances belonging to the same class. For example, when there are two chickens in the image, the semantic segmentation predicts all pixels of the whole of the two chickens as the category of "chickens". In contrast, example-level segmentation requires distinguishing which pixels belong to a first chicken and which pixels belong to a second chicken. It can be seen that employing example-level segmentation can further improve the accuracy of image recognition.
Optionally, the segmenting the current food material image by an example-level segmentation algorithm to generate a pixel area image of the current food material includes: inputting the current food material image subjected to preprocessing operation into a trained convolutional neural network to obtain a corresponding characteristic image; determining a plurality of first candidate interested areas corresponding to each pixel point in the characteristic image; performing binary classification and image bounding box regression processing on each first candidate region of interest input area proposal network to obtain a second candidate region of interest corresponding to each pixel point; and generating a pixel area image of the current food material image according to the classification of the second candidate region of interest and the regression processing of the image bounding box.
Optionally, the specific process may further include: inputting the current food material image subjected to preprocessing operation into a trained convolutional neural network to obtain a corresponding characteristic image; determining a plurality of first candidate interested areas corresponding to each pixel point in the characteristic image; performing binary classification and image bounding box regression processing on each first candidate region of interest input area proposal network to obtain a second candidate region of interest corresponding to each pixel point; and aligning the current food material image and the characteristic image according to the second candidate interest region, identifying the type of the current food material, and generating a pixel area image of the current food material according to the classification of the second candidate interest region and the regression processing of the image bounding box.
S4, obtaining a mould correction two-dimensional image corresponding to the size image and a food material correction two-dimensional image corresponding to the pixel area image according to the saved image distortion correction algorithm of the image acquisition equipment;
the distortion correction refers to lens distortion generated by inherent lens distortion of an optical lens, and a corrected two-dimensional image acquired by image acquisition equipment is obtained by using a distortion coefficient of a lens of the image acquisition equipment and through a distortion correction calibration formula.
Here, each image capturing device corresponds to a fixed image distortion coefficient, that is, a fixed image distortion correction algorithm.
Optionally, the image distortion coefficient may be obtained in advance by an image acquisition device configured in the oven, and the image distortion correction algorithm of the image acquisition device is stored, so that after the current food material image, the mold size image generated after the hough transform processing and the pixel area image of the current food material after the instance-level segmentation processing are respectively subjected to the image distortion correction algorithm, the corresponding mold correction two-dimensional image and the corresponding food material correction two-dimensional image are obtained.
Optionally, on the one hand, the current food material image acquired by the image acquisition device is processed by a hough transform algorithm to generate a mold size image, and then the mold corrected two-dimensional image is obtained by an image distortion correction algorithm.
Optionally, on the other hand, the current food material image acquired by the image acquisition device is subjected to instance-level segmentation processing to generate a pixel area image of the current food material, so that the position information of each pixel point can be determined, and then a food material correction two-dimensional image, namely a two-dimensional image carrying the actual size information of the image, is obtained through an image distortion correction algorithm, wherein the size information can be the length, the width and the like of the two-dimensional image.
In the embodiment of the invention, the image distortion correction algorithm of the image acquisition equipment in the oven needs to be stored in advance. Through an image distortion correction algorithm, the position information of each pixel point in the food material pixel area image acquired by the image acquisition equipment can be corresponding to the position information of each point in an actual scene.
S5, determining the three-dimensional image information of the die corresponding to the two-dimensional image for die correction and the three-dimensional image information of the food corresponding to the two-dimensional image for food correction according to the corresponding relation between the two-dimensional image and the three-dimensional image information;
the two-dimensional die correction image and the two-dimensional food correction image are both two-dimensional images carrying image size information, and the actual volume of the food material cannot be determined sufficiently. Therefore, the mold three-dimensional image information and the food material three-dimensional image information are determined according to the obtained mold correction two-dimensional image and food material correction two-dimensional image information and the corresponding relationship between the two-dimensional image and the three-dimensional image information.
Optionally, the correspondence between the two-dimensional image and the three-dimensional image information may be configured in advance, and since each food material generally baked by using an oven basically has a relatively fixed appearance characteristic, a database containing the correspondence between the two-dimensional image and the three-dimensional image of the food material frequently encountered by the oven can be obtained through a large amount of experimental data, and then the database is configured in advance in the program for identifying the two-dimensional image by the oven and is stored, so that the correspondence between the two-dimensional image and the three-dimensional image information is obtained. And the possibility of determining the three-dimensional image information of the mold corresponding to the two-dimensional image for mold correction and the three-dimensional image information of the food material corresponding to the two-dimensional image for food material correction is also provided.
And S6, determining the volume of the current food material according to the three-dimensional image information of the die and the three-dimensional image information of the food material.
Optionally, a database of correspondence between two-dimensional images and three-dimensional images of food materials frequently encountered by the oven obtained through a large amount of experimental data also contains correspondence information between three-dimensional image information and an object volume corresponding to the three-dimensional image information, so that the volume of the current food material can be further determined according to the three-dimensional image information of the mold and the three-dimensional image information of the food materials.
Optionally, in the process of determining the three-dimensional image information of the mold and the three-dimensional image information of the current food material according to the corresponding relationship between the two-dimensional image and the three-dimensional image information, the area of the two-dimensional image is obtained according to the length and width information carried in the corrected two-dimensional image, and the volume of the current food material is estimated according to the corresponding relationship between the area of the preset two-dimensional image and the thickness of the corresponding three-dimensional image, and the obtained length information, width information and thickness information of the three-dimensional image. Therefore, without adding any hardware facilities, the volume of the current food material can be finally determined through processing such as transformation, segmentation, correction and the like according to the image information of the current food material acquired by the image acquisition equipment.
Optionally, according to the volume of the current food material, a toasting prompt of the current food material is performed. The volume of eating the material and the corresponding data of baking temperature and baking time can be configured in advance for the oven, so that the operation prompts such as baking temperature and baking time can be given according to the determined volume of the current food material in the oven, namely, the baking prompt of the current food material is realized, the data support is provided for the intelligent operation of the oven through the recognition of the volume of the food material in the oven, the intelligent level of the oven is further improved, and the possibility is provided for one-key baking.
Fig. 2 is a block diagram illustrating a food volume identification apparatus in an oven, according to an exemplary embodiment. As shown in fig. 2, the apparatus includes: an acquisition unit 210, a transformation unit 220, a segmentation unit 230, a rectification unit 240, a transformation unit 250 and a volume identification unit 260, wherein,
an obtaining unit 210, configured to obtain, through an image acquisition device, a current food material image of a current food material borne by a mold in an oven;
the transformation unit 220 is configured to transform the current food material image through a hough transformation algorithm to generate a size image of the mold;
the segmentation unit 230 is configured to perform segmentation processing on the current food material image through an instance-level segmentation algorithm to generate a pixel area image of the current food material;
the correcting unit 240 is configured to correct the two-dimensional image for the mold corresponding to the size image and correct the two-dimensional image for the food material corresponding to the pixel area image according to the stored image distortion correction algorithm of the image acquisition device;
a conversion unit 250, configured to determine, according to a correspondence between the two-dimensional image and the three-dimensional image information, mold three-dimensional image information corresponding to the mold corrected two-dimensional image and food material three-dimensional image information corresponding to the food material corrected two-dimensional image;
and the volume identification unit 260 is used for determining the volume of the current food material according to the three-dimensional image information of the mold and the three-dimensional image information of the food material.
In an embodiment of the present invention, the transforming unit 220 is specifically configured to:
converting the current food material image into a gray level image, and processing through an edge detection operator to obtain a binary edge image;
carrying out Hough transform processing on the two-dimensional edge image;
screening one or more candidate peak values with peak values larger than a preset threshold value through a peak value detection function;
and generating a mould size image on the preset image according to the image characteristics corresponding to the one or more candidate peak values respectively.
In an embodiment of the present invention, the dividing unit 230 is specifically configured to:
inputting the current food material image subjected to preprocessing operation into a trained convolutional neural network to obtain a corresponding characteristic image;
determining a plurality of first candidate interested areas corresponding to each pixel point in the characteristic image;
performing binary classification and image bounding box regression processing on each first candidate region of interest input area proposal network to obtain a second candidate region of interest corresponding to each pixel point;
and generating a pixel area image of the current food material image according to the classification of the second candidate region of interest and the regression processing of the image bounding box.
Fig. 3 is a block diagram illustrating a food volume identification apparatus in an oven according to yet another exemplary embodiment. As shown in fig. 3, the apparatus includes: the obtaining unit 210, the transforming unit 220, the segmenting unit 230, the correcting unit 240, the transforming unit 250 and the volume recognizing unit 260 further include: a calibration unit 270 and a prompt unit 280, wherein,
a calibration unit 270, configured to obtain reference position information of at least three selected reference points on a reference map of the image capturing device;
the prompting unit 280 is configured to prompt baking of the current food material according to the volume of the current food material.
In an embodiment of the present invention, the calibration unit 270 is specifically configured to:
acquiring reference position information of at least three selected reference points on a reference image of the image acquisition equipment;
acquiring pixel position information of each first pixel point corresponding to each selected reference point on a known information shooting image of the image acquisition equipment;
and obtaining each coefficient in an image distortion correction calibration formula of the image acquisition equipment according to the reference position information and the pixel position information, and storing a corresponding image distortion correction algorithm.
FIG. 4 is a pictorial illustration of a fiducial showing an image capture device according to an exemplary embodiment. A reference map taken by the image capture device may be as shown in fig. 2. Finding three points (r) from the reference map1，s1)(r2，s2)(r3，s3) The coordinates of three corresponding points on the real scene shooting picture are (x)1，y1)(x2，y2)(x3，y3) The coordinates of the three pairs of points are substituted into the formula (1)
x′＝a00+a10x+a01y
y′＝b00+b10x+b01y-------------------------------------(1)
Equation (1) is a linear equation, where x and y are coordinates of a point on the captured image, respectively, and x 'and y' are coordinates of a corresponding point in the actual scene. And the matrix form corresponding to the formula (1) is the formulas (2) and (3):
by solving simultaneous equations or matrix inversion, each coefficient a can be obtainedij,bijI.e. the correction parameter, i-0, 1, or 2, and j-0, 1, or 2. Thus, through the coordinate points of the three pairs of pixels, the image distortion correction calibration formula can be determined.
Of course, a quadratic equation can also be used as an image distortion correction calibration formula, so that the distortion formula can be a binary quadratic polynomial and can be used for describing the relationship between the coordinate point (x, y) of the shot image and the coordinate (x ', y') of the corresponding point of the actual scene, and the mathematical expression is formula (4):
since the formula (4) includes 12 unknown correction parameters, it is necessary to determine 12 unknown correction parameters by solving a simultaneous equation or matrix inversion for 6 pixel coordinate points. Through a large number of experiments, coordinates of 6 pairs of known pixel points can be determined, and therefore an image distortion correction calibration formula is determined.
Therefore, the image distortion correction calibration formula of the image acquisition device is related to hardware and can be predetermined, and comprises the following steps: acquiring reference position information of at least three selected reference points on a reference image of the image acquisition equipment; acquiring pixel position information of each first pixel point corresponding to each selected reference point on a known information shooting image of the image acquisition equipment; and obtaining each coefficient in an image distortion correction calibration formula of the image acquisition equipment according to the reference position information and the pixel position information, and storing the corresponding image distortion correction calibration formula.
In the embodiment of the invention, the current correction two-dimensional image which is corresponding to the current mask image and carries the length and width information can be obtained by adopting the formula (1) or the formula (4) no matter the image distortion correction calibration formula. Preferably, the current mask image may include coordinates (x, y) of each pixel point; determining the actual scene midpoint coordinate (x ', y') corresponding to each pixel point coordinate (x, y) according to a formula (4); according to the point coordinates (x ', y') in each actual scene, a current correction two-dimensional image which corresponds to the current mask image and carries length and width information can be obtained;
wherein, aij，bijFor the correction parameters, i is 0, 1, or 2, and j is 0, 1, or 2.
The following sets the operation flow of the scheme into specific embodiments to illustrate the method provided by the embodiments of the present disclosure.
In this embodiment, the image distortion correction calibration formula shown in formula (4) has been previously stored in the oven, and the corresponding relationship between the two-dimensional image and the three-dimensional image information of the food material has also been previously stored.
In an embodiment of the present invention, a device for identifying a volume of food material in an oven is provided, where the device is used in an oven, and includes:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to:
acquiring a current food material image of a current food material loaded by a mould in an oven through image acquisition equipment;
performing transformation processing on the current food material image through a Hough transformation algorithm to generate a size image of the mold;
performing segmentation processing on the current food material image through an example-level segmentation algorithm to generate a pixel area image of the current food material;
according to a stored image distortion correction algorithm of the image acquisition equipment, obtaining a mould correction two-dimensional image corresponding to the size image and a food material correction two-dimensional image corresponding to the pixel area image;
determining the three-dimensional image information of the mold corresponding to the two-dimensional image for mold correction and the three-dimensional image information of the food material corresponding to the two-dimensional image for food material correction according to the corresponding relation between the two-dimensional image and the three-dimensional image information;
and determining the volume of the current food material according to the three-dimensional image information of the mold and the three-dimensional image information of the food material.
In one embodiment of the present invention, a computer-readable storage medium is provided, having stored thereon computer instructions, which when executed by a processor, implement the steps of the above-described method.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It is to be understood that the present invention is not limited to the procedures and structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (11)
1. A method of food volume identification in an oven, comprising:
acquiring a current food material image of a current food material loaded by a mould in an oven through image acquisition equipment;
performing transformation processing on the current food material image through a Hough transformation algorithm to generate a size image of the mold;
performing segmentation processing on the current food material image through an example-level segmentation algorithm to generate a pixel area image of the current food material;
obtaining a mould correction two-dimensional image corresponding to the size image and a food correction two-dimensional image corresponding to the pixel area image according to a stored image distortion correction algorithm of the image acquisition equipment;
determining the three-dimensional image information of the mold corresponding to the two-dimensional image for mold correction and the three-dimensional image information of the food material corresponding to the two-dimensional image for food material correction according to the corresponding relation between the two-dimensional image and the three-dimensional image information;
and determining the volume of the current food material according to the three-dimensional image information of the mold and the three-dimensional image information of the food material.
2. The method of claim 1, wherein the transforming the current food material image through a Hough transform algorithm to generate a size image of the mold comprises:
converting the current food material image into a gray level image, and processing through an edge detection operator to obtain a binary edge image;
carrying out Hough transform processing on the two-dimensional edge image;
screening one or more candidate peak values with peak values larger than a preset threshold value through a peak value detection function;
and generating the size image of the mold on a preset image according to the image characteristics corresponding to the one or more candidate peak values respectively.
3. The method of claim 1, wherein the segmenting the current food material image through an example-level segmentation algorithm to generate a pixel area image of the current food material comprises:
inputting the current food material image subjected to preprocessing operation into a trained convolutional neural network to obtain a corresponding characteristic image;
determining a plurality of first candidate interested areas corresponding to each pixel point in the characteristic image;
performing binary classification and image bounding box regression processing on each first candidate region of interest input area proposal network to obtain a second candidate region of interest corresponding to each pixel point;
and generating a pixel area image of the current food material image according to the classification of the second candidate region of interest and image bounding box regression processing.
4. The method of claim 1, wherein prior to obtaining the image of the current food material in the oven, further comprising:
acquiring reference position information of at least three selected reference points on a reference image of the image acquisition equipment;
acquiring pixel position information of each first pixel point corresponding to each selected reference point on a known information shooting image of the image acquisition equipment;
and obtaining each coefficient in an image distortion correction calibration formula of the image acquisition equipment according to the reference position information and the pixel position information, and storing a corresponding image distortion correction algorithm.
5. The method of claim 1, further comprising:
and according to the volume of the current food material, carrying out roasting prompt of the current food material.
6. An apparatus for food volume identification in an oven, comprising:
the acquiring unit is used for acquiring a current food material image of a current food material loaded by a mould in the oven through image acquisition equipment;
the transformation unit is used for carrying out transformation processing on the current food material image through a Hough transformation algorithm to generate a size image of the mold;
the segmentation unit is used for carrying out segmentation processing on the current food material image through an example-level segmentation algorithm to generate a pixel area image of the current food material;
the correction unit is used for correcting a two-dimensional image of the mould corresponding to the size image and a two-dimensional image of the food material corresponding to the pixel area image according to the saved image distortion correction algorithm of the image acquisition equipment;
the conversion unit is used for determining the three-dimensional image information of the die corresponding to the two-dimensional image for die correction and the three-dimensional image information of the food material corresponding to the two-dimensional image for food material correction according to the corresponding relation between the two-dimensional image and the three-dimensional image information;
and the volume identification unit is used for determining the volume of the current food material according to the mould three-dimensional image information and the food material three-dimensional image information.
7. The apparatus according to claim 6, wherein the transformation unit is specifically configured to:
converting the current food material image into a gray level image, and processing through an edge detection operator to obtain a binary edge image;
carrying out Hough transform processing on the two-dimensional edge image;
screening one or more candidate peak values with peak values larger than a preset threshold value through a peak value detection function;
and generating a size image of the mold on a preset image according to the image characteristics corresponding to the one or more candidate peak values respectively.
8. The apparatus according to claim 6, wherein the segmentation unit is specifically configured to:
inputting the current food material image subjected to preprocessing operation into a trained convolutional neural network to obtain a corresponding characteristic image;
determining a plurality of first candidate interested areas corresponding to each pixel point in the characteristic image;
performing binary classification and image bounding box regression processing on each first candidate region of interest input area proposal network to obtain a second candidate region of interest corresponding to each pixel point;
and generating a pixel area image of the current food material image according to the classification of the second candidate region of interest and image bounding box regression processing.
9. The apparatus according to claim 6, further comprising a calibration unit for:
acquiring reference position information of at least three selected reference points on a reference image of the image acquisition equipment;
acquiring pixel position information of each first pixel point corresponding to each selected reference point on a known information shooting image of the image acquisition equipment;
and obtaining each coefficient in an image distortion correction calibration formula of the image acquisition equipment according to the reference position information and the pixel position information, and storing a corresponding image distortion correction algorithm.
10. The apparatus of claim 6, further comprising a prompting unit configured to:
and according to the volume of the current food material, carrying out roasting prompt of the current food material.
11. A computer-readable storage medium having stored thereon computer instructions, characterized in that the instructions, when executed by a processor, implement the steps of the method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910198846.3A CN111696151A (en) | 2019-03-15 | 2019-03-15 | Method and device for identifying volume of food material in oven and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910198846.3A CN111696151A (en) | 2019-03-15 | 2019-03-15 | Method and device for identifying volume of food material in oven and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111696151A true CN111696151A (en) | 2020-09-22 |
Family
ID=72475319
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910198846.3A Pending CN111696151A (en) | 2019-03-15 | 2019-03-15 | Method and device for identifying volume of food material in oven and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111696151A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113213053A (en) * | 2021-05-12 | 2021-08-06 | 深圳市海柔创新科技有限公司 | Empty box warehousing method, system, equipment, electronic equipment and readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120135384A1 (en) * | 2010-11-26 | 2012-05-31 | Terumo Kabushiki Kaisha | Portable terminal, calorie estimation method, and calorie estimation program |
| CN107527324A (en) * | 2017-07-13 | 2017-12-29 | 江苏泽景汽车电子股份有限公司 | A kind of pattern distortion antidote of HUD |
| CN108198188A (en) * | 2017-12-28 | 2018-06-22 | 北京奇虎科技有限公司 | Food nutrition analysis method, device and computing device based on picture |
| CN108597582A (en) * | 2018-04-18 | 2018-09-28 | 中国科学院计算技术研究所 | A kind of method and apparatus for executing Faster R-CNN neural network computings |
| CN108846314A (en) * | 2018-05-08 | 2018-11-20 | 天津大学 | A kind of food materials identification system and food materials discrimination method based on deep learning |
| CN109064509A (en) * | 2018-06-29 | 2018-12-21 | 广州雅特智能科技有限公司 | The recognition methods of food volume and fuel value of food, device and system |
| CN109166146A (en) * | 2018-07-19 | 2019-01-08 | 上海斐讯数据通信技术有限公司 | A kind of food volume estimation method and system based on IMU and vision |
-
2019
- 2019-03-15 CN CN201910198846.3A patent/CN111696151A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120135384A1 (en) * | 2010-11-26 | 2012-05-31 | Terumo Kabushiki Kaisha | Portable terminal, calorie estimation method, and calorie estimation program |
| CN102565056A (en) * | 2010-11-26 | 2012-07-11 | 泰尔茂株式会社 | Portable terminal, calorie estimation method, and calorie estimation program |
| CN107527324A (en) * | 2017-07-13 | 2017-12-29 | 江苏泽景汽车电子股份有限公司 | A kind of pattern distortion antidote of HUD |
| CN108198188A (en) * | 2017-12-28 | 2018-06-22 | 北京奇虎科技有限公司 | Food nutrition analysis method, device and computing device based on picture |
| CN108597582A (en) * | 2018-04-18 | 2018-09-28 | 中国科学院计算技术研究所 | A kind of method and apparatus for executing Faster R-CNN neural network computings |
| CN108846314A (en) * | 2018-05-08 | 2018-11-20 | 天津大学 | A kind of food materials identification system and food materials discrimination method based on deep learning |
| CN109064509A (en) * | 2018-06-29 | 2018-12-21 | 广州雅特智能科技有限公司 | The recognition methods of food volume and fuel value of food, device and system |
| CN109166146A (en) * | 2018-07-19 | 2019-01-08 | 上海斐讯数据通信技术有限公司 | A kind of food volume estimation method and system based on IMU and vision |
Non-Patent Citations (1)
| Title |
|---|
| 贾永红, 北京理工大学出版社 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113213053A (en) * | 2021-05-12 | 2021-08-06 | 深圳市海柔创新科技有限公司 | Empty box warehousing method, system, equipment, electronic equipment and readable storage medium |
| CN113213053B (en) * | 2021-05-12 | 2023-02-28 | 深圳市海柔创新科技有限公司 | Empty box warehousing method, system, equipment, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3327668B1 (en) | Image-processing device and method for foreground mask correction for object segmentation | |
| US9135522B2 (en) | System and method for identifying scale invariant features of object outlines on images | |
| CN110546651B (en) | Method, system and computer readable medium for identifying objects | |
| CN109523583B (en) | Infrared and visible light image registration method for power equipment based on feedback mechanism | |
| CN107423306B (en) | Image retrieval method and device | |
| JPWO2016104712A1 (en) | Image processing apparatus, image processing method, and program | |
| CN116229189B (en) | Image processing method, device, equipment and storage medium based on fluorescence endoscope | |
| CN111914762A (en) | Gait information-based identity recognition method and device | |
| US20080226159A1 (en) | Method and System For Calculating Depth Information of Object in Image | |
| CN112766045A (en) | Scene change detection method, system, electronic device and storage medium | |
| CN111696151A (en) | Method and device for identifying volume of food material in oven and computer readable storage medium | |
| CN112991159B (en) | Face illumination quality evaluation method, system, server and computer readable medium | |
| Liu et al. | Splicing forgery exposure in digital image by detecting noise discrepancies | |
| CN110599514A (en) | Image segmentation method and device, electronic equipment and storage medium | |
| CN111611993A (en) | Method and device for identifying volume of food in refrigerator and computer storage medium | |
| JP7448035B2 (en) | Information processing device, three-dimensional position estimation method, and program | |
| JP6692321B2 (en) | Geometric fitting device, method, and program | |
| CN110991434B (en) | Self-service terminal certificate identification method and device | |
| CN111627069A (en) | Dot detection method, terminal and computer-readable storage medium | |
| CN107680083B (en) | Parallax determination method and parallax determination device | |
| CN114358131A (en) | Digital photo frame intelligent photo optimization processing system | |
| CN110874814A (en) | Image processing method, image processing device and terminal equipment | |
| CN113837106A (en) | Face recognition method, face recognition system, electronic equipment and storage medium | |
| CN106886791A (en) | Fat location recognition methods in a kind of two-dimensional ct picture based on condition random field | |
| CN117057377B (en) | Code identification matching method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |