CN111597862A - Dish category identification method and device and electronic equipment - Google Patents
Dish category identification method and device and electronic equipment Download PDFInfo
- Publication number
- CN111597862A CN111597862A CN201910128874.8A CN201910128874A CN111597862A CN 111597862 A CN111597862 A CN 111597862A CN 201910128874 A CN201910128874 A CN 201910128874A CN 111597862 A CN111597862 A CN 111597862A
- Authority
- CN
- China
- Prior art keywords
- dish
- target image
- dishes
- appearance
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a method and a device for identifying dish types and electronic equipment, wherein the method for identifying the dish types comprises the following steps: acquiring a target image containing a dish area; performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image; determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from appearance characteristic vectors of various pre-stored dishes; and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image. Therefore, by the technical scheme provided by the embodiment of the invention, the time consumed by dish type identification can be shortened, and the efficiency of dish type identification is improved.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a method and a device for identifying a dish category and electronic equipment.
Background
In the catering industry, there is a need to determine the dish category of dishes. For example, after a user takes a dish, the electronic device needs to identify a dish type of the dish taken by the user, and then can determine a price corresponding to the dish type, so that the user can pay for the dish taken by the user.
In the related art, the electronic device generally recognizes a barcode attached to a dish and acquires a dish category on the barcode.
However, in the related art, the process of identifying the dish type is complicated, and thus, the efficiency of identifying the dish type is low. Specifically, when identifying the type of the dish, the staff is required to find the bar code adhered to the dish, and the electronic device aligns with the bar code and identifies the bar code, so as to obtain the type of the dish. Because the bar code pasted on the dish is usually small, a worker needs to spend a certain time for searching the bar code pasted on the dish, the electronic equipment needs to find the bar code accurately, and the electronic equipment needs to spend a certain time for aligning the bar code, so that the time consumed for identifying the dish type is long, and the efficiency for identifying the dish type is low.
Disclosure of Invention
Embodiments of the present invention provide a method, an apparatus, and an electronic device for identifying a category of dishes, so as to shorten time consumed by identifying the category of the dishes and improve efficiency of identifying the category of the dishes. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a method for identifying a category of dishes, where the method includes:
acquiring a target image containing a dish area;
performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image;
determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from appearance characteristic vectors of various pre-stored dishes;
and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image.
Optionally, the step of performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image includes:
based on a pre-trained first convolution neural network, performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image, wherein the first convolution neural network is as follows: the method comprises the steps that training is carried out on the basis of a plurality of sample images and appearance characteristic vectors corresponding to the sample images, wherein the sample images comprise dish areas, and the appearance characteristic vector corresponding to each sample image is the appearance characteristic vector of dishes contained in the sample image.
Optionally, the step of obtaining the target image including the dish area includes:
monitoring whether a dinner plate for containing dishes exists in a preset area or not;
and when the dinner plate for containing dishes exists in the preset area, acquiring a target image of the area containing the dishes.
Optionally, after the step of acquiring the target image of the area containing the dishes when the dinner plate containing the dishes is monitored to exist in the preset area, the method further includes:
inputting the target image into a pre-trained second convolutional neural network to obtain a dish area contained in the target image, wherein the second convolutional neural network is as follows: the method comprises the steps that the method is obtained through training based on a plurality of sample images and dish areas contained in the sample images;
correspondingly, the step of performing appearance feature recognition on the target image based on the pre-trained first convolution neural network to obtain a target appearance feature vector of dishes contained in the target image includes:
and identifying the appearance characteristics of the dish area based on a pre-trained first convolution neural network to obtain a target appearance characteristic vector of the dishes contained in the dish area.
Optionally, the appearance characteristics of the dish comprise at least one of the following characteristics: color, texture and shape.
In a second aspect, an embodiment of the present invention provides an apparatus for identifying a category of dishes, where the apparatus includes:
the image acquisition module is used for acquiring a target image containing a dish area;
the characteristic identification module is used for identifying the appearance characteristic of the target image to obtain a target appearance characteristic vector of dishes contained in the target image;
the characteristic vector determining module is used for determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from appearance characteristic vectors of various pre-stored dishes;
and the dish type determining module is used for determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image.
Optionally, the feature vector determining module is specifically configured to:
based on a pre-trained first convolution neural network, performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image, wherein the first convolution neural network is as follows: the method comprises the steps that training is carried out on the basis of a plurality of sample images and appearance characteristic vectors corresponding to the sample images, wherein the sample images comprise dish areas, and the appearance characteristic vector corresponding to each sample image is the appearance characteristic vector of dishes contained in the sample image.
Optionally, the image obtaining module is specifically configured to:
monitoring whether a dinner plate for containing dishes exists in a preset area or not;
and when the dinner plate for containing dishes exists in the preset area, acquiring a target image of the area containing the dishes.
Optionally, the apparatus further comprises:
a dish area determining module, configured to, after the step of obtaining a target image including a dish area when it is monitored that a dinner plate containing dishes exists in the preset area, input the target image to a second convolutional neural network trained in advance to obtain the dish area included in the target image, where the second convolutional neural network is: the method comprises the steps that the method is obtained through training based on a plurality of sample images and dish areas contained in the sample images;
accordingly, the feature vector determination module is specifically configured to:
and identifying the appearance characteristics of the dish area based on a pre-trained first convolution neural network to obtain a target appearance characteristic vector of the dishes contained in the dish area.
Optionally, the appearance characteristics of the dish comprise at least one of the following characteristics: color, texture and shape.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;
a memory for storing a computer program;
and a processor configured to implement the dish type identification method according to the first aspect when executing the program stored in the memory.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method for identifying a category of dishes according to the first aspect is implemented.
According to the technical scheme provided by the embodiment of the invention, when the category of the dishes is identified, the target image containing the dish area is obtained; performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image; and determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from the appearance characteristic vectors of various pre-stored dishes, and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image. Therefore, according to the technical scheme provided by the embodiment of the invention, when the type of the dish is determined, only the target image including the dish area needs to be obtained, and unlike the related technology, a worker needs to search the bar code adhered to the dish and the electronic equipment needs to align to the bar code, so that the time consumed by dish type identification is shortened, the efficiency of dish type identification is improved, and the accuracy of dish type identification can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a method for identifying a category of dishes according to an embodiment of the present invention;
fig. 2 is a flowchart of another dish type identification method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a device for identifying a category of dishes according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the catering industry, there is a need to determine the dish category of dishes. For example, after a user takes a dish, the electronic device needs to identify a dish type of the dish taken by the user, and then can determine a price corresponding to the dish type, so that the user can pay for the dish taken by the user.
In the related art, the electronic device usually identifies the dish type through the RFID technology, but in the related art, in the process of identifying the dish type, a worker needs to find a barcode attached to the dish, and the electronic device needs to align with the barcode. The time required for the staff to search for the barcode pasted on the dish and the electronic device to align with the barcode is long, so that the time consumed for the electronic device to identify the dish type is long, and the efficiency of identifying the dish type is low.
In order to shorten the time consumed by dish type identification and improve the efficiency of dish type identification, the embodiment of the invention provides a dish type identification method and device and electronic equipment.
In the first aspect, a method for identifying a category of dishes according to an embodiment of the present invention is described below.
It should be noted that an execution subject of the method for identifying a category of dishes provided in the embodiment of the present invention may be a category of dishes identification apparatus, and the category of dishes identification apparatus may be run in an electronic device, where the electronic device may be an image capture device, such as a camera, the image capture device may be configured to obtain a target image including a dish area, and the electronic device may also be a background server communicatively connected to the camera, and the embodiment of the present invention does not limit the electronic device.
As shown in fig. 1, a method for identifying a category of dishes provided by an embodiment of the present invention may include the following steps:
and S110, acquiring a target image containing the dish area.
In identifying the category of the dishes, the image capturing apparatus may capture a target image including the area of the dishes, so that the electronic apparatus as the execution subject may acquire the target image including the area of the dishes.
For example, a camera supported by a bracket may be disposed on a checkout counter of a restaurant. When a dinner plate containing dishes is placed on the settlement table top, the camera can acquire a target image containing the dish area, so that the electronic equipment serving as the execution main body can acquire the target image containing the dish area.
In one embodiment, the step of acquiring the target image including the dish area may include:
monitoring whether a dinner plate for containing dishes exists in a preset area or not;
when the dinner plate for containing dishes exists in the preset area, a target image of the area containing the dishes is obtained.
In this embodiment, the electronic device can monitor whether a dinner plate for holding dishes exists in a preset area in real time, wherein the preset area can be a settlement table top; and when the dinner plate for holding dishes in the preset area is monitored, the dishes of the dish type to be identified exist, so that the target image of the area containing the dishes is obtained.
It should be noted that the preset area may be a settlement table, or may be another area, and the size of the preset area are not specifically limited in the embodiment of the present invention.
And S120, identifying the appearance characteristics of the target image to obtain a target appearance characteristic vector of the dishes contained in the target image.
After the electronic device serving as the execution subject obtains the target image, the electronic device may perform appearance feature recognition on the target image to obtain a target appearance feature vector of the dishes included in the target image. The mode of performing appearance feature recognition on the target image to obtain the target appearance feature vector may be various, for example, appearance feature recognition may be performed by using a pre-trained convolutional neural network to further obtain the target appearance feature vector. The embodiment of the invention identifies the appearance characteristic of the target image, and the specific implementation mode of obtaining the target appearance characteristic vector is not limited specifically.
In one embodiment, the step of performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes included in the target image may include:
based on a pre-trained first convolution neural network, performing appearance feature recognition on a target image to obtain a target appearance feature vector of dishes contained in the target image, wherein the first convolution neural network is as follows: the dish region training method comprises the steps that training is carried out on the basis of a plurality of sample images and appearance feature vectors corresponding to the sample images, wherein the sample images comprise dish regions, and the appearance feature vector corresponding to each sample image is the appearance feature vector of dishes contained in the sample image.
In this embodiment, before performing the appearance feature recognition on the target image, the first convolutional neural network may be trained based on appearance feature vectors corresponding to a plurality of sample images including the dish region and the plurality of sample images. The plurality of sample images including the dish area may be: the image acquisition equipment is arranged on the settlement table top and supported by the bracket. Therefore, when the appearance feature of the target image is identified, the trained first convolution neural network can be used for identifying the appearance feature of the target image, so that the target appearance feature vector of dishes contained in the target image can be obtained.
Wherein the appearance characteristics of the dish may comprise at least one of the following characteristics: color, texture and shape.
It should be noted that, in an embodiment, the pre-trained first convolutional neural network may include: a data input layer; convolutional layer a, wherein the convolutional core of convolutional layer a may be: 3x 3; pooling layer a (two-fold downsampling); convolutional layer b, whose convolution kernel can be: (1x1-3x3-1x1) in a cottleneck format; pooling layer b (two-fold down-sampling); convolutional layer c, whose convolution kernel can be: (1x1-3x3-1x1) in a cottleneck format; pooling layer c (two-fold downsampling); convolutional layer d, whose convolution kernel can be: (1x1-3x3-1x1-3x3-1x1) in the form of a cottleneck; pooling layer d (two-fold down-sampling); convolutional layer e, whose convolution kernel can be: (1x1-3x3-1x1-3x3-1x1) in the form of a cottleneck; and a feature output layer.
And S130, determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from the appearance characteristic vectors of various pre-stored dishes.
After the target appearance feature vector is obtained, a first reference feature vector matched with the target appearance feature vector can be determined from the appearance feature vectors of various pre-stored dishes. It can be understood that, since the first reference feature vector matches the target appearance feature vector, the dish category corresponding to the first reference feature vector also matches the dish category corresponding to the target appearance feature vector, that is, the dish category corresponding to the first reference feature vector is the same as the dish category corresponding to the target appearance feature vector.
And S140, determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image.
The first appearance characteristic vector is an appearance characteristic vector matched with the target appearance characteristic vector, so that the dish type corresponding to the first appearance characteristic vector is the dish type corresponding to the target appearance characteristic vector; since the target appearance feature vector is the appearance feature vector of the target image, the dish type corresponding to the target appearance feature vector can be determined as the dish type corresponding to the target image.
It can be understood that after the dish category corresponding to the target appearance feature vector is determined as the dish category corresponding to the target image, the dish category corresponding to the target image and the corresponding target dish price can be determined based on the preset correspondence between the dish category and the dish price; and displaying the dish type and the target dish price corresponding to the target image in a preset display screen so that the user can pay for the dishes to be purchased. Wherein, the settlement two-dimensional code on the usable display screen of user sweeps the sign indicating number and pays, or then, the user can use the camera on the display screen to brush face and pay
According to the technical scheme provided by the embodiment of the invention, when the category of the dishes is identified, the target image containing the dish area is obtained; performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image; and determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from the appearance characteristic vectors of various pre-stored dishes, and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image. Therefore, according to the technical scheme provided by the embodiment of the invention, when the type of the dish is determined, only the target image including the dish area needs to be obtained, and unlike the related technology, a worker needs to search the bar code adhered to the dish and the electronic equipment needs to align to the bar code, so that the time consumed by dish type identification is shortened, the efficiency of dish type identification is improved, and the accuracy of dish type identification can be improved.
It should be noted that, in practical application, the target image may be directly input to the first convolutional neural network trained in advance; it is reasonable that the dish region included in the target image is extracted first, and then the extracted dish region is input into the first convolutional neural network trained in advance. As shown in fig. 2, the method for identifying a category of dishes according to the embodiment of the present invention may include the following steps:
s210, when the dinner plate for containing dishes exists in the preset area, acquiring a target image of the area containing the dishes.
Step S210 has already been described in detail in the embodiment shown in fig. 1, and is not described herein again.
S220, inputting the target image into a pre-trained second convolutional neural network to obtain a dish area contained in the target image, wherein the second convolutional neural network is as follows: the method is obtained by training based on the plurality of sample images and the dish area contained in the plurality of sample images.
It will be appreciated that the dish area can be used to indicate the area in the tray that holds the dishes where the dishes are located. That is, the dish region is a region that can be used to identify the dish type in the target image. Therefore, after the target image is obtained, the target image can be input into the second convolutional neural network trained in advance to obtain the dish area contained in the target image, so that in the subsequent steps, the appearance feature recognition can be performed on the dish area to obtain the target appearance feature vector of the dish contained in the dish area.
And S230, identifying the appearance characteristics of the dish area based on the pre-trained first convolution neural network to obtain a target appearance characteristic vector of the dish contained in the dish area.
After the dish area of the target area is obtained, the appearance feature of the dish area can be identified based on a pre-trained first convolution neural network, and a target appearance feature vector of the dish contained in the dish area is obtained.
And S240, determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from the appearance characteristic vectors of various pre-stored dishes.
And S250, determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image.
Since step S240 in the embodiment shown in fig. 2 is the same as step S130 in the embodiment shown in fig. 1, and step S250 in the embodiment shown in fig. 2 is the same as step S140 in the embodiment shown in fig. 1, S240 and S250 are not described again here.
According to the technical scheme provided by the embodiment of the invention, when the category of the dishes is identified, the target image containing the dish area is obtained; performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image; and determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from the appearance characteristic vectors of various pre-stored dishes, and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image. Therefore, according to the technical scheme provided by the embodiment of the invention, when the type of the dish is determined, only the target image including the dish area needs to be obtained, and unlike the related technology, a worker needs to search the bar code adhered to the dish and the electronic equipment needs to align to the bar code, so that the time consumed by dish type identification is shortened, the efficiency of dish type identification is improved, and the accuracy of dish type identification can be improved.
In a second aspect, an embodiment of the present invention provides an apparatus for identifying a category of dishes, as shown in fig. 3, the apparatus includes:
an image acquisition module 310, configured to acquire a target image including a dish area;
the feature recognition module 320 is configured to perform appearance feature recognition on the target image to obtain a target appearance feature vector of dishes included in the target image;
the feature vector determining module 330 is configured to determine, from appearance feature vectors of various pre-stored dishes, a first appearance feature vector with a highest matching degree with the target appearance feature vector;
a dish type determining module 340, configured to determine a dish type corresponding to the first appearance feature vector as a dish type corresponding to the target image.
According to the technical scheme provided by the embodiment of the invention, when the category of the dishes is identified, the target image containing the dish area is obtained; performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image; and determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from the appearance characteristic vectors of various pre-stored dishes, and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image. Therefore, according to the technical scheme provided by the embodiment of the invention, when the type of the dish is determined, only the target image including the dish area needs to be obtained, and unlike the related technology, a worker needs to search the bar code adhered to the dish and the electronic equipment needs to align to the bar code, so that the time consumed by dish type identification is shortened, the efficiency of dish type identification is improved, and the accuracy of dish type identification can be improved.
Optionally, the feature vector determining module is specifically configured to:
based on a pre-trained first convolution neural network, performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image, wherein the first convolution neural network is as follows: the method comprises the steps that training is carried out on the basis of a plurality of sample images and appearance characteristic vectors corresponding to the sample images, wherein the sample images comprise dish areas, and the appearance characteristic vector corresponding to each sample image is the appearance characteristic vector of dishes contained in the sample image.
Optionally, the image obtaining module is specifically configured to:
monitoring whether a dinner plate for containing dishes exists in a preset area or not;
and when the dinner plate for containing dishes exists in the preset area, acquiring a target image of the area containing the dishes.
Optionally, the apparatus further comprises:
a dish area determining module, configured to, after the step of obtaining a target image including a dish area when it is monitored that a dinner plate containing dishes exists in the preset area, input the target image to a second convolutional neural network trained in advance to obtain the dish area included in the target image, where the second convolutional neural network is: the method comprises the steps that the method is obtained through training based on a plurality of sample images and dish areas contained in the sample images;
accordingly, the feature vector determination module is specifically configured to:
and identifying the appearance characteristics of the dish area based on a pre-trained first convolution neural network to obtain a target appearance characteristic vector of the dishes contained in the dish area.
Optionally, the appearance characteristics of the dish comprise at least one of the following characteristics: color, texture and shape.
In a third aspect, an embodiment of the present invention further provides an electronic device, as shown in fig. 4, including a processor 401, a communication interface 402, a memory 403, and a communication bus 404, where the processor 401, the communication interface 402, and the memory 403 complete mutual communication through the communication bus 404,
a memory 403 for storing a computer program;
the processor 401 is configured to implement the dish type identification method according to the first aspect when executing the program stored in the memory 403.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
According to the technical scheme provided by the embodiment of the invention, when the category of the dishes is identified, the target image containing the dish area is obtained; performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image; and determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from the appearance characteristic vectors of various pre-stored dishes, and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image. Therefore, according to the technical scheme provided by the embodiment of the invention, when the type of the dish is determined, only the target image including the dish area needs to be obtained, and unlike the related technology, a worker needs to search the bar code adhered to the dish and the electronic equipment needs to align to the bar code, so that the time consumed by dish type identification is shortened, the efficiency of dish type identification is improved, and the accuracy of dish type identification can be improved.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method for identifying a category of dishes according to the first aspect is implemented.
According to the technical scheme provided by the embodiment of the invention, when the category of the dishes is identified, the target image containing the dish area is obtained; performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image; and determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from the appearance characteristic vectors of various pre-stored dishes, and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image. Therefore, according to the technical scheme provided by the embodiment of the invention, when the type of the dish is determined, only the target image including the dish area needs to be obtained, and unlike the related technology, a worker needs to search the bar code adhered to the dish and the electronic equipment needs to align to the bar code, so that the time consumed by dish type identification is shortened, the efficiency of dish type identification is improved, and the accuracy of dish type identification can be improved.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the device, the electronic apparatus, and the storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (12)
1. A method for identifying a category of dishes, the method comprising:
acquiring a target image containing a dish area;
performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image;
determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from appearance characteristic vectors of various pre-stored dishes;
and determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image.
2. The method of claim 1, wherein the step of performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image comprises:
based on a pre-trained first convolution neural network, performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image, wherein the first convolution neural network is as follows: the method comprises the steps that training is carried out on the basis of a plurality of sample images and appearance characteristic vectors corresponding to the sample images, wherein the sample images comprise dish areas, and the appearance characteristic vector corresponding to each sample image is the appearance characteristic vector of dishes contained in the sample image.
3. The method of claim 2, wherein the step of obtaining the target image containing the dish area comprises:
monitoring whether a dinner plate for containing dishes exists in a preset area or not;
and when the dinner plate for containing dishes exists in the preset area, acquiring a target image of the area containing the dishes.
4. The method of claim 3, wherein after the step of obtaining the target image of the area containing the dish upon detecting the presence of the dish containing the dish within the predetermined area, the method further comprises:
inputting the target image into a pre-trained second convolutional neural network to obtain a dish area contained in the target image, wherein the second convolutional neural network is as follows: the method comprises the steps that the method is obtained through training based on a plurality of sample images and dish areas contained in the sample images;
correspondingly, the step of performing appearance feature recognition on the target image based on the pre-trained first convolution neural network to obtain a target appearance feature vector of dishes contained in the target image includes:
and identifying the appearance characteristics of the dish area based on a pre-trained first convolution neural network to obtain a target appearance characteristic vector of the dishes contained in the dish area.
5. Method according to any of claims 1 to 4, wherein the appearance characteristics of the dish comprise at least one of the following characteristics: color, texture and shape.
6. An apparatus for identifying a category of dishes, the apparatus comprising:
the image acquisition module is used for acquiring a target image containing a dish area;
the characteristic identification module is used for identifying the appearance characteristic of the target image to obtain a target appearance characteristic vector of dishes contained in the target image;
the characteristic vector determining module is used for determining a first appearance characteristic vector with the highest matching degree with the target appearance characteristic vector from appearance characteristic vectors of various pre-stored dishes;
and the dish type determining module is used for determining the dish type corresponding to the first appearance characteristic vector as the dish type corresponding to the target image.
7. The apparatus of claim 6, wherein the feature vector determination module is specifically configured to:
based on a pre-trained first convolution neural network, performing appearance feature recognition on the target image to obtain a target appearance feature vector of dishes contained in the target image, wherein the first convolution neural network is as follows: the method comprises the steps that training is carried out on the basis of a plurality of sample images and appearance characteristic vectors corresponding to the sample images, wherein the sample images comprise dish areas, and the appearance characteristic vector corresponding to each sample image is the appearance characteristic vector of dishes contained in the sample image.
8. The apparatus of claim 7, wherein the image acquisition module is specifically configured to:
monitoring whether a dinner plate for containing dishes exists in a preset area or not;
and when the dinner plate for containing dishes exists in the preset area, acquiring a target image of the area containing the dishes.
9. The apparatus of claim 8, further comprising:
a dish area determining module, configured to, after the step of obtaining a target image including a dish area when it is monitored that a dinner plate containing dishes exists in the preset area, input the target image to a second convolutional neural network trained in advance to obtain the dish area included in the target image, where the second convolutional neural network is: the method comprises the steps that the method is obtained through training based on a plurality of sample images and dish areas contained in the sample images;
accordingly, the feature vector determination module is specifically configured to:
and identifying the appearance characteristics of the dish area based on a pre-trained first convolution neural network to obtain a target appearance characteristic vector of the dishes contained in the dish area.
10. Device according to any one of claims 6 to 9, characterized in that the appearance characteristics of the dish comprise at least one of the following characteristics: color, texture and shape.
11. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 5 when executing a program stored in the memory.
12. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910128874.8A CN111597862A (en) | 2019-02-21 | 2019-02-21 | Dish category identification method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910128874.8A CN111597862A (en) | 2019-02-21 | 2019-02-21 | Dish category identification method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111597862A true CN111597862A (en) | 2020-08-28 |
Family
ID=72186816
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910128874.8A Pending CN111597862A (en) | 2019-02-21 | 2019-02-21 | Dish category identification method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111597862A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845527A (en) * | 2016-12-29 | 2017-06-13 | 南京江南博睿高新技术研究院有限公司 | A kind of vegetable recognition methods |
| CN106874913A (en) * | 2016-12-29 | 2017-06-20 | 南京江南博睿高新技术研究院有限公司 | A kind of vegetable detection method |
| CN107844790A (en) * | 2017-11-15 | 2018-03-27 | 上海捷售智能科技有限公司 | A kind of vegetable identification and POS and method based on image recognition |
| CN107992871A (en) * | 2017-12-21 | 2018-05-04 | 陕西伟景机器人科技有限公司 | The automatic accounting method in dining room based on image recognition |
| CN108256474A (en) * | 2018-01-17 | 2018-07-06 | 百度在线网络技术（北京）有限公司 | For identifying the method and apparatus of vegetable |
| CN108364420A (en) * | 2018-01-08 | 2018-08-03 | 阿里巴巴集团控股有限公司 | Image-recognizing method and device and electronic equipment |
| CN108491765A (en) * | 2018-03-05 | 2018-09-04 | 中国农业大学 | A kind of classifying identification method and system of vegetables image |
| CN108510677A (en) * | 2018-03-21 | 2018-09-07 | 潘跃瑞 | A kind of cash register system and data processing method |
| CN109214409A (en) * | 2018-07-10 | 2019-01-15 | 上海斐讯数据通信技术有限公司 | A kind of vegetable recognition methods and system |
-
2019
- 2019-02-21 CN CN201910128874.8A patent/CN111597862A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845527A (en) * | 2016-12-29 | 2017-06-13 | 南京江南博睿高新技术研究院有限公司 | A kind of vegetable recognition methods |
| CN106874913A (en) * | 2016-12-29 | 2017-06-20 | 南京江南博睿高新技术研究院有限公司 | A kind of vegetable detection method |
| CN107844790A (en) * | 2017-11-15 | 2018-03-27 | 上海捷售智能科技有限公司 | A kind of vegetable identification and POS and method based on image recognition |
| CN107992871A (en) * | 2017-12-21 | 2018-05-04 | 陕西伟景机器人科技有限公司 | The automatic accounting method in dining room based on image recognition |
| CN108364420A (en) * | 2018-01-08 | 2018-08-03 | 阿里巴巴集团控股有限公司 | Image-recognizing method and device and electronic equipment |
| CN108256474A (en) * | 2018-01-17 | 2018-07-06 | 百度在线网络技术（北京）有限公司 | For identifying the method and apparatus of vegetable |
| CN108491765A (en) * | 2018-03-05 | 2018-09-04 | 中国农业大学 | A kind of classifying identification method and system of vegetables image |
| CN108510677A (en) * | 2018-03-21 | 2018-09-07 | 潘跃瑞 | A kind of cash register system and data processing method |
| CN109214409A (en) * | 2018-07-10 | 2019-01-15 | 上海斐讯数据通信技术有限公司 | A kind of vegetable recognition methods and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109165645B (en) | Image processing method and device and related equipment | |
| CN110795584B (en) | User identifier generation method and device and terminal equipment | |
| CN111095401B (en) | Digital image capture session and metadata association | |
| JP7036401B2 (en) | Learning server, image collection support system for insufficient learning, and image estimation program for insufficient learning | |
| CN111274845A (en) | Method, device and system for identifying shelf display situation of store and electronic equipment | |
| CN109102324B (en) | Model training method, and red packet material laying prediction method and device based on model | |
| CN110940407A (en) | Intelligent electronic scale for Internet of things and transaction method, device and system thereof | |
| CN111126990B (en) | Automatic article identification method, settlement device, terminal and storage medium | |
| WO2020134411A1 (en) | Merchandise category recognition method, apparatus, and electronic device | |
| CN111105244B (en) | Refund-based service scheme determination method and refund-based service scheme determination device | |
| US9355395B2 (en) | POS terminal apparatus and commodity specification method | |
| CN111597862A (en) | Dish category identification method and device and electronic equipment | |
| KR20200101024A (en) | Service system of returning goods being able to pickup request using location information included in photo of returning goods | |
| CN109242601A (en) | Commodity information query method and system | |
| JP2015069327A (en) | Investigation system, investigation method, server, user terminal, program, and recording medium | |
| CN112950329A (en) | Commodity dynamic information generation method, device, equipment and computer readable medium | |
| CN115393838A (en) | Pointer instrument reading identification method and device, electronic equipment and storage medium | |
| CN112668558A (en) | Cash registering error correction method and device based on human-computer interaction | |
| JP2022014793A (en) | Information processing device, information processing method, and program | |
| JP2010191589A (en) | Action prediction apparatus, action prediction method, and program | |
| CN110827261A (en) | Image quality detection method and device, storage medium and electronic equipment | |
| CN111126110A (en) | Commodity information identification method, settlement method and device and unmanned retail system | |
| CN111626074A (en) | Face classification method and device | |
| CN109146030B (en) | Low-carbon consumption identification method and device and electronic equipment | |
| CN115331217B (en) | Dining settlement method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |