CN109857880B - Model-based data processing method and device and electronic equipment - Google Patents
Model-based data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN109857880B CN109857880B CN201910041435.3A CN201910041435A CN109857880B CN 109857880 B CN109857880 B CN 109857880B CN 201910041435 A CN201910041435 A CN 201910041435A CN 109857880 B CN109857880 B CN 109857880B
- Authority
- CN
- China
- Prior art keywords
- sample
- model
- classification information
- information
- storage platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The present invention relates to the field of data processing technologies, and in particular, to a data processing method and apparatus based on a model, and an electronic device. The method comprises the following steps: acquiring a picture containing a sample uploaded by a network user; obtaining characteristic information of the sample in the picture; loading candidate classification information of the characteristic information for a network user to select and confirm; storing the sample classification information selected and confirmed by the network user from the candidate classification information and the corresponding characteristic information thereof to a first storage platform for subsequent training of the model; loading meta data corresponding to the sample classification information and storing the meta data to a second storage platform; the characteristic information of the sample, the sample classification information and the meta data training model corresponding to the sample classification information are called, the characteristic information and the sample classification information about the sample are stored in different storage devices, and the phenomenon that the training platform calls the data to cause confusion and model training result errors due to the fact that the data are stored in the same storage medium can be well avoided.
Description
[ technical field ] A method for producing a semiconductor device
The present invention relates to the field of data processing technologies, and in particular, to a data processing method and apparatus based on a model, and an electronic device.
[ background of the invention ]
Nowadays, industries such as goods, food production and sales require classifying and detecting different kinds of goods based on characteristic information of the goods themselves, so as to distinguish different kinds of goods and store or settle prices of the goods. Therefore, when facing goods to be detected and classified, feature data information of a batch of sample articles is generally acquired, a model related to each article is built by using the feature data information, the model building needs feature information of the articles, classification information of the articles, sub-classification information of the articles and the like, when the model needs to be trained, matching and relevance among data become important, and if the data needed for training the model is stored unreasonably or distributed unreasonably, the quality of the trained model is affected.
[ summary of the invention ]
The method aims to solve the technical problem that the quality of a model obtained by training is affected due to unreasonable data storage at present.
The invention provides a data processing method based on a model for solving the technical problems, which comprises the following steps: s1, obtaining pictures containing samples uploaded by network users; s2, obtaining characteristic information of the sample in the picture; s3, loading the candidate classification information of the characteristic information for the network user to select and confirm; s4, storing the sample classification information selected and confirmed by the network user from the candidate classification information and the corresponding characteristic information thereof to a first storage platform for subsequent training of the model; s5, loading meta data corresponding to the sample classification information and storing the meta data to a second storage platform; s6, calling characteristic information of the sample, sample classification information and a meta data training model corresponding to the sample classification information; the first storage platform is a third-party storage platform, the second storage platform is a cloud management system, and the first storage platform is further used for storing the trained model.
Preferably, the feature information includes two-dimensional RGB data and depth data, the number of the samples is one or more with a space, and the meta data includes attribute information corresponding to each sample.
Preferably, the picture in step S1 further includes a carrier tray image carrying the sample, and in step S2, the characteristic information of the carrier tray image is also detected when the characteristic information of the sample is detected, and step S2 specifically includes the following steps: s21, converting the depth data into three-dimensional point cloud; s22, calculating a plane area corresponding to the carrying tray by using the three-dimensional point cloud; s23, stripping points higher than the plane area in the three-dimensional point cloud to obtain a three-dimensional point cloud corresponding to the sample; s24, generating a circumscribed rectangular frame to mark the sample based on the three-dimensional point cloud corresponding to the sample; and S25, converting the position information corresponding to each circumscribed rectangle frame into a standard labeling format.
Preferably, the depth vision-based data set acquisition method further comprises the steps of: and issuing the trained model to the user side.
Preferably, the model before being sent to the user side comprises the following steps: and the user side accesses the second storage platform to obtain the model serial number and judges whether the model needs to be updated according to the model serial number.
In order to solve the above technical problem, the present invention further provides a model-based data processing apparatus, including: an acquisition device: the method comprises the steps of configuring a picture containing a sample and uploaded by a network user; a detection module: the characteristic information of the sample and the object carrying plate image is obtained according to the picture; loading a module: configuring candidate classification information for loading the characteristic information for the network user to select and confirm; a first storage platform: the system is configured for storing sample classification information selected and confirmed by the network user from the candidate classification information and corresponding characteristic information; a second storage platform: the method comprises the steps that a, meta data corresponding to the sample classification information loaded by a network user are configured and stored; an automatic training platform: and configuring a meta data training model for calling the characteristic information of the sample, the sample classification information and the meta data corresponding to the sample classification information.
Preferably, the detection module comprises: the depth data processing module: configured to convert the depth data into a three-dimensional point cloud; a plane acquisition module: the three-dimensional point cloud is used for calculating a plane area corresponding to the objective plate image; a separation module: configuring to strip a portion of the picture above the planar area with the three-dimensional point cloud to obtain a three-dimensional point cloud representative of each sample; a labeling module: generating a circumscribed rectangular frame to mark the sample based on a three-dimensional point cloud corresponding to the sample; a conversion module: and the position information corresponding to each circumscribed rectangle frame is configured to be converted into a standard labeling format.
Preferably, the first storage platform is further configured to store the trained model and the acquired image data about each sample, and the first storage platform or the second storage platform is further configured to store a mapping relationship corresponding to the meta data and the sample classification information.
Preferably, the data processing apparatus of the model further includes a user side, and the user side accesses the second storage platform to obtain the model serial number and determines whether to send a model issue request to the first storage platform according to the model serial number of the second storage platform.
The invention also provides an electronic device comprising one or more processors; storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method as described above.
Compared with the prior art, the method comprises the steps that the network user selects confirmed sample classification information and corresponding characteristic information from candidate classification information to be stored in a first storage platform; the meta data corresponding to the sample classification information is loaded and stored in a second storage platform, the characteristic information and the sample classification information about the sample are stored in different storage devices, and when the model needs to be trained, the characteristic information, the sample classification information and the meta data corresponding to the sample classification information of the sample are called to train the model, so that the problem that the model training result is wrong due to confusion when the data is called by the training platform because the data is stored in the same storage medium can be well avoided; meanwhile, compared with meta data, the sample classification information and the corresponding characteristic information of the sample are generally complex, the data size is large, and the required storage medium and the meta data have large difference, so that the storage platform can be more reasonably utilized by separately storing the sample classification information and the characteristic information; furthermore, the sample classification information and the corresponding characteristic information thereof are often greatly different from the data types of the meta data, and the identification modes are different when the sample classification information and the characteristic information thereof are called for model training, so that the calling efficiency and the calling accuracy can be well prevented from being influenced by the identification modes of different types of data when the sample classification information and the corresponding characteristic information are separately stored; in addition, meta data generally represents attribute information of the sample, such as price information, formula information and other relatively confidential information, and can be operated by a general management department of a merchant, so that on one hand, the data can be kept secret, and on the other hand, the data can be conveniently controlled; furthermore, by the method, the characteristic information of the acquired picture, the sample classification information selected and confirmed by the network user from the candidate classification information and the corresponding characteristic information are stored in a first storage platform, meta data corresponding to the sample classification information are loaded and stored in a second storage platform, and the corresponding characteristic information, the classification information and the corresponding meta data are directly called for training during training, so that the closed loop of the whole step is completed.
The first storage platform is a third-party storage platform, the second storage platform is a cloud management system, the first storage platform is further used for storing the trained model, the third-party storage platform is suitable for storing data with high complexity or large data volume, and the cloud management system is suitable for storing data with simple relation.
The first storage platform is further used for storing the trained model and the acquired image data related to each sample, the first storage platform or the second storage platform is further used for storing the mapping relation corresponding to meta data and sample classification information, the image data and the meta data are often greatly different in data type, and when the image data and the meta data are called for model training, the recognition mode is greatly different, so that the recognition modes of different types of data can be better prevented from influencing calling efficiency and calling accuracy.
The data processing device of the model provided by the invention can automatically acquire the characteristic information and classification result of the picture and is trained by an automatic training platform by combining the meta data of the sample, and after the model is trained well, the meta data is directly issued to a user for use, so that the closed loop of the whole device is completed, and the data processing efficiency is improved.
[ description of the drawings ]
FIG. 1 is a schematic flow chart diagram of a model-based data processing method provided in a first embodiment of the present invention;
FIG. 2 is a schematic diagram of the structure of the process of FIG. 1 when a sample is included in the picture;
FIG. 3 is a schematic diagram of a structure of the picture processed by the flow chart of FIG. 1 when a plurality of samples are included;
fig. 4 is a schematic diagram of obtaining depth data of a picture in the model-based data processing method provided in the first embodiment of the present invention;
fig. 5 is a detailed flowchart in step S2 of the model-based data processing method provided in the first embodiment of the present invention;
FIG. 6a is a schematic illustration of the first embodiment of the present invention after the sample has been peeled from the carrier tray;
FIG. 6b is a schematic view of the first embodiment of the present invention after each sample is marked with a rectangular frame;
FIG. 7 is another schematic flow chart diagram of a model-based data processing method provided in the first embodiment of the present invention;
FIG. 8 is a block schematic diagram of a model-based data processing apparatus provided in a second embodiment of the present invention;
FIG. 9 is a block diagram of a detection module in a model-based data processing apparatus provided in a second embodiment of the present invention;
FIG. 10 is a schematic block diagram of a model-based data processing apparatus provided in a second embodiment of the present invention;
fig. 11 is a block schematic diagram of an electronic device provided in a third embodiment of the present invention;
FIG. 12 is a schematic block diagram of a computer system suitable for use with a server implementing an embodiment of the invention.
[ detailed description ] embodiments
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, a first embodiment of the present invention provides a model-based data processing method, including the following steps:
s1, obtaining pictures containing samples uploaded by network users;
specifically, in this step, the network user uploads one or more pictures, for example, through interaction with the user equipment, and the obtaining device for obtaining pictures obtains the pictures uploaded by the network user through interaction with the user equipment. For example, a network user uploads a picture through an application program, first logs in the application program, uploads one or more pictures stored in user equipment of the network user through a picture uploading function in the application program, or calls a photographing function program interface in the user equipment to photograph the picture in real time, uploads at least one newly photographed picture, and the obtaining device obtains the picture uploaded by the network user through one or more calls of the application program interface, or through other appointed manners.
It should be understood by those skilled in the art that the above-mentioned manner for obtaining the picture uploaded by the network user is only an example, and other manners for obtaining the picture uploaded by the network user, which may occur now or later, such as may be applicable to the present invention, should be included within the scope of the present invention, and are hereby incorporated by reference.
Referring to fig. 2 and 3, the number of samples M included in the pictures is one or more. When the number of the samples M is multiple, there are no multiple samples stacked on each other between each sample M, and the picture further includes a carrier tray image N for carrying the samples M, where the carrier tray image N has a planar area for placing the samples. The case where a plurality of samples are not stacked on top of each other is understood as: with a certain distance between each sample or with contact between the edges of each sample but without overlap. In the present invention, the sample M is particularly suitable for baked food, such as baked food like bread or biscuits, fruit, bottled drink or other objects with certain contours, and the bread is taken as an example in the present invention.
In particular, the picture may be obtained by a 2D camera, a 3D camera or other photographing device in this step. The 3D camera is a depth camera, which can be divided into three types according to its working principle: TOF cameras, RGB binocular cameras, and structured light cameras. Structured light cameras are preferred in this application.
Referring back to fig. 1 again, the model-based data processing method further includes:
and step S2, obtaining characteristic information of the sample in the picture.
In this step, the characteristic information of the sample in the picture is mainly obtained through detection by a detection module. The detection module may be an image recognition module provided in the photographing apparatus. The detection module identifies a plurality of objects by, for example, image recognition, identifying the color, brightness, pixel contrast of adjacent unit areas, and the like of the image in the picture, and further, the detection module can determine the outline of each sample according to the boundary point of the sudden change of the color, brightness, and the like in the pixel contrast, so that each sample can be marked. The marking method includes but is not limited to various methods such as frame selection, circle drawing, underlining, shading and the like.
In some specific embodiments, when the picture is obtained by a 3D camera, the feature information includes two-dimensional RGB data and depth data of the sample, and includes two-dimensional RGB data and depth data of a carrier tray image carrying the sample. Referring to fig. 4, for an example of bread, bread 300 is fed from the object tray 500 to the lower side of the camera 600, the camera 600 emits a plurality of random speckle infrared light spots 900 invisible to human eyes to the bread 300 and the object tray 500 through the depth sensor to obtain a light spot diagram as shown in fig. 4, the plurality of light spots 900 are distributed on the surface of the bread 300 and the object tray 500, and the camera 600 stores depth information of each light spot 900 to obtain depth data of a picture. Wherein the RGB data of the picture is obtained through an RGB camera. At this time, in order to peel each bread in the picture out of the tray 500 based on the obtained depth data and form image data regarding the standard of each bread 300. As shown in fig. 5, the step S2 includes at least the following steps:
s21, converting the depth data into three-dimensional point cloud;
in step S21, the depth data is processed and converted into a three-dimensional point cloud. It will be appreciated that a plurality of depth data corresponding to a plurality of points is obtained by the 3D camera, which depth data is converted into a three-dimensional point cloud corresponding to the plane in which the bread lies.
S22, acquiring a plane area corresponding to the carrying tray 500;
and establishing a plane between the points, and updating the plane through iterative calculation, and finally stopping updating the plane when the distance between the points as many as possible and the plane is minimum. Specifically, a plane is established among partial points in the three-dimensional point cloud, the shortest distance between the partial points and the plane is obtained through iterative calculation, the rest points in the three-dimensional point cloud are continuously brought into iterative calculation, the plane is continuously updated through repeated iterative calculation, and finally, when the distance between the partial points and the plane is the smallest, the plane is stopped being updated, the current plane can be determined as a placing plane corresponding to the bread 300, and the plane can be a plane for placing the bread, such as a carrying tray 500, a shopping cart bottom surface or a settlement counter.
It can be understood that, the model-based data processing method provided by the invention can be used for collecting image data of newly developed bread at the headquarters of the bakery for establishing a model to be issued to the branch shops for use. Namely, after a new bread is produced, the new bread is placed on the carrying tray, the carrying tray carrying the bread is placed on a fixed counter and is shot by a camera to obtain a picture, under the condition that the relative position of the counter and the camera is kept unchanged, the plane in the three-dimensional point cloud is a fixed plane, in the process of one or more times of calculation, the information of the estimated plane in the three-dimensional point cloud is stored and is provided for next image-based new bread image acquisition, namely, under the condition that the counter is kept unchanged, the plane in the three-dimensional point cloud is obtained to obtain the stored plane information.
It can be understood that, when the bread branch shop issues the price settlement after the model is trained, the customer often needs to place the chosen bread on the drawing tray and place the chosen bread on the counter to settle the account when settling the account, and at this time, the counter plane information of the corresponding bread branch shop can be obtained by the above method.
It can be understood that since the 3D camera cannot completely fill the picture with bread and planes, objects in the background environment are also present in the picture, and all areas in the picture include the carrier tray, bread and objects in the background environment.
From the above, it can be known that the three-dimensional point cloud covers the depth image, the plane of the bread and the object in the background environment. It can be understood that the points corresponding to the object under the background environment and the points corresponding to the plane area under the plane are removed, so that the calculation amount of the three-dimensional point cloud in the subsequent steps can be reduced, and the calculation efficiency can be improved. Therefore, referring to fig. 5 again, step S2 further includes:
s23, stripping points higher than the plane area in the three-dimensional point cloud to obtain a three-dimensional point cloud corresponding to the sample;
in this step, the point higher than the plane area is also referred to as the three-dimensional point cloud of the bread. After stripping as shown in figure 6 a.
S24, generating a circumscribed rectangular frame to mark the sample based on the three-dimensional point cloud corresponding to the sample;
as shown in fig. 6b, in this step, the circumscribed rectangular frame is generated in order to frame each sample, and each sample is marked with coordinate information of the edge of the rectangular frame to obtain position information representing the sample. Preferably, the sample is marked with the coordinates of the vertices of the rectangular box in the present invention. When the feature information, classification information, and meta data of a sample are matched with the sample image of the corresponding rectangular frame in the process of training a model using a plurality of images about the sample of one category and their corresponding feature information, classification information, and meta data, matching is mainly performed based on the position information and feature information of the marker. Alternatively, matching may be performed based on a mapping relationship between the position information and the feature information.
And S25, converting the position information corresponding to each circumscribed rectangle frame into a standard labeling format.
In this step, the position information, that is, the coordinate information, is mainly converted into an XML format, so that when the model is trained conveniently, the system can better recognize the position information and call corresponding feature information, classification information and meta data according to the position information to train the model. It should be noted that the format is not limited to XML, and other types that can be read by the system are also possible. Alternatively, this step may be omitted.
Referring back to fig. 1 again, the model-based data processing method further includes:
and step S3, loading the candidate classification information of the characteristic information for the network user to select and confirm.
In this step, specifically, after the characteristic information of the sample in the picture is obtained through step S2, candidate classification information of the characteristic information is loaded for the network user to select and confirm. The loading of the candidate classification information in this step may be realized, for example, by a loading means. The loading device determines one or more candidate classification information corresponding to the characteristic information by performing matching query in the database, and loads the one or more candidate classification information at the relevant position of the webpage or the application, so as to be selected by the network user. The database stores, for example, a mapping relationship between at least one candidate classification information and corresponding feature information in advance. Preferably, the mapping relationship between the candidate classification information and the corresponding feature information may be implemented by a model, and the model may be further trained by the feature information and the classification information.
In some other embodiments, the candidate classification information is pre-stored on the user equipment and set by the network user, no relevant mapping relation is set between the feature information and the candidate classification information, and the matching between the feature information and the candidate classification information is completely selected by the network user.
In this step, taking bread classification as an example, the candidate classification information corresponds to a bread classification, such as: egg tarts, cakes, sandwiches, and the like.
Referring back to fig. 1 again, the model-based data processing method further includes:
and step S4, storing the sample classification information selected and confirmed by the network user from the candidate classification information and the corresponding characteristic information thereof to a first storage platform for subsequent model training.
In this step, the first storage platform is a third-party storage platform. And the third-party storage platform provides long-term data storage, lookup, downloading and other services. And the third-party storage platform has a larger storage space, so that data with larger data volume or more complex dimensionality can be conveniently stored.
Referring back to fig. 1 again, the model-based data processing method further includes:
step S5, loading meta data corresponding to the sample classification information into a second storage platform.
In this step, the second storage platform is a cloud management system. meta data is attribute data of the sample corresponding to the sample classification information. For example, when the egg tart is classified, the corresponding meta data is corresponding to a, where a includes one or more of the following:
a1, price information;
a2, date of manufacture;
a3, taste information: grape flavor, strawberry flavor, blueberry flavor, etc.;
a4, promotional information: buy a delivery half dozen, a second beat half price, etc.
The meta data is simple relative to the sample classification information and the characteristic information, and does not need a large storage space platform, so that the meta data can be stored in a cloud management system.
Meanwhile, in this step, when meta data corresponding to the sample classification information is loaded, a mapping relationship corresponding to the sample classification information and the meta data is also loaded and stored on the first storage platform and/or the second storage platform, so that the sample classification information and the meta data corresponding thereto can be automatically retrieved in the subsequent model training process. The mapping relationship can be understood as: the sample classification information corresponds to an egg tart, the corresponding ID is 001, the corresponding meta data corresponds to the ID address of a, the ID address of a is 001, and the sample classification information is matched with the sample classification information 001.
Referring back to fig. 1 again, the processing method of the model further includes:
and step S6, calling characteristic information of the sample, sample classification information and meta data training models corresponding to the sample classification information.
In this step, a model is mainly trained on a training cloud platform, and after the characteristic information, the sample classification information and the corresponding meta data about the sample are loaded, the training cloud platform calls the characteristic information, the sample classification information and the meta data training model corresponding to the sample classification information of the sample. After the model is trained by the training cloud platform, the model is sent to the first storage platform to be stored for the user to download and use. And after the model is trained, sending the serial number corresponding to the model to a second storage platform.
Referring to fig. 7, the model-based data processing method further includes:
and step S7, issuing the trained model to the user terminal.
The user side can be understood as a merchant who needs to use the model for sample detection, classification or price settlement.
In some other embodiments, before sending the trained model to the user side, the method further includes: and the user side accesses the second storage platform to obtain the model serial number and judges whether the model needs to be updated according to the model serial number. And the user side occasionally accesses the second storage platform to check whether the new model serial number is updated well. When the user side checks that a new model needs to be updated, the user side sends request information to the first storage platform, and when the first storage platform receives the request information, the trained model is sent to the user side for use.
Referring to fig. 8, a second embodiment of the present invention provides a model-based data processing apparatus 200, comprising: the device comprises an acquisition device 201, a detection module 202 and a loading module 203.
The acquisition means 201: is configured to obtain pictures containing samples uploaded by network users and upload the obtained pictures to the detection module 202.
The detection module 202: the characteristic information of the sample and the characteristic information of the carrying disc are obtained according to the pictures;
in this step, the characteristic information includes two-dimensional RGB data and depth data of the sample, and also includes two-dimensional RGB data and depth data of the carrier tray image carrying the sample.
The loading module 203: the candidate classification information used for loading the feature information is configured for a network user to select and confirm, that is, the loading module 203 transmits the candidate classification information of the loaded object to the detection module 202 for the network user to select to match with the feature information of the sample obtained by the detection module 202.
A first storage platform 204, a second storage platform 205, and an automated training platform 206
The first storage platform 204: configuring a model which is used for storing sample classification information selected and confirmed by the network user from the candidate classification information and corresponding characteristic information and storing the trained model;
the second storage platform 205: and the system is configured for saving meta data corresponding to the sample classification information loaded by a network user. The meta data can be loaded through the loading module 203 as well. The loaded meta data may be saved at the second storage platform 205 through the detection module 202. It is understood that the meta data may also be directly uploaded to the second storage platform 205.
When the loading module 203 loads the meta data, the mapping relationship between the sample classification information and the meta data is loaded and stored in the second storage platform 205, but the mapping relationship between the sample classification information and the meta data may also be stored in the first storage platform 204.
Automated training platform 206: and configuring a meta data training model for calling the characteristic information of the sample, the sample classification information and the meta data corresponding to the sample classification information.
Referring to fig. 9, the detecting module 202 includes:
depth data processing module 2021: configured to convert the depth data into a three-dimensional point cloud;
plane acquisition module 2022: the three-dimensional point cloud is used for calculating a plane area corresponding to the objective plate image;
separation module 2023: stripping points higher than the plane area in the three-dimensional point cloud to obtain a three-dimensional point cloud corresponding to the sample;
marking module 2024: generating a circumscribed rectangular frame to mark the sample based on a three-dimensional point cloud corresponding to the sample;
an extracting module 2025 configured to extract a region corresponding to the circumscribed rectangle to form an image dataset representing each sample, the image dataset representing feature information and classification information of each sample.
Referring to fig. 10, the data processing apparatus 200 of the model further includes a user end 207, which sends a model serial number corresponding to the model to the second storage platform 206 after the automatic training platform 206 has trained the model. The user terminal 207 occasionally accesses the second storage platform 206 to check whether the new model serial number is updated. When the user terminal 207 checks that a new model needs to be updated, it sends a request message to the first storage platform 204, and when the first storage platform 204 receives the request message, the trained model is sent to the user terminal 207 for use. The model is used for commodity classification, detection or price settlement.
Referring to fig. 11, a third embodiment of the invention provides an electronic device 400, which includes one or more processors 402;
a storage device 401 for storing one or more programs,
when executed by the one or more processors 402, cause the one or more processors 402 to implement any of the steps of the model-based data processing method as provided by the first implementation.
The following exemplifies an application scenario of the data processing apparatus of the model of the present invention.
Taking bread production in the baking field as an example, the network user is a bread store, and the user side is a merchant selling bread developed by the store in each place. When a network user develops bread of a new variety, a main shop needs to collect feature information of the bread of the new variety, select sample classification information corresponding to a sample based on the feature information of the sample, further train a model of the new variety by combining meta data corresponding to the sample classification information, send the trained model to a merchant for use, and the merchant can use the model to settle accounts. It is understood that the collected sample characteristic information obtained by the network user is data of multiple samples in the same production batch of the same sample or multiple pictures obtained from different batches.
Specifically, when a network user produces a batch of new breads, a single bread or a plurality of breads are placed in the loading tray, and when a plurality of breads are provided, each bread is separated by a certain distance. Then the shooting equipment is used for shooting the bread and the carrying plate to obtain pictures. Alternatively, when photographed using a 3D camera, the carrier tray is photographed from a top view angle with respect to the carrier tray to well capture depth data with respect to each bread and carrier tray. Then, the acquiring device 201 acquires the photographed pictures and transmits the pictures to the detecting module 202, and the detecting module 202 detects the characteristic information corresponding to the object carrying disc and each bread. And the carrier module 203 loads candidate classification information corresponding to the bread. And the network user selects and confirms the sample classification information corresponding to the characteristic information of the bread.
Meanwhile, the detection module 202 sends the sample classification information selected and confirmed by the network user from the candidate classification information and the corresponding feature information thereof to a first storage platform for storage and preservation, so as to provide for subsequent training of the model.
After the image data of the new bread of each breed is collected, that is, after the set requirement of the number of model training samples is met, the automatic training platform 206 downloads the sample classification information and the corresponding feature information from the first storage platform 204 and the meta data from the second storage platform 205 for model training to be used by the user 207.
Referring now to fig. 12, a block diagram of a computer system 800 suitable for use in implementing a terminal device/server of an embodiment of the present application is shown. The terminal device/server shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 12, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
According to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program performs the above-described functions defined in the method of the present application when executed by the Central Processing Unit (CPU) 801. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor comprises an acquisition device, a detection module, a loading module, a first storage platform, a second storage platform and an automatic training platform. The names of these units do not in some cases constitute a limitation on the units themselves, for example, the capture means may also be described as "for capturing pictures containing samples uploaded by network users". As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: based on the picture containing the sample uploaded by the network user, obtaining the characteristic information of the sample and the characteristic information of the carrying disc according to the picture, loading the candidate classification information of the characteristic information for the network user to select and confirm, storing the sample classification information selected and confirmed by the network user from the candidate classification information and the corresponding characteristic information, further storing meta data corresponding to the sample classification information loaded by the network user, and calling the characteristic information of the sample, the sample classification information and the meta data training model corresponding to the sample classification information. And the trained model is issued to the user side for use.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. A model-based data processing method for food sales, characterized by: the method comprises the following steps:
s1, acquiring pictures including the object carrying disc and the sample uploaded by the network user;
s2, obtaining characteristic information of the sample in the picture, wherein the characteristic information comprises two-dimensional RGB data and depth data; the step S2 includes at least the following steps:
s21, converting the depth data into three-dimensional point cloud;
s22, acquiring a plane area corresponding to the carrying plate;
s23, stripping points higher than the plane area in the three-dimensional point cloud to obtain a three-dimensional point cloud corresponding to the sample so as to form characteristic information about the sample;
s3, loading the candidate classification information of the characteristic information for the network user to select and confirm;
s4, storing the sample classification information selected and confirmed by the network user from the candidate classification information and the corresponding characteristic information thereof to a first storage platform for subsequent training of the model;
s5, loading meta data corresponding to the sample classification information and storing the meta data to a second storage platform, wherein the meta data are attribute data of the sample corresponding to the sample classification information; and
s6, calling characteristic information of the sample, sample classification information and a meta data training model corresponding to the sample classification information; the first storage platform is a third-party storage platform, the second storage platform is a cloud management system, and the first storage platform is further used for storing the trained model; after the model is trained, sending the serial number corresponding to the model to a second storage platform;
s7, issuing the trained model to a user side;
before the trained model is sent to the user side, the method also comprises the following steps: based on whether the model needs to be updated or not, which is fed back after the user accesses and acquires the model serial number for the second storage platform, if so, the updated model is issued to the user;
the model training end is a commodity development headquarter, and the user end is a commodity sales subsection correspondingly.
2. The model-based data processing method for food sales of claim 1, characterized in that: the number of the samples is one or a plurality of samples which are not stacked with each other.
3. A model-based data processing method for food sales of claim 2, characterized in that: the picture in step S1 further includes a carrier tray image carrying the sample, and in step S2, the characteristic information of the carrier tray image is also detected when the characteristic information of the sample is detected, and step S2 specifically includes the following steps:
s24, generating a circumscribed rectangular frame to mark the sample based on the three-dimensional point cloud corresponding to the sample;
and S25, converting the position information corresponding to each circumscribed rectangle frame into a standard labeling format.
4. A model-based data processing method for food sales of claim 3, characterized in that: when matching the feature information, the sample classification information, and the meta data of a sample with the sample image of the corresponding rectangular frame in training a model using a plurality of images about the sample of one type and their corresponding feature information, sample classification information, and meta data, matching is performed based on the position information and the feature information of the marker, and matching is performed based on the mapping relationship between the position information and the feature information.
5. A data processing apparatus for a model of food sales, characterized by: it includes:
an acquisition device: the method comprises the steps of configuring a picture containing a sample and uploaded by a network user;
a detection module: the characteristic information of the sample and the object carrying plate image is obtained according to the picture;
loading a module: configuring candidate classification information for loading the characteristic information for the network user to select and confirm;
a first storage platform: the system is configured for storing sample classification information selected and confirmed by the network user from the candidate classification information and corresponding characteristic information;
a second storage platform: the method comprises the steps that a network user is configured to load meta data corresponding to the sample classification information, wherein the meta data are attribute data of a sample corresponding to the sample classification information; and
an automatic training platform: configuring a meta data training model used for calling the characteristic information of the sample, the sample classification information and the meta data corresponding to the sample classification information, and sending a serial number corresponding to the model to a second storage platform after the model is trained;
issuing the trained model to a user side;
before the trained model is sent to the user side, the method also comprises the following steps: based on whether the model needs to be updated or not, which is fed back after the user accesses and acquires the model serial number for the second storage platform, if so, the updated model is issued to the user;
the model training end is a commodity development headquarter, and the user end is a commodity sales subsection correspondingly.
6. The data processing apparatus for a model of food sales of claim 5, characterized by: the detection module comprises:
the depth data processing module: configured to convert the depth data into a three-dimensional point cloud;
a plane acquisition module: the three-dimensional point cloud is used for calculating a plane area corresponding to the objective plate image;
a separation module: configuring to strip a portion of the picture above the planar area with the three-dimensional point cloud to obtain a three-dimensional point cloud representative of each sample;
a labeling module: generating a circumscribed rectangular frame to mark the sample based on a three-dimensional point cloud corresponding to the sample;
a conversion module: and the position information corresponding to each circumscribed rectangle frame is configured to be converted into a standard labeling format.
7. The data processing apparatus for a model of food sales of claim 6, characterized by: the first storage platform is further used for storing the trained model and the acquired image data about each sample, and the first storage platform or the second storage platform is further used for storing the mapping relation corresponding to the meta data and the sample classification information.
8. Data processing device of a model for the sale of food items according to any one of claims 5 to 7, characterized in that: the data processing device of the model also comprises a user side, wherein the user side accesses the second storage platform to obtain the model serial number and judges whether to send a model issuing request to the first storage platform according to the model serial number of the second storage platform.
9. An electronic device, characterized in that: comprising one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910041435.3A CN109857880B (en) | 2019-01-16 | 2019-01-16 | Model-based data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910041435.3A CN109857880B (en) | 2019-01-16 | 2019-01-16 | Model-based data processing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109857880A CN109857880A (en) | 2019-06-07 |
| CN109857880B true CN109857880B (en) | 2022-04-05 |
Family
ID=66894949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910041435.3A Active CN109857880B (en) | 2019-01-16 | 2019-01-16 | Model-based data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109857880B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107562838A (en) * | 2017-08-24 | 2018-01-09 | 百度在线网络技术（北京）有限公司 | A kind of method and apparatus for image information collecting |
| CN108132054A (en) * | 2017-12-20 | 2018-06-08 | 百度在线网络技术（北京）有限公司 | For generating the method and apparatus of information |
| JP2018205937A (en) * | 2017-05-31 | 2018-12-27 | 日本放送協会 | Image retrieval device and program |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103412937B (en) * | 2013-08-22 | 2016-12-28 | 成都数之联科技有限公司 | A kind of search purchase method based on handheld terminal |
| CN106250382A (en) * | 2016-01-28 | 2016-12-21 | 新博卓畅技术（北京）有限公司 | A kind of metadata management automotive engine system and implementation method |
| CN107203561B (en) * | 2016-03-18 | 2020-11-24 | 创新先进技术有限公司 | Data processing method and equipment |
| CN107767456A (en) * | 2017-09-22 | 2018-03-06 | 福州大学 | A kind of object dimensional method for reconstructing based on RGB D cameras |
-
2019
- 2019-01-16 CN CN201910041435.3A patent/CN109857880B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2018205937A (en) * | 2017-05-31 | 2018-12-27 | 日本放送協会 | Image retrieval device and program |
| CN107562838A (en) * | 2017-08-24 | 2018-01-09 | 百度在线网络技术（北京）有限公司 | A kind of method and apparatus for image information collecting |
| CN108132054A (en) * | 2017-12-20 | 2018-06-08 | 百度在线网络技术（北京）有限公司 | For generating the method and apparatus of information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109857880A (en) | 2019-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108922026B (en) | Replenishment management method and device for vending machine and user terminal | |
| US10467454B2 (en) | Synchronization of image data from multiple three-dimensional cameras for image recognition | |
| US10116861B1 (en) | Guided image capture user interface | |
| CN107992820B (en) | Self-help goods selling method for container based on binocular vision | |
| EP3518149A1 (en) | Feedback loop for image-based recognition | |
| KR102358607B1 (en) | Artificial intelligence appraisal system, artificial intelligence appraisal method and storage medium | |
| US20150046299A1 (en) | Inventory Assessment with Mobile Devices | |
| CN111178379B (en) | Teaching data generation method and generation device, and commodity identification device | |
| CN110119914B (en) | Object replenishment processing method, device and system | |
| CN108960132B (en) | Method and device for purchasing commodities in open type vending machine | |
| WO2019080674A1 (en) | Self-service checkout device, method, apparatus, medium and electronic device | |
| US11869256B2 (en) | Separation of objects in images from three-dimensional cameras | |
| CN106203225B (en) | Pictorial element based on depth is deleted | |
| WO2020037762A1 (en) | Product information identification method and system | |
| CN110119915B (en) | Object warehousing processing method, device and system | |
| US11748787B2 (en) | Analysis method and system for the item on the supermarket shelf | |
| CN109857880B (en) | Model-based data processing method and device and electronic equipment | |
| JP2016024596A (en) | Information processor | |
| CN109166257B (en) | Shopping cart commodity verification method and device thereof | |
| CN115661624A (en) | Digital method and device for goods shelf and electronic equipment | |
| US20190019339A1 (en) | Systems and methods for dynamically displaying information about an object using augmented reality | |
| CN111126322A (en) | Article identification method, device and equipment applied to unmanned vending device | |
| CN116343198B (en) | Dish identification method and equipment based on multiple models | |
| US11727470B2 (en) | Optical scanning for weights and measures | |
| US20230237558A1 (en) | Object recognition systems and methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20191224 Address after: 200000 569n, building 1, No. 668, SHANGDA Road, Baoshan District, Shanghai Applicant after: Innovation wisdom (Shanghai) Technology Co., Ltd Address before: 315000 Lane 41, Tiantong South Road, Yinzhou District, Ningbo City, Zhejiang Province Applicant before: Innovation Qizhi (Ningbo) Technology Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |