CN109711472A - Training data generation method and device - Google Patents
Training data generation method and device Download PDFInfo
- Publication number
- CN109711472A CN109711472A CN201811628622.3A CN201811628622A CN109711472A CN 109711472 A CN109711472 A CN 109711472A CN 201811628622 A CN201811628622 A CN 201811628622A CN 109711472 A CN109711472 A CN 109711472A
- Authority
- CN
- China
- Prior art keywords
- image
- commodity
- training data
- projected image
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The present disclosure discloses a kind of training data generation method and devices, are related to field of image processing.This method comprises: obtaining the 3D model to be placed to the commodity identified in goods selling equipment；Any angle projection is carried out to the 3D model of commodity, obtains the 2D projected image of multiframe commodity；Image correlation algorithm processing is carried out to the 2D projected images of commodity, using treated image as the training data to be placed to the commodity identified in goods selling equipment.The present disclosure reduces artificial participations, and then improve the formation speed of training data.
Description
Technical field
This disclosure relates to field of image processing more particularly to a kind of training data generation method and device.
Background technique
In order to identify that the commodity increased newly in unmanned counter need to carry out model training in advance.For example, in laboratory environment
Under, staff's modelling customer behavior is taken repeatedly from counter by different angle, different location and different modes and is taken, put
Commodity are returned, and shoot the process using video camera, to obtain original image.Original image is uploaded into mark platform, is marked
The manual frame of personnel selects the rectangular area comprising the commodity, forms labeled data.Then staff is by labeled data and original
Picture is input to model and is trained.
Summary of the invention
The disclosure technical problem to be solved is to provide a kind of training data generation method and device, can be improved instruction
Practice the formation speed of data.
On the one hand according to the disclosure, a kind of training data generation method is proposed, comprising: obtain to be placed in goods selling equipment
The 3D model of the commodity identified；Any angle projection is carried out to the 3D model of commodity, obtains the 2D perspective view of multiframe commodity
Picture；Image correlation algorithm processing is carried out to the 2D projected image of commodity, image is used as to be placed to goods selling equipment by treated
The training data of the middle commodity identified.
In one embodiment, carrying out image correlation algorithm processing to the 2D projected image of commodity includes: the 2D to commodity
At least one processing in projected image progress lighting process, distortion processing and hand-held processing.
In one embodiment, carrying out lighting process to the 2D projected image of commodity includes: to obtain under different illumination conditions
Light change matrix；Each pixel in the 2D projected image of commodity is carried out at offset scaling based on light change matrix
Reason obtains 2D projected image of the commodity under different illumination conditions.
In one embodiment, it includes: true based on camera distortion parameter for carrying out distortion processing to the 2D projected image of commodity
Determine camera distortion matrix；2D projected image of the commodity under different illumination conditions is handled based on camera distortion matrix, is obtained
There is under different illumination conditions the 2D projected image of distortion effect to commodity.
In one embodiment, carrying out hand-held processing to the 2D projected image of commodity includes: to obtain hand-type texture mapping；Base
There is the 2D projected image of distortion effect to handle commodity in hand-type texture mapping under different illumination conditions, is handled
Image afterwards.
In one embodiment, this method further include: image is added at goods selling equipment environmental background to treated
Reason.
In one embodiment, image correlation algorithm processing is carried out to the 2D projected images of commodity, it will treated image
It include: to be added to the 2D projected image of commodity as the training data to be placed to the commodity identified in goods selling equipment
The processing of goods selling equipment environmental background, obtains primary simulation image；Primary simulation image is input to trained production confrontation
Virtual image is exported in network；Using virtual image as the training data to be placed to the commodity identified in goods selling equipment.
In one embodiment, this method further include: primary simulation image and target area image are input to production pair
In the generation network of anti-network, the aiming field virtual image based on primary simulation image is obtained；By aiming field virtual image and mesh
Mark area image is input in the judgement network of production confrontation network, is obtained and is determined network to aiming field virtual image and aiming field
The judgement result of the similarity degree of image；Calculate the penalty values of production confrontation network；The loss of network is fought according to production
Value is adjusted production confrontation network, to obtain trained production confrontation network.
In one embodiment, this method further include: determine opposite seat of the 2D projected image of commodity in different background
Mark information；Using relative co-ordinate information as the labeled data of commodity.
According to another aspect of the present disclosure, it is also proposed that a kind of training data generating means, comprising: 3D model acquiring unit,
It is configured as to be placed to the 3D model of the acquisition commodity identified in goods selling equipment；2D projected image acquiring unit, is matched
It is set to and any angle projection is carried out to the 3D model of commodity, obtain the 2D projected image of multiframe commodity；Image processing unit is matched
It is set to and image correlation algorithm processing is carried out to the 2D projected image of commodity, image is used as to be placed to goods selling equipment by treated
The training data of the middle commodity identified.
In one embodiment, image processing unit is configured as carrying out the 2D projected images of commodity lighting process, abnormal
Become at least one processing in processing and hand-held processing.
In one embodiment, image processing unit is configured as obtaining the light change matrix under different illumination conditions,
Offset scaling processing is carried out to each pixel in the 2D projected image of commodity based on light change matrix, obtains commodity not
With the 2D projected image under the conditions of illumination.
In one embodiment, image processing unit is configured as determining camera distortion matrix based on camera distortion parameter,
2D projected image of the commodity under different illumination conditions is handled based on camera distortion matrix, obtains commodity in different illumination
Under the conditions of have distortion effect 2D projected image.
In one embodiment, image processing unit is configured as obtaining hand-type texture mapping, is based on hand-type texture mapping
There is the 2D projected image of distortion effect to handle under different illumination conditions commodity, the image that obtains that treated.
In one embodiment, training data generating means further include: image processing unit is additionally configured to processing
Image afterwards is added the processing of goods selling equipment environmental background.
In one embodiment, image processing unit is configured as being added goods selling equipment to the 2D projected image of commodity
Environmental background processing, obtains primary simulation image, primary simulation image is input to defeated in trained production confrontation network
Virtual image out, using virtual image as the training data to be placed to the commodity identified in goods selling equipment.
In one embodiment, training data generating means further include: production fights network training unit, is configured
It is fought in the generation network of network for primary simulation image and target area image are input to production, obtains and be based on primary simulation
The aiming field virtual image of image；Aiming field virtual image and target area image are input to the judgement net of production confrontation network
In network, obtains and determine network to the judgement result of the similarity degree of aiming field virtual image and target area image；Calculate production
Fight the penalty values of network；The penalty values that network is fought according to production are adjusted production confrontation network, to be instructed
The production confrontation network perfected.
In one embodiment, training data generating means further include: markup information acquiring unit is configured to determine that
Relative co-ordinate information of the 2D projected image of commodity in different background, using relative co-ordinate information as the labeled data of commodity.
According to another aspect of the present disclosure, it is also proposed that a kind of training data generating means, comprising: memory；And coupling
To the processor of memory, processor is configured as generating based on for example above-mentioned training data of the instruction execution for being stored in memory
Method.
According to another aspect of the present disclosure, it is also proposed that a kind of computer readable storage medium is stored thereon with computer journey
The step of sequence instruction, which realizes above-mentioned training data generation method when being executed by processor.
Compared with prior art, the embodiment of the present disclosure first obtains to be placed to the 3D of the commodity identified in goods selling equipment
Model, then any angle projection is carried out to the 3D model of commodity, the 2D projected image of multiframe commodity is obtained, then to the 2D of commodity
Projected image carries out image correlation algorithm processing, using treated image as to be placed to the quotient identified in goods selling equipment
The training data of product reduces artificial participation, and then improves the formation speed of training data.
By the detailed description referring to the drawings to the exemplary embodiment of the disclosure, the other feature of the disclosure and its
Advantage will become apparent.
Detailed description of the invention
The attached drawing for constituting part of specification describes embodiment of the disclosure, and together with the description for solving
Release the principle of the disclosure.
The disclosure can be more clearly understood according to following detailed description referring to attached drawing, in which:
Fig. 1 is the flow diagram of one embodiment of disclosure training data generation method.
Fig. 2 is the flow diagram of another embodiment of disclosure training data generation method.
Fig. 3 is the flow diagram of the further embodiment of disclosure training data generation method.
Fig. 4 is the structural schematic diagram of one embodiment of disclosure training data generating means.
Fig. 5 is the structural schematic diagram of another embodiment of disclosure training data generating means.
Fig. 6 is the structural schematic diagram of the further embodiment of disclosure training data generating means.
Fig. 7 is the structural schematic diagram of another embodiment of disclosure training data generating means.
Specific embodiment
The various exemplary embodiments of the disclosure are described in detail now with reference to attached drawing.It should also be noted that unless in addition having
Body explanation, the unlimited system of component and the positioned opposite of step, numerical expression and the numerical value otherwise illustrated in these embodiments is originally
Scope of disclosure.
Be to the description only actually of at least one exemplary embodiment below it is illustrative, never as to the disclosure
And its application or any restrictions used.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable
In the case of, the technology, method and apparatus should be considered as authorizing part of specification.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
For the purposes, technical schemes and advantages of the disclosure are more clearly understood, below in conjunction with specific embodiment, and reference
The disclosure is further described in attached drawing.
In the related art, it is manually carried out in the process for obtaining original image and labeled data, needs to expend big
The time is measured, cause the period of more new data long and improves cost.
Fig. 1 is the flow diagram of one embodiment of disclosure training data generation method.
In step 110, the 3D model to be placed to the commodity identified in goods selling equipment is obtained.Goods selling equipment is, for example,
Self-service cabinet, shelf etc..
In one embodiment, commodity can be placed on to the center of a certain enclosed environment, use spatial digitizer
The commodity are directed at, and carry out surrounding shooting, the 3D model of the available commodity.
In another embodiment, single or multiple depth cameras can also be utilized, by adjusting commodity or camera angle
The mode of degree shoots multiple images with depth, to obtain the 3D model of the commodity.Wherein, depth camera, which can be, has master
The optical depth camera of dynamic projection hot spot structure, is also possible to the depth camera based on passive Binocular Vision Principle.
In step 120, any angle projection is carried out to the 3D model of commodity, obtains the 2D projected image of multiframe commodity.I.e.
It is 2D projected image by 3D model conversion.
In step 130, image correlation algorithm processing is carried out to the 2D projected image of commodity, will treated image is used as to
It is placed into the training data of the commodity identified in goods selling equipment.Due to the 2D projected image and true environment of obtained commodity
The commodity image information of middle shooting has certain difference, for example, light differential, camera distortion difference, hand-held difference etc., therefore,
It needs to carry out the 2D projected image of commodity the image procossings such as lighting process, distortion processing and hand-held processing, so that treated
Image is as far as possible close to the commodity image shot in true environment.
In this embodiment, the 3D model to be placed to the commodity identified in goods selling equipment is first obtained, then to commodity
3D model carry out any angle projection, obtain the 2D projected image of multiframe commodity, then the 2D projected images of commodity carried out
Image correlation algorithm processing, using treated image as the training number to be placed to the commodity identified in goods selling equipment
According to reducing artificial participation, and then improve the formation speed of training data.
Fig. 2 is the flow diagram of another embodiment of disclosure training data generation method.
In step 210, commodity are placed in articles holding table with certain posture.
In step 220, commodity are shot using depth camera.
In step 230, judges whether to complete shooting, if so, thening follow the steps 240, otherwise, continue to execute step 220.
In step 240, the 3D model of commodity is generated.
In step 250, the 3D model of commodity is projected into two-dimensional scene with any angle, obtains the 2D perspective view of commodity
Picture.Since the 3D model to commodity carries out any angle projection, the 2D projected image of multiframe commodity can be obtained.Then right
Each frame image carries out lighting process, distortion processing or hand-held processing, specific implementation such as step 260-2110.
In step 260, the light change matrix under different illumination conditions is obtained.
In step 270, offset contracting is carried out to each pixel in the 2D projected image of commodity based on light change matrix
Processing is put, 2D projected image of the commodity under different illumination conditions is obtained.
In one embodiment, it can use formula (1) and obtain 2D projected image of the commodity under different illumination conditions.
Wherein, (x, y) is the coordinate of the 2D projected image of commodity, and (x ', y ') is 2D of the commodity under different illumination conditions
The coordinate of projected image, sx、syIndicate zoom factor, dx、dyIndicate offset.
Lighting issues are to influence one of an important factor for unmanned counter commodity identify accuracy, under different illumination conditions, are clapped
The brightness for taking the photograph image has certain deviation.In a practical situation, scheme if being acquired under different illumination conditions to each commodity
Picture, it is both time- and labor-consuming, therefore, the light change matrix application under different illumination conditions can be projected in different angle and be generated
Image, offset scaling is carried out to each pixel color value in image, so that same commodity can be obtained in different illumination items
Image under part.
In one embodiment, the image information that can acquire a certain commodity under true illumination condition in advance, is then obtained
The 2D projected image for obtaining the commodity becomes according to the illumination under the available different illumination conditions of difference between image between pixel
Change matrix.In subsequent applications, can be directly using the light change matrix to the 2D projected image of the commodity of acquisition at
Reason.
In step 280, camera distortion matrix is determined based on camera distortion parameter.
When being 2D image by 3D model conversion, can be converted using projective transformation matrix, but obtained by this method
The image that is shot with different angle of image can be variant.Projective transformation matrix be based on ideal situation under the premise of obtain,
And in real life, due to the difference of the construction of camera, manufacture craft etc., will lead to the picture that camera is shot, there are a variety of non-
Linear distortion, therefore be only unable to accurate description with projective transformation matrix and be ultimately imaged relationship, need the distortion parameter using camera
The distortion matrix of camera is calculated, then 2D projected image is handled.
In step 290,2D projected image of the commodity under different illumination conditions is handled based on camera distortion matrix,
Obtain the 2D projected image that commodity have distortion effect under different illumination conditions.
For example, obtaining the 2D projected image that commodity have distortion effect under different illumination conditions using formula (2).
Wherein, (xp, yp) be different illumination conditions under 2D projected image coordinate, (xd, yd) it is commodity in different illumination
Under the conditions of have distortion effect 2D projected image coordinate, k1、k2、k3For coefficient of radial distortion, p1、p2For tangential distortion system
Number, r are with a distance from center of distortion, wherein r2=x2+y2。
In step 2100, hand-type texture mapping is obtained.
In step 2110, there is the 2D of distortion effect to project under different illumination conditions commodity based on hand-type texture mapping
Image is handled, the image that obtains that treated.2D projected image is handled using various hand-type texture mapping, so that raw
At image have the effect of that manpower takes commodity so that image is closer to the commodity image shot in true environment.
In step 2120, using treated image as the training data to be placed to the commodity identified in counter.
In this embodiment, the 3D model of commodity is first obtained, then any angle projection is carried out to the 3D model of commodity, is obtained
Then the 2D projected image of commodity carries out lighting process, distortion is handled and holds the images such as processing to the 2D projected image of commodity
Processing can be improved trained number using treated image as the training data to be placed to the commodity identified in counter
According to formation speed.
It in one embodiment, can also be to treated image is added goods selling equipment environmental background processing.It will
Treated, and image is placed into different backgrounds, so that treated image is as far as possible close to the commodity shot in true environment
Image.
Fig. 3 is the flow diagram of the further embodiment of disclosure training data generation method.
In step 310, the 3D model of commodity is obtained.
In step 320, any angle projection is carried out to the 3D model of commodity, obtains the 2D projected image of multiframe commodity.
In step 330, the 2D projected image of commodity is added the processing of counter environmental background, obtains primary simulation figure
Picture.The 2D projected image of commodity is placed into different environmental backgrounds and obtains primary simulation image.
In step 340, primary simulation image is input in trained production confrontation network and exports virtual image.
In one embodiment, it is necessary to be first trained to production confrontation network.Such as using primary simulation image and
Target area image to production confrontation network be trained so that production confrontation network in output virtual image as far as possible close to
True picture, so that the virtual image of output has the effect of similar illumination, camera distortion and hand-held commodity, wherein target
Area image is true picture.
It includes generating network and differentiation network that production, which fights network,.In one embodiment, by primary simulation image and
Target area image is input in the generation network of production confrontation network, obtains the aiming field virtual graph based on primary simulation image
Picture；Aiming field virtual image and target area image are input in the judgement network of production confrontation network, obtain and determine network
To the judgement result of the similarity degree of aiming field virtual image and target area image；Calculate the penalty values of production confrontation network；
The penalty values that network is fought according to production constantly adjust production confrontation network, until production fights network mould
Type convergence, to obtain trained production confrontation network.
In step 350, using virtual image as the training data to be placed to the commodity identified in counter.
In this embodiment, the 3D model of commodity is first obtained, then any angle projection is carried out to the 3D model of commodity, is obtained
Then the 2D projected image of commodity is carried out background process by the 2D projected image of multiframe commodity, obtain primary simulation image, and will
Primary simulation image is input in trained production confrontation network and exports virtual image, so that the virtual image of output
Close to the commodity image shot in true environment, reduce manual operation, improves training data formation efficiency, reduce into
This.
In another embodiment, relative co-ordinate information of the 2D projected image of commodity in different background is determined；By phase
Labeled data to coordinate information as commodity.For example, the 2D projected image of commodity to be placed on to different counter environmental backgrounds
In, since 2D projected image only includes commodity, when being synthesized 2D projected image and various backgrounds, can write down
The coordinate of commodity in the background obtains being similar to the coordinate that manually marks, therefore, the step of eliminating artificial mark, can be more
The acquisition labeled data of automation.
Fig. 4 is the structural schematic diagram of one embodiment of disclosure training data generating means.The training data generates dress
It sets including 3D model acquiring unit 410,2D projected image acquiring unit 420 and image processing unit 430.
The 3D mould that 3D model acquiring unit 410 is configured as obtaining to be placed to the commodity identified in goods selling equipment
Type.For example, obtaining the 3D model of commodity using spatial digitizer shooting commodity, or single or multiple depth cameras are used, led to
The mode of toning integral quotient product or camera angle shoots multiple images with depth, to obtain the 3D model of the commodity.
2D projected image acquiring unit 420 is configured as carrying out any angle projection to the 3D model of commodity, obtains multiframe
The 2D projected image of commodity.
Image processing unit 430 is configured as carrying out image correlation algorithm processing to the 2D projected image of commodity, will handle
Image afterwards is as the training data to be placed to the commodity identified in goods selling equipment.Due to the 2D projection of obtained commodity
The commodity image information shot in image and true environment has certain difference, for example, light differential, camera distortion difference, hand
Difference etc. is held, therefore, it is necessary to the 2D projected images to commodity to carry out at the images such as lighting process, distortion processing and hand-held processing
Reason, so that treated image is as far as possible close to the commodity image shot in true environment.
In this embodiment, the 3D model of commodity is first obtained, then any angle projection is carried out to the 3D model of commodity, is obtained
Then the 2D projected image of multiframe commodity carries out image procossing to the 2D projected images of commodity, will treated image is used as to
It is placed into the training data of the commodity identified in goods selling equipment, reduces artificial participation, and then improve training data
Formation speed.
In another embodiment of the disclosure, image processing unit 430 is configured as obtaining under different illumination conditions
Light change matrix carries out at offset scaling each pixel in the 2D projected image of commodity based on light change matrix
Reason obtains 2D projected image of the commodity under different illumination conditions.
Lighting issues are to influence one of an important factor for unmanned counter commodity identify accuracy, under different illumination conditions, are clapped
The brightness for taking the photograph image has certain deviation.In a practical situation, scheme if being acquired under different illumination conditions to each commodity
Picture, it is both time- and labor-consuming, therefore, the light change matrix application under different illumination conditions can be projected in different angle and be generated
Image, offset scaling is carried out to each pixel color value in image, so that same commodity can be obtained in different illumination items
Image under part.
In another embodiment, image processing unit 430 is configured as determining camera distortion based on camera distortion parameter
Matrix handles 2D projected image of the commodity under different illumination conditions based on camera distortion matrix, obtains commodity not
With the 2D projected image under the conditions of illumination with distortion effect.
When being 2D image by 3D model conversion, can be converted using projective transformation matrix, but obtained by this method
The image that is shot with different angle of image can be variant.Projective transformation matrix be based on ideal situation under the premise of obtain,
And in real life, due to the difference of the construction of camera, manufacture craft etc., will lead to the picture that camera is shot, there are a variety of non-
Linear distortion, therefore be only unable to accurate description with projective transformation matrix and be ultimately imaged relationship, need the distortion parameter using camera
The distortion matrix of camera is calculated, then 2D projected image is handled, obtain commodity has under different illumination conditions
The 2D projected image for the effect that distorts.
In another embodiment, image processing unit 430 is configured as obtaining hand-type texture mapping, is based on hand-type texture
Textures there is the 2D projected image of distortion effect to handle under different illumination conditions commodity, the image that obtains that treated.
For example, being handled using various hand-type texture mapping 2D projected image, so that there is the image generated manpower to take commodity
Effect, so that image is closer to the commodity image shot in true environment.
In this embodiment, the 3D model of commodity is first obtained, then any angle projection is carried out to the 3D model of commodity, is obtained
Then the 2D projected image of multiframe commodity handles and holds the images such as processing to the 2D projected image lighting process of commodity, distortion
Processing can be improved instruction using treated image as the training data to be placed to the commodity identified in goods selling equipment
Practice the formation speed of data.
In one embodiment, image processing unit 430 is additionally configured to that image is added dress of selling goods to treated
Set environmental background processing.Will treated that image is placed into different backgrounds so that treated image as far as possible close to
The commodity image shot in true environment.
Fig. 5 is the structural schematic diagram of another embodiment of disclosure training data generating means.The training data generates
Device includes outside 3D model acquiring unit 410,2D projected image acquiring unit 420 and image processing unit 430, further includes generating
Formula fights network training unit 510.
In the embodiment, image processing unit 430 is configured as the 2D projected image of commodity being added goods selling equipment
Environmental background processing, obtains primary simulation image, primary simulation image is input to defeated in trained production confrontation network
Virtual image out, using virtual image as the training data to be placed to the commodity identified in goods selling equipment.
Production confrontation network training unit 510 is configured as primary simulation image and target area image being input to generation
Formula is fought in the generation network of network, and the aiming field virtual image based on primary simulation image is obtained；By aiming field virtual image
It is input to target area image in the judgement network of production confrontation network, obtains and determine network to aiming field virtual image and mesh
Mark the judgement result of the similarity degree of area image；Calculate the penalty values of production confrontation network；Network is fought according to production
Penalty values are adjusted production confrontation network, to obtain trained production confrontation network.
In this embodiment, the 3D model of commodity is first obtained, then any angle projection is carried out to the 3D model of commodity, is obtained
Then the 2D projected image of commodity is carried out background process by the 2D projected image of multiframe commodity, obtain primary simulation image, and will
Primary simulation image is input in trained production confrontation network and exports virtual image, so that the virtual image of output
Close to the commodity image shot in true environment, reduce manual operation, improves training data formation efficiency.
Fig. 6 is the structural schematic diagram of the further embodiment of disclosure training data generating means.The training data generates
Device further includes markup information acquiring unit 610, is configured to determine that the 2D projected image of commodity is opposite in different background
Coordinate information, using relative co-ordinate information as the labeled data of commodity.For example, the 2D projected image of commodity is placed on different
In counter environmental background, since 2D projected image only includes commodity, synthesized by 2D projected image and various backgrounds
When, the coordinate of commodity in the background can be write down and obtain therefore eliminating the step manually marked similar to the coordinate manually marked
Suddenly, the acquisition labeled data that can more automate.
Fig. 7 is the structural schematic diagram of another embodiment of disclosure training data generating means.The training data generates
Device includes memory 710 and processor 720, wherein and processor 720 is configured as based on instruction stored in memory,
Execute the method in any one aforementioned embodiment.
Wherein, memory 710 is such as may include system storage, fixed non-volatile memory medium.System storage
Device is for example stored with operating system, application program, Boot loader (Boot Loader) and other programs etc..
Some embodiments of the present disclosure propose a kind of computer readable storage medium, are stored thereon with computer program, should
The method in any one aforementioned embodiment is realized when program is executed by processor.
Those skilled in the art should be understood that embodiment of the disclosure can provide as method, system or computer journey
Sequence product.Therefore, complete hardware embodiment, complete software embodiment or combining software and hardware aspects can be used in the disclosure
The form of embodiment.Moreover, it wherein includes the calculating of computer usable program code that the disclosure, which can be used in one or more,
Machine can use the meter implemented in non-transient storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of calculation machine program product.
The foregoing is merely the preferred embodiments of the disclosure, not to limit the disclosure, all spirit in the disclosure and
Within principle, any modification, equivalent replacement, improvement and so on be should be included within the protection scope of the disclosure.
Claims (20)
1. a kind of training data generation method, comprising:
Obtain the 3D model to be placed to the commodity identified in goods selling equipment；
Any angle projection is carried out to the 3D model of the commodity, obtains the 2D projected image of commodity described in multiframe；
Image correlation algorithm processing is carried out to the 2D projected image of the commodity, image is used as to be placed to selling goods by treated
The training data of the commodity identified in device.
2. training data generation method according to claim 1, wherein carry out image to the 2D projected image of the commodity
Related algorithm is handled
To at least one processing in the 2D projected image progress lighting process of the commodity, distortion processing and hand-held processing.
3. training data generation method according to claim 2, wherein carry out illumination to the 2D projected image of the commodity
Processing includes:
Obtain the light change matrix under different illumination conditions；
Offset scaling processing is carried out to each pixel in the 2D projected image of the commodity based on the light change matrix,
Obtain 2D projected image of the commodity under different illumination conditions.
4. training data generation method according to claim 3, wherein distort to the 2D projected image of the commodity
Processing includes:
Camera distortion matrix is determined based on camera distortion parameter；
2D projected image of the commodity under different illumination conditions is handled based on the camera distortion matrix, obtains institute
State the 2D projected image that commodity have distortion effect under different illumination conditions.
5. training data generation method according to claim 4, wherein held to the 2D projected image of the commodity
Processing includes:
Obtain hand-type texture mapping；
Based on the hand-type texture mapping to the commodity under different illumination conditions have distortion effect 2D projected image into
Row processing obtains treated the image.
6. training data generation method according to claim 2, further includes:
To treated image the is added goods selling equipment environmental background processing.
7. training data generation method according to claim 1, wherein carry out image to the 2D projected image of the commodity
Related algorithm processing, using treated image as the training data packet to be placed to the commodity identified in goods selling equipment
It includes:
The processing of goods selling equipment environmental background is added to the 2D projected image of the commodity, obtains primary simulation image；
The primary simulation image is input in trained production confrontation network and exports virtual image；
Using the virtual image as the training data to be placed to the commodity identified in goods selling equipment.
8. training data generation method according to claim 7, further includes:
The primary simulation image and target area image are input in the generation network of the production confrontation network, obtain base
In the aiming field virtual image of primary simulation image；
The aiming field virtual image and target area image are input in the judgement network of the production confrontation network, are obtained
It is described to determine network to the judgement result of the similarity degree of aiming field virtual image and target area image；
Calculate the penalty values of the production confrontation network；
The penalty values that network is fought according to the production are adjusted production confrontation network, to obtain trained life
An accepted way of doing sth fights network.
9. -8 any training data generation method according to claim 1, further includes:
Determine relative co-ordinate information of the 2D projected image of the commodity in different background；
Using the relative co-ordinate information as the labeled data of the commodity.
10. a kind of training data generating means, comprising:
3D model acquiring unit is configured as obtaining the 3D model to be placed to the commodity identified in goods selling equipment；
2D projected image acquiring unit is configured as carrying out any angle projection to the 3D model of the commodity, obtains multiframe institute
State the 2D projected image of commodity；
Image processing unit is configured as carrying out image correlation algorithm processing to the 2D projected image of the commodity, after processing
Image as the training data to be placed to the commodity identified in goods selling equipment.
11. training data generating means according to claim 10, wherein
Described image processing unit is configured as carrying out the 2D projected image of the commodity lighting process, distortion processing and hold
At least one processing in processing.
12. training data generating means according to claim 11, wherein
Described image processing unit is configured as obtaining the light change matrix under different illumination conditions, is based on the light change
Matrix carries out offset scaling processing to each pixel in the 2D projected image of the commodity, obtains the commodity and is not sharing the same light
2D projected image according under the conditions of.
13. training data generating means according to claim 12, wherein
Described image processing unit is configured as determining camera distortion matrix based on camera distortion parameter, is based on the camera distortion
Matrix handles 2D projected image of the commodity under different illumination conditions, obtains the commodity in different illumination conditions
The lower 2D projected image with distortion effect.
14. training data generating means according to claim 13, wherein
Described image processing unit is configured as obtaining hand-type texture mapping, is existed based on the hand-type texture mapping to the commodity
There is the 2D projected image of distortion effect to be handled under different illumination conditions, obtain treated the image.
15. training data generating means according to claim 11, further includes:
Described image processing unit is additionally configured to treated image the is added goods selling equipment environmental background processing.
16. training data generating means according to claim 10, wherein
Described image processing unit is configured as being added at goods selling equipment environmental background the 2D projected image of the commodity
Reason, obtains primary simulation image, and it is virtual that the primary simulation image is input to output in trained production confrontation network
Image, using the virtual image as the training data to be placed to the commodity identified in goods selling equipment.
17. training data generating means according to claim 16, further includes:
Production fights network training unit, is configured as the primary simulation image and target area image being input to the life
An accepted way of doing sth is fought in the generation network of network, and the aiming field virtual image based on primary simulation image is obtained；The aiming field is empty
Quasi- image and target area image are input in the judgement network of the production confrontation network, obtain the judgement network to target
The judgement result of the similarity degree of domain virtual image and target area image；Calculate the penalty values of the production confrontation network；Root
According to the penalty values of production confrontation network, production confrontation network is adjusted, to obtain trained production pair
Anti- network.
18. any training data generating means of 0-17 according to claim 1, further includes:
Markup information acquiring unit is configured to determine that relative coordinate letter of the 2D projected image of the commodity in different background
Breath, using the relative co-ordinate information as the labeled data of the commodity.
19. a kind of training data generating means, comprising:
Memory；And
It is coupled to the processor of the memory, the processor is configured to based on the instruction execution for being stored in the memory
Training data generation method as described in any one of claim 1 to 9.
20. a kind of computer readable storage medium, is stored thereon with computer program instructions, real when which is executed by processor
The step of existing claim 1 to 9 described in any item training data generation methods.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811628622.3A CN109711472B (en) | 2018-12-29 | 2018-12-29 | Training data generation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811628622.3A CN109711472B (en) | 2018-12-29 | 2018-12-29 | Training data generation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109711472A true CN109711472A (en) | 2019-05-03 |
| CN109711472B CN109711472B (en) | 2021-07-13 |
Family
ID=66259381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811628622.3A Active CN109711472B (en) | 2018-12-29 | 2018-12-29 | Training data generation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109711472B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110222757A (en) * | 2019-05-31 | 2019-09-10 | 华北电力大学（保定） | Based on insulator image pattern extending method, the system for generating confrontation network |
| CN111160261A (en) * | 2019-12-30 | 2020-05-15 | 北京每日优鲜电子商务有限公司 | Sample image labeling method and device for automatic sales counter and storage medium |
| CN112529097A (en) * | 2020-12-23 | 2021-03-19 | 北京百度网讯科技有限公司 | Sample image generation method and device and electronic equipment |
| CN113469644A (en) * | 2021-06-18 | 2021-10-01 | 深圳市点购电子商务控股股份有限公司 | Shelf life reminding method, automatic vending device, computer equipment and storage medium |
| CN113506400A (en) * | 2021-07-05 | 2021-10-15 | 深圳市点购电子商务控股股份有限公司 | Automatic vending method, automatic vending device, computer equipment and storage medium |
| US11568578B2 (en) | 2020-12-28 | 2023-01-31 | Industrial Technology Research Institute | Method for generating goods modeling data and goods modeling data generation device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2002041269A1 (en) * | 2000-11-15 | 2002-05-23 | Technology Tree Co., Ltd. | Vending machine with half mirror signboard |
| CN101271469A (en) * | 2008-05-10 | 2008-09-24 | 深圳先进技术研究院 | Two-dimension image recognition based on three-dimensional model warehouse and object reconstruction method |
| CN105930382A (en) * | 2016-04-14 | 2016-09-07 | 严进龙 | Method for searching for 3D model with 2D pictures |
| CN108009222A (en) * | 2017-11-23 | 2018-05-08 | 浙江工业大学 | Method for searching three-dimension model based on more excellent view and depth convolutional neural networks |
| CN108399432A (en) * | 2018-02-28 | 2018-08-14 | 成都果小美网络科技有限公司 | Object detecting method and device |
| CN108492451A (en) * | 2018-03-12 | 2018-09-04 | 远瞳（上海）智能技术有限公司 | Automatic vending method |
| CN108509848A (en) * | 2018-02-13 | 2018-09-07 | 视辰信息科技(上海)有限公司 | The real-time detection method and system of three-dimension object |
| CN108961547A (en) * | 2018-06-29 | 2018-12-07 | 深圳和而泰数据资源与云技术有限公司 | A kind of commodity recognition method, self-service machine and computer readable storage medium |
-
2018
- 2018-12-29 CN CN201811628622.3A patent/CN109711472B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2002041269A1 (en) * | 2000-11-15 | 2002-05-23 | Technology Tree Co., Ltd. | Vending machine with half mirror signboard |
| CN101271469A (en) * | 2008-05-10 | 2008-09-24 | 深圳先进技术研究院 | Two-dimension image recognition based on three-dimensional model warehouse and object reconstruction method |
| CN105930382A (en) * | 2016-04-14 | 2016-09-07 | 严进龙 | Method for searching for 3D model with 2D pictures |
| CN108009222A (en) * | 2017-11-23 | 2018-05-08 | 浙江工业大学 | Method for searching three-dimension model based on more excellent view and depth convolutional neural networks |
| CN108509848A (en) * | 2018-02-13 | 2018-09-07 | 视辰信息科技(上海)有限公司 | The real-time detection method and system of three-dimension object |
| CN108399432A (en) * | 2018-02-28 | 2018-08-14 | 成都果小美网络科技有限公司 | Object detecting method and device |
| CN108492451A (en) * | 2018-03-12 | 2018-09-04 | 远瞳（上海）智能技术有限公司 | Automatic vending method |
| CN108961547A (en) * | 2018-06-29 | 2018-12-07 | 深圳和而泰数据资源与云技术有限公司 | A kind of commodity recognition method, self-service machine and computer readable storage medium |
Non-Patent Citations (4)
| Title |
|---|
| GUAN PANG，ULRICH NEUMANN: "3D point cloud object detection with multi-view convolutional neural network", 《2016 23RD INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR)》 * |
| 于洪伟: "基于Triplet-CNN的三维模型检索", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 李启炎: "《AutoCAD 2000三维建模与深入运用》", 31 January 2002, 同济大学出版社 * |
| 马利庄: "《数字动画创作与后期视频处理技术》", 30 September 2018 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110222757A (en) * | 2019-05-31 | 2019-09-10 | 华北电力大学（保定） | Based on insulator image pattern extending method, the system for generating confrontation network |
| CN111160261A (en) * | 2019-12-30 | 2020-05-15 | 北京每日优鲜电子商务有限公司 | Sample image labeling method and device for automatic sales counter and storage medium |
| CN112529097A (en) * | 2020-12-23 | 2021-03-19 | 北京百度网讯科技有限公司 | Sample image generation method and device and electronic equipment |
| CN112529097B (en) * | 2020-12-23 | 2024-03-26 | 北京百度网讯科技有限公司 | Sample image generation method and device and electronic equipment |
| US11568578B2 (en) | 2020-12-28 | 2023-01-31 | Industrial Technology Research Institute | Method for generating goods modeling data and goods modeling data generation device |
| CN113469644A (en) * | 2021-06-18 | 2021-10-01 | 深圳市点购电子商务控股股份有限公司 | Shelf life reminding method, automatic vending device, computer equipment and storage medium |
| CN113506400A (en) * | 2021-07-05 | 2021-10-15 | 深圳市点购电子商务控股股份有限公司 | Automatic vending method, automatic vending device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109711472B (en) | 2021-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109711472A (en) | Training data generation method and device | |
| Zheng et al. | Hybridfusion: Real-time performance capture using a single depth sensor and sparse imus | |
| CN108509848B (en) | The real-time detection method and system of three-dimension object | |
| US11640672B2 (en) | Method and system for wireless ultra-low footprint body scanning | |
| CN106030661B (en) | The independent 3D scene texture background in the visual field | |
| CN103140879B (en) | Information presentation device, digital camera, head mounted display, projecting apparatus, information demonstrating method and information are presented program | |
| CN104781849B (en) | Monocular vision positions the fast initialization with building figure (SLAM) simultaneously | |
| CN103181157B (en) | Unmarked augmented reality system and its operating method based on flatness of the response | |
| JP5746477B2 (en) | Model generation device, three-dimensional measurement device, control method thereof, and program | |
| Pujades et al. | The virtual caliper: rapid creation of metrically accurate avatars from 3D measurements | |
| CN107315470B (en) | Graphic processing method, processor and virtual reality system | |
| DK2686834T3 (en) | IMPROVED VIRTUAL TEST ON SIMULATION DEVICE | |
| US8571698B2 (en) | Simple techniques for three-dimensional modeling | |
| CN109313814A (en) | Camera calibration system | |
| CN105094335B (en) | Situation extracting method, object positioning method and its system | |
| JP2002058045A (en) | System and method for entering real object into virtual three-dimensional space | |
| CN108364319A (en) | Scale determines method, apparatus, storage medium and equipment | |
| WO2008145952A1 (en) | Apparatus and method of body characterisation | |
| JP2017096725A (en) | Information processor and information processing method | |
| WO2021238171A1 (en) | Image registration method and related model training method, device and apparatus | |
| JP2018129009A (en) | Image compositing device, image compositing method, and computer program | |
| JP4109075B2 (en) | Method for measuring the rotation characteristics and flight characteristics of a sphere, and a device for measuring the rotation characteristics and flight characteristics of a sphere | |
| CN107633532A (en) | A kind of point cloud fusion method and system based on white light scanning instrument | |
| CN112669436A (en) | Deep learning sample generation method based on 3D point cloud | |
| AU2017260525B2 (en) | Method and system for body scanning and display of biometric data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |