CN110378952A - A kind of image processing method and device - Google Patents
A kind of image processing method and device Download PDFInfo
- Publication number
- CN110378952A CN110378952A CN201910619862.5A CN201910619862A CN110378952A CN 110378952 A CN110378952 A CN 110378952A CN 201910619862 A CN201910619862 A CN 201910619862A CN 110378952 A CN110378952 A CN 110378952A
- Authority
- CN
- China
- Prior art keywords
- image
- under test
- object under
- pixel
- target area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a kind of image processing method and devices, wherein method includes: the first image and the second image for obtaining photographic device shooting object under test and obtaining by the scene where object under test when predeterminated position, the corresponding target area of object under test is determined from the first image, and according in the first image target area and the corresponding depth information of the second objective area in image, determine the volume of object under test.In the embodiment of the present invention, by being measured in real time under motion state, so that the dependence to environment is smaller, the object volume under complex environment is detected so as to realize, and no setting is required measuring station and fixed test position, so as to improve the efficiency of deposit；In addition, shooting to obtain a color image by photographic device and a depth image can determine the volume of object under test, it is not necessarily to staff, and no setting is required that multiple photographic devices are easy to implement so that cost is relatively low, and easy to operate.
Description
Technical field
The present invention relates to financial technology (Fintech) technical field more particularly to a kind of image processing methods and device.
Background technique
With the development of computer technology, more and more technical applications are in financial field, and traditional financial industry is also
Gradually change to financial technology (Fintech), however, due to the safety of financial industry, requirement of real-time, so that financial technology
To technology, higher requirements are also raised in field.Financial technology field generally involves the Auto-Memory problem of cargo, such as
Collateral security is stored in front of warehouse, it usually needs the volume (and/or quantity) of detection collateral security in advance is pledged so as to basis
The volume of product is the shelf that collateral security distributes corresponding volume, improves the flexibility of collateral security Auto-Memory.In that case, if
The accuracy of the volume detection of collateral security is lower, may will affect the efficiency of collateral security Auto-Memory, in some instances it may even be possible to can make certainly
The wasting of resources of the process check or warehouse of dynamic deposit.
In a kind of existing implementation, measuring station, detection can be set in a certain position on route of depositing in advance
Fixed test position is provided in standing, and can in each orientation (such as front, rear, the left and right) around fixed test position
Image of multiple camera is respectively set.In specific implementation, before collateral security is stored in warehouse, it can be controlled and be pitched by staff
Vehicle carries collateral security and moves to the fixed test position in measuring station, in this way, the photographic device around fixed test position can be shot
Collateral security obtains plurality of pictures, can determine the volume of collateral security by analyzing multiple images i.e..However, this kind of implementation due to
Needing to guarantee that collateral security remains static can shoot, thus the process of depositing may need to take a substantial amount of time, manpower at
Originally with material resources cost, cause the efficiency of deposit lower；And this kind of implementation to the condition depended around fixed test position compared with
Greatly, it cannot achieve and the object volume under complex environment detected.
To sum up, a kind of image processing method is needed at present, and the object volume under complex environment is detected to realize,
And improve the efficiency of deposit.
Summary of the invention
The embodiment of the present invention provides a kind of image processing method and device, to realize to the physical quantities under complex environment
And/or volume is detected, and improves the efficiency of deposit.
In a first aspect, a kind of image processing method provided in an embodiment of the present invention, comprising:
Obtain that scene when photographic device shooting object under test passes through predeterminated position where the object under test obtains the
One image and the second image, scene corresponding color image of the first image where the object under test, described second
Image is the corresponding depth image of scene where the object under test；Further, according to pixel each in the first image
Characteristic value, the corresponding target area of the object under test is determined from the first image, and then according to the target area
And the corresponding depth information in target area described in second image, obtain the volume of the object under test.
In above-mentioned design, by realizing measurement process in real time under motion state, so that above-mentioned design is to environment
It relies on smaller, the object volume under complex environment is detected so as to realize, and no setting is required measuring station and fixed inspection
Location, so as to improve the efficiency of deposit；In addition, above-mentioned design shoots to obtain a color image and one by photographic device
Depth image is the volume that can determine object under test, is not necessarily to staff, and no setting is required multiple photographic devices, thus cost
It is lower and easy to operate, it is easy to implement.
It is described that the corresponding target area of the object under test is obtained from the first image in a kind of possible design
Domain, comprising: the first image is inputted into the first preset model, so that the first preset model is according to each in the first image
The first image is divided into one or more subgraphs by the characteristic value of pixel；It is every in one or more of subgraphs
A subgraph includes at least one object；Further, obtain from one or more of subgraphs includes the determinand
The target subgraph of body, and the target subgraph is inputted into the second preset model, obtain the corresponding target of the object under test
Region.
In above-mentioned design, simple model analysis is carried out to the first image by the first preset model, it can be from multiple areas
Obtain the rough region where object under test in domain, so by the rough region where the second preset model measuring targets into
The analysis of row accurate model, the precise region where available object under test；It follows that above-mentioned design is pre- by setting first
If model and the second preset model, it may not need and accurate model analysis is carried out to entire first image, so as to realize mould
The efficiency of model analysis is improved while the accuracy of type analysis.
It is described that the target subgraph is inputted into the second preset model in a kind of possible design, it obtains described to be measured
The corresponding target area of object, comprising: it is down-sampled to target subgraph progress, the first subgraph is obtained, and described in determination
Classification belonging to multiple first pixels that first subgraph includes, and then first subgraph is up-sampled to obtain
Two subgraphs；Second subgraph includes multiple second pixels, the first pixel of each of the multiple first pixel
At least one corresponding second pixel, classification belonging at least one corresponding second pixel of each first pixel with
Classification belonging to each first pixel is identical；Further, it is pre- for classification being chosen from the multiple second pixel
If the second pixel of one or more targets of classification, and according to one or more of the second pixels of target obtain it is described to
Survey the corresponding target area of object；The pre-set categories are classification belonging to the object under test.
In above-mentioned design, lesser first son of resolution ratio is converted by target subgraph by using down-sampled mode
Image, so that image procossing can be reduced by analyzing to obtain classification belonging to pixel based on lesser first subgraph of resolution ratio
Complexity improves the efficiency of image procossing；And it is larger by using the mode of up-sampling by the first subgraph to convert resolution ratio
The second subgraph, the pixel of block of pixels and the first subgraph on the second subgraph can be made mutually to map, in this way, can
To determine the classification of each pixel in the second subgraph according to mapping relations, the accuracy of image procossing thereby may be ensured that.
In a kind of possible design, the target area according to the target area and second image
Corresponding depth information determines the volume of the object under test, comprising: the target area is divided into one or more nets
Lattice, according to the corresponding depth letter of one or more grid described in one or more of grids and second image
Breath determines the volume of one or more of grids, sums up to obtain to the volume of one or more of grids described to be measured
The volume of object.
It, can be according to the depth information of multiple grids by the way that target area is divided into multiple grids in above-mentioned design
The volume of multiple grids is determined respectively, to sum up the body that object under test can be obtained by the volume to multiple grids
Product；The process obtains the volume of object under test by unit of grid, can make the volume of determining obtained object under test more
Accurately, and realize that logic is relatively simple, so as to improve the efficiency of processing.
It is described that the corresponding target area of the object under test is determined from the first image in a kind of possible design
Before, also the first image is filtered.
In above-mentioned design, by being filtered in advance to the first image before handling the first image, it can be adapted for
What the scene of the first image dynamic fuzzy, i.e. the first image can obtain for shooting in motion process；In this way, above-mentioned design can be with
It executes during measuring targets are deposited, (for example is needed in stationary state without changing original work flow
Execute, or need to move to measuring station execution), so as to improve the efficiency of image procossing.
Second aspect, a kind of image processing apparatus provided in an embodiment of the present invention, described device include:
Module is obtained, when for obtaining photographic device shooting object under test by predeterminated position where the object under test
The first image and the second image that scene obtains, the first image are the corresponding cromogram of scene where the object under test
Picture, second image are the corresponding depth image of scene where the object under test；
Determining module determines institute for the characteristic value according to pixel each in the first image from the first image
State the corresponding target area of object under test；
Processing module is used for the corresponding depth in target area according to the target area and second image
Information obtains the volume of the object under test.
In a kind of possible design, the determining module is specifically used for: the first image is inputted the first default mould
Type, so that the first image is divided into one according to the characteristic value of pixel each in the first image by the first preset model
A or multiple subgraphs；Each subgraph in one or more of subgraphs includes at least one object；Further, from
The target subgraph including the object under test is obtained in one or more of subgraphs, and the target subgraph is inputted
Second preset model obtains the corresponding target area of the object under test.
In a kind of possible design, the determining module is specifically used for: it is down-sampled to target subgraph progress, it obtains
To the first subgraph, and determine classification belonging to multiple first pixels that first subgraph includes, and then to described the
One subgraph is up-sampled to obtain the second subgraph；Second subgraph includes multiple second pixels, and the multiple
The first pixel of each of one pixel corresponds at least one second pixel, each first pixel corresponding at least one
Classification belonging to a second pixel is identical as classification belonging to each first pixel；Further, from the multiple
The second pixel of one or more targets that classification is pre-set categories is chosen in second pixel, and according to one or more of
The second pixel of target obtains the corresponding target area of the object under test；The pre-set categories are belonging to the object under test
Classification.
In a kind of possible design, the processing module is specifically used for: the target area is divided into one or more
A grid, according to the corresponding depth of one or more grid described in one or more of grids and second image
Degree information determines the volume of one or more of grids, sums up to obtain to the volume of one or more of grids described
The volume of object under test.
In a kind of possible design, the determining module determines that the object under test is corresponding from the first image
Before target area, the determining module is also filtered the first image.
The third aspect, a kind of computer readable storage medium provided in an embodiment of the present invention, including instruction, when it is being calculated
When being run on the processor of machine, so that the processor of computer executes the image processing method as described in above-mentioned first aspect is any
Method.
Fourth aspect, a kind of computer program product provided in an embodiment of the present invention make when run on a computer
Obtain image processing method of the computer execution as described in above-mentioned first aspect is any.
The aspects of the invention or other aspects can more straightforwards in the following description.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without any creative labor, it can also be obtained according to these attached drawings
His attached drawing.
Fig. 1 is a kind of corresponding configuration diagram of Auto-Memory system provided in an embodiment of the present invention；
Fig. 2 is a kind of corresponding flow diagram of image processing method provided in an embodiment of the present invention；
Fig. 3 is the schematic diagram that a kind of first preset model provided in an embodiment of the present invention handles the first image；
Fig. 4 is a kind of structural schematic diagram of image processing apparatus provided in an embodiment of the present invention.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to the present invention make into
It is described in detail to one step, it is clear that described embodiments are only a part of the embodiments of the present invention, rather than whole implementation
Example.Based on the embodiments of the present invention, obtained by those of ordinary skill in the art without making creative efforts
All other embodiment, shall fall within the protection scope of the present invention.
Financial technology (Fintech), which refers to, behind information technology involvement financial field, to be that financial field bring is a kind of new
Creative Science and Technology Co. Ltd, assist realizing that financial operation, transaction execute and financial system is improved by using advanced information technology, can
To promote treatment effeciency, the business scale of financial system, and cost and financial risks can be reduced.In general, financial technology
Field would generally be related to largely transaction and the friendship between data, such as user and the transaction of trade company and data, different trade companies
How easy and data etc. are traded from a large amount of using the means of science and technology and excavate feature required for financial field in data,
The always target of financial technology field pursuit.
Fig. 1 is a kind of corresponding configuration diagram of Auto-Memory system provided in an embodiment of the present invention, as shown in Figure 1, should
System may include warehouse 100, deposit route 110 and carrier 120, and can be set on carrier 120 placed side by side has a row or more
Collateral security 130 is arranged, the quantity of any two rows of collateral securities in a row or multi-row collateral security 130 can be identical or or not
Together, a row or multi-row collateral security 130 can be placed in sequence, or can also arbitrarily place, and specifically be not construed as limiting；Accordingly
Multiple cabinets have can be set in warehouse 100 in ground, and multiple cabinets can be used for storing collateral security.
In the embodiment of the present invention, binocular camera, binocular camera can be set on the predeterminated position for route 110 of depositing
In can be set there are two camera, such as camera shown in FIG. 1 141 and camera 142；Wherein, the position of photographic device
It can be rule of thumb configured, or can also be configured according to actual scene by those skilled in the art, do not made specifically
It limits.In the embodiment of the present invention, predeterminated position can on deposit route 110 close to warehouse 100 position (such as position a),
To which camera 141 and camera 142 also can be set in position a；In this way, when carrier 120 carries a row or multi-row collateral security
130 when moving across position a along deposit route 110, and camera 141 and camera 142 can shoot to obtain a row or multi-row
The corresponding image of scene where collateral security 130.In one example, camera 141 can be optical camera, camera
142 can be depth camera；In this way, being moved when carrier 120 carries a row or multi-row collateral security 130 along deposit route 110
When by position a, camera 141 can shoot to obtain the corresponding color image of scene where a row or multi-row collateral security 130,
Camera 142 can shoot to obtain the corresponding depth image of scene where a row or multi-row collateral security 130.
In one possible implementation, camera 141 and camera 142 can (Fig. 1 not be carried out with server respectively
Signal) connection, server can obtain the color image that camera 141 is shot according to predetermined period and camera 142 is shot
Depth image, and then the volume of a row or multi-row collateral security 130 can be determined based on color image and depth image.Further
Ground, also can be set photographic device in warehouse 100, and server can be according to the photographic device in warehouse 100 from multiple cabinets
In select the spare cabinet being in idle condition, and then can be according to the volume of a row or multi-row collateral security 130 from guest machine
Selection target cabinet in cabinet；In this way, server can send control instruction to carrier 120 (or staff on carrier),
So that carrier 120 carries a row or multi-row collateral security 130 and moves to the position where target cabinet, and pledged a row or multi-row
Product 130 are stored on target cabinet.
Based on system architecture illustrated in Figure 1, Fig. 2 is that a kind of image processing method provided in an embodiment of the present invention is corresponding
Flow diagram, the executing subject of this method can be server.This method comprises:
Step 201, scene when obtaining photographic device shooting object under test by predeterminated position where the object under test
Obtained the first image and the second image.
In one example, camera 141 and camera 142 can according to the scene around a of predetermined period camera site,
In this way, if a certain moment carrier 120 carries a row or multi-row collateral security 130 by position a, camera 141 and camera 142
The corresponding color image of scene and the depth image around a row or multi-row collateral security 130 can be shot to obtain respectively.Wherein, in advance
If the period can be rule of thumb configured by those skilled in the art, or can also be configured according to actual needs, tool
Body is not construed as limiting.
In another example, induction coil can be set at the position a of deposit route 110, if a certain moment carrier
120 carry a row or multi-row collateral security 130 by position a, then induction coil can report induction information to server；In this way,
Server can control the scene around 142 camera site a of camera 141 and camera, to obtain a row or multi-row pledge
The corresponding color image of scene and depth image around product 130.
It should be noted that carrier 120 can not stop after deposit line-of-road movement to position a in the embodiment of the present invention
Only, in this way, camera 141 and camera 142 can shoot the field around a row or multi-row collateral security 130 being kept in motion
Scape obtains the color image and depth image under motion state；It, can be with by realizing measurement process in real time under motion state
Improve the efficiency of deposit.
Step 202, according to the characteristic value of pixel each in color image, object under test is determined from the color image
Corresponding target area.
In one possible implementation, if server gets color image and depth map under being kept in motion
Picture then can be used default filtering mode and be filtered to color image；Wherein, default filtering mode can be by art technology
Personnel are rule of thumb configured, and are specifically not construed as limiting.The embodiment of the present invention can using Wiener filter to color image into
Row filtering；Specifically, color image can be inputted Wiener filter by server, since color image is useful signal and nothing
With the sum of noise, therefore, Wiener filter can parse that color image obtains the corresponding matrix of image respectively and noise is corresponding
Matrix, further, Wiener filter can determine the part of image according to the corresponding matrix of image and the corresponding matrix of noise
Variance, and the filtering parameter of filter output can be adjusted according to the local variance of image, guarantee the output letter of Wiener filter
Number and desired signal difference mean-square value it is minimum, in this way, being filtered using the filtering parameter of output to color image, can obtain
To preferable filter effect.
In the embodiment of the present invention, by being filtered to the color image under being kept in motion, it can remove in fortune
Dynamic fuzzy in the color image of dynamic state, so as to improve the essence for executing image procossing based on filtered color image
Exactness；That is, the image processing method in the embodiment of the present invention can execute during carrier 120 moves, and nothing
The original work flow of carrier 120 need to be changed, so as to improve the efficiency of image procossing.
In one example, server in the color image obtained under remaining static or can obtain filtered
After color image, alignment operation can be carried out to color image and depth image, such as can be by color image and depth image
On same object be aligned；Further, server, can be according to cromogram after alignment color image and depth image
The characteristic value of each pixel as in determines the corresponding target area of object under test from the color image.The present invention is implemented
In example, by alignment operation, so that each pixel on color image can correspond to a depth on depth image
Information, i.e. color image and depth image can have one-to-one relationship.
In one possible implementation, server can according to the mode of deep learning from color image determine to
The corresponding target area of object is surveyed, for example, color image directly can be inputted trained target image language in advance by server
Justice segmentation (such as target U-NET3) model, and target U-NET3 model can be identified on color image not with different colours
Same object, in this way, server can (this mode be known as first and shows using the corresponding color region of object under test as target area
Example)；Alternatively, color image first can be inputted trained target detection (such as target YOLO2) model in advance by server, and
Target YOLO2 model can identify the region where the different objects on color image with different block diagrams, further, service
Device can be by area image (for ease of description, the referred to as corresponding son of object under test where the corresponding block diagram of object under test
Image) trained image, semantic divides (such as target U-NET3) model in advance for input, so that target U-NET3 model can be with
The different objects on the corresponding subgraph of object under test are identified with different colours, in this way, server can be by object under test pair
The corresponding color region of object under test is as target area on the subgraph answered (this mode is known as the second example)；Alternatively, service
Device can also determine the corresponding target area of object under test using other deep learning modes from color image, not limit specifically
It is fixed.
In the embodiment of the present invention, simple model analysis, Ke Yicong are carried out to color image by using target YOLO2 model
The rough region where object under test is obtained in color image, and then by using where target U-NET3 model measuring targets
Rough region carry out accurate model analysis, the precise region where available object under test；It follows that the present invention is implemented
Example may not need by setting target YOLO2 model and target U-NET3 model and carry out accurate model point to entire color image
Analysis, so as to analyzed in implementation model accuracy while improve model analysis efficiency.
The specific implementation process of the corresponding target area of determining object under test is described by taking the second example as an example below.
In the embodiment of the present invention, target YOLO2 model can be obtained for server according to the training of multiple first sample images
's.Specifically, the scene of the available camera 141 of server and camera 142 around different moments camera site a obtains
The quantity of the multiple first sample images arrived, collateral security included by each first sample image can be different, and each first
Marked block diagram, the block diagram where carrier and the block diagram where staff where collateral security out on sample image.Into one
Step ground, server can be used multiple first sample image training YOLO2 models, obtain target YOLO2 model, target YOLO2
Model can be to the image tagged block diagram of input target YOLO2 model.
Correspondingly, target U-NET3 model can obtain for server according to the training of multiple second sample images.Specifically
Say that server can be from corresponding multiple second sample graphs of block diagram where obtaining collateral security in multiple first sample images in ground
Picture may include color different collateral security region and background area on each second sample image, or also may include face
The quantity in color difference carrier region and/or staff region, collateral security included by each second sample image can be different,
And in multiple second sample images collateral security region color it is identical.Further, multiple second samples can be used in server
Image trains U-NET3 model, obtains target U-NET3 model, and target U-NET3 model can be to input target U-NET3 model
Image on collateral security region, carrier region, staff region, background area mark different colors respectively.
In specific implementation, color image can be inputted target YOLO2 model by server, in this way, target YOLO2 model can
To mark multiple block diagrams on color image according to the one or more objects for including on color image；Wherein, any two frame
Included object can be different in figure.For example, as shown in figure 3, if on color image include " automobile ", " bicycle " and
" dog " three objects, then target YOLO2 model can mark block diagram a on color image1, block diagram a2With block diagram a3, wherein frame
Scheme a1In may include " automobile ", block diagram a2In may include " bicycle ", block diagram a3In may include " dog ".
In one example, it can determine which block diagram is the pixel on color image belong to using sliding window mode.Specifically
Ground says, can be directed to each pixel and 9 windows are arranged, by by other pictures in each pixel and 9 windows
Vegetarian refreshments combines, and can analyze and determines classification belonging to each pixel.In this example, picture is determined by using sliding window mode
The classification of vegetarian refreshments can make pixel correspond to multiple classifications, so, it is ensured that the integrality of the block diagram where object.
It should be noted that above-mentioned is only a kind of illustrative simple declaration, cited by window quantity be only for
Convenient for illustrating scheme, the restriction to scheme is not constituted, in specific implementation, the quantity of window can be greater than 9, such as can
Think 16, or might be less that 9, for example can be 4, is specifically not construed as limiting.
It at least may include that collateral security is corresponding on the color image of target YOLO2 model output in the embodiment of the present invention
The corresponding block diagram of block diagram, carrier, the corresponding block diagram of staff and background, further, server can be from color image
The target subgraph where the corresponding block diagram of collateral security is obtained, and target subgraph can be inputted into target U-NET3 model, such as
This, the one or more object tags for including on target subgraph can be different colors by target U-NET3 model；Wherein,
The color of any two object can be different.For example, collateral security is corresponding on the target subgraph of target U-NET3 model output
It can be yellow, the corresponding region of staff can be that black, background are corresponding that region, which can be red, the corresponding region of carrier,
Region can for blue.
In one example, target U-NET3 model is on target subgraph to the inside of different object tag different colours
Realization process can be as shown in step a~step c:
Step a, it is down-sampled to the progress of target subgraph, obtain the first image.
Specifically, if the resolution ratio of target subgraph be greater than the first default resolution ratio, can to target subgraph into
Row is down-sampled to obtain the first image, so that the resolution ratio of the first image is less than or equal to the first default resolution ratio.For example, can incite somebody to action
Down-sampled the first image for being 28*28 of the target subgraph that resolution ratio is 128*128；Wherein, every on the first image of 28*28
A pixel can correspond to a block of pixels on the target subgraph of 128*128.Correspondingly, if the resolution of target subgraph
Rate is less than or equal to the first default resolution ratio, then can not carry out to target subgraph down-sampled.
Step b determines classification belonging to each pixel on the first image.
In specific implementation, any pixel point (such as first pixel) being directed on the first image of 28*28 can be obtained
The characteristic value of the first pixel is taken, if the collateral security characteristic value stored in the characteristic value of the first pixel and target U-NET3 model
Matching can then determine that classification belonging to the pixel is collateral security；Alternatively, if the characteristic value of the first pixel and target U-
The carrier features value matching stored in NET3 model can then determine that classification belonging to the pixel is carrier；Alternatively, if first
The staff's characteristic value stored in the characteristic value of pixel and target U-NET3 model matches, then can determine the pixel institute
The classification of category is staff；Alternatively, if the background characteristics stored in the characteristic value of the first pixel and target U-NET3 model
Value matching can then determine that classification belonging to the pixel is background.
Step c, according to classification belonging to pixel each on the first image, to the different object tags on target subgraph
Different colours.
It,, can be from the first figure in specific implementation for marking the corresponding region of collateral security in a kind of possible situation
Classification belonging to determining in multiple pixels of picture is one or more target pixel points of collateral security, and then from target subgraph
In find one or more target pixel blocks corresponding with one or more target pixel points, and can be by one or more targets
Block of pixels is labeled as red.
In alternatively possible situation, determine that affiliated classification is to pledge in multiple pixels from the first image
After one or more target pixel points of product, the first image can also be up-sampled to obtain the second image, the second image
Size is identical as the size of target subgraph, and each pixel on the first image can correspond to a picture on the second image
Plain block；In this way, one or more object pixels corresponding with one or more target pixel points can be found from the second image
Block, and can be by one or more target pixel blocks labeled as red.
In the embodiment of the present invention, resolution ratio lesser first is converted by target subgraph by using down-sampled mode
Subgraph improves figure so that the complexity of image procossing can be reduced by carrying out analysis based on lesser first subgraph of resolution ratio
As the efficiency of processing；And by the way that the pixel of block of pixels and the first subgraph on target subgraph is mapped, Ke Yibao
Demonstrate,prove the accuracy of image procossing.
Step 203, believed according to the corresponding target area of object under test and the corresponding depth of the second objective area in image
Breath, determines the volume of object under test.
In one possible implementation, server determines the corresponding target area of collateral security in color image
Afterwards, depth areas corresponding with target area can be intercepted from depth image, and then can be carried out a cloud to depth areas and be turned
It changes；In this way, depth areas can be converted into depth information corresponding with target area.It is shown in Figure 1, due to camera 141
Being arranged above collateral security with camera 142, i.e., color image and depth image are to obtain at the top of shooting collateral security, because
This, depth information can identify the top of collateral security at a distance from camera 142.
In the embodiment of the present invention, target area can be divided into one or more grids by server；Wherein, each grid
It may include a pixel, or also may include multiple pixels, for example, each grid may include 1*1 pixel, or
Person also may include 2*2 pixel, or can also include 3*2 pixel, specifically be not construed as limiting.In the embodiment of the present invention, it is
Guarantee the accuracy of measurement volume, each grid can be set includes 1*1 pixel.
Further, server can obtain the corresponding depth information of each grid, each grid pair from depth areas
The depth information answered can identify the top of the corresponding collateral security of each grid (h as shown in figure 1 at a distance from camera 1421)；
In this way, passing through 120 loading end of carrier measured in advance (h as shown in figure 1 at a distance from camera 1422) and each net
The top of the corresponding collateral security of lattice (i.e. h at a distance from camera 1421), the height of the corresponding collateral security of available each grid
Spend (i.e. h1-h2)。
Further, server can be according to pair of the length and width of the length of color image, width and actual scene
It should be related to, determine the ratio of color image, and then can be according to the ratio of color image by the length of grid each in target area
The length and width that the corresponding collateral security of each grid is converted into width is spent, so as to pledge according to each grid is corresponding
Height (the i.e. h of the length and width of product and the corresponding collateral security of each grid1-h2), determine the volume of each grid.In this way,
Server can sum it up the volume of one or more grids included by the corresponding target area of collateral security, obtain collateral security
Volume.It, can be according to the depth information of multiple grids point by the way that target area is divided into multiple grids in the embodiment of the present invention
The volume of multiple grids is not determined, to sum up the volume that object under test can be obtained by the volume to multiple grids；
The process obtains the volume of object under test by unit of grid, can make the volume for determining obtained object under test compared with subject to
Really, and realize that logic is relatively simple, so as to improve the efficiency of processing.
In the above embodiment of the present invention, obtains photographic device shooting object under test and pass through determinand when predeterminated position
The first image and the second image that scene where body obtains, the first image are that the scene where the object under test is corresponding
Color image, second image be the object under test where the corresponding depth image of scene；Further, according to institute
The characteristic value for stating each pixel in the first image determines the corresponding target area of the object under test from the first image
Domain, and the corresponding depth information in target area according to the target area and second image, determine it is described to
Survey the volume of object.In the embodiment of the present invention, by realizing measurement process in real time under motion state so as to environment according to
Rely smaller, the object volume under complex environment is detected so as to realize, and no setting is required measuring station and fixed test
Position, so as to improve the efficiency of deposit；In addition, shooting to obtain a color image and a depth image by photographic device
It can determine the volume of object under test, be not necessarily to staff, and no setting is required multiple photographic devices, so that cost is relatively low, and grasp
Make simply, to be easy to implement.
For above method process, the embodiment of the present invention also provides a kind of image processing apparatus, the particular content of the device
It is referred to above method implementation.
Fig. 4 is a kind of structural schematic diagram of image processing apparatus provided in an embodiment of the present invention, comprising:
Module 401 is obtained, object under test institute when for obtaining photographic device shooting object under test by predeterminated position
Scene obtained the first image and the second image, the first image is the corresponding coloured silk of scene where the object under test
Chromatic graph picture, second image are the corresponding depth image of scene where the object under test；
Determining module 402, for the characteristic value according to pixel each in the first image, from the first image really
Determine the corresponding target area of the object under test；
Processing module 403, it is corresponding for the target area according to the target area and second image
Depth information determines the volume of the object under test.
Optionally, the determining module 402 is specifically used for:
The first image is inputted into the first preset model, so that the first preset model is according to each in the first image
The first image is divided into one or more subgraphs by the characteristic value of pixel；It is every in one or more of subgraphs
A subgraph includes at least one object；
Obtained from one or more of subgraphs include the object under test target subgraph, and by the target
Subgraph inputs the second preset model, obtains the corresponding target area of the object under test.
Optionally, the determining module 402 is specifically used for:
It is down-sampled to target subgraph progress, the first subgraph is obtained, and determine that first subgraph includes
Classification belonging to multiple first pixels, and then first subgraph is up-sampled to obtain the second subgraph；Described
Two subgraphs include multiple second pixels, the first pixel of each of the multiple first pixel correspond at least one second
Pixel, classification and each first pixel belonging at least one corresponding second pixel of each first pixel
Classification belonging to point is identical；
The second pixel of one or more targets that classification is pre-set categories is chosen from the multiple second pixel, and
The corresponding target area of the object under test is obtained according to one or more of the second pixels of target；The pre-set categories are
Classification belonging to the object under test.
Optionally, the processing module 403 is specifically used for:
The target area is divided into one or more grids, according to one or more of grids and described second
The corresponding depth information of one or more grid described in image determines the volume of one or more of grids, to described
The volume of one or more grids sums up to obtain the volume of the object under test.
Optionally, the determining module 402 determines the corresponding target area of the object under test from the first image
Before, the determining module 402 is also used to:
The first image is filtered.
It can be seen from the above: in the above embodiment of the present invention, where acquisition photographic device measuring targets
The first image and the second image that scene is shot, the first image are that the scene where the object under test is corresponding
Color image, second image be the object under test where the corresponding depth image of scene；Further, according to institute
The characteristic value for stating each pixel in the first image determines the corresponding target area of the object under test from the first image
Domain, and the corresponding depth information in target area according to the target area and second image, determine it is described to
Survey the volume of object.In the embodiment of the present invention, by realizing measurement process in real time under motion state so as to environment according to
Rely smaller, the object volume under complex environment is detected so as to realize, and no setting is required measuring station and fixed test
Position, so as to improve the efficiency of deposit；In addition, shooting to obtain a color image and a depth image by photographic device
It can determine the volume of object under test, be not necessarily to staff, and no setting is required multiple photographic devices, so that cost is relatively low, and grasp
Make simply, to be easy to implement.
Based on the same inventive concept, the embodiment of the invention also provides a kind of computer readable storage mediums, including instruct,
When it runs on the processor of computer, so that the processor of computer executes at the image as described in Fig. 1 or Fig. 1 is any
Reason method.
Based on the same inventive concept, the embodiment of the invention also provides a kind of computer program products, when it is in computer
When upper operation, so that computer executes the image processing method as described in Fig. 1 or Fig. 1 is any.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the present invention
Form.It is deposited moreover, the present invention can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (12)
1. a kind of image processing method, which is characterized in that the described method includes:
Obtain the first figure that scene when photographic device shooting object under test passes through predeterminated position where the object under test obtains
Picture and the second image, the first image are the corresponding color image of scene where the object under test, second image
For the corresponding depth image of scene where the object under test；
According to the characteristic value of pixel each in the first image, the corresponding mesh of the object under test is determined from the first image
Mark region；
According to the corresponding depth information in target area described in the target area and second image, obtain described to be measured
The volume of object.
2. the method according to claim 1, wherein the feature according to pixel each in the first image
Value determines the corresponding target area of the object under test from the first image, comprising:
The first image is inputted into the first preset model, so that the first preset model is according to pixel each in the first image
The first image is divided into one or more subgraphs by the characteristic value of point；Every height in one or more of subgraphs
Image includes at least one object；
Obtained from one or more of subgraphs include the object under test target subgraph, and by the target subgraph
As the second preset model of input, the corresponding target area of the object under test is obtained.
3. according to the method described in claim 2, it is characterized in that, described input the second default mould for the target subgraph
Type obtains the corresponding target area of the object under test, comprising:
It is down-sampled to target subgraph progress, the first subgraph is obtained, and determine that first subgraph includes multiple
Classification belonging to first pixel, and then first subgraph is up-sampled to obtain the second subgraph；Second son
Image includes multiple second pixels, and the first pixel of each of the multiple first pixel corresponds at least one second pixel
Point, classification and each first pixel institute belonging at least one corresponding second pixel of each first pixel
The classification of category is identical；
Classification is chosen from the multiple second pixel as the second pixel of one or more targets of pre-set categories, and according to
One or more of the second pixels of target obtain the corresponding target area of the object under test；The pre-set categories are described
Classification belonging to object under test.
4. the method according to claim 1, wherein described according to the target area and second image
Described in the corresponding depth information in target area, determine the volume of the object under test, comprising:
The target area is divided into one or more grids, according to one or more of grids and second image
Described in the corresponding depth information of one or more grid determine the volumes of one or more of grids, to one
Or the volume of multiple grids sums up to obtain the volume of the object under test.
5. method according to claim 1 to 4, which is characterized in that described to be determined from the first image
Before the corresponding target area of the object under test, further includes:
The first image is filtered.
6. a kind of image processing apparatus, which is characterized in that described device includes:
Module is obtained, scene when for obtaining photographic device shooting object under test by predeterminated position where the object under test
Obtained the first image and the second image, the first image are the corresponding color image of scene where the object under test,
Second image is the corresponding depth image of scene where the object under test；
Determining module, for the characteristic value according to pixel each in the first image, determined from the first image it is described to
Survey the corresponding target area of object；
Processing module, for the corresponding depth letter in the target area according to the target area and second image
Breath, obtains the volume of the object under test.
7. device according to claim 6, which is characterized in that the determining module is specifically used for:
The first image is inputted into the first preset model, so that the first preset model is according to pixel each in the first image
The first image is divided into one or more subgraphs by the characteristic value of point；Every height in one or more of subgraphs
Image includes at least one object；
Obtained from one or more of subgraphs include the object under test target subgraph, and by the target subgraph
As the second preset model of input, the corresponding target area of the object under test is obtained.
8. device according to claim 7, which is characterized in that the determining module is specifically used for:
It is down-sampled to target subgraph progress, the first subgraph is obtained, and determine that first subgraph includes multiple
Classification belonging to first pixel, and then first subgraph is up-sampled to obtain the second subgraph；Second son
Image includes multiple second pixels, and the first pixel of each of the multiple first pixel corresponds at least one second pixel
Point, classification and each first pixel institute belonging at least one corresponding second pixel of each first pixel
The classification of category is identical；
Classification is chosen from the multiple second pixel as the second pixel of one or more targets of pre-set categories, and according to
One or more of the second pixels of target obtain the corresponding target area of the object under test；The pre-set categories are described
Classification belonging to object under test.
9. device according to claim 6, which is characterized in that the processing module is specifically used for:
The target area is divided into one or more grids, according to one or more of grids and second image
Described in the corresponding depth information of one or more grid determine the volumes of one or more of grids, to one
Or the volume of multiple grids sums up to obtain the volume of the object under test.
10. device according to any one of claims 6 to 9, which is characterized in that the determining module is from first figure
Before determining the corresponding target area of the object under test as in, the determining module is also used to:
The first image is filtered.
11. a kind of computer readable storage medium, which is characterized in that including instruction, when it runs on the processor of computer
When, so that the processor of computer executes such as method described in any one of claim 1 to 5.
12. a kind of computer program product, which is characterized in that when run on a computer, so that computer is executed as weighed
Benefit requires 1 to 5 described in any item methods.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910619862.5A CN110378952A (en) | 2019-07-10 | 2019-07-10 | A kind of image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910619862.5A CN110378952A (en) | 2019-07-10 | 2019-07-10 | A kind of image processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110378952A true CN110378952A (en) | 2019-10-25 |
Family
ID=68250875
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910619862.5A Pending CN110378952A (en) | 2019-07-10 | 2019-07-10 | A kind of image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378952A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111524180A (en) * | 2020-04-23 | 2020-08-11 | Oppo广东移动通信有限公司 | Object volume calculation method and device, electronic equipment and storage medium |
| CN114088207A (en) * | 2020-07-17 | 2022-02-25 | 北京京东尚科信息技术有限公司 | Temperature detection method and system |
| WO2023124442A1 (en) * | 2021-12-29 | 2023-07-06 | 京东方科技集团股份有限公司 | Method and device for measuring depth of accumulated water |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120001902A1 (en) * | 2010-07-02 | 2012-01-05 | Samsung Electronics Co., Ltd. | Apparatus and method for bidirectionally inpainting occlusion area based on predicted volume |
| CN107067431A (en) * | 2017-01-16 | 2017-08-18 | 河海大学常州校区 | A kind of object volume computational methods based on Kinect |
| CN108876236A (en) * | 2018-05-31 | 2018-11-23 | 清华大学 | Logistics system of returning goods of receiving based on UAV Intelligent express delivery cabinet |
| CN109147216A (en) * | 2018-07-25 | 2019-01-04 | 上海与德科技有限公司 | Intelligent express delivery cabinet, express delivery are sent with charge free and collection methods, device, server and medium |
| CN109978939A (en) * | 2019-05-21 | 2019-07-05 | 芜湖智久机器人有限公司 | A kind of dimension of object estimating system and method, warehouse capacity estimating system and method |
-
2019
- 2019-07-10 CN CN201910619862.5A patent/CN110378952A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120001902A1 (en) * | 2010-07-02 | 2012-01-05 | Samsung Electronics Co., Ltd. | Apparatus and method for bidirectionally inpainting occlusion area based on predicted volume |
| CN107067431A (en) * | 2017-01-16 | 2017-08-18 | 河海大学常州校区 | A kind of object volume computational methods based on Kinect |
| CN108876236A (en) * | 2018-05-31 | 2018-11-23 | 清华大学 | Logistics system of returning goods of receiving based on UAV Intelligent express delivery cabinet |
| CN109147216A (en) * | 2018-07-25 | 2019-01-04 | 上海与德科技有限公司 | Intelligent express delivery cabinet, express delivery are sent with charge free and collection methods, device, server and medium |
| CN109978939A (en) * | 2019-05-21 | 2019-07-05 | 芜湖智久机器人有限公司 | A kind of dimension of object estimating system and method, warehouse capacity estimating system and method |
Non-Patent Citations (3)
| Title |
|---|
| LIANG HAN: "Livestock detection in aerial images using a fully convolutional network", 《COMPUTATIONAL VISUAL MEDIA》 * |
| 杨顺辽等: "《数字信号处理实现与实践》", 30 September 2011 * |
| 马伟等: "新型乳腺磁共振增强图像肿瘤区域的自动分割模型", 《中国生物医学工程学报》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111524180A (en) * | 2020-04-23 | 2020-08-11 | Oppo广东移动通信有限公司 | Object volume calculation method and device, electronic equipment and storage medium |
| CN114088207A (en) * | 2020-07-17 | 2022-02-25 | 北京京东尚科信息技术有限公司 | Temperature detection method and system |
| WO2023124442A1 (en) * | 2021-12-29 | 2023-07-06 | 京东方科技集团股份有限公司 | Method and device for measuring depth of accumulated water |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110264468B (en) | Point cloud data mark, parted pattern determination, object detection method and relevant device | |
| CN108154168B (en) | Comprehensive cargo inspection system and method | |
| CN110023994B (en) | Histopathological image analysis | |
| CN106707293B (en) | Obstacle recognition method and device for vehicle | |
| CN108875600A (en) | A kind of information of vehicles detection and tracking method, apparatus and computer storage medium based on YOLO | |
| CN108154105B (en) | Underwater biological detection and identification method and device, server and terminal equipment | |
| CN109447169A (en) | The training method of image processing method and its model, device and electronic system | |
| CN110378952A (en) | A kind of image processing method and device | |
| CN110443969A (en) | A kind of fire point detecting method, device, electronic equipment and storage medium | |
| CN108898109A (en) | The determination methods, devices and systems of article attention rate | |
| CN111242852A (en) | Boundary aware object removal and content filling | |
| CN107358182A (en) | Pedestrian detection method and terminal device | |
| CN110443862A (en) | Lithologic map filling method and system based on unmanned aerial vehicle and electronic equipment | |
| CN108961547A (en) | A kind of commodity recognition method, self-service machine and computer readable storage medium | |
| CN109508583A (en) | A kind of acquisition methods and device of distribution trend | |
| CN109840503A (en) | A kind of method and device of determining information | |
| CN109154938A (en) | Using discrete non-trace location data by the entity classification in digitized map | |
| CN109740609A (en) | A kind of gauge detection method and device | |
| CN109684910A (en) | A kind of method and system of network detection transmission line of electricity ground surface environment variation | |
| CN113191204A (en) | Multi-scale blocking pedestrian detection method and system | |
| CN109543617A (en) | The detection method of intelligent vehicle movement traffic information based on YOLO target detection technique | |
| Yadav et al. | An improved deep learning-based optimal object detection system from images | |
| CN110490058A (en) | Training method, device, system and the computer-readable medium of pedestrian detection model | |
| Nassar et al. | Assessing data quality of annotations with Krippendorff alpha for applications in computer vision | |
| CN109492573A (en) | A kind of pointer read method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |