CN110390731A - Image processing method, device, computer readable storage medium and electronic equipment - Google Patents
Image processing method, device, computer readable storage medium and electronic equipment Download PDFInfo
- Publication number
- CN110390731A CN110390731A CN201910636694.0A CN201910636694A CN110390731A CN 110390731 A CN110390731 A CN 110390731A CN 201910636694 A CN201910636694 A CN 201910636694A CN 110390731 A CN110390731 A CN 110390731A
- Authority
- CN
- China
- Prior art keywords
- image
- target object
- processed
- target
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0641—Shopping interfaces
- G06Q30/0643—Graphical representation of items or shoppers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
Abstract
The embodiment of the present disclosure discloses a kind of image processing method and device, wherein this method comprises: obtain include target object image image to be processed；Target object image is removed from image to be processed；Image to be processed after removal target object image is restored, obtains going back original image；Acquisition is inserted into subject image, and will be inserted into subject image insertion and go back the target position in original image, wherein target position is the position for removing target object image；Adjustment is inserted into subject image in the size gone back in original image to target size.The embodiment of the present disclosure realizes the replacement to target object image, makes the image of insertion in going back original image explicitly more naturally, helping that user is made quickly, neatly to check the effect for replacing the object of target object image instruction.
Description
Technical field
This disclosure relates to field of computer technology, especially a kind of image processing method, device, computer-readable storage medium
Matter and electronic equipment.
Background technique
Currently, Internet technology is that people's lives bring a large amount of convenience, it is clear that internet can be used in people
It lookes at various information, provides reference for the every aspect of life.For example, can not imagine this when user browses furniture on line
Furniture put in own home be it is what kind of, therefore, it is necessary to by it is this imagination become vision present.And current technology
In, user can not be allowed only to need to clap Zhang Quanjing's figure, just existing furniture image is taken away and lays out the image of new furniture, to give
The true visual effect of user.
Summary of the invention
In order to solve the above-mentioned technical problem, the disclosure is proposed.Embodiment of the disclosure provides a kind of image processing method
Method, device, computer readable storage medium and electronic equipment.
Embodiment of the disclosure provides a kind of image processing method, this method comprises: obtaining includes target object image
Image to be processed；Target object image is removed from image to be processed；To the figure to be processed after removal target object image
As being restored, obtain going back original image；Acquisition is inserted into subject image, and will be inserted into subject image insertion and go back in original image
Target position, wherein target position is the position for removing target object image；Adjustment is inserted into subject image in going back original image
Size to target size.
In some embodiments, the image to be processed after removal target object image is restored, obtains going back original image,
Include: the image restoring model that the image to be processed input after removal target object image is trained in advance, obtains going back original image.
In some embodiments, image to be processed has corresponding depth information；And by target object image to from
It is removed in reason image, comprising: be based on depth information, determine the corresponding three-dimensional point cloud information of image to be processed；Believe from three-dimensional point cloud
In breath, determine for characterizing the three-dimensional point cloud information of target object image as target three-dimensional point cloud information；Based on target three-dimensional
Point cloud information determines position of the target object image in image to be processed；According to position, by target object image to be processed
It is removed in image.
In some embodiments, subject image is inserted into corresponding three-dimensional dimension information；And adjustment is inserted into object
Body image is in the size gone back in original image to target size, comprising: the corresponding three-dimensional point cloud information of original image is gone back in determination；Based on to
It is inserted into the corresponding three-dimensional dimension information of subject image three-dimensional point cloud information corresponding with original image is gone back, adjustment is inserted into subject image
In the size gone back in original image to target size.
In some embodiments, method further include: determine and go back the colouring information of original image and be inserted into the face of subject image
Color information；Adjustment is inserted into the colouring information of subject image, so as to be inserted into the colouring information of subject image and go back original image
Matches color information.
According to the other side of the embodiment of the present disclosure, a kind of image processing apparatus is provided, which includes: first to obtain
Modulus block, for obtaining the image to be processed including target object image；Remove module, for by target object image to from
It is removed in reason image；Recovery module obtains also original image for restoring to the image to be processed after removal target object image
Picture；Second obtains module, is inserted into subject image for obtaining, and be inserted into the target gone back in original image for subject image is inserted into
Position, wherein target position is the position for removing target object image；The first adjustment module is inserted into object figure for adjusting
As in the size gone back in original image to target size.
In some embodiments, recovery module is further used for: the image to be processed after removal target object image is defeated
Enter image restoring model trained in advance, obtains going back original image.
In some embodiments, image to be processed has corresponding depth information；And remove module includes: first determining
Unit determines the corresponding three-dimensional point cloud information of image to be processed for being based on depth information；Second determination unit is used for from three
It ties up in point cloud information, determines for characterizing the three-dimensional point cloud information of target object image as target three-dimensional point cloud information；Third
Determination unit determines position of the target object image in image to be processed for being based on target three-dimensional point cloud information；It removes single
Member, for according to position, target object image to be removed from image to be processed.
In some embodiments, subject image is inserted into corresponding three-dimensional dimension information；And the first adjustment module
Include: the 4th determination unit, goes back the corresponding three-dimensional point cloud information of original image for determination；Adjustment unit is inserted into for being based on
The corresponding three-dimensional dimension information of subject image three-dimensional point cloud information corresponding with original image is gone back, adjustment are inserted into subject image also
Size in original image is to target size.
In some embodiments, device further include: determining module for the determining colouring information for going back original image and is inserted into
The colouring information of subject image；Second adjustment module, for adjusting the colouring information for being inserted into subject image, so as to be inserted into object
The colouring information of body image and the matches color information for going back original image.
According to the other side of the embodiment of the present disclosure, a kind of computer readable storage medium is provided, it is computer-readable
Storage medium is stored with computer program, and computer program is for executing above-mentioned image processing method.
According to the other side of the embodiment of the present disclosure, a kind of electronic equipment is provided, electronic equipment includes: processor；
Memory for storage processor executable instruction；Processor for reading executable instruction from memory, and executes and refers to
It enables to realize above-mentioned image processing method.
Based on disclosure image processing method provided by the above embodiment, device, computer readable storage medium and electronics
Equipment by removing target object image from image to be processed, then restores the image to be processed after removal, then
Subject image insertion will be inserted into and go back the position of the removal target object image in original image, and adjust the ruler of the image of insertion
It is very little.To realize the replacement to target object image, make the image of insertion in going back original image explicitly more naturally, helping
In make user quickly, neatly check replace target object image instruction object effect.
Below by drawings and examples, the technical solution of the disclosure is described in further detail.
Detailed description of the invention
The embodiment of the present disclosure is described in more detail in conjunction with the accompanying drawings, the above-mentioned and other purposes of the disclosure,
Feature and advantage will be apparent.Attached drawing is used to provide to further understand the embodiment of the present disclosure, and constitutes explanation
A part of book is used to explain the disclosure together with the embodiment of the present disclosure, does not constitute the limitation to the disclosure.In the accompanying drawings,
Identical reference label typically represents same parts or step.
Fig. 1 is the system diagram that the disclosure is applicable in.
Fig. 2 is the flow diagram for the image processing method that one exemplary embodiment of the disclosure provides.
Fig. 3 is the flow diagram for the image processing method that disclosure another exemplary embodiment provides.
Fig. 4 is the structural schematic diagram for the image processing apparatus that one exemplary embodiment of the disclosure provides.
Fig. 5 is the structural schematic diagram for the image processing apparatus that disclosure another exemplary embodiment provides.
Fig. 6 is the structure chart for the electronic equipment that one exemplary embodiment of the disclosure provides.
Specific embodiment
In the following, will be described in detail by referring to the drawings according to an example embodiment of the present disclosure.Obviously, described embodiment is only
It is only a part of this disclosure embodiment, rather than the whole embodiments of the disclosure, it should be appreciated that the disclosure is not by described herein
The limitation of example embodiment.
It should also be noted that unless specifically stated otherwise, the opposite cloth of the component and step that otherwise illustrate in these embodiments
It sets, numerical expression and the unlimited the scope of the present disclosure processed of numerical value.
It will be understood by those skilled in the art that the terms such as " first ", " second " in the embodiment of the present disclosure are only used for distinguishing
Different step, equipment or module etc., neither represent any particular technology meaning, also do not indicate that the inevitable logic between them is suitable
Sequence.
It should also be understood that in the embodiments of the present disclosure, " multiple " can refer to two or more, and "at least one" can refer to
One, two or more.
It should also be understood that for the either component, data or the structure that are referred in the embodiment of the present disclosure, clearly limit no or
Person may be generally understood to one or more in the case where context provides opposite enlightenment.
In addition, term "and/or" in the disclosure, only a kind of incidence relation for describing affiliated partner, expression can be deposited
In three kinds of relationships, for example, A and/or B, can indicate: individualism A exists simultaneously A and B, these three situations of individualism B.
In addition, character "/" in the disclosure, typicallys represent the relationship that forward-backward correlation object is a kind of "or".
It should also be understood that the disclosure highlights the difference between each embodiment to the description of each embodiment,
Same or similar place can be referred to mutually, for sake of simplicity, no longer repeating one by one.
Simultaneously, it should be appreciated that for ease of description, the size of various pieces shown in attached drawing is not according to reality
Proportionate relationship draw.
Be to the description only actually of at least one exemplary embodiment below it is illustrative, never as to the disclosure
And its application or any restrictions used.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable
In the case of, the technology, method and apparatus should be considered as part of specification.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
The embodiment of the present disclosure can be applied to the electronic equipments such as terminal device, computer system, server, can with it is numerous
Other general or specialized computing system environments or configuration operate together.Suitable for electric with terminal device, computer system, server etc.
The example of well-known terminal device, computing system, environment and/or configuration that sub- equipment is used together includes but is not limited to:
Personal computer system, thin client, thick client computer, hand-held or laptop devices, is based on microprocessor at server computer system
System, set-top box, programmable consumer electronics, NetPC Network PC, minicomputer system, large computer system and
Distributed cloud computing technology environment, etc. including above-mentioned any system.
The electronic equipments such as terminal device, computer system, server can be in the department of computer science executed by computer system
It is described under the general context of system executable instruction (such as program module).In general, program module may include routine, program, mesh
Beacon course sequence, component, logic, data structure etc., they execute specific task or realize specific abstract data type.Meter
Calculation machine systems/servers can be implemented in distributed cloud computing environment, and in distributed cloud computing environment, task is by by logical
What the remote processing devices of communication network link executed.In distributed cloud computing environment, it includes storage that program module, which can be located at,
On the Local or Remote computing system storage medium of equipment.
Application is summarized
Currently, Internet technology is that people's lives bring a large amount of convenience, it is clear that internet can be used in people
It lookes at various information, provides reference for the every aspect of life.For example, can not imagine this when user browses furniture on line
Furniture put in own home be it is what kind of, therefore, it is necessary to by it is this imagination become vision present.I.e. user only needs to clap
Zhang Quanjing's figure, takes and lays out existing furniture image the image of new furniture away, to give the true visual effect of user.
Exemplary system
Fig. 1 is shown can be using the image processing method of embodiment of the disclosure or the exemplary system of image processing apparatus
System framework 100.
As shown in Figure 1, system architecture 100 may include terminal device 101, network 102 and server 103.Network 102 is used
To provide the medium of communication link between terminal device 101 and server 103.Network 102 may include various connection types,
Such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 101 and be interacted by network 102 with server 103, to receive or send message etc..
Various telecommunication customer end applications, such as the application of shopping class, searching class application, image procossing can be installed on terminal device 101
Class application, instant messaging tools etc..
Terminal device 101 can be various electronic equipments, including but not limited to such as mobile phone, laptop, number
Word radio receiver, PDA (personal digital assistant), PAD (tablet computer), PMP (portable media player), car-mounted terminal
The fixed terminal of the mobile terminal of (such as vehicle mounted guidance terminal) etc. and such as number TV, desktop computer etc..
Server 103 can be to provide the server of various services, such as at the image uploaded to terminal device 101
The backstage image processing server of reason.Backstage image processing server can be handled the image received, be handled
As a result (such as the image for being inserted into new subject image).
It should be noted that image processing method provided by embodiment of the disclosure can be executed by server 103,
It can be executed by terminal device 101, correspondingly, image processing apparatus can be set in server 103, also can be set in end
In end equipment 101.
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
Illustrative methods
Fig. 2 is the flow diagram for the image processing method that one exemplary embodiment of the disclosure provides.The present embodiment can answer
On electronic equipment (terminal device 101 as shown in Figure 1 or server 103), as shown in Fig. 2, this method includes following step
It is rapid:
Step 201, the image to be processed including target object image is obtained.
In the present embodiment, electronic equipment can be from the figure to be processed for remotely or from local acquisition including target object image
Picture.Wherein, image to be processed can be the image that camera shoots the space comprising target object.Target object
The target object of image instruction can be the objects such as furniture, clothes.Above-mentioned camera can be various types of cameras.
For example, it may be the first-class camera that can shoot depth image of such as binocular camera shooting.
Step 202, target object image is removed from image to be processed.
In the present embodiment, electronic equipment can remove target object image from image to be processed.Specifically, as
Example, electronic equipment can use existing object detection method (such as object detection method based on convolutional neural networks),
It determines target object image first from image to be processed, the corresponding pixel of target object image is deleted.Remove object
Occurs the region of a blank in image to be processed after body image, which is the profile of target object image.
Step 203, the image to be processed after removal target object image is restored, obtains going back original image.
In the present embodiment, electronic equipment can restore the image to be processed after removal target object image, obtain
To going back original image.Specifically, electronic equipment can use various modes (such as neural network model) and restore to image.Also
Image after original may be implemented after removing target object image, and the region of blank is reverted to background image, makes to go back original image
Display it is more natural.For example, target object image is furniture image, set since furniture is placed against the wall, after removing furniture image,
Occur white space, by remove furniture image after image restoring to be processed after, can by white space metope image completion,
To realize the visual effect for removing furniture.
In some optional implementations, electronic equipment can be defeated by the image to be processed after removal target object image
Enter image restoring model trained in advance, obtains going back original image.Wherein, image restoring model contains white space for characterizing
Image and the corresponding relationship for going back original image.
Specifically, as an example, image restoring model can be based on the training of existing image inpainting algorithm
Obtained model.Electronic equipment can use machine learning method, include by the training sample in preset training sample set
Sample restore before image as input, before being restored with the sample of input after the reduction of image corresponding sample image as it is expected
Output, is trained initial model (such as convolutional neural networks of various structures), also for each sample for training input
Image before original, available reality output.Wherein, reality output is the data of initial model reality output, for characterizing reduction
Image afterwards.Then, above-mentioned executing subject can use gradient descent method and back propagation, based on reality output and it is expected defeated
Out, the parameter for adjusting initial model, using the model obtained after each adjusting parameter as the initial model of training next time, and full
In the case where the preset trained termination condition of foot, terminate training, so that training obtains image restoring model.
Herein, electronic equipment can using batch training method initial model is trained, can also using with
The method of machine training is trained initial model, and the embodiment of the present disclosure does not limit this.
It should be noted that the training termination condition here preset at can include but is not limited to it is at least one of following: training
Time is more than preset duration；Frequency of training is more than preset times；It utilizes preset loss function (such as cross entropy loss function)
It calculates resulting penalty values and is less than default penalty values threshold value.
This optional implementation can be restored more accurately by using image restoring model and remove target object figure
Image to be processed as after helps more accurately to show for user so that the obtained display effect for going back original image is more preferable
Replace the effect of target object.
Step 204, it obtains and is inserted into subject image, and subject image insertion will be inserted into and go back the target position in original image
It sets.
In the present embodiment, electronic equipment is available is inserted into subject image, and will be inserted into subject image insertion also
Target position in original image.Wherein, target position is the position for removing target object image, i.e., above-mentioned target object image exists
Position where in image to be processed.Electronic equipment can obtain from Local or Remote and be inserted into subject image, be inserted into object
Image can be the image of various objects, for example, it may be the furniture image that user selects from furniture image library.Above-mentioned target
Position can be characterized with coordinate, for example, can be using the coordinate of the geometric center point of target object image as target position.
Step 205, adjustment is inserted into subject image in the size gone back in original image to target size.
In the present embodiment, the adjustable subject image that is inserted into of electronic equipment is in the size gone back in original image to target ruler
It is very little.Wherein, target size is to be inserted into image in the size gone back in original image.
As an example, target size can be the size of above-mentioned target object image (such as comprising target object image
The size of minimum rectangle or the overall size of target object image).Target size is also possible to electronic equipment according to going back original image
In target position where region be inserted into subject image size determine size.For example, electronic equipment can determine
The size in the region comprising target position and not comprising other subject images, then it is based on above-mentioned size, treat insertion subject image
Processing is zoomed in and out, thus by its size adjusting to target size (such as preset ratio of the size of above-mentioned zone).Target ruler
The very little electronic equipment that is also possible to is according to the proportionate relationship of the actual size of target object and the size of target object image, Yi Jigen
Size determined by actual size and aforementioned proportion relationship according to the object for being inserted into subject image instruction.
Optionally, target size can also be the operation that the size for being inserted into image is manually adjusted based on user, be adjusted
Size extremely.
In some optional implementations, following steps are can also be performed in electronic equipment:
It goes back the colouring information of original image firstly, determining and is inserted into the colouring information of subject image.Wherein, colouring information can
To include but is not limited to following at least one: brightness, tone, contrast etc..
Then, adjustment is inserted into the colouring information of subject image, so as to be inserted into colouring information and the reduction of subject image
The matches color information of image.Specifically, as an example, the various components that the adjustable colouring information of electronic equipment includes, with
The various components that each component for making the colouring information for being inserted into subject image include includes with the colouring information for going back original image it
Difference is in preset range.This optional implementation can make to be inserted into subject image and go back the color characteristic of original image more
It is close, so that the display effect for going back original image after insertion image is more naturally, help preferably to provide replacement for user
The reference of object.
The method provided by the above embodiment of the disclosure, by the way that target object image is removed from image to be processed, then
Image to be processed after removal is restored, subject image then will be inserted into and be inserted into the removal target object gone back in original image
The position of image, and adjust the size of the image of insertion.To realize the replacement to target object image, make the image of insertion
Explicitly more naturally, helping that user is made quickly, neatly to check that replacing target object image indicates in going back original image
Object effect.
With further reference to Fig. 3, the process signal of another embodiment of image insertion method image processing method is shown
Figure.In the present embodiment, image to be processed has corresponding depth information.As shown in figure 3, in above-mentioned embodiment illustrated in fig. 2
On the basis of, step 202 may include steps of:
Step 2021, it is based on depth information, determines the corresponding three-dimensional point cloud information of image to be processed.
In the present embodiment, electronic equipment can be based on depth information, determine the corresponding three-dimensional point cloud letter of image to be processed
Breath.Wherein, depth information is used to characterize the distance between the imaging surface of subject image and camera in image to be processed.It is logical
Often, image to be processed can be the depth image of depth camera shooting, and each pixel in depth image corresponds to a depth
Angle value, depth value are used to characterize the distance between position and imaging surface of camera of pixel instruction.
The distance that electronic equipment can be characterized according to depth information, determines three-dimensional point cloud information.Wherein, three-dimensional point cloud information
It may include D coordinates value, corresponding to the point in three-dimensional space.
Step 2022, from three-dimensional point cloud information, the three-dimensional point cloud information conduct for characterizing target object image is determined
Target three-dimensional point cloud information.
In the present embodiment, electronic equipment can be determined from three-dimensional point cloud information for characterizing target object image
Three-dimensional point cloud information is as target three-dimensional point cloud information.Specifically, the three-dimensional point cloud information of target object image can be divided
For multiple subsets, each subset corresponds to an object, and electronic equipment can be according to the specified of user, from three-dimensional point cloud information
Determine three-dimensional point cloud information that the subset of target object includes as target three-dimensional point cloud information.
Step 2023, it is based on target three-dimensional point cloud information, determines position of the target object image in image to be processed.
In the present embodiment, electronic equipment can be based on target three-dimensional point cloud information, determine target object image wait locate
Manage the position in image.Specifically, electronic equipment can determine the side of target object image according to target three-dimensional point cloud information
Boundary, the region which surrounds are the position of target object image.As an example, machine learning can be based on, magnanimity
Markd data are supplied to computer and are learnt, thus make it have can from cloud identification tag body point cloud number
According to boundary ability, will label article identified from the point cloud data of overall model.For example, utilizing various not similar shapes
Shape trains computer with markd sofa point cloud data, and to have it can identify it in the point cloud data of input
In include sofa point cloud data boundary.
Step 2024, according to above-mentioned position, target object image is removed from image to be processed.
In the present embodiment, electronic equipment can remove target object image according to position from image to be processed.Tool
Body, the pixel in region that electronic equipment can surround above-mentioned boundary is removed from image to be processed (for example, can incite somebody to action
The color of pixel in the region is set as pre-set color, such as white).
In some optional implementations, subject image is inserted into corresponding three-dimensional dimension information.In general, to be inserted
Enter the threedimensional model that subject image can correspond to pre-establish, which has corresponding three-dimensional dimension information.As
Example, three-dimensional dimension information, which can be, is in advance based on what the three dimensional point cloud for being inserted into subject image determined.
Step 205 may include following sub-step:
Firstly, the corresponding three-dimensional point cloud information of original image is gone back in determination.It is depth image generally, due to image to be processed, because
This, going back original image is also depth image, the method that electronic equipment can use the three-dimensional point cloud information such as determination image to be processed,
The corresponding three-dimensional point cloud information of original image is gone back in determination.
Then, based on being inserted into the corresponding three-dimensional dimension information of subject image corresponding with original image is gone back three-dimensional point cloud letter
Breath, adjustment are inserted into subject image in the size gone back in original image to target size.Specifically, electronic equipment is adjustable to be inserted
The size for entering the corresponding threedimensional model of subject image makes the size of threedimensional model adapt to go back where the target position in original image
The three-dimensional space in region.
As an example, the target size in this optional implementation can be the three-dimensional of target three-dimensional point cloud information characterization
Size.
As another example, target size, which can be, is based on being inserted into the corresponding three-dimensional dimension information of image and goes back original image
Picture size determined by ratio between the space three-dimensional dimension information of corresponding three-dimensional point cloud information characterization.For example, to be inserted
Entering the corresponding three-dimensional dimension information of image includes " 1 meter of height ", goes back the space three of the corresponding three-dimensional point cloud information characterization of original image
Tieing up dimension information includes " 3 meters of height ", then adjustment is inserted into image in the size gone back in original image, so that being inserted into the height of image
It spends and the ratio between the height of going back original image is 1:3.
This optional implementation, by corresponding with original image is gone back based on the corresponding three-dimensional dimension of subject image is inserted into
Three-dimensional point cloud information, adjustment are inserted into the size of subject image, realize and are inserted into the ruler of subject image in three-dimensional space adjustment
It is very little, help to make to be inserted into subject image with go back that original image is more preferable to be merged, accurately shown to user and replace target object
Effect afterwards.
The method that above-mentioned Fig. 3 corresponding embodiment provides highlights the step of removing target object image based on depth information,
To efficiently utilize three-dimensional point cloud information, target object image is accurately removed from image to be processed, helps to tie
The three-dimensional dimension information for being inserted into subject image is closed, more accurately target object image is replaced with and is inserted into subject image, from
And keep the effect of the replaced image presented to user more preferable.
Any image processing method that the embodiment of the present disclosure provides can have data-handling capacity by any suitable
Equipment execute, including but not limited to: terminal device and server etc..Alternatively, any image that the embodiment of the present disclosure provides
Processing method can be executed by processor, be implemented as processor executes the disclosure by the command adapted thereto for calling memory to store
Any image processing method that example refers to.Hereafter repeat no more.
Exemplary means
Fig. 4 is the structural schematic diagram for the image processing apparatus that one exemplary embodiment of the disclosure provides.The present embodiment can answer
With on an electronic device, as shown in figure 4, image processing apparatus includes: the first acquisition module 401, include object for acquisition
The image to be processed of body image；Remove module 402, for removing target object image from image to be processed；Recovery module
403, for restoring the image to be processed after removal target object image, obtain going back original image；Second obtains module
404, it is inserted into subject image for obtaining, and subject image insertion will be inserted into and go back the target position in original image, wherein mesh
Mark is set to the position for removing target object image；The first adjustment module 405 is inserted into subject image in also original image for adjusting
Size as in is to target size.
In the present embodiment, first obtain module 401 can from it is long-range or from local obtain include target object image
Image to be processed.Wherein, image to be processed can be the image that camera shoots the space comprising target object.
The target object of target object image instruction can be the objects such as furniture, clothes.Above-mentioned camera can be various types
Camera.For example, it may be the first-class camera that can shoot depth image of such as binocular camera shooting.
In the present embodiment, remove module 402 can remove target object image from image to be processed.Specifically,
As an example, remove module 402 can use existing object detection method (such as the target detection based on convolutional neural networks
Method), it determines target object image first from image to be processed, the corresponding pixel of target object image is deleted.Remove
Occurs the region of a blank in image to be processed after target object image, which is the profile of target object image.
In the present embodiment, recovery module 403 can restore the image to be processed after removal target object image,
It obtains going back original image.Specifically, recovery module 403 can use various modes (such as neural network model) and go back to image
It is former.Image after reduction may be implemented after removing target object image, and the region of blank is reverted to background image, makes to restore
The display of image is more natural.For example, target object image is furniture image, set since furniture is placed against the wall, when removal furniture figure
As after, there is white space, it, can be by white space metope image after the image restoring to be processed after removal furniture image
Filling, to realize the visual effect for removing furniture.
In the present embodiment, it second obtains that module 404 is available is inserted into subject image, and subject image will be inserted into
The target position in original image is gone back in insertion.Wherein, target position is the position for removing target object image, i.e., above-mentioned target object
Position of the image where in image to be processed.Second acquisition module 404 can be obtained from Local or Remote is inserted into object figure
Picture, being inserted into subject image can be the image of various objects, for example, it may be the furniture that user selects from furniture image library
Image.Above-mentioned target position can be characterized with coordinate, for example, can using the coordinate of the geometric center point of target object image as
Target position.
In the present embodiment, the adjustable subject image that is inserted into of the first adjustment module 405 is in the size gone back in original image
To target size.
As an example, target size can be the size of above-mentioned target object image (such as comprising target object image
The size of minimum rectangle or the overall size of target object image).Target size is also possible to 405 basis of the first adjustment module
The size that the region where target position also in original image is determined with the size for being inserted into subject image.For example, the first adjustment
Module 405 can determine the size in the region comprising target position and not comprising other subject images, then be based on above-mentioned size, right
It is inserted into subject image and zooms in and out processing, so that (such as the size of above-mentioned zone is pre- by its size adjusting to target size
If ratio).
It is the structural schematic diagram for the image processing apparatus that disclosure another exemplary embodiment provides referring to Fig. 5, Fig. 5.
In some optional implementations, recovery module 403 can be further used for: after removing target object image
The trained in advance image restoring model of image to be processed input, obtain going back original image.
In some optional implementations, image to be processed has corresponding depth information；And remove module 402 can
To include: the first determination unit 4021, for being based on depth information, the corresponding three-dimensional point cloud information of image to be processed is determined；The
Two determination units 4022, for from three-dimensional point cloud information, determining that the three-dimensional point cloud information for characterizing target object image is made
For target three-dimensional point cloud information；Third determination unit 4023 determines target object image for being based on target three-dimensional point cloud information
Position in image to be processed；Unit 4024 is removed, for according to position, target object image to be moved from image to be processed
It removes.
In some optional implementations, subject image is inserted into corresponding three-dimensional dimension information；And first
Adjusting module 405 may include: the 4th determination unit 4051, go back the corresponding three-dimensional point cloud information of original image for determination；Adjustment
Unit 4052 is inserted into the corresponding three-dimensional dimension information of subject image three-dimensional point cloud letter corresponding with original image is gone back for being based on
Breath, adjustment are inserted into subject image in the size gone back in original image to target size.
In some optional implementations, device can also comprise determining that module 406, go back original image for determination
Colouring information and the colouring information for being inserted into subject image；Second adjustment module 407, for adjusting the face for being inserted into subject image
Color information, so as to be inserted into the colouring information of subject image and go back the matches color information of original image.
Disclosure image processing apparatus provided by the above embodiment, by moving target object image from image to be processed
It removes, then the image to be processed after removal is restored, then will be inserted into subject image and be inserted into the removal mesh gone back in original image
The position of subject image is marked, and adjusts the size of the image of insertion.To realize the replacement to target object image, make to be inserted into
Image in going back original image explicitly more naturally, help to make user quickly, neatly check and replace target object figure
As the effect of the object of instruction.
Example electronic device
In the following, being described with reference to Figure 6 the electronic equipment according to the embodiment of the present disclosure.The electronic equipment can be such as Fig. 1 institute
Any of terminal device 101 and server 103 for showing or both or with their independent stand-alone devices, the stand-alone device
It can be communicated with terminal device 101 and server 103, to receive the collected input signal of institute from them.
Fig. 6 illustrates the block diagram of the electronic equipment according to the embodiment of the present disclosure.
As shown in fig. 6, electronic equipment 600 includes one or more processors 601 and memory 602.
Processor 601 can be central processing unit (CPU) or have data-handling capacity and/or instruction execution capability
Other forms processing unit, and can control the other assemblies in electronic equipment 600 to execute desired function.
Memory 602 may include one or more computer program products, and computer program product may include various
The computer readable storage medium of form, such as volatile memory and/or nonvolatile memory.Volatile memory is for example
It may include random access memory (RAM) and/or cache memory (cache) etc..Nonvolatile memory for example may be used
To include read-only memory (ROM), hard disk, flash memory etc..It can store one or more meters on computer readable storage medium
Calculation machine program instruction, processor 601 can run program instruction, at the image of each embodiment to realize the disclosure above
Reason method and/or other desired functions.Such as input signal, letter can also be stored in a computer-readable storage medium
The various contents such as number component, noise component(s).Processor 601 is configured to read the computer journey stored in memory 602
Sequence, to execute the method and step as shown in figure any in Fig. 2-5.
In one example, electronic equipment 600 can also include: input unit 603 and output device 604, these components
It is interconnected by bindiny mechanism's (not shown) of bus system and/or other forms.
For example, the input unit 603 can be camera shooting when the electronic equipment is terminal device 101 or server 103
The equipment such as head, mouse, keyboard are used for input picture.When the electronic equipment is stand-alone device, which be can be
Communication network connector, for receiving inputted image from terminal device 101 and server 103.
The output device 604 can be output to the outside various information, including the image after insertion subject image.The output is set
Standby 604 may include such as display, loudspeaker, printer and communication network and its remote output devices connected
Deng.
Certainly, to put it more simply, illustrated only in Fig. 6 it is some in component related with the disclosure in the electronic equipment 600,
The component of such as bus, input/output interface etc. is omitted.In addition to this, according to concrete application situation, electronic equipment 600
It can also include any other component appropriate.
Illustrative computer program product and computer readable storage medium
Other than the above method and equipment, embodiment of the disclosure can also be computer program product comprising meter
Calculation machine program instruction, it is above-mentioned that the computer program instructions make the processor execute this specification when being run by processor
According to the step in the image processing method of the various embodiments of the disclosure described in " illustrative methods " part.
The computer program product can be write with any combination of one or more programming languages for holding
The program code of row embodiment of the present disclosure operation, described program design language includes object oriented program language, such as
Java, C++ etc. further include conventional procedural programming language, such as " C " language or similar programming language.Journey
Sequence code can be executed fully on the user computing device, partly execute on a user device, be independent soft as one
Part packet executes, part executes on a remote computing or completely in remote computing device on the user computing device for part
Or it is executed on server.
In addition, embodiment of the disclosure can also be computer readable storage medium, it is stored thereon with computer program and refers to
It enables, the computer program instructions make the processor execute above-mentioned " the exemplary side of this specification when being run by processor
According to the step in the image processing method of the various embodiments of the disclosure described in method " part.
The computer readable storage medium can be using any combination of one or more readable mediums.Readable medium can
To be readable signal medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can include but is not limited to electricity, magnetic, light, electricity
Magnetic, the system of infrared ray or semiconductor, device or device, or any above combination.Readable storage medium storing program for executing it is more specific
Example (non exhaustive list) includes: the electrical connection with one or more conducting wires, portable disc, hard disk, random access memory
Device (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc
Read-only memory (CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
The basic principle of the disclosure is described in conjunction with specific embodiments above, however, it is desirable to, it is noted that in the disclosure
The advantages of referring to, advantage, effect etc. are only exemplary rather than limitation, must not believe that these advantages, advantage, effect etc. are the disclosure
Each embodiment is prerequisite.In addition, detail disclosed above is merely to exemplary effect and the work being easy to understand
With, rather than limit, it is that must be realized using above-mentioned concrete details that above-mentioned details, which is not intended to limit the disclosure,.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with its
The difference of its embodiment, the same or similar part cross-reference between each embodiment.For system embodiment
For, since it is substantially corresponding with embodiment of the method, so being described relatively simple, referring to the portion of embodiment of the method in place of correlation
It defends oneself bright.
Device involved in the disclosure, device, equipment, system block diagram only as illustrative example and be not intended to
It is required that or hint must be attached in such a way that box illustrates, arrange, configure.As those skilled in the art will appreciate that
, it can be connected by any way, arrange, configure these devices, device, equipment, system.Such as "include", "comprise", " tool
" etc. word be open vocabulary, refer to " including but not limited to ", and can be used interchangeably with it.Vocabulary used herein above
"or" and "and" refer to vocabulary "and/or", and can be used interchangeably with it, unless it is not such that context, which is explicitly indicated,.Here made
Vocabulary " such as " refers to phrase " such as, but not limited to ", and can be used interchangeably with it.
Disclosed method and device may be achieved in many ways.For example, can by software, hardware, firmware or
Software, hardware, firmware any combination realize disclosed method and device.The said sequence of the step of for the method
Merely to be illustrated, the step of disclosed method, is not limited to sequence described in detail above, special unless otherwise
It does not mentionlet alone bright.In addition, in some embodiments, also the disclosure can be embodied as to record program in the recording medium, these programs
Including for realizing according to the machine readable instructions of disclosed method.Thus, the disclosure also covers storage for executing basis
The recording medium of the program of disclosed method.
It may also be noted that each component or each step are can to decompose in the device of the disclosure, device and method
And/or reconfigure.These decompose and/or reconfigure the equivalent scheme that should be regarded as the disclosure.
The above description of disclosed aspect is provided so that any person skilled in the art can make or use this
It is open.Various modifications in terms of these are readily apparent to those skilled in the art, and are defined herein
General Principle can be applied to other aspect without departing from the scope of the present disclosure.Therefore, the disclosure is not intended to be limited to
Aspect shown in this, but according to principle disclosed herein and the consistent widest range of novel feature.
In order to which purpose of illustration and description has been presented for above description.In addition, this description is not intended to the reality of the disclosure
It applies example and is restricted to form disclosed herein.Although already discussed above multiple exemplary aspects and embodiment, this field skill
Its certain modifications, modification, change, addition and sub-portfolio will be recognized in art personnel.
Claims (10)
1. a kind of image processing method, comprising:
Obtain the image to be processed including target object image；
The target object image is removed from the image to be processed；
Image to be processed after the removal target object image is restored, obtains going back original image；
Acquisition is inserted into subject image, and described be inserted into described in subject image insertion is gone back the target position in original image,
In, the target position is the position for removing the target object image
Subject image is inserted into described in adjustment in the size gone back in original image to target size.
2. according to the method described in claim 1, wherein, the described pair of image to be processed removed after the target object image into
Row reduction, obtains going back original image, comprising:
The input of the image to be processed after target object image image restoring model trained in advance will be removed, also original image is obtained
Picture.
3. according to the method described in claim 1, wherein, the image to be processed has corresponding depth information；And
It is described to remove the target object image from the image to be processed, comprising:
Based on the depth information, the corresponding three-dimensional point cloud information of the image to be processed is determined；
From the three-dimensional point cloud information, determine for characterizing the three-dimensional point cloud information of the target object image as target three
Tie up point cloud information；
Based on the target three-dimensional point cloud information, position of the target object image in the image to be processed is determined；
According to the position, the target object image is removed from the image to be processed.
4. according to the method described in claim 3, wherein, the subject image that is inserted into is with corresponding three-dimensional dimension information；
And
Subject image is inserted into described in the adjustment in the size gone back in original image to target size, comprising:
The corresponding three-dimensional point cloud information of original image is gone back described in determination；
Based on it is described be inserted into the corresponding three-dimensional dimension information of subject image and it is described go back the corresponding three-dimensional point cloud information of original image,
Subject image is inserted into described in adjustment in the size gone back in original image to target size.
5. method described in one of -4 according to claim 1, wherein the method also includes:
The colouring information and the colouring information for being inserted into subject image of original image are gone back described in determination；
The colouring information of subject image is inserted into described in adjustment, so that the colouring information for being inserted into subject image is gone back with described
The matches color information of original image.
6. a kind of image processing apparatus, comprising:
First obtains module, for obtaining the image to be processed including target object image；
Remove module, for removing the target object image from the image to be processed；
Recovery module obtains going back original image for restoring the image to be processed after the removal target object image；
Second obtains module, is inserted into subject image for obtaining, and is inserted into the subject image insertion also original image for described
Target position as in, wherein the target position is the position for removing the target object image；
The first adjustment module described is inserted into subject image in the size gone back in original image to target size for adjusting.
7. device according to claim 6, wherein the recovery module is further used for:
The input of the image to be processed after target object image image restoring model trained in advance will be removed, also original image is obtained
Picture.
8. device according to claim 6, wherein the image to be processed has corresponding depth information；And
The remove module includes:
First determination unit determines the corresponding three-dimensional point cloud information of the image to be processed for being based on the depth information；
Second determination unit, for determining the three-dimensional for characterizing the target object image from the three-dimensional point cloud information
Point cloud information is as target three-dimensional point cloud information；
Third determination unit determines the target object image described wait locate for being based on the target three-dimensional point cloud information
Manage the position in image；
Unit is removed, for according to the position, the target object image to be removed from the image to be processed.
9. a kind of computer readable storage medium, the storage medium is stored with computer program, and the computer program is used for
Execute any method of the claims 1-5.
10. a kind of electronic equipment, the electronic equipment include:
Processor；
For storing the memory of the processor-executable instruction；
The processor, for reading the executable instruction from the memory, and it is above-mentioned to realize to execute described instruction
Any method of claim 1-5.
Priority Applications (8)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910636694.0A CN110390731B (en) | 2019-07-15 | 2019-07-15 | Image processing method, image processing device, computer-readable storage medium and electronic equipment |
| US16/929,082 US10956626B2 (en) | 2019-07-15 | 2020-07-14 | Artificial intelligence systems and methods for interior design |
| AU2020315029A AU2020315029B2 (en) | 2019-07-15 | 2020-07-15 | Artificial intelligence systems and methods for interior design |
| PCT/CN2020/102215 WO2021008566A1 (en) | 2019-07-15 | 2020-07-15 | Artificial intelligence systems and methods for interior design |
| JP2022502901A JP7325602B2 (en) | 2019-07-15 | 2020-07-15 | Artificial intelligence system and method for interior design |
| CA3147320A CA3147320A1 (en) | 2019-07-15 | 2020-07-15 | Artificial intelligence systems and methods for interior design |
| US17/180,415 US20210173968A1 (en) | 2019-07-15 | 2021-02-19 | Artificial intelligence systems and methods for interior design |
| US17/180,377 US20210173967A1 (en) | 2019-07-15 | 2021-02-19 | Artificial intelligence systems and methods for interior design |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910636694.0A CN110390731B (en) | 2019-07-15 | 2019-07-15 | Image processing method, image processing device, computer-readable storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110390731A true CN110390731A (en) | 2019-10-29 |
| CN110390731B CN110390731B (en) | 2020-07-24 |
Family
ID=68285065
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910636694.0A Active CN110390731B (en) | 2019-07-15 | 2019-07-15 | Image processing method, image processing device, computer-readable storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110390731B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113239445A (en) * | 2021-06-11 | 2021-08-10 | 重庆电子工程职业学院 | AR-based indoor pipeline information display method and system |
| WO2023051143A1 (en) * | 2021-09-28 | 2023-04-06 | 北京地平线信息技术有限公司 | Data desensitization method and apparatus |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150178962A1 (en) * | 2013-12-24 | 2015-06-25 | Konica Minolta, Inc. | Image processing apparatus, image forming apparatus, image generating method, and medium |
| US20160198097A1 (en) * | 2015-01-05 | 2016-07-07 | GenMe, Inc. | System and method for inserting objects into an image or sequence of images |
| CN107194870A (en) * | 2017-05-24 | 2017-09-22 | 北京大学深圳研究生院 | A kind of image scene reconstructing method based on conspicuousness object detection |
| CN109035385A (en) * | 2018-06-06 | 2018-12-18 | 链家网（北京）科技有限公司 | Panorama sketch screenshot method and system in a kind of source of houses virtual three-dimensional model |
-
2019
- 2019-07-15 CN CN201910636694.0A patent/CN110390731B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150178962A1 (en) * | 2013-12-24 | 2015-06-25 | Konica Minolta, Inc. | Image processing apparatus, image forming apparatus, image generating method, and medium |
| US20160198097A1 (en) * | 2015-01-05 | 2016-07-07 | GenMe, Inc. | System and method for inserting objects into an image or sequence of images |
| CN107194870A (en) * | 2017-05-24 | 2017-09-22 | 北京大学深圳研究生院 | A kind of image scene reconstructing method based on conspicuousness object detection |
| CN109035385A (en) * | 2018-06-06 | 2018-12-18 | 链家网（北京）科技有限公司 | Panorama sketch screenshot method and system in a kind of source of houses virtual three-dimensional model |
Non-Patent Citations (2)
| Title |
|---|
| 匿名: "酷家乐3D云设计", 《HTTP://WWW.IQIYI.COM/W_19RSV9UPY9.HTML#CURID=6174573709_3E989FBC70EFCDC43BF7EAF5873E5761》 * |
| 匿名: "酷家乐云设计软件如何添加和替换家具", 《HTTPS://WWW.7DOWN.COM/ARTICLE/91793.HTML》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113239445A (en) * | 2021-06-11 | 2021-08-10 | 重庆电子工程职业学院 | AR-based indoor pipeline information display method and system |
| CN113239445B (en) * | 2021-06-11 | 2023-08-01 | 重庆电子工程职业学院 | AR-based indoor pipeline information display method and system |
| WO2023051143A1 (en) * | 2021-09-28 | 2023-04-06 | 北京地平线信息技术有限公司 | Data desensitization method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110390731B (en) | 2020-07-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11978245B2 (en) | Method and apparatus for generating image | |
| CN108830235B (en) | Method and apparatus for generating information | |
| CN110503703B (en) | Method and apparatus for generating image | |
| US11270099B2 (en) | Method and apparatus for generating facial feature | |
| US10832037B2 (en) | Method and apparatus for detecting image type | |
| CN109255337B (en) | Face key point detection method and device | |
| CN108337505B (en) | Information acquisition method and device | |
| CN112954450B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN108492364A (en) | The method and apparatus for generating model for generating image | |
| CN108198044A (en) | Methods of exhibiting, device, medium and the electronic equipment of merchandise news | |
| CN111612842B (en) | Method and device for generating pose estimation model | |
| CN105512637A (en) | Image processing method and electric device | |
| CN110059623B (en) | Method and apparatus for generating information | |
| CN108388889B (en) | Method and device for analyzing face image | |
| CN108229375B (en) | Method and device for detecting face image | |
| CN110472558B (en) | Image processing method and device | |
| CN110059624B (en) | Method and apparatus for detecting living body | |
| US10628998B2 (en) | System and method for three dimensional object reconstruction and quality monitoring | |
| CN110390731A (en) | Image processing method, device, computer readable storage medium and electronic equipment | |
| CN112639870B (en) | Image processing device, image processing method, and image processing program | |
| CN113033677A (en) | Video classification method and device, electronic equipment and storage medium | |
| CN110516598B (en) | Method and apparatus for generating image | |
| CN110673717A (en) | Method and apparatus for controlling output device | |
| CN108921138B (en) | Method and apparatus for generating information | |
| CN110415318B (en) | Image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20191211 Address after: 100085 Floor 102-1, Building No. 35, West Second Banner Road, Haidian District, Beijing Applicant after: Seashell Housing (Beijing) Technology Co., Ltd. Address before: 300 457 days Unit 5, Room 1, 112, Room 1, Office Building C, Nangang Industrial Zone, Binhai New Area Economic and Technological Development Zone, Tianjin Applicant before: Shell Technology Co., Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |