CN109035288A - A kind of image processing method and device, equipment and storage medium - Google Patents

A kind of image processing method and device, equipment and storage medium Download PDF

Info

Publication number
CN109035288A
CN109035288A CN201810848300.3A CN201810848300A CN109035288A CN 109035288 A CN109035288 A CN 109035288A CN 201810848300 A CN201810848300 A CN 201810848300A CN 109035288 A CN109035288 A CN 109035288A
Authority
CN
China
Prior art keywords
image
identifier
attribute
target object
synthesized
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810848300.3A
Other languages
Chinese (zh)
Other versions
CN109035288B (en
Inventor
陈晨
陶然
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to CN201810848300.3A priority Critical patent/CN109035288B/en
Publication of CN109035288A publication Critical patent/CN109035288A/en
Application granted granted Critical
Publication of CN109035288B publication Critical patent/CN109035288B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/194Segmentation; Edge detection involving foreground-background segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/14Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038Image mosaicing, e.g. composing plane images from plane sub-images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the invention discloses a kind of image processing methods, which comprises obtains the first image, wherein includes identifier in the first image;Obtain the second image;Target object in second image is carried out to scratch figure, obtains target object;According to the attribute of identifiers value, the target object and the first image are synthesized, the image after being synthesized.The embodiment of the present invention also discloses a kind of image processing apparatus, equipment and storage medium simultaneously.

Description

A kind of image processing method and device, equipment and storage medium
Technical field
The present invention relates to artificial intelligence field more particularly to a kind of image processing methods and device, equipment and storage medium.
Background technique
Currently, most of social categories apply (APP, Application) or instant messaging class application, all support recorded video Perhaps it shoots picture or is sent after directly choosing local existing video or picture progress simple process, such as QQ, wechat Deng.In the recent period, human face detection tech, face recognition technology maturation make some class APP that dress up, such as swash and sprout, U.S. face camera, obtain Quick development is arrived.APP can use recognition of face and human face detection tech and do special face processing for picture, such as certainly Dynamic makeup and automatic installation decorations.
However the video of these clients, picture processing mode are single, automatic makeup and the automatic decorations and other effects that install additional also lack Weary interest, how by the video of recording, picture, perhaps in the local video directly chosen, picture or real-time shooting Hold, or application comprising material, such as cartoon video, decoration etc., synthesized, so that the content after synthesizing is more rich It is rich, three-dimensional, interesting, it is those skilled in the art's urgent problem to be solved.
Summary of the invention
In view of this, the embodiment of the present invention be solve the problems, such as it is existing in the prior art at least one and a kind of image is provided Processing method and processing device, equipment and storage medium.
The technical solution of the embodiment of the present invention is achieved in that
The embodiment of the present invention provides a kind of image processing method, which comprises
The first image is obtained, wherein includes identifier in the first image;
Obtain the second image;
Target object in second image is carried out to scratch figure, obtains target object;
According to the attribute of identifiers value, the target object and the first image are synthesized, after being synthesized Image.
In embodiments of the present invention, the first image of the acquisition wherein includes identifier in the first image, comprising:
Target area in the pretreatment image of acquisition is carried out to scratch figure, obtains white space;
The identifier is added to the white space according to the attribute of identifiers value, obtains the first image.
In embodiments of the present invention, described that the identifier is added to by the blank according to the attribute of identifiers value Region obtains the first image, comprising:
The identifier is added to the white space;
Attribute of identifiers value after addition is adjusted, the first image is obtained.
In embodiments of the present invention, the attribute of identifiers includes position, size and/or the direction of identifier, described Attribute of identifiers value after addition is adjusted, the first image is obtained, comprising:
The position of identifier after addition, size and/or direction are configured, the first image is obtained;
In embodiments of the present invention, the attribute of identifiers includes position, size and/or the direction of identifier, described Attribute of identifiers value after addition is adjusted, the first image is obtained, further includes:
According to the image feature information of the pretreatment image, to the position of the identifier after addition, size, and/or side To being adjusted, the first image is obtained.
In embodiments of the present invention, the attribute of identifiers value includes at least the position of identifier, described according to Attribute of identifiers value, the target object and the first image are synthesized, the image after being synthesized, comprising:
Position of the target object in the first image is adjusted according to the position of the identifier, after obtaining synthesis Image.
In embodiments of the present invention, the attribute value further includes size, described according to the attribute of identifiers value, by institute It states target object to be synthesized with the first image, the image after being synthesized, further includes:
According to display size of the target object described in the size adjusting of the identifier in the first image, closed Image after.
In embodiments of the present invention, the attribute value further includes direction, described according to the attribute of identifiers value, by institute It states target object to be synthesized with the first image, the image after being synthesized, further includes:
Direction of the target object in the first image is adjusted according to the direction of the identifier, after obtaining synthesis Image.
In embodiments of the present invention, second image includes facial image, and the target object includes face, and described Direction of the target object in the first image is adjusted according to the direction of the identifier, the image after being synthesized, packet It includes:
According to the facial image, the key point information of face is determined;
According to the key point information of the face, the three-dimensional information of face is determined;
According to the three-dimensional information of the face, the inceptive direction of the face is determined;
The face is adjusted in the first image according to the inceptive direction of the direction of the identifier and the face Direction, the image after being synthesized.
In embodiments of the present invention, the first image of the acquisition wherein includes identifier in the first image, comprising:
Target area in the frame image of first video of acquisition is carried out to scratch figure, obtains the frame figure with white space Picture;
The identifier is added to the white space of the frame image according to the attribute of identifiers value;
Using each frame image with identifier as the first image.
In embodiments of the present invention, the second image of the acquisition, comprising:
Obtain the second video of image acquisition device acquisition;
Using the present frame in second video as second image.
In embodiments of the present invention, the identifier includes placeholder.
The embodiment of the present invention provides a kind of image processing apparatus, and described image processing unit includes: the first acquisition module, Two obtain module, scratch module and synthesis module, in which:
Described first obtains module, wherein includes identifier in the first image for obtaining the first image;
Described second obtains module, for obtaining the second image;
The stingy module scratches figure for carrying out to the target object in second image, obtains target object;
The synthesis module, for according to the attribute of identifiers value, the target object and the first image to be carried out Synthesis, the image after being synthesized.
In embodiments of the present invention, described first module is obtained, comprising:
First stingy figure unit carries out scratching figure for the target area in the pretreatment image to acquisition, obtains white space;
First adding unit, for the identifier to be added to the blank area according to the attribute of identifiers value Domain obtains the first image.
In embodiments of the present invention, first adding unit, comprising:
First addition component, for the identifier to be added to the white space;
The first adjustment component obtains the first image for being adjusted to the attribute of identifiers value after addition.
In embodiments of the present invention, the attribute of identifiers includes position, size and/or the direction of identifier, described The first adjustment component, comprising:
First setting subassembly, is configured for position, size and/or the direction to the identifier after addition, obtains First image;
In embodiments of the present invention, the attribute of identifiers includes position, size and/or the direction of identifier, described The first adjustment component, further includes:
The first adjustment subassembly, for first according to the image feature information of the pretreatment image, to the mark after addition Position, size, and/or the direction for knowing symbol are adjusted, and obtain the first image.
In embodiments of the present invention, position of the attribute of identifiers value including at least identifier, the synthesis module, Include:
First synthesis unit, for adjusting the target object in the first image according to the position of the identifier Position, the image after being synthesized.
In embodiments of the present invention, the attribute value further includes size, the synthesis module, further includes:
Second synthesis unit, for the target object according to the size adjusting of the identifier in the first image Display size, the image after being synthesized.
In embodiments of the present invention, the attribute value further includes direction, the synthesis module, further includes:
Third synthesis unit, for adjusting the target object in the first image according to the direction of the identifier Direction, the image after being synthesized.
In embodiments of the present invention, second image includes facial image, and the target object includes face, and described Three synthesis units, comprising:
First determines component, for determining the key point information of face according to the facial image;
Second determines that component determines the three-dimensional information of face for the key point information according to the face;
Third determines component, for the three-dimensional information according to the face, determines the inceptive direction of the face;
First compound component, for adjusting the face according to the direction of the identifier and the inceptive direction of the face Direction in the first image, the image after being synthesized.
In embodiments of the present invention, described first module is obtained, comprising:
Second stingy figure unit carries out scratching figure for the target area in the frame image to the first video of acquisition, be had There is the frame image of white space;
Second adding unit, for the identifier to be added to the frame image according to the attribute of identifiers value White space;
First execution unit, for that will have each frame image of identifier as the first image.
In embodiments of the present invention, described second module is obtained, comprising:
First acquisition unit, for obtaining the second video of image acquisition device acquisition;
Second execution unit, for using the present frame in second video as second image.
In embodiments of the present invention, the identifier includes placeholder.
The embodiment of the present invention provides a kind of computer storage medium, which is characterized in that deposits in the computer storage medium Computer executable instructions are contained, after which is performed, can be realized method and step as described above.
The embodiment of the present invention provides a kind of computer equipment, which is characterized in that the computer equipment include memory and Processor is stored with computer executable instructions on the memory, and the processor runs the computer on the memory Method and step as described above can be realized when executable instruction.
The embodiment of the present invention provides a kind of image processing method and device, equipment and storage medium, wherein obtains the first figure Picture wherein includes identifier in the first image;Obtain the second image;Target object in second image is carried out stingy Figure, obtains target object;According to the attribute of identifiers value, the target object and the first image are synthesized, obtained Image after synthesis;So can it is simple, easily and efficiently by image perhaps image target object or video carry out Synthesis, so that the image after synthesis is more abundant, three-dimensional, interesting.
Detailed description of the invention
Fig. 1 is a kind of system architecture schematic diagram;
Fig. 2A is the implementation process schematic diagram one of image processing method of the embodiment of the present invention;
Fig. 2 B is the first image of embodiment of the present invention schematic diagram;
Fig. 2 C is the second image of embodiment of the present invention schematic diagram;
Fig. 2 D is the image schematic diagram one after synthesis of the embodiment of the present invention;
Fig. 3 A is the implementation process schematic diagram two of image processing method of the embodiment of the present invention;
Fig. 3 B is facial image of embodiment of the present invention schematic diagram;
Fig. 3 C is the image schematic diagram two after synthesis of the embodiment of the present invention;
Fig. 4 is the composed structure schematic diagram of image processing apparatus of the embodiment of the present invention;
Fig. 5 is a kind of hardware entities schematic diagram of image processing equipment of the embodiment of the present invention.
Specific embodiment
The technical solution of the present invention is further elaborated with reference to the accompanying drawings and examples.
Fig. 1 is a kind of system architecture schematic diagram, and Fig. 1 shows a communication system, which includes terminal 11 and server 12, it is wherein installed in terminal 11 and has run various clients (such as application program (APP, Application)), client packet It includes social software, instant message applications, share the clients such as software from media software, information, server is corresponding to client Server.In this example, terminal 11 and server 12 can be one or more, and therefore, above system includes one or more A terminal 11 for being equipped with client and one or more servers 12, these terminals 11 and server 12 are connected by network 13 It connects.In embodiments of the present invention, network side server 12 can be interacted with terminal 11 by client, and terminal 11 will be pending The material of cloth is to server 12, and then server releases the material to be released received.Wherein, material package to be released Include the combination of image, video and image and video.
It should be noted that the system architecture that some embodiments of the present invention can be proposed based on Fig. 1.
The embodiment of the present invention provides a kind of image processing method, is applied to terminal, and the function that this method is realized can lead to The processor caller code crossed in terminal realizes that certain program code can be stored in computer storage medium, During realization, which can be used as client, so as to install at the terminal, and be transported by the terminal Row.From the above, it can be seen that the terminal includes at least pocessor and storage media.Fig. 2A is image processing method of the embodiment of the present invention The implementation process schematic diagram one of method, as shown in Figure 2 A, this method comprises:
Step S201, the first image is obtained, wherein includes identifier in the first image;
Here, the first image can be the image of captured in real-time, is also possible to the picture being locally stored, is also possible to The video of recording is also possible to the material carried in application APP, such as a cartoon picture, one section of cartoon video etc., sheet In inventive embodiments and with no restrictions.
In the present embodiment, client includes the various clients that can be uploaded and issue multimedia messages, such as social software, Instant message applications shares the clients such as software, information upload software from media software, information, and user will by social software Picture after synthesis perhaps video distribution go out user can also be by instant message applications by the picture or video after synthesis It is sent to oneself good friend, user can certainly be by uploading picture or video after synthesis from media software, and user may be used also To share picture or video after software division synthesis by information.
In general, terminal can be various types of equipment with information processing capability, institute in the process of implementation Terminal includes mobile terminal and fixed terminal again, such as the mobile terminal may include mobile phone, personal digital assistant (Personal Digital Assistant, PDA), navigator, digital telephone, visual telephone, smartwatch, Intelligent bracelet, Wearable device, tablet computer etc..Server can be mobile terminal such as mobile phone, tablet computer, notes during realization This computer, fixed terminal such as personal computer and server cluster etc. have the calculating equipment of information processing capability.
Here, the identifier can be placeholder etc..Wherein, the placeholder is exactly as its name suggests first to occupy one Fixed position, then the symbol of content is added inside, the editor with all kinds of documents in computer is widely applied.Wherein, described Placeholder contains image placeholder again, and described image placeholder is exactly in some webpage or article, because can not find suitably Image is just first looked for the image temporarily replaced, is placed on the position of final image, as interim substitution, it be it is interim, The figure of substitute.The effect of image placeholder is that a position is occupied on webpage, when being not ready the picture to be inserted into When, first can account for a position with image placeholder, to mark, in addition to can mark be inserted into picture position, can also To mark the size for being inserted into picture, the direction for being inserted into picture can also be marked.
It should be noted that, when the identifier is placeholder, what is referred to is exactly image occupy-place in the embodiment of the present invention Symbol.
Step S202, the second image is obtained;
Here, second image can be the image of captured in real-time, is also possible to the picture being locally stored, is also possible to The video of recording is also possible to the material carried in application APP, such as a cartoon picture, one section of cartoon video etc., sheet In inventive embodiments and with no restrictions.
Step S203, the target object in second image is carried out scratching figure, obtains target object;
Here, the target object can be any one object that can be shown in the second image.For example, in a width In facial image, the target object can be face;In the image shot in a width body-building, the target object be can be Just in the people of body-building;In a width animal photography image, the target object can be a certain or a few animals, can also Be a certain animal body a part.In the embodiment of the present invention, to the target object and it is not particularly limited.
Step S204, according to the attribute of identifiers value, the target object and the first image is synthesized, obtained Image after synthesis.
Here, the attribute of identifiers value, the including but not limited to value of the position of identifier marking, identifier marking Direction value, the value of the size of identifier marking, wherein the value of the position of the identifier marking, indicate it is to be inserted Enter the position in the image of picture in post synthesis;The value in the direction of the identifier marking, expression are inserted into picture and are synthesizing The direction in image afterwards;The value of the size of the identifier marking, expression are inserted into the image of picture in post synthesis Size.
In image processing method provided in an embodiment of the present invention, by obtaining the first image, wherein the first image Upper includes identifier;Obtain the second image;Target object in second image is carried out to scratch figure, obtains target object;Root According to the attribute of identifiers value, the target object and the first image are synthesized, the image after being synthesized;In this way, Can it is simple, easily and efficiently by image perhaps image target object or video synthesize so that after synthesis Image is more abundant, three-dimensional, interesting.
Based on above method embodiment, the embodiment of the present invention provides a kind of image processing method again, this method comprises:
Step S21, the target area in the pretreatment image of acquisition is carried out scratching figure, obtains white space;
Here, the pretreatment image can be the network materials of acquisition, be also possible to the picture of oneself shooting.Fig. 2 B For the first image schematic diagram of the embodiment of the present invention, as shown in Figure 2 B, picture 21 is pretreatment image, to the target area in picture 21 Domain 22 carries out scratching figure, obtains white space 24.Then, identifier is added to the white space 24, and to the identifier Attribute value be adjusted, so that it may obtain the first image 23.Target area 22 in pretreatment image 21 is where butterfly Region can select different target areas according to different effects in actual use.
Step S22, the identifier is added to by the white space according to the attribute of identifiers value, obtains first Image;
Here it is possible to which identifier is added in white space 20, the image of identifier is added in white space 20, As the first image.
Here, the attribute of identifiers value, including but not limited to, the value of the position of identifier marking, identifier mark The value in the direction of note, the value of the size of identifier marking.
In the embodiment of the present invention, the identifier is added to by the white space according to the attribute of identifiers value, May include two ways: identifier can be first placed in the first image by first way, then according to the mark The attribute value for knowing symbol adjusts the value in the position of identifier, size and/or direction, is located at white space;Second of side Formula, can also be according to the position of the identifier, and the value in size and/or direction adds the identifier with the attribute value It is added to the white space.
Here, the step S22 can be realized by the mode in following steps:
Step S221, the identifier is added to the white space;
Step S222, the attribute of identifiers value after addition is adjusted, obtains the first image.
Here, after identifier being added to the white space, it is also necessary to the attribute of identifiers value is adjusted, Just available first image.
Here, the attribute of identifiers includes position, size and/or the direction of identifier, in turn, the step S222 It can be realized by two ways, wherein
The first implementation is realized according to the following steps:
Step S2221a, the position of the identifier after addition, size and/or direction are configured, obtain the first image;
In the embodiment of the present invention, the position of the identifier after addition, size and/or direction are configured, obtain first Image refers to that user or too development personnel can be according to oneself wishes, position, ruler to the identifier after addition Very little and/or direction is configured, to achieve the effect that oneself to want.
Second of implementation is realized according to the following steps:
Step S2221b, position, ruler according to the image feature information of the pretreatment image, to the identifier after addition Very little, and/or direction is adjusted, and obtains the first image.
In the embodiment of the present invention, according to the image feature information of the pretreatment image, to the position of the identifier after addition Set, size, and/or direction are adjusted, and obtain the first image, referring to can be according to the characteristic information of pretreatment image, people For the position of the identifier after addition, size, and/or direction are adjusted, obtain the first image;It can also be according to pre- place The characteristic information for managing image, is adjusted the position of the identifier after addition, size, and/or direction using program, obtains the One image.
Wherein, described image characteristic information, including but not limited to, the color characteristic of image, textural characteristics, shape feature and Spatial relation characteristics.The color characteristic is a kind of global characteristics, describes the table of scenery corresponding to image or image-region Surface properties.The textural characteristics are also a kind of global characteristics, it also illustrates the surface of scenery corresponding to image or image-region Property.But since texture is a kind of characteristic of body surface, it can not reflect the essential attribute of object completely, so only High-level picture material can not be obtained using textural characteristics.Different from color characteristic, textural characteristics are not based on pixel Feature, it need statistics calculating is carried out in the region comprising multiple pixels.The shape feature has two class representation methods, One kind is contour feature, and another kind of is provincial characteristics.Outer boundary of the contour feature of image mainly for object, and the area of image Characteristic of field is then related to entire shape area.So-called spatial relationship refers to the phase between the multiple targets split in image Mutual spatial position or relative direction relationship, these relationships can also be divided into connection/syntople, overlapping/overlapping relation and comprising/ Containment relationship etc..
Step S23, the second image is obtained;
Here, second image can be the image of camera shooting, be also possible to from the image locally obtained, It can be the image obtained from network.
Step S24, the target object in second image is carried out scratching figure, obtains target object;
Fig. 2 C is the second image of embodiment of the present invention schematic diagram, and as shown in Figure 2 C, picture 25 is the second image, and picture 26 is Target object, the target object are the face part of face in figure, to the target object 26 in second image 25 into Row scratches figure, obtains target object 26.Currently, can use program to the target in the second image there are many kinds of the methods of stingy figure Object carries out scratching figure, also can use all kinds of APP and carries out scratching figure to the target object in the second image, for example, software PhotoShop, software Knockout, software PhotoImpact etc., those skilled in the art can select according to actual needs Suitable method carries out scratching figure, and which is not described herein again.
Step S25, the described attribute of identifiers value includes at least the position of identifier, adjusts institute according to the position of identifier State position of the target object in the first image, the image after being synthesized;
Step S26, the described attribute value further includes size, according to target object described in the size adjusting of the identifier in institute State the display size in the first image, the image after being synthesized;
Step S27, the described attribute value further includes direction, adjusts the target object in institute according to the direction of the identifier State the direction in the first image, the image after being synthesized.
Fig. 2 D is the image schematic diagram one after synthesis of the embodiment of the present invention, as shown in Figure 2 D, first to being added to Fig. 2 B blank The size of identifier in region 24, direction, position are adjusted;Then institute in Fig. 2 C is adjusted according to the position of the identifier Position of the target object 26 in the first image is stated, according to target object described in the size adjusting of the identifier described Display size in first image adjusts side of the target object in the first image according to the direction of the identifier To image 27 after being synthesized.Wherein, position of the target object in the image after the synthesis and the identifier Position consistency, size of the target object in the image after the synthesis is consistent with the size of the identifier, described Direction of the target object in the image after the synthesis is consistent with the direction of the identifier.
In other embodiments, the method also includes:
Second image includes facial image, and the target object includes face, the side according to the identifier Image to direction of the adjustment target object in the first image, after being synthesized, comprising:
According to the facial image, the key point information of face is determined;
In the embodiment of the present invention, the face key point information, including but not limited to, the location information of face on face, Such as human eye, nose, mouth, eyebrow etc., wherein on the same face, and may include multiple key point informations.
According to the key point information of the face, the three-dimensional information of face is determined;
According to the three-dimensional information of the face, the inceptive direction of the face is determined;
Here, the inceptive direction of the face, i.e., the direction of the described face before synthesis, for example, when face level is put It sets, the picture that shoots when camera is also horizontally arranged, the picture placed with face skew, shot when camera is horizontal positioned, with The picture shot when face is horizontal positioned, camera skew is placed, wherein the inceptive direction of face is all different.
The face is adjusted in the first image according to the inceptive direction of the direction of the identifier and the face Direction, the image after being synthesized.
Here it is possible to adjust the face described according to the direction of the inceptive direction of the face and the identifier Direction in first image, the image after being synthesized, wherein direction of the face in the first image and the mark The direction for knowing symbol is consistent.
Based on above method embodiment, the embodiment of the present invention provides a kind of image processing method again, this method comprises:
Step S21, the target area in the frame image of the first video of acquisition is carried out scratching figure, is obtained with white space Frame image;
Step S22, the identifier is added to the white space of the frame image according to the attribute of identifiers value;
Step S23, using each frame image with identifier as the first image;
Here, the first image can be the frame image carried out in treated video to the first video, wherein institute The first video is stated, the first video present in video material library in program application is can be, is also possible to the view uploaded by user Frequently, it the present invention and is not particularly limited.
Here, the step S22 can be realized by the mode in following steps:
Step S221, the identifier is added to the white space;
Step S222, the attribute of identifiers value after addition is adjusted, obtains the first image.
Here, the attribute of identifiers includes position, size and/or the direction of identifier, in turn, the step S222 It can be realized by two ways, wherein
The first implementation is realized according to the following steps:
Step S2221a, the position of the identifier after addition, size and/or direction are configured, obtain the first image;
Second of implementation is realized according to the following steps:
Step S2221b, position, ruler according to the image feature information of the pretreatment image, to the identifier after addition Very little, and/or direction is adjusted, and obtains the first image.
Step S24, the second video of image acquisition device acquisition is obtained;
Step S25, using the present frame in second video as second image;
Step S26, the target object in second image is carried out scratching figure, obtains target object;
Here, second image can be the frame image carried out in treated video to the second video, wherein institute The second video is stated, the second video present in video material library in program application is can be, is also possible to the view uploaded by user Frequently, it is also possible to image acquisition device, such as camera, the video of acquisition, the present invention is simultaneously not particularly limited.
Step S27, the described attribute of identifiers value includes at least the position of identifier, adjusts institute according to the position of identifier State position of the target object in the first image, the image after being synthesized;
Step S28, the described attribute value further includes size, according to target object described in the size adjusting of the identifier in institute State the display size in the first image, the image after being synthesized;
Step S29, the described attribute value further includes direction, adjusts the target object in institute according to the direction of the identifier State the direction in the first image, the image after being synthesized;
In the embodiment of the present invention, the identifier may include placeholder etc..
In the embodiment of the present invention, when the first image is the frame image in the first video, second image is When frame image in the second video, the image after the synthesis is also the frame image in video, i.e., by first video In frame image and the frame image in the second video synthesized according to certain rules, the image after being synthesized, into And the video after being synthesized.
In the embodiment of the present invention, when the first image is frame image (wherein, the video bag in the first video Include N frame), and when second image is a width individual picture, can by first video each frame image and institute The target object stated in the second image is synthesized, the N frame image after being synthesized, and then the video after being synthesized.Wherein, The content of background image (i.e. image in the first image in addition to white space) in image after synthesis can be continuous variation , and the content of target object in the image after synthesizing does not change, and target object only has position, direction, size can With change.
In the embodiment of the present invention, when second image is frame image (wherein, the video bag in the second video Include N frame), it, can will be in each frame image in second video and when the first image is a width individual picture Target object is synthesized with the first image, the N frame image after being synthesized, and then the video after being synthesized.Wherein, The content of background image (i.e. image in the first image in addition to white space) in image after synthesis does not change, and Content, position, direction, the size of the target object in image after synthesis are all can be continually changing.
Based on above method embodiment, the embodiment of the present invention provides a kind of image processing method again, and Fig. 3 A is that the present invention is real The implementation process schematic diagram two of an image processing method is applied, as shown in Figure 3A, this method comprises:
Step S301, basic material is obtained;
Step S302, the frame image in the basic material is carried out scratching figure, obtains white space;
In the embodiment of the present invention, the basic material of acquisition can be a video, and the corresponding sequence frame of the video is added Add specific markers, the sequence frame after addition label is labeled as to scratch the use of face function.
Step S303, face placeholder is added in the white space;
Step S304, the size and Orientation for adjusting face placeholder described in each frame image, then adjusts the face Position of the placeholder in the white space;
In the embodiment of the present invention, the step S301 to step S305 in described image processing method can be with the shape of small tool State is realized.Fig. 3 B is facial image of embodiment of the present invention schematic diagram, and as shown in Figure 3B, picture 31 is that mobile phone camera is clapped in real time The facial image taken the photograph, face account for substantial portion of position in the picture.Picture 32 is to be placed into the facial image Designated position 33 in screen, i.e., no matter face when shooting is can to force the people that will be taken in which of screen Face is put into screen designated position fixed size and selects the direction of face, and direction can be 360 degree rotation, and be real-time.
Here, the size and Orientation for adjusting the placeholder, then adjusts the placeholder in the white space In position, refer to be exactly can be with an effect of de-regulation placeholder, so that it may guarantee this block tune effect and actually see Effect be consistent.
Here, the basic material includes many frames, it is contemplated that the position of the face of real-time detection between every frame and every frame May be different with size, consider for different-effect, the every frame needs of this block are individually set, and are arranged one by one, until all All setting completed for face placeholder in frame image.It supports in small tool to each frame sign of placeholder and position, side To different, designer is facilitated to design more effects.
Step S305, the basic material finished to adjusting, is saved and is exported;
Here, the basic material finished to adjusting, is saved and is exported, referring to being organized into can make for rendering engine Material package.
Step S306, the facial image of camera shooting is measured in real time;
Step S307, according to the position of the face placeholder, the face that will test is added to the element substantially after output In material;
Step S308, the facial image of the face placeholder and real-time detection is compared, the face detected Size be adjusted to it is in the same size with placeholder;
Step S309, the facial image of the face placeholder and real-time detection is compared, the face detected Direction be adjusted to it is consistent with the direction of placeholder.
Fig. 3 C is the image schematic diagram two after synthesis of the embodiment of the present invention, and as shown in Figure 3 C, picture 34 to picture 38 is not In the same time real-time detection to face and basic material synthesized after image.The basic material into picture 38 of picture 34 includes Content and the content of face that arrives of real-time detection, be all different.It, can when camera is shot less than facial image To show the other content of shooting, can also be shown as scratching the basic material content before figure.It can be by frame each in basic material Image and the facial image of real-time detection are synthesized, can also be by the face of partial frame image and real-time detection in basic material Image is synthesized.From picture 34 into picture 38 as can be seen that the face part including face in image after synthesis, and It is not that the entire facial image shot can be detected and be identified to face, obtain the five of face to realize this effect Official.
In the embodiment of the present invention, the facial image of the face placeholder and real-time detection is compared, detecting Face direction be adjusted to it is consistent with the direction of placeholder, wherein including recognition of face, determine face key point and face Posture information recycles human face posture information therein to be adjusted the angle of face, and the human face posture information includes people Face three-dimensional information.
It should be noted that Face datection involved in present example and the method for recognition of face can have very much, mesh Before, also have many APP that can directly carry out Face datection and recognition of face, those skilled in the art can according to actual needs, Suitable Face datection and face identification method are selected, which is not described herein again.
Here, the expression of the step S306 to step S309, real-time detection face, and the face that will test is put Designated position in screen, the size of face, direction with design in advance it is identical.
In image processing method provided in an embodiment of the present invention, by obtaining basic material;To in the basic material Frame image carry out scratch figure, obtain white space;Face placeholder is added in the white space;It adjusts in each frame image Then the size and Orientation of the face placeholder adjusts position of the face placeholder in the white space;It exchanges The basic material finished is saved, is saved and is exported;The facial image of camera shooting is measured in real time;According to the people The position of face placeholder, the face that will test are added in the basic material after output;By the face placeholder and in real time The facial image of detection compares, and the size of the face detected is adjusted in the same size with placeholder;By the people Face placeholder and the facial image of real-time detection compare, and the direction of the face detected is adjusted to the direction with placeholder Unanimously;It so, it is possible simply, easily and efficiently to synthesize the facial image of captured in real-time and basic material, so that synthesis Image afterwards is more abundant, three-dimensional, interesting.
The embodiment of the present invention provides a kind of image processing apparatus, and Fig. 4 is the composition of image processing apparatus of the embodiment of the present invention Structural schematic diagram, as shown in figure 4, described device 400 includes: that the first acquisition module 401, second obtains module 402, scratches module 403 and synthesis module 404, in which:
Described first obtains module 401, wherein includes identifier in the first image for obtaining the first image;
Described second obtains module 402, for obtaining the second image;
The stingy module 403 scratches figure for carrying out to the target object in second image, obtains target object;
The synthesis module 404, for according to the attribute of identifiers value, by the target object and the first image into Row synthesis, the image after being synthesized.
In other embodiments, described first module is obtained, comprising:
First stingy figure unit carries out scratching figure for the target area in the pretreatment image to acquisition, obtains white space;
First adding unit, for the identifier to be added to the blank area according to the attribute of identifiers value Domain obtains the first image.
In other embodiments, first adding unit, comprising:
First addition component, for the identifier to be added to the white space;
The first adjustment component obtains the first image for being adjusted to the attribute of identifiers value after addition.
In other embodiments, the attribute of identifiers includes position, size and/or the direction of identifier, and described One adjustment component, comprising:
First setting subassembly, is configured for position, size and/or the direction to the identifier after addition, obtains First image;
In other embodiments, the attribute of identifiers includes position, size and/or the direction of identifier, and described One adjustment component, further includes:
The first adjustment subassembly, for first according to the image feature information of the pretreatment image, to the mark after addition Position, size, and/or the direction for knowing symbol are adjusted, and obtain the first image.
In other embodiments, the attribute of identifiers value includes at least the position of identifier, the synthesis module, packet It includes:
First synthesis unit, for adjusting the target object in the first image according to the position of the identifier Position, the image after being synthesized.
In other embodiments, the attribute value further includes size, the synthesis module, further includes:
Second synthesis unit, for the target object according to the size adjusting of the identifier in the first image Display size, the image after being synthesized.
In other embodiments, the attribute value further includes direction, the synthesis module, further includes:
Third synthesis unit, for adjusting the target object in the first image according to the direction of the identifier Direction, the image after being synthesized.
In other embodiments, second image includes facial image, and the target object includes face, the third Synthesis unit, comprising:
First determines component, for determining the key point information of face according to the facial image;
Second determines that component determines the three-dimensional information of face for the key point information according to the face;
Third determines component, for the three-dimensional information according to the face, determines the inceptive direction of the face;
First compound component, for adjusting the face according to the direction of the identifier and the inceptive direction of the face Direction in the first image, the image after being synthesized.
In other embodiments, described first module is obtained, comprising:
Second stingy figure unit carries out scratching figure for the target area in the frame image to the first video of acquisition, be had There is the frame image of white space;
Second adding unit, for the identifier to be added to the frame image according to the attribute of identifiers value White space;
First execution unit, for that will have each frame image of identifier as the first image.
In other embodiments, described second module is obtained, comprising:
First acquisition unit, for obtaining the second video of image acquisition device acquisition;
Second execution unit, for using the present frame in second video as second image.
In other embodiments, the identifier includes placeholder.
It need to be noted that: the description of apparatus above embodiment, be with the description of above method embodiment it is similar, With the similar beneficial effect of same embodiment of the method.For undisclosed technical detail in apparatus of the present invention embodiment, please refer to The description of embodiment of the present invention method and understand.
In the embodiment of the present invention, if realizing above-mentioned image processing method in the form of software function module, and conduct Independent product when selling or using, also can store in a computer readable storage medium.Based on this understanding, Substantially the part that contributes to existing technology can be with the shape of software product in other words for the technical solution of the embodiment of the present invention Formula embodies, which is stored in a storage medium, including some instructions are used so that a calculating Equipment executes all or part of each embodiment the method for the present invention.And storage medium above-mentioned includes: USB flash disk, moves firmly The various media that can store program code such as disk, ROM (Read Only Memory, read-only memory), magnetic or disk. It is combined in this way, the embodiment of the present invention is not limited to any specific hardware and software.
Accordingly, the embodiment of the present invention provides a kind of computer equipment, which includes memory and processor, described to deposit Reservoir is stored with the computer program that can be run on a processor, and the processor realizes image processing method when executing described program Step in method.
Accordingly, the embodiment of the present invention provides a kind of computer readable storage medium, is stored thereon with computer program, should The step in image processing method is realized when computer program is executed by processor.
It need to be noted that: the description of medium stored above and apparatus embodiments, with retouching for above method embodiment It is similar for stating, and has with embodiment of the method similar beneficial effect.For in storage medium of the present invention and apparatus embodiments not The technical detail of disclosure please refers to the description of embodiment of the present invention method and understands.
It should be noted that Fig. 5 is a kind of hardware entities schematic diagram of image processing equipment of the embodiment of the present invention, such as Fig. 5 Shown, the hardware entities of the image processing equipment 500 include: memory 501, communication bus 502 and processor 503, wherein
Memory 501 is configured to store the instruction and application that can be performed by processor 503, can also cache device to be processed 503 and image processing equipment 500 in each module is to be processed or processed data, FLASH (flash memory) or RAM can be passed through (Random Access Memory, random access storage device) is realized.
Communication bus 502 can make image processing equipment 500 by network and other terminals or server communication, can be with Realize the connection communication between processor 503 and memory 501.
The overall operation of the usually control image processing equipment 500 of processor 503.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row His property includes, so that the process, method, article or the device that include a series of elements not only include those elements, and And further include other elements that are not explicitly listed, or further include for this process, method, article or device institute it is intrinsic Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including being somebody's turn to do There is also other identical elements in the process, method of element, article or device.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art The part contributed out can be embodied in the form of software products, which is stored in a storage medium In (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal device (can be mobile phone, computer, clothes Business device, air conditioner or the network equipment etc.) execute method described in each embodiment of the present invention.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (device) and computer program product Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates, Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one The step of function of being specified in a box or multiple boxes.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills Art field, is included within the scope of the present invention.

Claims (10)

1. a kind of image processing method, which is characterized in that the described method includes:
The first image is obtained, wherein includes identifier in the first image;
Obtain the second image;
Target object in second image is carried out to scratch figure, obtains target object;
According to the attribute of identifiers value, the target object and the first image are synthesized, the image after being synthesized.
2. the method according to claim 1, wherein the first image of the acquisition, wherein in the first image Including identifier, comprising:
Target area in the pretreatment image of acquisition is carried out to scratch figure, obtains white space;
The identifier is added to the white space according to the attribute of identifiers value, obtains the first image.
3. according to the method described in claim 2, it is characterized in that, it is described according to the attribute of identifiers value by the mark Symbol is added to the white space, obtains the first image, comprising:
The identifier is added to the white space;
Attribute of identifiers value after addition is adjusted, the first image is obtained.
4. according to the method described in claim 3, it is characterized in that, the attribute of identifiers includes the position of identifier, ruler Very little and/or direction, the attribute of identifiers value after described pair of addition are adjusted, and obtain the first image, comprising:
The position of identifier after addition, size and/or direction are configured, the first image is obtained.
5. according to the method described in claim 3, it is characterized in that, the attribute of identifiers includes the position of identifier, ruler Very little and/or direction, the attribute of identifiers value after described pair of addition are adjusted, and obtain the first image, further includes:
According to the image feature information of the pretreatment image, to the position of the identifier after addition, size, and/or direction into Row adjustment, obtains the first image.
6. method according to any one of claims 1 to 5, which is characterized in that the attribute of identifiers value includes at least The position of identifier, it is described according to the attribute of identifiers value, the target object and the first image are synthesized, obtained Image after synthesis, comprising:
Position of the target object in the first image is adjusted according to the position of the identifier, the figure after being synthesized Picture.
7. described according to the mark according to the method described in claim 6, it is characterized in that, the attribute value further includes size The attribute value for knowing symbol, the target object and the first image is synthesized, the image after being synthesized, further includes:
According to display size of the target object described in the size adjusting of the identifier in the first image, after obtaining synthesis Image.
8. a kind of image processing apparatus, which is characterized in that described image processing unit includes: the first acquisition module, the second acquisition Module scratches module and synthesis module, in which:
Described first obtains module, wherein includes identifier in the first image for obtaining the first image;
Described second obtains module, for obtaining the second image;
The stingy module scratches figure for carrying out to the target object in second image, obtains target object;
The synthesis module, for according to the attribute of identifiers value, the target object and the first image to be synthesized, Image after being synthesized.
9. a kind of computer storage medium, which is characterized in that be stored with the executable finger of computer in the computer storage medium It enables, after which is performed, can be realized the described in any item method and steps of claim 1 to 7.
10. a kind of computer equipment, which is characterized in that the computer equipment includes memory and processor, the memory On be stored with computer executable instructions, can be realized when the processor runs the computer executable instructions on the memory The described in any item method and steps of claim 1 to 7.
CN201810848300.3A 2018-07-27 2018-07-27 Image processing method and device, equipment and storage medium Active CN109035288B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810848300.3A CN109035288B (en) 2018-07-27 2018-07-27 Image processing method and device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810848300.3A CN109035288B (en) 2018-07-27 2018-07-27 Image processing method and device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN109035288A true CN109035288A (en) 2018-12-18
CN109035288B CN109035288B (en) 2024-04-16

Family

ID=64647437

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810848300.3A Active CN109035288B (en) 2018-07-27 2018-07-27 Image processing method and device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN109035288B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110012229A (en) * 2019-04-12 2019-07-12 维沃移动通信有限公司 A kind of image processing method and terminal
CN110111238A (en) * 2019-04-24 2019-08-09 薄涛 Image processing method, device, equipment and its storage medium
CN110136363A (en) * 2019-05-08 2019-08-16 深圳市朗形网络科技有限公司 Self-service entertainment video production and DIY text create souvenir equipment for customizing
CN111738930A (en) * 2020-05-12 2020-10-02 北京三快在线科技有限公司 Face image synthesis method and device, electronic equipment and storage medium
CN112291590A (en) * 2020-10-30 2021-01-29 北京字节跳动网络技术有限公司 Video processing method and device
CN112308809A (en) * 2019-08-20 2021-02-02 北京字节跳动网络技术有限公司 Image synthesis method and device, computer equipment and storage medium
CN112529765A (en) * 2019-09-02 2021-03-19 阿里巴巴集团控股有限公司 Image processing method, apparatus and storage medium
CN113791721A (en) * 2021-08-31 2021-12-14 北京达佳互联信息技术有限公司 Picture processing method and device, electronic equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070296736A1 (en) * 2006-06-26 2007-12-27 Agfa Inc. System and method for scaling overlay images
CN102763099A (en) * 2009-10-05 2012-10-31 法布塔利生产股份有限公司 Interactive electronic document
CN105701762A (en) * 2015-12-30 2016-06-22 联想(北京)有限公司 Picture processing method and electronic equipment
CN107330408A (en) * 2017-06-30 2017-11-07 北京金山安全软件有限公司 Video processing method and device, electronic equipment and storage medium
CN107959843A (en) * 2017-12-25 2018-04-24 广东欧珀移动通信有限公司 Image processing method and device, computer-readable recording medium and computer equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070296736A1 (en) * 2006-06-26 2007-12-27 Agfa Inc. System and method for scaling overlay images
CN102763099A (en) * 2009-10-05 2012-10-31 法布塔利生产股份有限公司 Interactive electronic document
CN105701762A (en) * 2015-12-30 2016-06-22 联想(北京)有限公司 Picture processing method and electronic equipment
CN107330408A (en) * 2017-06-30 2017-11-07 北京金山安全软件有限公司 Video processing method and device, electronic equipment and storage medium
CN107959843A (en) * 2017-12-25 2018-04-24 广东欧珀移动通信有限公司 Image processing method and device, computer-readable recording medium and computer equipment

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110012229A (en) * 2019-04-12 2019-07-12 维沃移动通信有限公司 A kind of image processing method and terminal
CN110012229B (en) * 2019-04-12 2021-01-08 维沃移动通信有限公司 Image processing method and terminal
CN110111238A (en) * 2019-04-24 2019-08-09 薄涛 Image processing method, device, equipment and its storage medium
CN110136363A (en) * 2019-05-08 2019-08-16 深圳市朗形网络科技有限公司 Self-service entertainment video production and DIY text create souvenir equipment for customizing
CN112308809A (en) * 2019-08-20 2021-02-02 北京字节跳动网络技术有限公司 Image synthesis method and device, computer equipment and storage medium
CN112308809B (en) * 2019-08-20 2024-06-14 北京字节跳动网络技术有限公司 Image synthesis method, device, computer equipment and storage medium
CN112529765A (en) * 2019-09-02 2021-03-19 阿里巴巴集团控股有限公司 Image processing method, apparatus and storage medium
CN111738930A (en) * 2020-05-12 2020-10-02 北京三快在线科技有限公司 Face image synthesis method and device, electronic equipment and storage medium
CN112291590A (en) * 2020-10-30 2021-01-29 北京字节跳动网络技术有限公司 Video processing method and device
CN113791721A (en) * 2021-08-31 2021-12-14 北京达佳互联信息技术有限公司 Picture processing method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN109035288B (en) 2024-04-16

Similar Documents

Publication Publication Date Title
CN109035288A (en) A kind of image processing method and device, equipment and storage medium
US10339689B2 (en) Intelligent camera
US10937216B2 (en) Intelligent camera
CN113287118A (en) System and method for face reproduction
US11688145B2 (en) Virtualizing content
CN110290425A (en) A kind of method for processing video frequency, device and storage medium
KR102624635B1 (en) 3D data generation in messaging systems
US8135724B2 (en) Digital media recasting
JP2022166078A (en) Composing and realizing viewer's interaction with digital media
CN110636365B (en) Video character adding method and device, electronic equipment and storage medium
CN110300316A (en) Method, apparatus, electronic equipment and the storage medium of pushed information are implanted into video
TW201924317A (en) Video processing method and device based on augmented reality, and electronic equipment
CN106507170A (en) A kind of method for processing video frequency and device
CN107862718A (en) 4D holographic video method for catching
KR20230162972A (en) Face compositing in augmented reality content for advertising
JP5878523B2 (en) Content processing apparatus and integrated circuit, method and program thereof
KR20230162971A (en) Face compositing in overlaid augmented reality content
CN110471528A (en) A kind of costume changing method, device, system and the storage medium of virtual spectators
WO2009076876A1 (en) Cartoon image generation, implantation method and system
CN112511815B (en) Image or video generation method and device
CN109064526A (en) A kind of method and device generating picture mosaic
CN112604279A (en) Special effect display method and device
Liu et al. Stereo-based bokeh effects for photography
Tous Pictonaut: movie cartoonization using 3D human pose estimation and GANs
Fei et al. Split: Single portrait lighting estimation via a tetrad of face intrinsics

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant