CN111223045B - Jigsaw method and device and terminal equipment - Google Patents
Jigsaw method and device and terminal equipment Download PDFInfo
- Publication number
- CN111223045B CN111223045B CN201911120026.9A CN201911120026A CN111223045B CN 111223045 B CN111223045 B CN 111223045B CN 201911120026 A CN201911120026 A CN 201911120026A CN 111223045 B CN111223045 B CN 111223045B
- Authority
- CN
- China
- Prior art keywords
- image
- user
- jigsaw
- images
- characteristic information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The application is applicable to the technical field of image processing and provides a jigsaw method, a jigsaw device and terminal equipment. According to the embodiment of the application, the image characteristic information of all the material images in at least one material image set is acquired, the target image is divided into the preset number of sub-image areas corresponding to the width of the preset color gamut range, the jigsaw characteristic information of the target image is generated, then the material images with the image characteristic information matched with the jigsaw characteristic information are screened out from the at least one material image set according to the jigsaw characteristic information and the image characteristic information, the selected material images are spliced to form the splicing effect graph of the target image, the material images with the image characteristic information matched with the jigsaw characteristic information can be screened out from the at least one material image set, the selected material images are used as the selected material images, and the splicing effect graph of the target image is spliced, so that the flexibility is good.
Description
Technical Field
The application belongs to the technical field of image processing, and particularly relates to a jigsaw method, a jigsaw device and terminal equipment.
Background
With the continuous popularization of terminal devices such as desktop computers, notebook computers, mobile phones, tablet computers, personal digital assistants, and the like, various types of application software such as word processing software, media playing software, aided design software, information management software, and graphic art designing software are installed in these terminal devices to realize various application functions, which has become a popular trend and brings great convenience to daily production and life of people. For example, montage image making (Andreamosaic) software can splice a plurality of images to form a montage effect image of a target image, which is of great interest.
However, in the existing image processing software, all images in a folder are used as material images, and then are spliced to form a splicing effect diagram in a simple image combination mode, so that the material images cannot be changed at will, and the flexibility is poor.
Content of the application
In view of this, the embodiments of the present application provide a jigsaw method, apparatus, and terminal device, so as to solve the problem that the existing image processing software cannot change the material image at will, and has poor flexibility.
A first aspect of an embodiment of the present application provides a jigsaw method, including:
acquiring image characteristic information of all material images in at least one material image set;
dividing a target image into a preset number of sub-image areas corresponding to the width of a preset color gamut range, and generating jigsaw characteristic information of the target image;
according to the jigsaw characteristic information and the image characteristic information, screening out material images with image characteristic information matched with the jigsaw characteristic information from the at least one material image set as selected material images;
and splicing the selected material images to form a splicing effect diagram of the target image.
A second aspect of embodiments of the present application provides a jigsaw device, comprising:
The acquisition module is used for acquiring the image characteristic information of all the material images in at least one material image set;
the generation module is used for dividing the target image into a preset number of sub-image areas corresponding to the width of a preset color gamut range, and generating jigsaw characteristic information of the target image;
the screening module is used for screening out material images with image characteristic information matched with the jigsaw characteristic information from the at least one material image set according to the jigsaw characteristic information and the image characteristic information, and taking the material images as selected material images;
and the jigsaw module is used for splicing the selected material images to form a splicing effect diagram of the target image.
A third aspect of the corresponding preset number of embodiments of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the puzzle method according to the first aspect of the embodiments of the present application when the computer program is executed.
A fourth aspect of the embodiments of the present application provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the puzzle method according to the first aspect of the embodiments of the present application.
According to the embodiment of the application, the image characteristic information of all the material images in at least one material image set is acquired, the target image is divided into the preset number of sub-image areas corresponding to the width of the preset color gamut range, the jigsaw characteristic information of the target image is generated, then the material images with the image characteristic information matched with the jigsaw characteristic information are screened out from the at least one material image set according to the jigsaw characteristic information and the image characteristic information, the selected material images are spliced to form the splicing effect graph of the target image, the material images with the image characteristic information matched with the jigsaw characteristic information can be screened out from the at least one material image set, the selected material images are used as the selected material images, and the splicing effect graph of the target image is spliced, so that the flexibility is good.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments or the description of the prior art, it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a first process of a jigsaw method according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of an image preview interface provided by an embodiment of the present application;
FIG. 3 is a schematic diagram of an atlas selection process provided by an embodiment of the present application;
fig. 4 is a schematic diagram of a second flow of the jigsaw method according to the embodiment of the present application;
fig. 5 is a schematic diagram of a third flow of a jigsaw method according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of a fourth flow chart of a jigsaw method according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram of a fifth flow chart of a jigsaw method according to an embodiment of the present disclosure;
fig. 8 is a schematic diagram of a jigsaw interface provided in an embodiment of the present application;
fig. 9 is a schematic diagram of a sixth flow chart of a jigsaw method according to an embodiment of the present disclosure;
fig. 10 is a schematic diagram of a seventh flow of a jigsaw method according to an embodiment of the present disclosure;
fig. 11 is a schematic diagram of an eighth flow chart of a jigsaw method according to an embodiment of the present disclosure;
fig. 12 is a schematic structural view of a jigsaw device according to an embodiment of the present disclosure;
fig. 13 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
The term "comprising" in the description of the present application and the claims and in the above figures, as well as any variants thereof, is intended to cover a non-exclusive inclusion. For example, a process, method, or system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed but may optionally include additional steps or elements not listed or inherent to such process, method, article, or apparatus. Furthermore, the terms "first," "second," and "third," etc. are used for distinguishing between different objects and not for describing a particular sequential order.
The embodiment of the application provides a jigsaw method, which is applied to a desktop computer, a notebook computer, a mobile phone, a tablet computer and a personal digital assistant comprising a display screen and a man-machine interaction device, or a terminal device such as a computer host, a (cloud) server and the like which can be in communication connection with the display screen and the man-machine interaction device, and can be executed by a processor of the terminal device when a corresponding computer program is executed. The jigsaw method is used for screening out material images with image characteristic information matched with the jigsaw characteristic information from the material image set, taking the material images as selected material images, and splicing the selected material images to form a splicing effect image of the target image.
In application, the processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), field-programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
As shown in fig. 1, the jigsaw method provided in the embodiment of the present application includes:
step S101, obtaining image characteristic information of all material images in at least one material image set.
In the application, at least one material image set for performing the jigsaw may be stored in advance in the storage medium of the terminal device, and each material image set may include only the same type of material image. In step S101, image feature information of all material images in at least one material image set selected by the user may be acquired according to the actual needs of the user. The image characteristic information may include a size and a color value of the material image.
In one embodiment, step S101 includes:
acquiring image characteristic information of all material images in at least one material image set selected by a user; wherein one of the material images includes the same type of material image.
In one embodiment, prior to step S101, comprising:
and when detecting the jigsaw starting operation of the user on the target image, entering a jigsaw interface of the target image.
In an application, the target image may be an image displayed in a desktop interface of the terminal device, an image preview interface of an album, an arbitrary folder interface, or a tile production software (e.g., montage production software) interface. The jigsaw starting operation can be any triggering operation on the target image or the interface for displaying the target image, and can be set according to actual needs. For example, a tile start button or icon may be provided in the interface for displaying the target image, and the tile start operation may be any touch operation such as clicking, pressing or touching the tile start button or icon. The tile actuation operation may also be a voice control operation or a gesture control operation.
As shown in fig. 2, an image preview interface 1 is exemplarily shown, and a target image 11 and a tile start button 12 are displayed in the image preview interface 1.
In one embodiment, step S101 includes:
entering an atlas selection interface when detecting the triggering operation of a user on an atlas selection area displayed on the jigsaw interface;
and when the selection operation of the user on at least one material image set displayed on the image set selection interface is detected, acquiring the image characteristic information of all material images in the at least one material image set selected by the user.
In the application, the album selection area may be an album selection button or icon, and the trigger operation may be any touch operation such as clicking, pressing, or touching the album selection button or icon. The selection operation may be any touch operation such as clicking, pressing, or touching a selection frame in any material diagram set in at least one material diagram set displayed on the diagram set selection interface. The triggering operation or the selecting operation may also be a voice control operation or a gesture control operation.
In the application, after the user finishes selecting the material atlas, thumbnail images of all the material atlas selected by the user are displayed in the jigsaw interface according to the selection sequence of the user, the user can continue to select other material atlas, or select the selected material atlas can be eliminated, and the thumbnail image of each material atlas is used for displaying part of material images in the material atlas.
In the application, the user can select the material atlas according to the self-defined priority order, the system (i.e. the computer program or the operating system executing the jigsaw method) defaults that the priority of the material atlas selected by the user is higher than that of the material atlas selected by the user, and in the subsequent jigsaw process, the system can screen the material images available for carrying out jigsaw from the material atlas with high priority according to the priority order.
As shown in fig. 3, a schematic diagram illustrating a set selection process is exemplary. The illustration in fig. 3 shows an album selection area 31 displayed in the puzzle interface 3; a plurality of material atlases (six selectable material atlases are exemplarily shown in fig. 3, and are respectively shown as a material atlas one, a material atlas two, a material atlas three, a material atlas four, a material atlas five and a material atlas six) are displayed in the atlas selection interface 32, and selection boxes (a distance box is exemplarily shown in fig. 3) are displayed in each material atlas; after the user completes the material album selecting operation, thumbnails of all the material albums selected by the user (two material albums are exemplarily shown in fig. 3, and three material images in each material album are illustrated in the thumbnail of the material album, which are respectively represented as a material image one, a material image two, and a material image three) and an album selecting area 31 are displayed in the tile interface 3 in the order of selection by the user.
As shown in fig. 4, in one embodiment, before step S101, it includes:
step S401, carrying out image recognition on the material images to be classified, and dividing the material images containing the same elements into the same type;
step S402, at least one material image set is generated, and the material images of the same type are added into the same material image set.
In the application, in order to facilitate the user to select the required material atlas, the material images need to be classified in advance, the material images containing different elements are divided into different types, the material images of the same type are added into the same material atlas, and the user can name each material atlas. The elements in the material image may be faces or objects, and the material image may also include a specific scene composed of a plurality of elements.
As shown in fig. 5, in one embodiment, step S401 includes:
step S501, when detecting the template material image selection operation of a user, acquiring the material image selected by the user as a template material image;
step S502, carrying out image recognition on the template material image to obtain elements in the template material image as target elements;
step S503, performing image recognition on the material images to be classified according to the target element, and dividing the material images containing the target element into the same type.
In the application, when a user needs to screen out a material image containing a certain specific element from material images to be classified and add the material image into the same material image set, a material image containing the specific element can be selected as a template material image, so that a system can automatically identify the specific element in the template material image, and then identify other material images containing the specific element from a plurality of material images to be classified.
In application, the material image selected by the user in step S501 may be one of the material images to be classified or already classified, and the template material image selecting operation may be any triggering operation on the material image selected by the user. In step S502 and step S503, according to the different element types in the template material image, a corresponding image recognition algorithm may be used to perform image recognition, for example, when the element is a face, a face recognition algorithm may be used; when the element is an article, an article identification algorithm can be adopted; when the material image includes a specific scene, a scene recognition algorithm may be employed.
In the application, the user can sequentially select the material images containing different elements or scenes as template material images, so that the system can sequentially generate different material atlas according to each template material image. The user can also select a plurality of material images containing different elements or scenes at a time as template material images, so that the system can quickly generate a plurality of different material image sets according to all the template material images at a time.
Step S102, dividing a target image into a preset number of sub-image areas corresponding to the width of a preset color gamut range, and generating jigsaw characteristic information of the target image.
In the application, the preset color gamut range can be preset by a user in a customized way, and if the preset color gamut range is not preset by the user, the default color gamut range of the system is adopted or the color gamut range which is set by the user last time is adopted. The width of the preset color gamut range is inversely related to the number of sub-image areas and the number of material images required to stitch to form the target image. The wider the preset color gamut range, the fewer the required color gradation, the fewer the number of sub-image areas and the number of required material images, and conversely, the fewer the number of sub-image areas and the number of required material images.
In the application, the user may self-define and adjust the preset color gamut according to the number of the material images, for example, if the preset color gamut set by the user is initially set so that the target image is divided into a first number (for example, 80×60) of sub-image areas, the first number of material images is required correspondingly; if the number of the material images in the material image set selected by the user is insufficient, the user can adjust the preset color gamut range to widen the preset color gamut range more widely, so that the target image is divided into a second number (for example, 40×30) of sub-image areas, and a second number Zhang Sucai of images is correspondingly required; if the number of material images in the user-selected material image set is still insufficient, the user may continue to adjust the preset color gamut range until the number of material images in the user-selected material image set is sufficient. Under the condition that the number of the material images in the material image set selected by the user is insufficient and the user does not adjust the preset color gamut range, the system can default to adopt a mode of multiplexing the material images to complete the jigsaw, and the same material images are arranged in at least two sub-image areas according to actual needs to be reused.
In an application, the tile interface displays a gamut range setting region, which may be a gamut range input box, a gamut range adjustment progress bar, a gamut range selection button, or an icon.
In application, the jigsaw characteristic information comprises the number of sub-image areas and the size and color value of each sub-image area, and when the jigsaw is performed, one sub-image area needs to be correspondingly provided with a material image matched with the size and color value of the sub-image area.
Step S103, according to the jigsaw characteristic information and the image characteristic information, material images with the image characteristic information matched with the jigsaw characteristic information are screened out from the at least one material image set to be used as selected material images.
In the application, the material atlas may be a material atlas selected by a user, or may be a material atlas selected by a default of the system. The size and color value of the material images in the material image set do not necessarily match the size and color value of the sub-image area, and therefore, it is necessary to screen the material images whose image feature information matches with the tile feature information as selected material images for tile matching.
And step S104, splicing the selected material images to form a splicing effect diagram of the target image.
In the application, the stitching effect map is an image formed by stitching a plurality of selected material images, which is identical to the picture of the target image.
As shown in fig. 6, in one embodiment, step S103 includes:
step S601, selecting, from the at least one material image set, a material image matched with the color value of each sub-image area as a selected material image, where one sub-image corresponds to one selected material image, according to the color value of each sub-image area;
step S104 includes:
step S602, clipping or amplifying the selected material image corresponding to each sub-image area according to the size of each sub-image area, so that the size of each sub-image area is the same as the size of the corresponding selected material image;
and step 603, placing the selected material image corresponding to each sub-image area at a corresponding position in the target image according to the position of each sub-image area, and splicing to form a splicing effect diagram of the target image.
In the application, in step S601, the color values are matched, which means that the color values are the same or belong to the same color gamut; in step S602, when the sub-image area and the corresponding selected material image are different in size, the corresponding selected material image needs to be cut or enlarged so that the sizes of the sub-image area and the corresponding selected material image are the same. The clipping may be performed centrally or centered on the position of the target element in the selected material image, so that the selected material image after clipping contains the target element and has the same size as the sub-image region.
As shown in fig. 7, in one embodiment, after step S103, it includes:
step S701, displaying the jigsaw information of the target image on a jigsaw interface; the mosaic information comprises at least one material image set forming the mosaic effect image and the duty ratio of selected material images in each material image set in the mosaic effect image.
In the application, after the mosaic effect graph of the target image is formed by stitching, the mosaic information of the target image is displayed on the mosaic interface, and the mosaic information can include at least one material graph set for forming the mosaic effect graph and the ratio of the selected material image in each material graph set in the mosaic effect graph, and can also include the thumbnail of the mosaic effect graph and the thumbnail of the target image.
As shown in fig. 8, an album selecting area 31, a thumbnail 33 of a mosaic effect image, material albums constituting the mosaic effect image (two material albums are exemplarily shown in fig. 3, and three selected material images in each material album are schematically shown in the thumbnail of the material album, which are respectively represented as a selected material image one, a selected material image two, and a selected material image three) and the ratio of the selected material images in each material album in the mosaic effect image (the ratio corresponding to the two material albums is illustratively shown in fig. 3 as a ratio of one and a ratio of two, respectively) are exemplarily shown in the mosaic interface 3.
As shown in fig. 9, in one embodiment, after step S701, it includes:
step S901, when the sum of the duty ratios of the selected material images in each material image set in the stitching effect image is smaller than 1, prompting the user to change the preset color gamut range or increase the material images;
step S902, when the color gamut range changing operation of a user is detected, dividing the target image into a preset number of sub-image areas corresponding to the width of the changed preset color gamut range, and regenerating jigsaw characteristic information of the target image;
step S903, when detecting the material image adding operation of the user, identifying whether the type of the added image belongs to the type of the material image in at least one material image set selected by the user, and prompting the identification result; wherein the material image adding operation includes an image photographing operation or an image importing operation;
step S904, when the type of the added image of the user belongs to the type of at least one material image in the material image set selected by the user, prompting the user whether to take the added image as a new added material image;
step S905, when detecting that the user uses the added image as a trigger operation of the newly added material image, updating the stitching effect map and the tile information according to the newly added material image.
In the application, in step S901, when the number of the material images in at least one material image set selected by the user is smaller than the number of sub-image areas in the target image, the stitching effect image is an unfinished semi-finished product, the sum of the percentages of the selected material images in the stitching effect image is smaller than 1, the system may display the thumbnail of the unfinished stitching effect image on the jigsaw interface, at this time, the system may prompt the user to change the preset color gamut range, and the user may adjust the preset color gamut range to a wider range or increase the material images. In step S902, after the user changes the preset color gamut range, the system re-executes steps S102 to S104 and S701 according to the changed preset color gamut range. In step S903, when the user starts the camera application to perform the image capturing operation to capture an image, or performs the image importing operation to import an image through wireless (e.g. WiFi or bluetooth) or wired (e.g. USB interface) communication, the system automatically identifies the type of the element in the image added by the user by using the image identification algorithm, and then prompts the user to identify the result. In step S905, when the user takes the added image as a newly added material image, the newly added material image is added to the corresponding material image set according to the type of the newly added material image, and steps S103, S104, and S701 are re-executed according to the newly added material image to update the mosaic effect map and the mosaic information.
As shown in fig. 10, in one embodiment, after step S701, it includes:
step S1001, displaying a zoomed-in photographing area on the zoomed-out interface when the sum of the duty ratios of the selected material images in the material images set in the zoomed-in effect images is less than 1;
step S1002, when detecting the triggering operation of the user to pull up the photographing area, starting a camera application and entering a camera preview interface;
step S1003, identifying whether the type of the preview image displayed in the camera preview interface belongs to the type of the material image in at least one material image set selected by the user, and prompting the identification result;
step S1004, when detecting a shooting operation of the user on the preview image, shooting the preview image as a newly added material image;
step S1005, updating the mosaic effect map and the mosaic information according to the newly added material image.
In the application, in step S1001, when the number of the material images in at least one material image set selected by the user is smaller than the number of sub-image areas in the target image, the stitching effect image is an unfinished semi-finished product, the sum of the percentages of the selected material images in the stitching effect image is smaller than 1, the system may display the thumbnail of the unfinished stitching effect image on the jigsaw interface, at this time, the system may display a pull-up photographing area on the jigsaw interface, and the pull-up photographing area may be a pull-up photographing button or icon, so that the user may increase the material images by photographing. In step S1002, after detecting a trigger operation of pulling up the photographing region by the user, the system starts the camera application and enters the camera preview interface. In step S1003, the system automatically recognizes the type of the element in the image added by the user using the image recognition algorithm, and then prompts the user for the recognition result. In step S1004, the user knows, according to the identification result, that the type of the preview image belongs to the type of the material image in the material image set selected by the user, and can perform a shooting operation on the preview image after the type of the preview image is used as the material image, and the preview image is shot as a newly added material image. In step S1005, the newly added material image is added to the corresponding material image set according to the type of the newly added material image, and steps S103, S104, and S701 are re-executed according to the newly added material image to update the mosaic effect map and the tile information.
In an application, the system may issue the prompt by at least one of a voice prompt, a light prompt, a sound prompt, and a pop-up prompt.
As shown in fig. 8, a pull-up photographing button 34 is exemplarily shown displayed in the jigsaw interface 3.
As shown in fig. 11, in one embodiment, after step S104, the method includes:
step 1101, displaying a preview image of the spliced effect image when a preview operation of a user is detected;
and step 1102, when detecting the local amplifying operation of the user on the sub-image area in the preview image, amplifying and displaying the selected material image positioned in the sub-image area through an image local analyzing technology.
In the application, in step S1101, a preview area may be displayed on the tile interface, the preview area may be a preview button or icon, and the preview operation may be a trigger operation of the preview area by the user. In step S1102, the system can enlarge and display a selected material image in a sub-image area that the user wants to enlarge by the image local analysis technique.
As shown in fig. 8, a preview button 35 is exemplarily shown displayed in the tile interface 3.
In one embodiment, after step S1102, it includes:
and editing the selected material image when the editing operation of the user on the selected material image in the sub-image area is detected.
In the application, the user can also perform editing operations such as deleting, replacing, adding an identification (e.g., graffiti) and the like on the selected material image located in the sub-image area.
In one embodiment, after step S104, it includes:
displaying a jigsaw starting area in the target image;
entering the jigsaw interface when detecting the triggering operation of a user on the jigsaw starting area;
distinguishing and displaying at least one material image set selected by a user and a material image set not selected by the user;
and distinguishing and displaying the selected material image and the unselected material image.
In the application, after the mosaic is spliced to form the mosaic effect graph of the target image, a mosaic starting area can be displayed in the target image, the mosaic starting area can be a mosaic starting button or an icon, the mosaic starting area can be used for identifying that the target image has a corresponding mosaic effect graph, and a user can enable a system to display a mosaic interface by triggering the mosaic starting area, so that the mosaic information can be viewed in the mosaic interface. The selected material image and the other material images (i.e. the non-selected material images) except the selected material image can be displayed in a distinguishing way, so that the user can know which material image set and the material image are used for splicing to form a splicing effect image, and when the user needs to start the splicing operation on other target images to obtain the corresponding splicing effect image, the material image set or the material image which is used for forming the splicing effect image can not be selected any more. The distinguishing display may be performed by setting distinguishing marks in different material atlas or different material images, and the distinguishing marks may include at least one of different graphics, different colors, and different characters.
According to the embodiment of the application, the image characteristic information of all the material images in at least one material image set is acquired, the target image is divided into the preset number of sub-image areas corresponding to the width of the preset color gamut range, the jigsaw characteristic information of the target image is generated, then the material images with the image characteristic information matched with the jigsaw characteristic information are screened out from the at least one material image set according to the jigsaw characteristic information and the image characteristic information, the selected material images are spliced to form the splicing effect graph of the target image, the material images with the image characteristic information matched with the jigsaw characteristic information can be screened out from the at least one material image set, the selected material images are used as the selected material images, and the splicing effect graph of the target image is spliced, so that the flexibility is good.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not limit the implementation process of the embodiment of the present application in any way.
As shown in fig. 12, an embodiment of the present application provides a jigsaw device 12 for performing the jigsaw method of the above-described embodiment. The puzzle 12 may be a terminal device or may be a virtual device (virtual appliance) in a processor of the terminal device. A puzzle device 12 comprising:
An acquiring module 121, configured to acquire image feature information of all material images in at least one material image set;
the generating module 122 is configured to divide a target image into a preset number of sub-image areas corresponding to a width of a preset color gamut range, and generate jigsaw feature information of the target image;
a screening module 123, configured to screen, according to the tile feature information and the image feature information, a material image whose image feature information matches the tile feature information from the at least one material image set as a selected material image;
and the jigsaw module 124 is used for splicing the selected material images to form a splicing effect diagram of the target image.
In one embodiment, the puzzle device further includes:
and the display module is used for entering a jigsaw interface of the target image when detecting the jigsaw starting operation of the user on the target image.
In one embodiment, the puzzle device further includes:
the classification module is used for carrying out image recognition on the material images to be classified and dividing the material images containing the same elements into the same type;
and the atlas generating module is used for generating at least one material atlas and adding the material images of the same type into the same material atlas.
In one embodiment, the display module is further configured to display tile information of the target image at a tile interface; the mosaic information comprises at least one material image set forming the mosaic effect image and the duty ratio of selected material images in each material image set in the mosaic effect image.
In one embodiment, the puzzle device further includes:
the prompting module is used for prompting a user to change the preset color gamut range or increase the material images when the sum of the duty ratio of the selected material images in each material image set in the splicing effect image is smaller than 1;
the generation module is further used for dividing the target image into a preset number of sub-image areas corresponding to the width of the preset color gamut range after modification when the color gamut range modification operation of the user is detected, and regenerating jigsaw characteristic information of the target image;
the prompting module is also used for identifying whether the type of the image added by the user belongs to the type of the material image in at least one material image set selected by the user when the material image adding operation of the user is detected, and prompting the identification result; wherein the material image adding operation includes an image photographing operation or an image importing operation;
The prompting module is further used for prompting the user whether to take the added image as a newly added material image when the type of the added image of the user belongs to the type of the material image in the at least one material image set selected by the user;
and the updating module is used for updating the splicing effect graph and the jigsaw information according to the newly added material image when detecting that the user takes the added image as the trigger operation of the newly added material image.
In one embodiment, the display module is further configured to:
when the sum of the duty ratios of the selected material images in each material image set in the splicing effect image is smaller than 1, a drawing photographing area is displayed on the jigsaw interface;
when the triggering operation of the user on the pulling-up photographing area is detected, starting a camera application and entering a camera preview interface;
the prompting module is also used for identifying whether the type of the preview image displayed in the camera preview interface belongs to the type of the material image in at least one material image set selected by a user, and prompting an identification result;
the jigsaw device further comprises:
the shooting module is used for shooting the preview image as a newly added material image when detecting shooting operation of a user on the preview image;
The updating module is also used for updating the splicing effect graph and the jigsaw information according to the newly added material image.
In one embodiment, the puzzle device further includes:
the preview module is used for displaying a preview image of the spliced effect image when the preview operation of the user is detected;
and the amplifying module is used for amplifying and displaying the selected material image positioned in the sub-image area through an image local analysis technology when detecting the local amplifying operation of the user on the sub-image area in the preview image.
In one embodiment, the puzzle device further includes:
and the editing module is used for editing the selected material image when detecting the editing operation of the user on the selected material image positioned in the sub-image area.
In one embodiment, the display module is further configured to:
displaying a jigsaw starting area in the target image;
entering the jigsaw interface when detecting the triggering operation of a user on the jigsaw starting area;
distinguishing and displaying the material atlas selected by the user and the material atlas not selected by the user;
and distinguishing and displaying the material images meeting the requirements of the jigsaw characteristic information and the material images not meeting the requirements of the jigsaw characteristic information.
In application, each module in the jigsaw device can also be a software program module in a processor of the terminal device, can be respectively realized by different processors, and can also be realized by different logic circuit structures in the processors.
According to the embodiment of the application, the image characteristic information of all the material images in at least one material image set is acquired, the target image is divided into the preset number of sub-image areas corresponding to the width of the preset color gamut range, the jigsaw characteristic information of the target image is generated, then the material images with the image characteristic information matched with the jigsaw characteristic information are screened out from the at least one material image set according to the jigsaw characteristic information and the image characteristic information, the selected material images are spliced to form the splicing effect graph of the target image, the material images with the image characteristic information matched with the jigsaw characteristic information can be screened out from the at least one material image set, the selected material images are used as the selected material images, and the splicing effect graph of the target image is spliced, so that the flexibility is good.
As shown in fig. 13, an embodiment of the present application further provides a terminal device 13, including: a processor 130, a memory 131 and a computer program 132, such as a jigsaw program, stored in the memory 131 and executable on the processor 130. The processor 130, when executing the computer program 132, implements the steps of the various tile method embodiments described above, such as steps S101 to S104 shown in fig. 1. Alternatively, the processor 130 may implement the functions of the modules in the above-described apparatus embodiments, such as the functions of the modules 121 to 124 shown in fig. 12, when executing the computer program 132.
For example, the computer program 132 may be partitioned into one or more modules that are stored in the memory 131 and executed by the processor 130 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program 132 in the terminal device 13. For example, the computer program 132 may be divided into an acquisition module, a generation module, a screening module, and a jigsaw module, each of which specifically functions as follows:
the acquisition module is used for acquiring the image characteristic information of all the material images in at least one material image set;
the generation module is used for dividing the target image into a preset number of sub-image areas corresponding to the width of a preset color gamut range, and generating jigsaw characteristic information of the target image;
the screening module is used for screening out material images with image characteristic information matched with the jigsaw characteristic information from the at least one material image set according to the jigsaw characteristic information and the image characteristic information, and taking the material images as selected material images;
and the jigsaw module is used for splicing the selected material images to form a splicing effect diagram of the target image.
The terminal device 13 may be a computing device such as a desktop computer, a notebook computer, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor 130, a memory 131. It will be appreciated by those skilled in the art that fig. 13 is merely an example of a terminal device 13 and does not constitute a limitation of the terminal device 13, and may include more or less components than illustrated, or may combine certain components, or different components, e.g., the terminal device may further include an input-output device, a network access device, a bus, etc.
The processor 130 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 131 may be an internal storage unit of the terminal device 13, for example, a hard disk or a memory of the terminal device 13. The memory 131 may also be an external storage device of the terminal device 13, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the terminal device 13. Further, the memory 131 may also include both an internal storage unit and an external storage device of the terminal device 13. The memory 131 is used for storing the computer program and other programs and data required by the terminal device. The memory 131 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the apparatus/terminal device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated modules, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each method embodiment described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (11)
1. A method of puzzle, comprising:
acquiring image characteristic information of all material images in at least one material image set; the image characteristic information comprises a size and a color value;
dividing a target image into a preset number of sub-image areas corresponding to the width of a preset color gamut range, and generating jigsaw characteristic information of the target image; the jigsaw characteristic information comprises the size, the position and the color value of each sub-image area;
according to the jigsaw characteristic information and the image characteristic information, screening out material images with image characteristic information matched with the jigsaw characteristic information from the at least one material image set as selected material images;
Splicing the selected material images to form a splicing effect diagram of the target image;
the width of the preset color gamut range can be customized according to the number of the material images; specific: and if the number of the material images in the material image set selected by the user is smaller than the preset number corresponding to the width of the preset color gamut range, adjusting the preset color gamut range until the number of the material images in the material image set selected by the user is larger than or equal to the preset number corresponding to the width of the preset color gamut range.
2. The method of claim 1, wherein prior to obtaining image feature information for all of the material images in the at least one material atlas, comprising:
carrying out image recognition on the material images to be classified, and dividing the material images containing the same elements into the same type;
at least one material atlas is generated, and the material images of the same type are added to the same material atlas.
3. The mosaic method according to claim 2, wherein performing image recognition on the image to be classified, dividing the material images containing the same elements into the same type, comprises:
when detecting the template material image selection operation of a user, acquiring the material image selected by the user as a template material image;
Performing image recognition on the template material image to obtain elements in the template material image as target elements;
and carrying out image recognition on the material images to be classified according to the target elements, and dividing the material images containing the target elements into the same type.
4. The method of claim 1, wherein obtaining image feature information for all material images in at least one material atlas comprises:
entering an atlas selection interface when detecting the triggering operation of a user on an atlas selection area displayed on the jigsaw interface;
and when the selection operation of the user on at least one material image set displayed on the image set selection interface is detected, acquiring the image characteristic information of all material images in the at least one material image set selected by the user.
5. The method of claim 1, wherein selecting a material image from the at least one material atlas based on the tile feature information and the image feature information, the material image having image feature information matching the tile feature information, as the selected material image, comprises:
screening out a material image matched with the color value of each sub-image area from the at least one material image set according to the color value of each sub-image area, wherein one sub-image area corresponds to one selected material image;
Splicing the selected material images to form a splicing effect diagram of the target image, wherein the splicing effect diagram comprises the following steps:
cutting or amplifying selected material images corresponding to each sub-image area according to the size of each sub-image area, so that the size of each sub-image area is the same as the size of the corresponding selected material image;
and according to the position of each sub-image area, placing the selected material image corresponding to each sub-image area at the corresponding position in the target image, and splicing to form a splicing effect diagram of the target image.
6. The method of any one of claims 1 to 5, wherein after stitching the selected material images to form the stitching effect map of the target image, the method comprises:
displaying the jigsaw information of the target image on a jigsaw interface; the mosaic information comprises at least one material image set forming the mosaic effect image and the duty ratio of selected material images in each material image set in the mosaic effect image.
7. The method of tiling as claimed in claim 6, wherein after tile information of said target image is displayed at a tile interface, comprising:
when the sum of the duty ratio of the selected material images in each material image set in the splicing effect image is smaller than 1, prompting a user to change the preset color gamut range or increase the material images;
When the color gamut range changing operation of the user is detected, dividing the target image into a preset number of sub-image areas corresponding to the width of the changed preset color gamut range, and regenerating jigsaw characteristic information of the target image;
when the material image adding operation of the user is detected, identifying whether the type of the image added by the user belongs to the type of the material image in at least one material image set selected by the user, and prompting an identification result; wherein the material image adding operation includes an image photographing operation or an image importing operation;
when the type of the added image of the user belongs to the type of at least one material image in the material image set selected by the user, prompting the user whether to take the added image as a newly added material image;
when the trigger operation of taking the added image as the newly added material image by the user is detected, the splicing effect diagram and the jigsaw information are updated according to the newly added material image.
8. The method of tiling as claimed in claim 6, wherein after tile information of said target image is displayed at a tile interface, comprising:
when the sum of the duty ratios of the selected material images in each material image set in the splicing effect image is smaller than 1, a drawing photographing area is displayed on the jigsaw interface;
When the triggering operation of the user on the pulling-up photographing area is detected, starting a camera application and entering a camera preview interface;
identifying whether the type of the preview image displayed in the camera preview interface belongs to the type of the material image in at least one material image set selected by a user, and prompting an identification result;
when detecting shooting operation of a user on the preview image, shooting the preview image as a newly added material image;
and updating the splicing effect diagram and the jigsaw information according to the newly added material image.
9. The method of any one of claims 1 to 5, wherein after stitching the selected material images to form the stitching effect map of the target image, the method comprises:
when the preview operation of the user is detected, displaying a preview image of the spliced effect image;
and when the local amplifying operation of the user on the sub-image area in the preview image is detected, amplifying and displaying the selected material image positioned in the sub-image area through an image local analyzing technology.
10. A puzzle device, comprising:
the acquisition module is used for acquiring the image characteristic information of all the material images in at least one material image set; the image characteristic information comprises a size and a color value;
The generation module is used for dividing the target image into a preset number of sub-image areas corresponding to the width of a preset color gamut range, and generating jigsaw characteristic information of the target image; the jigsaw characteristic information comprises the size, the position and the color value of each sub-image area;
the screening module is used for screening out material images with image characteristic information matched with the jigsaw characteristic information from the at least one material image set according to the jigsaw characteristic information and the image characteristic information, and taking the material images as selected material images;
the jigsaw module is used for splicing the selected material images to form a splicing effect diagram of the target image;
the width of the preset color gamut range can be customized according to the number of the material images; specific: and if the number of the material images in the material image set selected by the user is smaller than the preset number corresponding to the width of the preset color gamut range, adjusting the preset color gamut range until the number of the material images in the material image set selected by the user is larger than or equal to the preset number corresponding to the width of the preset color gamut range.
11. Terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the puzzle method of any one of claims 1 to 9 when the computer program is executed.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911120026.9A CN111223045B (en) | 2019-11-15 | 2019-11-15 | Jigsaw method and device and terminal equipment |
| PCT/CN2020/125800 WO2021093623A1 (en) | 2019-11-15 | 2020-11-02 | Image splicing method and apparatus, and terminal device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911120026.9A CN111223045B (en) | 2019-11-15 | 2019-11-15 | Jigsaw method and device and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111223045A CN111223045A (en) | 2020-06-02 |
| CN111223045B true CN111223045B (en) | 2023-06-30 |
Family
ID=70828991
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911120026.9A Active CN111223045B (en) | 2019-11-15 | 2019-11-15 | Jigsaw method and device and terminal equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111223045B (en) |
| WO (1) | WO2021093623A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111223045B (en) * | 2019-11-15 | 2023-06-30 | Oppo广东移动通信有限公司 | Jigsaw method and device and terminal equipment |
| CN111915744A (en) * | 2020-08-31 | 2020-11-10 | 深圳传音控股股份有限公司 | Interaction method, terminal and storage medium for augmented reality image |
| CN114797084A (en) * | 2021-01-22 | 2022-07-29 | 北京字跳网络技术有限公司 | Graphic display method, apparatus, device and medium |
| CN113599800A (en) * | 2021-08-17 | 2021-11-05 | 京东方科技集团股份有限公司 | Data interaction method, device and system and terminal operation equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105578145A (en) * | 2015-12-30 | 2016-05-11 | 天津德勤和创科技发展有限公司 | Method for real-time intelligent fusion of three-dimensional virtual scene and video monitoring |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009129160A (en) * | 2007-11-22 | 2009-06-11 | Nikon Corp | Image processor |
| US9251763B2 (en) * | 2012-05-25 | 2016-02-02 | Picmonkey, Llc | System and method for image collage editing |
| CN103634650B (en) * | 2013-11-06 | 2017-12-22 | 康佳集团股份有限公司 | A kind of image processing method and system based on intelligent television platform |
| CN104637040B (en) * | 2013-11-12 | 2017-11-21 | 上海携程商务有限公司 | The method and device of picture splicing is realized using Photoshop |
| CN103606159B (en) * | 2013-11-28 | 2017-06-13 | 广东威创视讯科技股份有限公司 | A kind of combination analysis method and system |
| CN106780325B (en) * | 2016-11-29 | 2018-11-30 | 维沃移动通信有限公司 | A kind of picture joining method and mobile terminal |
| CN107330858B (en) * | 2017-06-30 | 2020-12-04 | 北京乐蜜科技有限责任公司 | Picture processing method and device, electronic equipment and storage medium |
| CN109064526A (en) * | 2018-06-07 | 2018-12-21 | 珠海格力电器股份有限公司 | A kind of method and device generating picture mosaic |
| CN111223045B (en) * | 2019-11-15 | 2023-06-30 | Oppo广东移动通信有限公司 | Jigsaw method and device and terminal equipment |
-
2019
- 2019-11-15 CN CN201911120026.9A patent/CN111223045B/en active Active
-
2020
- 2020-11-02 WO PCT/CN2020/125800 patent/WO2021093623A1/en active Application Filing
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105578145A (en) * | 2015-12-30 | 2016-05-11 | 天津德勤和创科技发展有限公司 | Method for real-time intelligent fusion of three-dimensional virtual scene and video monitoring |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021093623A1 (en) | 2021-05-20 |
| CN111223045A (en) | 2020-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111223045B (en) | Jigsaw method and device and terminal equipment | |
| CN106687902B (en) | Image display, visualization and management based on content analysis | |
| US20150074576A1 (en) | Information processing methods and electronic devices | |
| US10140005B2 (en) | Causing elements to be displayed | |
| CN112954196A (en) | Shooting method, shooting device, electronic equipment and readable storage medium | |
| CN104932773A (en) | Information processing method and electronic equipment | |
| US10162507B2 (en) | Display control apparatus, display control system, a method of controlling display, and program | |
| CN105677731A (en) | Method, device, terminal and server for displaying picture previews | |
| CN108924440B (en) | Sticker display method, device, terminal and computer-readable storage medium | |
| EP4057633A1 (en) | Video processing method and apparatus, and terminal device | |
| US11770603B2 (en) | Image display method having visual effect of increasing size of target image, mobile terminal, and computer-readable storage medium | |
| CN111371999A (en) | Image management method, device, terminal and storage medium | |
| KR20120026836A (en) | Method and apparatus for displaying data object, and computer readable storage medium | |
| CN111885298B (en) | Image processing method and device | |
| CN110704377B (en) | Multimedia file processing method and device, processor and electronic device | |
| CN112162814A (en) | Image display method and device and electronic equipment | |
| CN112330728A (en) | Image processing method, image processing device, electronic equipment and readable storage medium | |
| CN115344159A (en) | File processing method and device, electronic equipment and readable storage medium | |
| CN110688035B (en) | Photo album processing method, photo album processing device and mobile terminal | |
| WO2013177863A1 (en) | Colour-based symbol selection method and device | |
| CN106778507A (en) | Text extraction method and device | |
| CN104657146A (en) | Method and device for shooting based on intelligent equipment and intelligent equipment | |
| CN111325656B (en) | Image processing method, image processing device and terminal equipment | |
| US20140153836A1 (en) | Electronic device and image processing method | |
| CN117274057A (en) | Image stitching method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |