CN108241645B

CN108241645B - Image processing method and device

Info

Publication number: CN108241645B
Application number: CN201611208759.4A
Authority: CN
Inventors: 宋翔宇; 郭德安; 曾晋川; 黄桂洲; 江启泉
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2016-12-23
Filing date: 2016-12-23
Publication date: 2020-03-17
Anticipated expiration: 2036-12-23
Also published as: CN108241645A

Abstract

The invention discloses an image processing method and device; the method comprises the following steps: identifying images meeting similar conditions from an image set, and dividing the images meeting similar conditions in the image set into corresponding groups; extracting feature points from each image of the group, marking the feature points matched with any two images in the group, and determining a feature area including the feature points marked for the corresponding image in each image; determining a feature region transformation relation between the feature region of the reference image and the feature regions of other images in the group and a difference region of the other images relative to the reference image by taking the image with the minimum feature region in the group as the reference image; storing the images in the group in such a manner that the reference image, the transformation relation, and the difference region are stored. By implementing the invention, the cost of storage space caused by storing the image can be saved, and the efficient searching and use of the image can be supported.

Description

Image processing method and device

Technical Field

The present invention relates to image processing technologies, and in particular, to an image processing method and apparatus.

Background

The current terminals such as smart phones and tablet computers generally have a shooting function, and a camera is configured for a user to shoot images for storage, and can be shared anytime and anywhere. However, with the increasing resolution of cameras, the need for storage space for stored images is increasing.

In order to reduce the storage space of an image shot by a user, save the cost of the storage space of a terminal and avoid influencing the application that the user can only install a limited number in the terminal, the following schemes are available at present:

1) the user manually arranges and deletes unnecessary images stored in the terminal, which requires manual operation, the user is difficult to screen out the images to be deleted in a short time, the efficiency is low, and the user does not want to delete the images in most cases, and even if a large number of similar images are shot by using a continuous shooting mode, the user still wants to remain in the terminal.

2) Although the storage space overhead can be saved by compressing each image, compressing the image inevitably affects the quality of the image, and it is difficult to meet the requirement of the user on a high-quality image in some occasions.

3) The local image of the terminal is backed up through the cloud storage service, and the local partial image of the terminal is deleted, so that although the storage space of the terminal can be saved, a user needs to log in the cloud for downloading when needing to use the image, and therefore, the efficiency of obtaining the image by the user is influenced by uncertain factors such as networking conditions, and the situation that the user can efficiently search and use the shot image is difficult to guarantee.

Therefore, for the condition that a user generally stores a large number of images at present, how to realize the purpose of saving the cost of storage space caused by storing the images and supporting the user to quickly search and use the shot images is not an effective solution in the related technology.

Disclosure of Invention

Embodiments of the present invention provide an image processing method and apparatus, which can save the overhead of a storage space caused by storing an image, and support efficient searching and use of the image.

The technical scheme of the embodiment of the invention is realized as follows:

in a first aspect, an embodiment of the present invention provides a graphics processing method, including:

identifying images meeting similar conditions from an image set, and dividing the images meeting similar conditions in the image set into corresponding groups;

extracting feature points from each image in the group, marking the feature points matched with any two images in the group, and determining a feature area formed by the marked feature points in each image;

determining a feature region transformation relation between the feature region of the reference image and the feature regions of other images in the group and a difference region of the other images relative to the reference image by taking the image with the minimum feature region in the group as the reference image;

storing the images in the group in such a manner that the reference image, the transformation relation, and the difference region are stored.

In a second aspect, an embodiment of the present invention provides an image processing apparatus, including:

the grouping unit is used for identifying images meeting similar conditions from an image set and dividing the images meeting similar conditions in the image set into corresponding groups;

the characteristic unit is used for extracting characteristic points from each image in the group, marking the characteristic points matched with any two images in the group and determining a characteristic area formed by the marked characteristic points in each image;

a transformation unit, configured to determine, using an image with a smallest feature region in the group as a reference image, a feature region transformation relationship between the feature region of the reference image and another image in the group, and a difference region between the another image and the reference image;

a storage unit configured to store the images in the group in a manner that the reference image, the transformation relation, and the difference region are stored.

In a third aspect, an embodiment of the present invention provides an image processing apparatus, including a memory and a processor; the memory stores executable instructions for executing the image processing method provided by the embodiment of the invention.

In a fourth aspect, an embodiment of the present invention provides a storage medium storing executable instructions for executing an image processing method provided by an embodiment of the present invention.

The embodiment of the invention has the following beneficial effects:

by identifying images meeting similar conditions, dividing similar images in an image set into the same group, such as the group of continuous shooting photos, taking a characteristic region of a reference image as a reference, storing other images in a mode of converting the reference characteristic region into a conversion relation of the characteristic region of other images and a difference region, and establishing the overall association of a series of images in the group; for similar images in the group, original image data of each image does not need to be stored respectively, and storage in a difference area and transformation relation mode is replaced, so that the expense of storage space can be obviously saved;

the transformation relation and the difference region support fast and accurate reduction of other images in the group from the characteristic region of the reference image, and the mode of storing the images does not involve compression processing of original image data of other images, so that the quality of other images in the group is ensured, and the problem of image quality loss caused by compression of the images in the related technology is solved.

Drawings

FIG. 1-1 is an alternative processing diagram of an image processing method provided by an embodiment of the invention;

fig. 1-2 are an alternative schematic diagram of grouping image sets according to an embodiment of the present invention;

FIGS. 1-3 are schematic diagrams of an alternative method for matching feature points of an image m and an image n according to an embodiment of the present invention;

FIGS. 1-4 are schematic diagrams of different transformation relationships between feature regions of a reference image and feature regions of other images according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an alternative software and hardware structure of an image processing apparatus according to an embodiment of the present invention;

fig. 3 is a schematic diagram of an alternative functional structure of an image processing apparatus according to an embodiment of the present invention;

FIG. 4-1 is a schematic diagram of an alternative process for performing similar photo recognition according to an embodiment of the present invention;

fig. 4-2 is an alternative flow diagram of feature point extraction, reference picture determination, and merged storage of grouped pictures according to the embodiment of the present invention;

4-3 are alternative flow diagrams for rendering a presentation and a boundary-compensated photograph provided by embodiments of the present invention;

FIG. 5 is an alternative display diagram of a similar grouping of photographs provided by an embodiment of the present invention;

fig. 6 is an alternative schematic diagram of grouping and merging storage of photos according to an embodiment of the present invention.

Detailed Description

The present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the examples provided herein are merely illustrative of the present invention and are not intended to limit the present invention. In addition, the following embodiments are provided as some embodiments for implementing the invention, not all embodiments for implementing the invention, and those skilled in the art will not make creative efforts to recombine technical solutions of the following embodiments and other embodiments based on implementing the invention all belong to the protection scope of the invention.

It should be noted that, in the embodiments of the present invention, the terms "comprises", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion, so that a method or apparatus including a series of elements includes not only the explicitly recited elements but also other elements not explicitly listed or inherent to the method or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other related elements in a method or apparatus that comprises the element (e.g., steps in a method or elements in an apparatus).

For example, the image processing method provided by the embodiment of the present invention includes a series of steps, but the image processing method provided by the embodiment of the present invention is not limited to the described steps, and similarly, the file transfer device-apparatus provided by the embodiment of the present invention includes a series of units, but the file transfer device provided by the embodiment of the present invention is not limited to include the explicitly described units, and may also include units that are required to be set for acquiring related information or performing processing based on the information.

Before further detailed description of the present invention, terms and expressions referred to in the embodiments of the present invention are described, and the terms and expressions referred to in the embodiments of the present invention are applicable to the following explanations.

1) An image, comprising: the picture is created by a graphic application program under the operation of a designer and is composed of simple points, straight lines, curves, circles, square frames and other basic elements, and is also called a vector graph; images in digital form of actual scenes acquired by a sensing part in a digital camera, scanner, or the like, also referred to as electronic photographs (hereinafter simply referred to as photographs), may also include various filter effects added by a graphic application.

2) And the color histogram features are extracted based on the color histogram of the image, and represent feature vectors of the distribution situation of the pixels of the image in different sections of the color space.

3) Perceptual Hash Algorithm (PHA), a fingerprint (fingerprint) string is generated for each image, i.e., Perceptual Hash features pHash, and then the degree of closeness between fingerprints of different photos is compared to represent the degree of similarity between images.

4) Hamming distance (Hamming distance) used for quantizing and representing if the data bits which are different are not more than 5, the two photos are very similar; if it is greater than 10, it is indicated that these are two different photographs.

5) The characteristic points are more obvious points with identification in the image, such as contour points, bright points in darker areas, dark points in lighter areas and the like.

6) An ORB (ORiented Brief) feature extraction algorithm, wherein the ORB adopts a FAST (Features From accepted Segment Test) algorithm to detect FAST feature points, the FAST feature points are detected From the image by using the FAST feature point detection method, a plurality of feature points with the maximum Harris corner response value are selected From the FAST feature points as ORB feature points by using a Harris corner measurement method, and the ORB feature points are described by using a BRIEF feature descriptor.

7) Image Data (Image Data) is a set of values representing, as numerical values, the values of pixels (pixels) in a color space, such as the values of pixels in a Red-Green-Blue (RGB) color space, in an Image.

At present, the scheme of saving the storage space of the stored image generally adopts an image compression technology, and as for the image compression technology, the individual image is compressed, and the condition of storing a large number of similar images is not considered.

For example, when a user takes an image, multiple photos are often taken at one time in order to pick the best image, for example, multiple photos are taken by using a continuous shooting function, so that a large number of similar or identical regions exist in the taken photos, and for example, a large number of identical elements (such as circles, straight lines, squares, and the like) exist in vector pictures downloaded from a network by the user.

In view of the foregoing problems, an embodiment of the present invention provides an image processing method, which provides a scheme for fusing and storing a plurality of images, and referring to fig. 1-1, fig. 1-1 is an optional schematic flow diagram of the image processing method provided in the embodiment of the present invention, and includes the following steps:

step 101, identifying images meeting similar conditions from the image set, and dividing the images meeting similar conditions in the image set into corresponding groups.

Referring to fig. 1-2, fig. 1-2 are alternative schematic diagrams of grouping image sets according to an embodiment of the present invention, and as an example of identifying images meeting similar conditions, a scheme is provided that supports a user to manually group image sets, and the images are divided into corresponding groups according to user specifications.

In the case of similar images (e.g., continuously shot photographs) in an image collection, similar or identical regions (also referred to as feature regions) exist in the images, and various forms of features may be extracted from the images and compared to determine the degree of similarity between the images.

Of course, images meeting similar conditions can also be identified from the image set and divided into corresponding groups, which will be described below.

In one embodiment, each image in the image set is converted into a color space, such as RGB, a color space, or alternatively, each image may be converted into a Hue, color Saturation and brightness (HIS, Hue, Saturation and Intensity) space, a Hue, color Saturation and Value (HSV, Hue, Saturation, Value) space, a Luv space, a Lab space, and the like, histogram features of the images in the corresponding spaces are calculated according to the distribution of the images in the color space, and images with the same or similar histogram features (such as images with cosine similarity exceeding a cosine similarity threshold) are divided into the same group by comparing the histogram features of the images.

For example, when a user performs a continuous shooting operation at two different positions by a mobile phone camera, the photos continuously shot at each position are divided into two groups by the above-described process of recognizing similar images and grouping.

In another embodiment, the perceptual hash feature pHash of each image in the image set is extracted through PHA, the Hamming distance of pHash of any two images is compared (the similarity degree of the two images and the Hamming distance of pHash of the two images), and the images of which pHash exceeds the Hamming distance threshold value between the images are divided into the same group.

In practical applications, images with SIFT Feature similarity exceeding the SIFT similarity threshold may also be classified into the same group by extracting SIFT features from the images through Scale Invariant Feature Transform (SIFT).

In addition, the above-mentioned conditions for dividing the image set into groups may be used in combination, for example, images that satisfy the following similarity conditions are identified from the image set and divided into the same group: the pHash of the image exceeds a Hamming distance threshold image; the cosine similarity of the histogram feature of the image exceeds a cosine similarity threshold.

In another embodiment, given that similar images have the same or similar attributes in time and location, the comparison may be based on the attributes of time and/or location of the images in the group and filtering may be performed, including 1) filtering images from the group whose shot location distance exceeds the shot location distance threshold; 2) images whose photographing time interval exceeds the photographing time interval are filtered from the packets. It will be appreciated that the above filtration means may be used alternatively or in combination.

For example, after preliminarily grouping the photos stored in the electronic album, the terminal may filter the photos in the group based on at least one of the shooting time and the shooting position of the photos, and if there is a photo in the group with a larger difference between the shooting time of the photo and the shooting time of other photos, the terminal may present the result of the grouping when the user views the electronic album, and prompt the user to confirm whether the photo with the larger difference between the shooting times continues to be retained in the group.

And 102, extracting feature points from each grouped image, marking the feature points matched with any two images in the group, and determining a feature area comprising the feature points marked aiming at the corresponding image in each image.

In one embodiment, feature points that match any two images constitute a feature point pair, for example, denoted as < image M feature point, image n feature point >, where 1 ≦ M, 1 ≦ j ≦ M, i ≠ j, M being the number of photos in the grouping; traversing the feature points of the image m and the feature points of the image n, comparing the distance (Euclidean example) between the feature points of the image m and the feature points of the image n in the feature point pair with a feature point distance threshold, if the distance is greater than the feature point distance threshold, indicating that corresponding actual objects of the feature points in the feature point pair in the image m and the image n are different, and if the distance is less than the feature point distance threshold, indicating that corresponding actual objects of the feature points in the feature point pair in the image m and the image n are the same, such as paint points of an object, inflection points of a wall surface and the like, filtering the feature points greater than the feature point distance threshold, and marking the remaining feature points after filtering for each grouped image.

For the feature points marked in each image in the group, the minimum region including the feature points in the image is determined as the feature region of the corresponding image, for example, the feature region may be a minimum convex polygon including the feature points in the image, hereinafter also referred to as a convex hull region.

Since the feature points labeled for each image are matched with the feature points of the other images in the group, the feature points of the same object must exist in the feature region of each image, for example, when the group is a facial photo taken continuously for a task, the feature region of each photo must include the imaging region of the face of the person in the corresponding photo.

In another embodiment, in order to reduce errors and further eliminate false feature point matching, for a feature point pair < image m feature point, image n feature point > formed by feature points remaining after filtering of image m and image n, distances of the feature points of image m and the feature points of image n are calculated in a traversing manner, the determined distances are less than a distance from the feature point to a threshold, and the distances less than the distance from the feature point to the threshold are averaged to obtain an average minimum distance, if the average minimum distance is less than the average minimum distance threshold; referring to fig. 1-3, fig. 1-3 are optional schematic diagrams of matching feature points of an image m and an image n according to an embodiment of the present invention, and if yes, as shown in fig. 1-3 matching (1), it indicates that the feature points of the image m and the feature points of the image n form a good match as a whole, mark the feature points remaining after filtering of any two images, and if no, as shown in fig. 1-3 matching (2), it indicates that there is an error that is difficult to ignore in the matching between the feature points of the image m and the feature points of the image n, and discard the feature points of the image m and the feature points of the image n.

And 103, taking the image with the minimum characteristic region in the group as a reference image, and determining the characteristic region transformation relation between the characteristic region of the reference image and the characteristic regions of other images in the group and the difference regions of other images relative to the reference image.

Since the feature points of the same object inevitably exist in the feature regions of the respective images, the minimum feature region obtained by the comparison is common to the respective images in the group for the feature regions of the respective images in the group, for example, in the case of a continuously shot photograph of the face of a person, the minimum feature region is an imaged region of the face of the person in the photograph.

The feature region of other images and the minimum feature region may be consistent, for example, continuous pictures are taken under the condition that the shooting angle of the photographer and the pose of the photographer are not changed, and the images of the faces of people in the pictures are consistent; referring to fig. 1-4, fig. 1-4 are schematic diagrams illustrating different transformation relationships between a feature region of a reference image and feature regions of other images, where the feature region of the reference image may need to be transformed to be consistent with the feature regions of the other images, such as at least one transformation selected from rotation, translation, stretching and scaling, a matching region in the other images is cropped with the feature region of the reference image, transformation is performed between the feature region of the reference image and the cropped region, and an angle of rotation, a uniqueness of translation, a stretching amplitude and a scaling (hereinafter also referred to as transformation data) are recorded in the transformation relationship, and of course, when the feature region of the reference image is not transformed and is consistent with the feature regions of the other images, a "special" transformation relationship may be recorded, i.e., the feature region of the reference image is consistent with the feature regions of the other images without transformation, the special transformation relationship is indicated by an appointed field such as "1", or is not recorded by default to indicate that the feature region of the reference image is consistent with the feature regions of the other images.

For the other images except the reference image in the group, the characteristic region can be obtained by transforming the characteristic region of the reference image, and a part of the region is the region not included in the reference image, namely the region left after the matched region in the other images is cut out by the characteristic region of the reference image,

step 104, storing the images in the group in a manner of storing the reference image, the transformation relation and the difference area.

For grouped images, the following way is used for storage:

1) reference images, i.e. raw image data of reference images

2) The characteristic region of the reference image is the region where the characteristic region of the reference image is distributed in the reference image;

3) the conversion relation of the characteristic areas of other images records the conversion required for converting the characteristic area of the reference image into the characteristic area of other images;

4) the difference region of the other pattern is the original image data of the region that the reference image does not have.

As can be seen from the above storage manner for the images in the group, only the original image data of the difference region of the reference image and other images needs to be stored, only the transformation relation needs to be stored for the feature regions of other images,

in another embodiment, the reference image and other images in the presentation group are illustrated.

Step 105, pre-rendering the canvas, and loading the reference image on the canvas.

If the image to be presented is the reference image, step 105 is executed and ended, and if the image to be presented is another image, the

subsequent steps

106 and 107 are executed.

And 106, reading the transformation relation and the difference area of the other images to be presented, and carrying out affine transformation on the characteristic area of the reference image based on the transformation relation to form the characteristic area of the other images to be presented.

And step 107, loading the read difference area on the canvas based on the position of the feature area after affine transformation.

In order to avoid the problem that the boundaries of the feature regions and the difference regions of other images to be presented are too sharp, so that the other images present a very obvious fit feeling, the boundaries can be subjected to fuzzy processing, or the feature values of the boundaries can be calculated, and then the smooth transition is performed on the boundaries by adopting a smooth transformation function, so that the effect transition of the boundaries is more natural.

The embodiment of the present invention further provides an image processing apparatus for implementing the image processing method, where the apparatus may be based on a terminal on a user side, such as a smart phone, a tablet computer, and a notebook computer, and may also be a server deployed in a network for providing a file transfer background service, and of course, the apparatus may also be distributed and deployed in the terminal on the user side and the server on the network side.

On the hardware level, the hardware resources of the implementation device include computing resources such as a processor and a memory, and communication resources such as a network interface; at the software level, the device may be embodied as executable instructions (including executable instructions such as programs, modules) stored in a storage medium.

As above, referring to an optional software and hardware structure diagram of the image processing apparatus 10 provided in the embodiment of the present invention shown in fig. 2, the image processing apparatus 10 includes a hardware layer, a driver layer, an operating system layer, and an application layer. However, it should be understood by those skilled in the art that the structure of the image processing apparatus 10 shown in fig. 2 is merely an example, and does not constitute a limitation on the structure of the image processing apparatus 10. For example, the image processing apparatus 10 may be provided with more components than those shown in fig. 2 according to the implementation requirement, or may omit some components according to the implementation requirement.

The hardware layers of the image processing apparatus 10 include a processor 11, an input/output interface 13, a storage medium 14, and a network interface 12, and the components can communicate via a system bus connection.

The processor 11 may be implemented by a Central Processing Unit (CPU), a Microprocessor (MCU), an Application Specific Integrated Circuit (ASIC), or a Field-Programmable Gate Array (FPGA).

The input/output interface 13 may be implemented using input/output devices such as a display screen, a touch screen, a speaker, etc.

The storage medium 14 includes nonvolatile storage media such as a flash memory, a hard disk, and an optical disk, and volatile storage media such as a Double Data Rate (DDR) dynamic cache, where the nonvolatile storage media store a file system structure of an operating system (including an operating system library and an operating system program), and also store Data of each application in an application layer.

For example, the storage medium 14 may be provided in a centralized manner with other components of the image processing apparatus 10, or may be provided in a distributed manner with respect to other components in the image processing apparatus 10. The network interface 12 provides the processor 11 with external data such as an access capability of the storage medium 14 set in a remote place, and the network interface 12 may perform Near Field Communication (NFC) based on Near field communication (Near field communication), Bluetooth (Bluetooth) based on Bluetooth (Bluetooth), ZigBee (ZigBee) based on NFC, and may also implement communication in communication systems such as CDMA and WCDMA, and in evolution systems thereof.

The driver layer includes middleware 15 for the operating system 16 to recognize and communicate with the components of the hardware layer, such as a set of drivers for the components of the hardware layer.

The operating system 16 is used for providing a graphical interface facing a user, and exemplarily comprises a plug-in icon, a desktop background and an application icon, and the operating system 16 supports the user to control the device via the graphical interface, and the embodiment of the present invention does not limit the software environment of the device, such as the type and the version of the operating system, and may be, for example, a Linux operating system, a UNIX operating system or other operating systems.

The application layer includes a layer provided with an application, such as an electronic album 17, packaged for executing a graphic processing method for fuse-storing images stored in the storage medium 14 in which partially identical regions exist.

For the case where the image processing apparatus 10 is deployed in a terminal, the application layer may include, for example, an instant messaging client 19, which provides a function of transmitting files to and sharing files with friends of the user, and for example, a network disk client 18, which provides a server for uploading files local to the image processing apparatus 10 (e.g., personal files such as user images and videos, and operating system setting files and installation package files) to the cloud, and restoring the files from the cloud to the image processing apparatus 10 when necessary. For the case where the image processing apparatus 10 is deployed in a server, the application layer may include business logic (e.g., user data management, message management) for supporting instant messaging services, for example.

Referring to fig. 3, fig. 3 is a schematic diagram of an optional functional structure of the image processing apparatus 10 according to an embodiment of the present invention, which includes:

a grouping unit 21, configured to identify images that meet similar conditions from the image set, and divide the images that meet similar conditions in the image set into corresponding groups;

a feature unit 22, configured to extract feature points from each of the grouped images, mark feature points that match any two of the grouped images, and determine a feature region in each of the images that includes the feature points marked for the corresponding image;

a transformation unit 23, configured to determine a transformation relationship between the feature region of the reference image and the feature regions of other images in the group and a difference region between the other images and the reference image, by using the image with the smallest feature region in the group as the reference image;

a storage unit 24 for storing the images in the group in such a manner that the reference image, the transformation relation, and the difference area are stored.

In one embodiment, the grouping unit 21 is further configured to identify an image from the image set, which satisfies at least one of the following similarity conditions:

the cosine similarity of the histogram features of the image exceeds a cosine similarity threshold;

the hamming distance of the perceptual hash features of the image is less than a hamming distance threshold.

In one embodiment, the grouping unit 21 is further configured to, after dividing the images in the image set that meet the similar condition into corresponding groups, perform at least one of the following filtering operations:

filtering images of which the shooting position distance exceeds a shooting position distance threshold value from the grouping;

images having a photographing time interval exceeding a photographing time interval threshold are filtered from the packets.

In one embodiment, the feature unit 22 is further configured to form a feature point pair from feature points matching any two images, compare a distance between the feature points in the feature point pair with a feature point distance threshold, filter feature points larger than the feature point distance, and mark the remaining feature points after filtering.

In an embodiment, the feature unit 22 is further configured to calculate a distance smaller than a feature point distance threshold in the distance between the feature points remaining after filtering of any two images, and average the calculated distances to obtain an average minimum distance;

judging whether the average minimum distance threshold is smaller than the average minimum distance;

if so, marking the feature points left after filtering of any two images,

and if not, discarding the feature points left after filtering of any two images.

In one embodiment, the transforming unit 23 is further configured to crop a matching region in the other image with the feature region of the reference image, transform between the feature region of the reference image and the cropped region, and determine a transformed rotation angle, a transformed shift displacement, and a transformed scale;

and marking the residual area after the matched area in the other image is cut as the difference area of the other image relative to the reference image.

In one embodiment, further comprising:

and a first rendering unit 25, configured to pre-render a canvas on which the reference image is loaded when the image to be rendered is the reference image.

In one embodiment, further comprising:

a second rendering unit 26, configured to pre-render a canvas and load a reference image on the canvas when the image to be rendered is another image; reading the transformation relation and the difference area of other images to be presented, and transforming the characteristic area of the reference image based on the transformation relation to form the characteristic area of the other images to be presented; and loading the read difference region on the canvas based on the position of the transformed feature region.

In an embodiment, the second presenting unit 26 is further configured to perform a blurring process or a smooth transition process on the boundary between the feature region and the difference region of the other image to be presented.

The following description will be given with reference to an example of fused storage of similar photos in photos (i.e., electronic photos) in an electronic album, which mainly relates to several links of similar photo recognition, feature point extraction, reference photo determination, fused storage of grouped photos, rendering presentation, and edge compensation, which are described separately.

First, similar photo recognition

Similar photographs are grouped together into groups of similar photographs using techniques similar to pHash or color histograms.

In addition, generally, there is a certain similarity between the associated photos in the shooting position and shooting time, and the shooting time of the photos can be represented by the change of light or the change of background on the photos, so these factors are considered together, for example, in the grouping of similar photos, any 2 photos satisfy the following similar conditions:

1) the Hamming distance of pHash of the photo is less than 8;

2) the cosine similarity of the histogram feature of the photo is not less than 0.96;

3) the distance difference of the shooting positions of the pictures is not more than 10 meters;

4) the photographs were taken within 2 hours of each other.

Referring to fig. 4-1, fig. 4-1 is a schematic view of an alternative process for performing similar photo recognition according to an embodiment of the present invention, which includes the following steps:

step 201, reading N photos of the whole photo album.

The preprocessing of step 202 to step 210 is performed on the read photos, and similar photos are divided into the same group.

In step 202, the color histogram feature of each photo is calculated.

The color histogram description is the probability that different colors appear at different positions throughout the picture, and does not focus on the spatial location of each color, i.e., the color histogram does not describe objects or objects in the image, and is particularly suitable for describing pictures that are difficult to segment automatically.

The color histogram may be based on the distribution of different color spaces of the photograph. The commonly used color space is the RGB color space, and in addition, the color histogram can also be calculated based on the distribution of the photograph in the HIS space, HSV space, Luv space, and Lab space.

When calculating the color histogram feature of a photo, the color space distributed by the photo needs to be divided into a plurality of small (e.g. 8) intervals, and each small interval becomes a container (bin) of the histogram. This process is called color quantization (colorquantization). Then, the number of the pixels mapped in each section in the pixels of the photo is calculated, and the distribution condition of the pixels of the photo in different sections is obtained and expressed by a feature vector.

Taking the calculation of the color histogram features of the photo in the HSV color space as an example, H represents hue, S represents color, and V represents saturation, if the HSV color space is divided into 16 parts in the H dimension, 8 parts in the S dimension, and 4 parts in the V dimension, a feature vector representing the distribution of the pixels of the photo in the 512 intervals is finally obtained, and the dimension of the feature vector is 512(16 × 8 × 4). The dimensionality of the feature vector can be adjusted according to the precision of the feature vector of the photo, and in practical application, the real-time requirement for fusion storage of the photo can be referred to (if the real-time requirement is not high and enough computing resources are available, the feature vector with relatively large dimensionality can be used).

In step 203, the pHash of each photo is calculated.

The picture is reduced to a predetermined size, such as a size of 8x8, for a total of 64 pixels. The details of the photo are removed, only basic information such as structure, brightness and the like is kept, and photo differences caused by different sizes and proportions are abandoned.

The colors of the picture are simplified, and the reduced picture is converted into a preset number of levels, such as 64-level gray scale. That is, all pixels have 64 colors in total.

The average of the gray levels of the 64 pixels of each photograph is calculated and compared to the average of the gray levels for each pixel of the photograph. If the average value is larger than or equal to the average value, the comparison result is marked as 1; if the comparison result is less than the average value, the comparison result is recorded as 0, and the comparison results of all the pixels of the picture are combined together to form 64-bit pHash which is used as the fingerprint of the picture.

And step 204, traversing the photos i and j (j is more than or equal to 1 and less than or equal to N, i is not equal to j), and calculating the Hamming distance of pHash values of any two photos and the cosine similarity of the color histogram features for any two photos i (i is more than or equal to 1 and less than or equal to N, N is the number of photos in the group k) and j (j is more than or equal to 1 and less than or equal to N, i is not equal to j).

And selecting a photo i (i is more than or equal to 1 and less than or equal to N), calculating the Hamming distance of pHash values of the photo i and the photo j (j is more than or equal to 1 and less than or equal to N, i is not equal to j), and calculating the cosine similarity of the color histograms of the photo i and the photo j.

The Hamming distance between picture i and picture j refers to the number of digits of different numerical values of pHash (such as 64-digit pHash) of picture i and pHash of picture j, and the similarity degree between picture i and picture j is inversely related to the Hamming distance.

The cosine similarity of the color histograms of the photo i and the photo j refers to a cosine value of an included angle between a feature vector of the color histogram feature of the photo i and a feature vector of the color histogram feature of the photo i, and the similarity of the photo i and the photo j is positively correlated with the cosine similarity.

Step 205, judging that the Hamming distance of the pHash values of the picture i and the picture j is less than 8, and the cosine similarity of the histogram features of the picture i and the picture j is more than 0.96, if so, executing step 206; otherwise, go back to step 204 and continue to traverse photos i and j.

The numerical values used in the above determination are examples, and may be adjusted according to the need for identifying the similarity accuracy of the photos.

And step 206, judging that the two photos of the photo i and the photo j do not belong to the same group, returning to the step 204, and continuously traversing the photo i and the photo j.

After determining that the photo i and the photo j do not belong to the same photo group, if j is smaller than N, j equals j +1, and the process returns to step 204 to continue to compare the degree of similarity between the photo i and other photos; if j is equal to N and i is less than or equal to N, i is equal to i +1, and the method returns to the step 204 to select the next photo from the album and compare the next photo with other photos in the album to obtain a similar degree; if i is equal to N and j is equal to N, the photo traversal comparison is ended, and N photos in the album are divided into corresponding groups.

After the Hamming distance based on the pHash value of the photo and the cosine similarity of the histogram feature are grouped as much as possible, the grouping result can be further adjusted based on the photographing time and photographing position of the photo.

Step 207, dividing the two photos of the photo i and the photo j into the same group.

After the photo i and the photo j are grouped into the same photo group, if j is smaller than N, j equals j +1, and the process returns to step 204 to continue to compare the degree of similarity between the photo i and other photos; if j is equal to N and i is less than or equal to N, i is equal to i +1, and the method returns to the step 204 to select the next photo from the album and compare the next photo with other photos in the album to obtain a similar degree; if i is equal to N and j is equal to N, the photo traversal comparison is ended, and N photos in the album are divided into corresponding photo groups.

Step 208, for each group of photos, judging any two photos i and j in the group, judging whether the photos meet the conditions that the shooting positions and the shooting time are close, namely whether the distance difference between the shooting positions is not more than 10 meters, and the shooting time of the photos is within 2 hours, if so, executing step 210; otherwise, step 209 is performed.

Step 209, removing the photos which do not meet the conditions from the grouping; or presenting the result of each group, prompting the photos which do not meet the conditions of the shooting position and the shooting time, and processing according to the user instruction.

And for the photos which do not meet the conditions of close shooting positions and shooting time in the group, the photos do not participate in the process of fusion storage with other photos in the group.

And step 210, obtaining a photo grouping result, and taking the current photo in the group as the photo finally included in the group.

Secondly, extracting characteristic points, determining reference photos and storing grouped photos

Referring to fig. 4-2, fig. 4-2 is a schematic diagram of an optional process of feature point extraction, reference picture determination, and merging and storing grouped pictures according to an embodiment of the present invention, including the following steps:

step 301, traversing the photos in the groups of similar photos, and for each group K (K is more than or equal to 1 and less than or equal to K, and K is the number of the groups of similar photos), selecting one photo M in the group K (M is more than or equal to 1 and less than or equal to M, and M is the number of the photos in the group K).

And step 302, for the group k, randomly selecting a photo n (M is not equal to n, n is more than or equal to 1 and less than or equal to M) except the photo M in the group in a traversal mode, traversing the photo M and the photo n, and extracting feature points from any two photos of the photo M and the photo n by adopting an ORB feature extraction algorithm.

For a photo n (M ≠ n, n is more than or equal to 1 and less than or equal to M) selected in a traversal mode in the grouping k, feature points are respectively extracted from the photo M and the photo n by using an ORB feature extraction algorithm, the feature points are represented by feature description vectors, the feature description vectors of the feature points of the photo M form a feature description matrix for describing features of the photo M, and the feature description vectors of the feature points of the photo n form a feature description matrix for describing features of the photo candidate n.

The reason for extracting the feature points by adopting the ORB feature extraction algorithm is that the FAST feature detection algorithm is used for detecting the feature points in the photo, the positions of the feature points in the photo can be quickly determined, and the BRIEF feature descriptor is used for describing the feature points, so that the extraction speed advantage is very obvious, and the method has the following advantages: 1) the method is not sensitive to noise, and when the picture has noise, the characteristic points can still be accurately extracted; 2) rotation invariance, i.e. the features extracted from the photograph have rotation invariance; and also has 3) scale invariance.

Step 303, for the picture m and the picture n in the group k, traversing and calculating the distance between the feature points in any two pictures, namely the feature point in the picture m and the feature point in the picture n.

Let the feature description matrix of photo m and photo n correspond to descriptor sA and descriptor sB, and use the matching matrix to match descriptor sA and descriptor sB, i.e. match any feature point R (R is greater than or equal to 1 and less than or equal to R, R is the number of feature points in photo m) of photo m with any feature point S (S is greater than or equal to 1 and less than or equal to S, S is the number of feature points in photo n), to obtain the distance (Euclidean distance) between the feature point R of photo m and the feature point S of photo n.

And step 304, filtering the characteristic points which do not meet the preset condition in the picture m and the picture n based on the distance of the characteristic points.

For a feature point pair < r, s > consisting of any feature point r in the picture m and any feature point s in the picture n, determining the distance (Euclidean distance) between the feature point r and the feature point s by traversing r and s, comparing the distance between the feature point r and the feature point s with a feature point distance threshold value, judging whether the distance is smaller than the feature point distance threshold value, if so, retaining the feature point in the feature point pair, otherwise, filtering the feature point in the feature point pair. As an example, the feature point distance threshold is max (2 × minimum distance, 0.02).

Step 305, comparing the average minimum distance between the feature points r 'remaining after filtering in the picture m and the feature points s' remaining after filtering in the picture n with the average minimum distance threshold; if the average distance is less than the average distance threshold, go to step 306; otherwise, go back to step 302 to continue traversing photos m and n.

And step 306, recording the characteristic points r 'of the picture m and the characteristic points s' of the picture n.

To reduce errors and eliminate false feature point matches, in step 305, the average minimum distance between the feature point r 'in picture m and the feature point s' in picture n is compared with an average minimum distance threshold to determine whether the average minimum distance between the feature point r 'remaining after filtering in picture m and the feature point s' remaining after filtering in picture n is less than the average minimum distance threshold (e.g., takes the value of 10):

1) if the average distance is smaller than the average distance threshold value, recording the characteristic point r 'in the picture m and the characteristic point s' in the picture n, and turning to the step 307;

2) if not, judging whether the pictures n in the group K are traversed, namely judging whether n is equal to K (the number of the pictures in the group K), if not, judging that n is n +1, and iteratively executing the steps 302 to 306 according to the picture m and a new picture n in the group K until all the pictures n in the group K are traversed.

Wherein, for the average minimum distance, the sum of the distances (certainly less than the distance threshold) between the feature points r 'and s' in the feature point pair < r ', s', in the feature point pair < r ', s' > and the feature points r 'in the feature point pair < r', s '> can be compared with the number of the feature point pairs < r', s '> by traversing the feature point pair < r', s 'in the picture m and the feature point s' in the picture n, and the average minimum distance is expressed as: the average minimum distance ═ the number of distance/feature point pairs < r ', s' > where Σ is smaller than max (2 × minimum distance, 0.02).

Step 307, determining whether any two photos in the group K, i.e. the photo m and the photo n, have been traversed, i.e. determining whether n is equal to K (the number of photos in the group K), if not, n is equal to n +1, and iteratively executing steps 302 to 306 according to the photo m and the new photo in the group K until all the photos n in the group K are traversed.

Step 308, traversing the photos M in the group k (M is more than or equal to 1 and less than or equal to M, wherein M is the number of the photos in the group k), performing convex hull operation on the feature points recorded by the photos M in the group k to obtain a convex hull area of the photos M, and calculating the area of the convex hull area, namely the convex hull area.

The convex hull area refers to the area of the smallest convex polygon M (M is more than or equal to 1 and less than or equal to M, M is the number of photos in the group k) including the feature points of the photo M (i.e. the feature points recorded for the photo M), and the area of the smallest convex polygon M (M is more than or equal to 1 and less than or equal to M, M is the number of photos in the group k) is calculated as the corresponding convex hull area.

Step 309, comparing the convex hull areas of the photos of the group k, determining the convex hull area with the smallest area, and recording the photo with the smallest convex hull area as the reference photo of the group k.

The processing of removing the region of the reference picture similar to the other pictures in the group k (i.e., the smallest convex polygon of the reference picture) from the reference picture to transform the reference picture into the other pictures in the group k is the least transformation (or transition) performed on the reference picture in the group k than on the other pictures in the group k that are selected as the reference picture (and transformed into the other pictures in the group k).

Step 310 determines whether the convex hull region of the reference picture needs to be transformed (and the data transformed) and the difference region that needs to be covered in the reference picture when the reference picture is transformed into another picture.

The other photos in the group k are clipped according to the convex hull region by taking the convex hull region of the reference photo as a reference, and the regions remaining after each photo is clipped are stretched to be the same through affine transformation (including at least one of rotation and translation).

The other photos within the group k can be divided into two such partial regions: 1) the first region is the same as or similar to the convex hull region of the reference photo, and the similar region can be obtained by transforming (transforming by at least one of rotation, translation, stretching and scaling) the convex hull region of the reference photo; 2) the second area is a difference area that the reference photograph does not have, and when another photograph needs to be presented, the second area of the other photograph needs to be overlaid on the reference photograph.

Step 311, for the group k, store based on the reference photograph: the reference picture of the group k, information of the convex hull region of the reference picture, and a region where the other pictures in the group k are distinguished from the reference picture (i.e., a distinguishing region) are stored.

Theoretically, the time complexity of the multiple operations performed in step 310 is high, and is almost impossible to be practically applied; in practical applications, the step 310 is iterated once or twice, as long as it is ensured that the total area of the convex polygons obtained by clipping other photos occupies at least a predetermined proportion (e.g. 70%) of the corresponding photos, and the photos in the group can adopt the storage scheme provided by the embodiment of the present invention.

For the photos in the group k, the photos are stored as follows:

1) a reference picture including original image data of the reference picture and a convex hull region (position in the reference picture) of the reference picture;

2) the other photos, the first area data is used for recording the relation between the first area and the convex hull area of the reference photo, such as the relation is the same as the convex hull area of the reference area, or the first area data is obtained by converting the conversion data and recording the corresponding conversion data; a second region, the other photographs including raw image data of the difference region with respect to the reference photograph.

And step 312, judging whether the packet k is processed in a traversal mode, if not, determining that k is k +1, returning to step 301 to process the next packet, and otherwise, ending.

For example, for an alternative display diagram of similar photo grouping provided by the embodiment of the present invention shown in fig. 5, taking the photo (1) in fig. 5 as a reference photo, the corresponding convex hull region may be formed by covering the difference region on the basis of the photo (1) as shown in the photos (7) to (8) for other photos, i.e. photos (2) to (6), and it can be seen that, for the conversion from the reference photo (1) to the photos (2) to (6), since the convex hull region (10) of the reference photo (1) does not translate or rotate with respect to the convex hull region (9) of the photos (2) to (9), for the photos (2) to (6), the first region data does not include the conversion data, and the second region includes the difference region of the photos (2) to (6) with respect to the photo (1), comprises a graphic element which needs to be overlapped in the convex hull area of the photo (1) and a graphic element which needs to be overlapped outside the convex hull area in the photo (1).

Three, rendering and edge compensation

Referring to fig. 4-3, fig. 4-3 is a schematic flow chart of an alternative method for rendering a presentation and a boundary compensation photo according to an embodiment of the present invention, including the following steps:

step 401, pre-rendering the canvas, and loading the reference photo.

The step 401 is executed to display the reference photos in the group k, and the following description will be made of the other photos in the display group.

Step 402, inquiring whether the photo to be presented (the other photos except the reference photo in the group k) exists in the first area data, if so, executing step 403, otherwise, executing step 404.

If the data of the first area of the photo to be presented has no transformation data, the first area of the photo to be presented is the same as the convex hull area of the reference photo, and the photo to be presented can be directly reproduced by multiplexing the convex hull area of the reference photo.

And 403, transforming the convex hull region of the reference photo according to the transformation data to form a first region of the photo to be presented, and rendering by replacing the corresponding convex hull region of the reference photo.

If the first region data of the photo to be presented comprises the transformation data, the first region of the photo to be presented is obtained by transforming the convex hull region of the reference photo, and the convex hull region of the reference photo is transformed (such as at least one of rotation, translation and scaling) based on the transformation data to form and render the first region to be presented.

And step 404, rendering the second area of the photo to be presented by replacing the corresponding area of the reference photo.

Step 405a, calculating the boundary characteristics of the first area and the second area of the photo to be presented, and performing smooth transition on the boundary by using a smooth function.

In order to avoid the problem that the joint of the first area and the second area is sharp and the photo shows obvious fit, the characteristic value of the boundary can be calculated, and then the smooth transition function is adopted to carry out smooth transition on the boundary, so that the effect transition of the boundary is more natural.

Step 405b, blur processing is performed on the boundary.

In order to avoid the problem that the joint of the first area and the second area is sharp and the photo presents a very obvious fit feeling, the joint boundary of the first area and the second area can be blurred due to the fact that the joint of the first area and the second area is not the content main area of the photo, so that the sharp display effect of the boundary is eliminated, and compared with the mode that the boundary is excessively processed, the resource overhead can be saved.

Referring to fig. 6 again, fig. 6 is an optional schematic diagram of grouping and merging storage of photos according to an embodiment of the present invention, and an application of the image information processing apparatus is described.

Scene 1) an image information processing device is deployed on a mobile phone at a user side, and is provided as an application for processing images on an application level, such as an electronic photo album, which stores various photos taken by a user through the mobile phone, generally, a large number of photos taken in a continuous shooting mode or photos taken by manually pressing a shutter for multiple times, wherein the photos stored in the electronic photo album can be checked after the user opens the electronic photo album, when the user wants to save the storage space of the mobile phone, the photos are grouped in a manual mode or are grouped through an automatic grouping function of the electronic photo album, and photos with larger differences between the shooting time and/or the shooting position in the groups prompted by the electronic photo album can be adjusted according to a grouping result; after the grouping is determined, the reference photograph, the conversion relationship between the other photographs and the reference photograph (hereinafter also referred to as conversion data), and the difference region are identified for each of the groups. If the photos in the group are stored locally in the mobile phone, only the reference photos and the original data of the difference area need to be stored, and the original image data of the feature areas of other images are stored instead of the storage transformation relation.

In addition, when other images need to be presented, the electronic photo album can be quickly presented through the transformation relation, and the quality of the photos is guaranteed because algorithm storage is not adopted.

In summary, the embodiments of the present invention have the following beneficial effects:

the scheme for fusing and storing the images, which is provided by the embodiment of the invention, is different from the scheme for compressing a single image, and realizes a compression mode based on a series of images on the whole by analyzing the incidence relation among the images.

The embodiment of the invention comprehensively uses a series of technologies to achieve the aim: the method comprises the steps of photo similarity identification, feature point extraction, reference image determination, change area storage, rendering presentation and boundary compensation.

Similar images are found by using pHash and a histogram, feature points of each group of similar images are extracted by a specially optimized ORB method, a reference image of one group of similar images is found by using a convex hull, and then only the reference image is stored, other images are relative to a reference image area, and the transformation relation of the convex hull area of other images relative to the convex hull area of the reference image is stored if necessary.

When the image is presented, the image is combined and rendered in a canvas mode, and for the boundary area, smooth transformation or fuzzy processing can be adopted, so that the image looks more natural after being rendered.

Those skilled in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media capable of storing program codes, such as a removable Memory device, a Random Access Memory (RAM), a Read-Only Memory (ROM), a magnetic disk, and an optical disk.

Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a RAM, a ROM, a magnetic or optical disk, or other various media that can store program code.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims

1. An image processing method, comprising:

identifying images meeting similar conditions from an image set, and dividing the images meeting similar conditions in the image set into corresponding groups based on light change or background change on the images;

extracting feature points from each image of the group, and marking the feature points matched with any two images in the group;

determining a minimum convex polygon including the marked feature points in each image as a feature region of the corresponding image;

comparing the areas of the characteristic regions of the images in the group to determine a minimum characteristic region, and taking the image with the minimum characteristic region in the group as a reference image;

cutting out matched regions in the other grouped images by using the characteristic region of the reference image, and determining the transformation relation between the characteristic region of the reference image and the cut-out regions;

taking the remaining area after the matched area in the other image is cut out as the difference area of the other image relative to the reference image;

2. The method of claim 1, wherein identifying images from the set of images that meet similar criteria comprises:

identifying images from the set of images that satisfy at least one of the following similarity conditions:

the cosine similarity of the histogram feature of the image exceeds a cosine similarity threshold;

3. The method of claim 1, further comprising:

after dividing the images meeting the similar conditions in the image set into corresponding groups, performing at least one of the following filtering operations:

filtering images from the packets whose shot location distance exceeds a shot location distance threshold;

and filtering the images of which the shooting time interval exceeds the shooting time interval threshold value from the packets.

4. The method of claim 1, wherein said marking feature points where any two images in the group match comprises:

and forming a characteristic point pair by the characteristic points matched with any two images, comparing the distance between the characteristic points in the characteristic point pair with a characteristic point distance threshold, filtering the characteristic points which are greater than the characteristic point distance threshold, and marking the residual characteristic points after filtering.

5. The method of claim 4, wherein the marking the feature points remaining after filtering comprises:

calculating the distance between the residual characteristic points of any two images after filtering to be smaller than the distance between the characteristic points and a threshold value, and calculating the average value of the calculated distances to obtain the average minimum distance;

judging whether the average minimum distance is smaller than an average minimum distance threshold value;

if so, marking the feature points of any two images which are left after filtering,

6. The method of claim 1, wherein the cropping matching regions in the grouped other images with the feature region of the reference image and determining the transformation relationship between the feature region of the reference image and the cropped regions comprises:

and cutting out the matched regions in the other grouped images by using the characteristic region of the reference image, converting between the characteristic region of the reference image and the cut-out region, and determining the converted rotation angle, movement displacement and scale conversion.

7. The method of claim 1, further comprising:

and when the image to be presented is the reference image, pre-rendering a canvas, and loading the reference image on the canvas.

8. The method of claim 1, further comprising:

when the image to be presented is the other image, prerendering a canvas, and loading the reference image on the canvas;

reading the transformation relation and the difference area of the other images to be presented, and transforming the characteristic area of the reference image based on the transformation relation to form the characteristic area of the other images to be presented;

loading the read difference region on the canvas based on the location of the transformed feature region.

9. The method of claim 8, further comprising:

and carrying out fuzzy processing or smooth transition processing on the boundary of the other image characteristic region to be presented and the difference region.

10. An image processing apparatus characterized by comprising:

the grouping unit is used for identifying images meeting similar conditions from an image set and dividing the images meeting the similar conditions in the image set into corresponding groups based on the change of light or the change of background on the images;

the characteristic unit is used for extracting characteristic points from each image of the group and marking the characteristic points matched with any two images in the group; determining a minimum convex polygon including the marked feature points in each image as a feature region of the corresponding image;

a transformation unit to:

11. The apparatus of claim 10,

the grouping unit is further configured to identify an image satisfying at least one of the following similarity conditions from the image set:

12. The apparatus of claim 10,

the grouping unit is further configured to perform at least one of the following filtering operations after dividing the images in the image set that meet the similar condition into corresponding groups:

13. The apparatus of claim 10,

the feature unit is further configured to form a feature point pair from feature points matched with any two images, compare a distance between the feature points in the feature point pair with a feature point distance threshold, filter feature points larger than the feature point distance threshold, and mark remaining feature points after filtering.

14. The apparatus of claim 13,

the feature unit is further configured to calculate a distance between the feature points remaining after filtering of the two arbitrary images, which is smaller than a distance threshold between the feature points, and calculate an average value of the calculated distances to obtain an average minimum distance;

15. The apparatus of claim 10,

the transformation unit is further configured to crop a matching region in the other grouped images with the feature region of the reference image, transform between the feature region of the reference image and the cropped region, and determine a rotation angle, a shift displacement, and a scale transformation of the transformation.

16. The apparatus of claim 10, further comprising:

and the first presentation unit is used for prerendering a canvas when the image to be presented is the reference image, and loading the reference image on the canvas.

17. The apparatus of claim 10, further comprising:

the second presentation unit is used for prerendering a canvas and loading the reference image on the canvas when the image to be presented is the other image;

18. The apparatus of claim 17,

the second presenting unit is further configured to perform blurring processing or smooth transition processing on the boundary between the other image feature region to be presented and the difference region.

19. A computer storage medium having stored thereon executable instructions for implementing the image processing method of any one of claims 1 to 9 when executed.