WO2023142732A1

WO2023142732A1 - Image processing method and apparatus, and electronic device

Info

Publication number: WO2023142732A1
Application number: PCT/CN2022/138573
Authority: WO
Inventors: 李政; 陈刚
Original assignee: 华为技术有限公司
Priority date: 2022-01-28
Filing date: 2022-12-13
Publication date: 2023-08-03
Also published as: CN114627000A

Abstract

Provided are an image processing method and apparatus, and an electronic device. In the method, an electronic device stores image blocks corresponding to a panoramic image at various viewpoints, and a mapping relationship between features of an image of each angle of view at the viewpoints and identifiers of the image blocks corresponding to the panoramic image at the viewpoints. The image of each angle of view at the viewpoints and the image blocks corresponding to the panoramic image at the viewpoints are all obtained on the basis of the panoramic image at the viewpoints, wherein the panoramic image at the viewpoints is a high-definition image. Compared with the method for storing high-definition images of different angles of view at various viewpoints in the prior art, the method in the present application can reduce storage overheads. On this basis, image processing can also be realized, so as to obtain an image with higher definition.

Description

Image processing method, device and electronic equipment

This application claims the priority of the Chinese patent application with the application number 202210109463.6 and the application title "Image Processing Method, Device and Electronic Equipment" filed with the China Patent Office on January 28, 2022, the entire contents of which are incorporated by reference in this application middle.

technical field

The embodiments of the present application relate to the field of image processing, and in particular, to an image processing method, device, and electronic equipment.

Background technique

With the development of terminals, many terminals can support high-magnification photography at present, such as 30 times and 50 times. However, due to the limitations of the structure of some terminals, such as mobile phones, which require a relatively thin body, when the terminal uses a high magnification to take pictures, the captured images have low definition and fuzzy details.

In order to improve the clarity of images taken by the terminal at a high magnification, a large number of high-definition images of different shooting positions and different shooting angles at the same shooting position may be stored in advance. After taking pictures with a high magnification, the terminal can obtain the high-definition image with the greatest similarity to the image taken by the terminal among the pre-stored high-definition images, so that the terminal can display the high-definition image. At present, this method needs to store a large number of high-definition images, and the storage overhead is large.

Contents of the invention

Embodiments of the present application provide an image processing method, device, and electronic device, which can reduce storage overhead.

In the first aspect, the embodiment of the present application provides an image processing method, and the execution subject of the method may be an electronic device or a chip in the electronic device, and the cloud electronic device is taken as an example for description below. The electronic device stores the image blocks corresponding to the panoramic images under each viewpoint, and the mapping relationship between the features of the images of each viewpoint under the viewpoints and the identifiers of the image blocks corresponding to the panoramic images under the viewpoints, and the viewpoints The images of each angle of view and the image blocks corresponding to the panoramic images at each viewpoint are all obtained based on the panoramic images at each viewpoint, and the panoramic images at each viewpoint are high-definition images. If it is based on a single-lens reflex camera, etc., it can obtain high-definition images. The image is obtained from the image captured by the device.

In this method, the electronic device can obtain the first image to be processed, and can extract the features in the first image, and then obtain the similarity between the features of the image of each viewing angle under each viewpoint and the feature of the first image , where the image with the largest similarity has the highest similarity with the feature of the first image. The electronic device may determine the target identifier of the feature map of the image of the viewing angle corresponding to the maximum similarity according to the feature of the image of the viewing angle corresponding to the maximum similarity and the mapping relationship.

Because the image blocks corresponding to the panoramic images under each viewpoint are obtained based on the panoramic images under each viewpoint, after obtaining the target identification of the feature map of the image of the viewing angle corresponding to the maximum similarity, the corresponding image block can be identified according to the target , acquiring a second image, where the definition of the second image is higher than the definition of the first image. The second image is obtained according to the image block in the panoramic image to which the image block corresponding to the target identifier belongs.

Wherein, because the resolution of the panoramic images at each viewpoint is greater than or equal to the preset resolution, that is, the resolution of the panoramic images at each viewpoint is higher than that of the first image, the corresponding panoramic images at each viewpoint The image blocks are obtained based on the panoramic images under the various viewpoints, so the definition of the second image obtained based on the image blocks corresponding to the target identifier is greater than or equal to the preset definition. Therefore, using this method, the terminal can obtain a high-definition second image, and because the electronic device stores the image blocks corresponding to the panoramic images at each viewpoint, as well as the features of the images at each viewpoint at each viewpoint and the panoramic images at each viewpoint The mapping relationship of the identifiers of the image blocks corresponding to the image can reduce the storage overhead compared with the way of storing high-definition images of different viewing angles under each viewpoint in the prior art.

In a possible implementation manner, in order to reduce the amount of calculation of the similarity of the electronic device, when the electronic device acquires the first image, it may also acquire the position (ie, viewpoint) of the device that captures the first image. In this way, the electronic device may determine a target viewpoint within a preset range from the location, and then acquire the similarity between the feature of the image at each viewing angle under the target viewpoint and the feature of the first image. This can reduce the calculation amount of the terminal to calculate the similarity, and only need to calculate the similarity between the feature of the image of each viewing angle and the feature of the first image at the target viewpoint within the preset range from the position of the terminal, without obtaining A degree of similarity between the feature of the image of each viewing angle under each viewpoint and the feature of the first image.

In a possible implementation manner, the mapping relationship includes a first index relationship and a second index relationship, and the second index relationship is: the feature of the image of each viewing angle under each viewpoint and the The mapping relationship of the central point of the image of each viewing angle, the first index relationship is: the mapping relationship between the central point of the image of each viewing angle under each viewing point and the identifier of the image block corresponding to the panoramic image under each viewing point . Wherein, after the electronic device acquires the features of the image of the viewing angle corresponding to the maximum similarity, it may determine the viewing angle corresponding to the maximum similarity according to the features of the image of the viewing angle corresponding to the maximum similarity and the second index relationship. The center point of the feature map of the image of the image, and then according to the center point of the feature map of the image of the viewing angle corresponding to the maximum similarity, and the first index relationship, the target identifier is determined.

In one embodiment, the features of the images of each viewpoint under each viewpoint, the first index relationship, and the second index relationship stored in the electronic device may be preset in the electronic device by the staff, or acquired by the electronic device itself.

Wherein, in one embodiment, the electronic device may acquire, according to the panoramic images under the various viewpoints, the features of the images of each viewpoint under the viewpoints, the first index relationship, and the second index relationship, Furthermore, image features of each viewing angle under each viewing point, the first index relationship, and the second index relationship are stored.

The electronic device may use back-projection transformation on the panoramic images under each viewpoint to obtain images of multiple viewing angles (highly overlapping images) under each viewpoint, and the center point of the image of each viewing angle under each viewpoint is in the corresponding The coordinate position of the panoramic image, the overlap ratio between the images of the adjacent viewing angles under each viewpoint is greater than the preset overlapping ratio, and then the features of the images of each viewing angle under the various viewpoints are extracted.

Wherein, the electronic device may directly perform back-projection transformation on the panoramic images at each viewpoint.

Alternatively, in an embodiment, the electronic device may acquire panoramic images at various viewpoints according to the low-overlapping images, and then perform back-projection transformation on the panoramic images at various viewpoints. In this embodiment, the electronic device may use the panorama image stitching technology to obtain the panoramic images under each viewpoint according to the pre-collected images of multiple viewpoints under each viewpoint, and the pre-collected images of adjacent viewpoints under each viewpoint The overlapping ratio between the images is less than the preset overlapping ratio, that is, low overlapping images.

Specifically, the electronic device may use a sliding window with a second preset size to slide in the panoramic image in the panoramic image at each viewpoint, and use back-projection transformation to sequentially obtain partial panoramic images in the sliding window The center point of the image of the corresponding viewing angle and the image of the corresponding viewing angle of the partial panoramic image is at the coordinate position of the corresponding panoramic image, and the image of each viewing angle under each viewing point has the second preset size.

Wherein, the electronic device may construct the second index relationship according to the coordinate position of the center point of the image of each angle of view under each viewpoint in the corresponding panoramic image, and the characteristics of the image of each angle of view under each viewpoint. as well as,

The electronic device may cut the panoramic image under each viewpoint to obtain image blocks corresponding to the panoramic image under each viewpoint, and according to the coordinate position of the center point of the image of each angle of view under each viewpoint is in the corresponding panoramic image , and the image blocks corresponding to the panoramic images under the various viewpoints, construct the first index relationship. In a possible implementation manner, the image blocks corresponding to the panoramic images under each viewpoint have a first preset size.

In the above example, the electronic device adopts the panorama image stitching technology, and according to the pre-collected images of multiple viewing angles under each viewpoint, obtains the panoramic images under each viewpoint, that is, the panoramic images under each viewpoint in the first viewing plane. The pre-collected image of each viewing angle is projected onto the second viewing plane to which the panoramic image belongs, so as to obtain the panoramic image at each viewing point. During the projection process, the electronic device may also obtain the transformation relationship between the first viewing plane and the second viewing plane.

In a possible implementation manner, after obtaining the target identifier, the electronic device may use back projection transformation to project the image block corresponding to the target identifier to the first viewing plane according to the transformation relationship, to obtain the second Two images.

In this way, the electronic device can obtain the second image on the same viewing plane as the image captured by the terminal, and there is no difference in the viewing plane when the user watches it. For the user, the conversion from the first image to the second image is not perceived by the user, which can improve user experience .

In a possible scenario, the electronic device can be the cloud, and the terminal can capture the first image, but the definition of the first image is not high. Therefore, in this scenario, the terminal can send the first image to the cloud and capture the first image. The image is the location of the terminal. In this way, the cloud can obtain the first image and the location where the first image was shot, and then use the method described in the above possible implementation manners to obtain the second image. After the cloud obtains the second image, it can send the second image to the terminal, so that the terminal can display and store the second image, so that the user can see the high-definition second image on the terminal, and can Improve user experience.

In a second aspect, the embodiment of the present application provides an image processing method, and the executing subject for executing the method may be a terminal or a chip in the terminal, and the following uses a terminal as an example for description.

The terminal uses the first image captured by the first magnification, the first magnification is greater than or equal to the preset magnification, and the terminal sends the first image to the electronic device. The terminal receives the second image from the electronic device, and may display the second image in response to the image display instruction, and the definition of the second image is higher than that of the first image.

In a possible implementation manner, the terminal stores the image blocks corresponding to the panoramic images under each viewpoint, and the features of the images of each viewpoint under the viewpoints and the identification of the image blocks corresponding to the panoramic images under each viewpoint The mapping relationship of each angle of view under each viewpoint and the image block corresponding to the panoramic image under each viewpoint are obtained based on the panoramic image under each viewpoint, and the definition of the panoramic image under each viewpoint is greater than or equal to Preset sharpness.

In response to the first image taken with the first magnification, the terminal may acquire the similarity between the features of the image of each viewing angle under the various viewpoints and the feature of the first image, and the The feature, and the mapping relationship, determine the target identifier of the feature map of the image of the viewing angle corresponding to the maximum similarity. The terminal acquires a second image according to the image block corresponding to the target identifier, and the definition of the second image is higher than that of the first image, and then can display the second image in response to an image display instruction.

In a possible implementation manner, the acquiring the similarity between the feature of the image of each viewing angle under each viewing point and the feature of the first image includes: determining a distance within a preset range from the terminal A target viewpoint, and obtaining a similarity between the features of the image of each viewing angle under the target viewpoint and the features of the first image.

In a possible implementation manner, the mapping relationship includes a first index relationship and a second index relationship, and the second index relationship is: the feature of the image of each viewing angle under each viewpoint and the The mapping relationship of the central point of the image of each viewing angle, the first index relationship is: the mapping relationship between the central point of the image of each viewing angle under each viewing point and the identifier of the image block corresponding to the panoramic image under each viewing point .

According to the feature of the image of the angle of view corresponding to the maximum similarity and the mapping relationship, determining the target identifier of the feature map of the image of the angle of view corresponding to the maximum similarity includes: according to the image of the angle of view corresponding to the maximum similarity The feature of the image, and the second index relationship, determine the center point of the feature map of the image of the viewing angle corresponding to the maximum similarity; according to the center point of the feature map of the image of the viewing angle corresponding to the maximum similarity, and the The first index relationship is used to determine the target identifier.

In a possible implementation manner, the method further includes: according to the panoramic images under each viewpoint, acquiring the features of images of each viewpoint under each viewpoint, the first index relationship, and the second Index relationship: store the feature of the image of each viewing angle under each viewpoint, the first index relationship, and the second index relationship.

In a possible implementation manner, the acquiring the feature of the image of each view angle under each view point according to the panoramic image under each view point includes: applying back projection transformation to the panoramic image under each view point to obtain The images of multiple viewing angles under each viewing point, and the center point of the image of each viewing angle under each viewing point are at the coordinate position of the corresponding panoramic image, and the overlapping ratio between images of adjacent viewing angles under each viewing point is greater than Presetting the overlapping ratio; extracting the feature of the image of each view angle under each view point.

In a possible implementation manner, the back-projection transformation is used for the panoramic images under each viewpoint to obtain images of multiple viewing angles under each viewpoint, and the center point of the image of each viewing angle under each viewpoint The coordinate position of the corresponding panoramic image includes: in the panoramic image under each viewpoint, using a sliding window with a second preset size to slide in the panoramic image, and using back-projection transformation to sequentially obtain the sliding window The center point of the image of the angle of view corresponding to the partial panoramic image and the image of the angle of view corresponding to the partial panoramic image is at the coordinate position of the corresponding panoramic image, and the image of each angle of view under each viewpoint has the second preset size.

In a possible implementation manner, obtaining the second index relationship includes: according to the coordinate position of the center point of the image of each viewing angle under the various viewpoints in the corresponding panoramic image, and the The feature of the image of the viewing angle is used to construct the second index relationship.

In a possible implementation manner, obtaining the first index relationship includes: cutting the panoramic image under each viewpoint to obtain image blocks corresponding to the panoramic image under each viewpoint; The center point of the image of each viewing angle is at the coordinate position of the corresponding panoramic image, and the image block corresponding to the panoramic image under each viewing angle, to construct the first index relationship.

In a possible implementation manner, the image blocks corresponding to the panoramic images under each viewpoint have a first preset size.

In a possible implementation manner, before acquiring the features of images of each view angle under each view point, the first index relationship, and the second index relationship according to the panoramic images under each view point, It also includes: using panoramic image stitching technology, according to the pre-collected images of multiple viewing angles under each viewpoint, obtaining the panoramic image under each viewpoint, and the overlap between the pre-collected images of adjacent viewing angles under each viewpoint rate is less than the preset overlap rate.

In a possible implementation manner, using the panoramic image stitching technology to acquire the panoramic images under each viewpoint according to the pre-collected images of multiple viewpoints under each viewpoint includes: placing the panoramic image in the first viewing plane The pre-collected images of each viewing angle under each viewpoint are projected onto the second viewing plane to which the panoramic image belongs, so as to obtain the panoramic image under each viewing point, and the first viewing plane and the second viewing plane transform relationship.

In a possible implementation manner, the acquiring the second image according to the image block corresponding to the target identifier includes: projecting the image block corresponding to the target identifier onto the image block corresponding to the target identifier according to the transformation relationship The first viewing plane is used to obtain the second image.

In a third aspect, the embodiment of the present application provides an image processing apparatus, and the image processing apparatus may be an electronic device or a chip in the electronic device. The image processing device includes:

A processing module, configured to obtain the similarity between the features of the image of each viewing angle under the various viewpoints and the features of the first image, and determine the The target identifier of the feature map of the image of the viewing angle corresponding to the maximum similarity, and according to the image block corresponding to the target identifier, acquire a second image, the definition of the second image is higher than the definition of the first image .

In a possible implementation manner, the processing module is specifically configured to acquire the first image, a location where the first image is taken, determine a target viewpoint within a preset range from the location, and acquire the A degree of similarity between the feature of the image of each viewing angle under the target viewpoint and the feature of the first image.

The processing module is specifically configured to determine the center point of the feature map of the image of the viewing angle corresponding to the maximum similarity according to the features of the image of the viewing angle corresponding to the maximum similarity and the second index relationship, and according to the The center point of the image feature map of the viewing angle corresponding to the maximum similarity, and the first index relationship determine the target identifier.

In a possible implementation manner, the processing module is further configured to acquire, according to the panoramic images under each viewpoint, the features of images of each viewpoint under each viewpoint, the first index relationship, and the second index relationship.

A storage module, configured to store features of images of each view angle under each view point, the first index relationship, and the second index relationship.

In a possible implementation manner, the processing module is specifically configured to perform back-projection transformation on the panoramic images under each viewpoint to obtain images of multiple angles of view under each viewpoint, and images of each angle of view under each viewpoint. The center point of the image is at the coordinate position of the corresponding panoramic image, and the overlapping ratio between images of adjacent viewing angles under each viewing point is greater than a preset overlapping ratio; extracting features of images of each viewing angle under each viewing point.

In a possible implementation manner, the processing module is specifically configured to use a sliding window with a second preset size to slide in the panoramic image in the panoramic image at each viewpoint, and use back-projection transformation to sequentially obtain The image of the angle of view corresponding to the partial panoramic image in the sliding window and the center point of the image of the angle of view corresponding to the partial panoramic image are at the coordinate position of the corresponding panoramic image, and the image of each angle of view under each viewpoint has the Second default size.

In a possible implementation manner, the processing module is specifically configured to, according to the coordinate position of the center point of the image of each angle of view under each viewpoint in the corresponding panoramic image, and the coordinate position of the image of each angle of view under each viewpoint features, constructing the second index relationship.

In a possible implementation manner, the processing module is specifically configured to cut the panoramic image under each viewpoint to obtain image blocks corresponding to the panoramic image under each viewpoint, and The center point of the image is at the coordinate position of the corresponding panoramic image, and the image blocks corresponding to the panoramic image under each viewpoint are used to construct the first index relationship.

In a possible implementation manner, the processing module is further configured to use the panoramic image stitching technology to acquire the panoramic images under each viewpoint according to the pre-collected images of multiple viewpoints under each viewpoint, and the panoramic images under each viewpoint are The overlap ratio between pre-collected images of adjacent viewing angles is smaller than the preset overlap ratio.

In a possible implementation manner, the processing module is specifically configured to project the pre-collected image of each viewing angle under each viewpoint in the first viewing plane to the second viewing plane to which the panoramic image belongs, so as to obtain the The panoramic image under each viewpoint, and the transformation relationship between the first viewing plane and the second viewing plane.

In a possible implementation manner, the processing module is specifically configured to, according to the transformation relationship, project the image block corresponding to the target identifier to the first viewing plane by using back-projection transformation to obtain the second image.

In a possible implementation manner, the transceiver module is configured to receive the first image from the terminal and the location of the terminal when the terminal captures the first image, and send the second image to the terminal.

In a fourth aspect, the embodiment of the present application provides an image processing device, and the image processing device may be a terminal or a chip in the terminal. The image processing device includes:

In a possible implementation manner, the terminal stores the image blocks corresponding to the panoramic images under each viewpoint, and the features of the images of each viewpoint under the viewpoints and the identification of the image blocks corresponding to the panoramic images under each viewpoint The mapping relationship between the images of each viewing angle under the viewpoints and the image blocks corresponding to the panoramic images under the viewpoints are all obtained based on the panoramic images under the viewpoints.

The photographing module is used to capture the first image with the first magnification, and the first magnification is smaller than the preset magnification.

A processing module, configured to obtain the similarity between the features of the image of each viewing angle under the various viewpoints and the features of the first image, and determine according to the features of the image of the viewing angle corresponding to the maximum similarity and the mapping relationship Acquiring a second image according to the object identifier of the feature map of the image of the viewing angle corresponding to the maximum similarity, and the image block corresponding to the object identifier. The definition of the second image is higher than that of the first image.

The display module is configured to display the second image in response to the image display instruction.

In a possible implementation manner, the processing module is specifically configured to determine a target viewpoint within a preset range from the terminal, and acquire features of an image of each viewing angle under the target viewpoint and the first image similarity of features.

In a fifth aspect, the embodiment of the present application provides an electronic device, and the electronic device may be the above-mentioned cloud or terminal. The electronic device may include: a processor and a memory. The memory is used to store computer-executable program codes, and the program codes include instructions; when the processor executes the instructions, the instructions cause the electronic device to execute the methods in the first aspect and the second aspect.

In a sixth aspect, the embodiments of the present application provide a computer program product including instructions, which, when run on a computer, cause the computer to execute the methods in the first aspect and the second aspect above.

In the seventh aspect, the embodiment of the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned first aspect and the second aspect. method.

For the beneficial effects of the possible implementation manners of the above-mentioned second aspect to the seventh aspect, reference may be made to the beneficial effects brought by the above-mentioned first aspect, which will not be repeated here.

Description of drawings

FIG. 1 is a schematic diagram of a scene applicable to an embodiment of the present application;

2 is a schematic diagram of high-definition images at different shooting angles at a shooting position stored in the cloud in the prior art;

Fig. 3 is a schematic diagram of an image processing method in the prior art;

Fig. 4 is a schematic diagram of cloud storage image block and index relationship provided by the embodiment of the present application;

Fig. 5 is another schematic diagram of cloud storage image block and index relationship provided by the embodiment of the present application;

FIG. 6 is a schematic diagram of images of different viewing angles acquired by the cloud under the viewpoint provided by the embodiment of the present application;

FIG. 7 is a schematic flowchart of an embodiment of an image processing method provided in the embodiment of the present application;

FIG. 8 is a schematic diagram of a variation of the camera interface provided by the embodiment of the present application;

FIG. 9 is a schematic flowchart of an embodiment of an image processing method provided in an embodiment of the present application;

FIG. 10 is a schematic diagram of a panoramic image stored in the cloud provided by an embodiment of the present application;

FIG. 11 is a schematic flowchart of another embodiment of the image processing method provided by the embodiment of the present application;

FIG. 12 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

FIG. 13 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

Definitions of terms involved in the embodiments of this application:

Single point of view: The point of view can understand the position of the camera device (such as a mobile phone) when taking pictures, and a single point of view is a position.

Panoramic image stitching technology: it is to stitch multiple images into a large-scale image. Exemplarily, in this embodiment of the application, multiple high-definition images are spliced into one panoramic image. Here is a brief introduction to the principle of panoramic image stitching technology. Panoramic image stitching may include but not limited to 4 steps, the 4 steps are: detecting and extracting image features and key points, matching key points of two images, using random sampling The consensus algorithm (random sample consensus, RANSAC) estimates the homography matrix, and stitches the image.

In one embodiment, the specific implementation of the panoramic image stitching technology may include: using a scale-invariant feature transform (scale-invariant feature transform, SIFT) local description operator to detect key points and features (feature descriptors or SIFT features) in the image ), and matching feature descriptors between two images, that is, using features to match keypoints of two images. Then, the RANSAC algorithm is used to estimate the homography matrix (homography estimation) using the key points matched on the two images, that is, to match one of the images with the other image through correlation.

After estimating the homography matrix, perspective transformation (perspective transformation) can be used. For example, you can input the homography matrix, the image you want to distort, and the shape of the output image, and then use the height of the image by obtaining the sum of the widths of the two images For determining the derived shape of the output image, for details, reference may be made to relevant descriptions in the prior art of perspective transformation. Perspective transformation can be understood as: projecting an image to a new viewing plane. Perspective transformation is also called projective mapping or projection transformation.

Projection transformation: You can refer to the description of perspective transformation.

Back projection transformation: Projection transformation refers to the process of projecting an image to a new viewing plane, and back projection transformation refers to projecting the image of the new viewing plane to the original viewing plane of the image. It should be understood that in the process of projection transformation, the transformation relationship (such as transformation matrix) between the original viewing plane and the new viewing plane can be obtained, and the process of back-projection transformation adopts the "original viewing plane to new viewing plane The transformation relationship between ", project the image of the new viewing plane to the original viewing plane of the image.

In an embodiment, the back-projection transformation may be called back-projection mapping, inverse perspective transformation, or inverse perspective transformation. For details of the back-projection transformation, reference may be made to relevant descriptions in the prior art of back-projection transformation.

High-definition image: such as a high-definition image captured by a SLR camera, the definition of the high-definition image is greater than the preset definition. In one embodiment, if the resolutions of the images are the same, the higher the code rate is, the higher the resolution is. In this scenario, the preset code rate can be used to represent the preset resolution. The parameters representing clarity are not limited in the embodiments of the present application.

Photo magnification: Refers to the zoom magnification.

High magnification: The zoom magnification used when taking pictures is greater than the preset magnification. The preset magnification depends on the camera capability of the terminal, and the preset magnifications of different terminals may be the same or different. In one embodiment, the preset magnification may be 5, for example.

FIG. 1 is a schematic diagram of a scene applicable to an embodiment of the present application. Figure 1 compares the terminal with a mobile phone and a SLR camera, and uses both the mobile phone and the SLR camera to capture computer screens as an example. Referring to a in Figure 1, when the user uses a SLR camera to take pictures with a magnification of 30, he can get a high-definition image. For example, the user can clearly see the text "one two three four" on the computer screen in the image. When the user uses a mobile phone to take a photo with a magnification of 30 (that is, 30x in b in Figure 1), the resolution of the image obtained by taking the photo is low, and the user cannot clearly see the text on the computer screen, and can only see a few shadows block, as shown in b in Figure 1. It should be understood that, for the convenience of illustrating the images captured by the mobile phone and the SLR camera, the images captured by the mobile phone and the SLR camera at a magnification of 30 are shown on the right side of a in FIG. 1 and the right side of b in FIG. 1 respectively.

It should be understood that, in the embodiment of the present application, the user takes a photo with a high magnification, which may be understood as: the user takes a photo with a magnification greater than a preset magnification. For the preset magnification, refer to the related description in the above term explanation.

In order to improve the clarity of images obtained by the terminal using high-magnification photography, a large number of high-definition images can be stored in the cloud in advance in the prior art. High-definition images include: high-definition images taken at different shooting locations, and high-definition images taken at the same shooting location at different shooting angles. Exemplarily, FIG. 2 is a schematic diagram of high-definition images taken at different shooting angles at the shooting position of A stored in the cloud in the prior art. It should be understood that the black rectangle in FIG. illustrate. In one embodiment, in the prior art, the overlap rate of the frames of the high-definition images stored in the cloud is greater than or equal to the first overlap rate, such as 80%. In one embodiment, the shooting location may be called a viewpoint, and the shooting angle may be called a viewing angle. In other words, in the prior art, high-definition images taken from different viewpoints and high-definition images taken from different viewpoints at the same viewpoint are stored in the cloud.

Referring to Fig. 3, in the prior art, when the terminal uses a high magnification to take pictures, the captured image can be sent to the cloud, and the cloud obtains the similarity between each high-definition image stored in the cloud and the image from the terminal, and then the highest similarity High-definition images are fed back to the terminal. After the terminal receives the high-definition image from the cloud, it can display the high-definition image, and the user can see the high-definition image obtained by the terminal taking pictures with a high magnification. Exemplarily, in FIG. 3, the terminal sends the low-resolution image shown in b in FIG. 1 to the cloud, and the cloud can feed back a high-resolution image to the terminal, such as an image displaying the words "one two three four". Although the method in the prior art can enable the terminal to obtain high-definition images when taking pictures at high magnifications, the cloud needs to store a large number of high-definition images, occupying a large amount of storage space, and the storage cost of the cloud is large.

In an embodiment, the number of high-definition images stored in the cloud may be reduced, for example, storing high-definition images with an overlapping rate less than a second overlapping rate, such as 20%. In this way, because the number of high-definition images stored in the cloud is small, the shooting angle corresponding to the image at the same shooting position is reduced, so based on the method of comparing the similarity between each high-definition image stored in the cloud and the image from the terminal, the feedback to the terminal The shooting angles of high-definition images and the images captured by the actual terminal are different, causing users to look like images taken at different shooting angles. The feedback accuracy of high-definition images is low, resulting in poor user experience.

Based on the above problems, on the one hand, in the embodiment of the present application, panoramic images (or image blocks divided by panoramic images) under different single viewpoints (or viewpoints) can be stored in the cloud. For a single viewpoint, the high-definition images stored in the cloud Changing multiple high-definition images into one panoramic image (or multiple image blocks corresponding to one panoramic image) can reduce storage overhead on the cloud. On the other hand, in order to ensure that accurate high-definition images are fed back to the terminal, it is also necessary to store high-definition images from different shooting angles under the same viewpoint. The characteristics of high-definition images from different shooting angles can not only ensure the accuracy of feedback, but also reduce the storage overhead of the cloud.

In an embodiment, the terminal in the embodiment of the present application may be called a user equipment, and the terminal has a camera function and supports high-magnification camera. When the terminal in the embodiment of the present application uses a high magnification to take a picture, the definition of the image obtained by taking the picture is low. It should be understood that a high magnification can be understood as a photographing magnification greater than a preset magnification. Exemplarily, the terminal can be a mobile phone, a tablet computer (portable android device, PAD), a personal digital assistant (PDA), a handheld device with a wireless communication function, a computing device, or a wearable device. Reality (VR) terminal equipment, augmented reality (augmented reality, AR) terminal equipment, terminals in a smart home (smart home), etc., the form of the terminal is not specifically limited in the embodiment of the present application.

In an embodiment, the cloud may be a server, or a server cluster. Exemplarily, the server may be a server corresponding to a photographing application program, or a server corresponding to an application program carrying a photographing function, and the embodiment of the present application does not specifically limit the form of the cloud.

Before introducing the image processing method provided by the embodiment of this application, the content stored in the cloud will be described first:

In one embodiment, multiple image blocks corresponding to panoramic images under different viewpoints are stored in the cloud, wherein there is no overlap between the multiple image blocks corresponding to panoramic images under the same viewpoint or the overlapping ratio is less than the third overlapping ratio. Exemplarily, the third overlapping ratio may be a smaller value such as 20%, 10%, or the like. It should be understood that the panoramic image is a high-definition image, and multiple image blocks corresponding to the panoramic image are also high-definition image blocks.

In one embodiment, panoramic images from different viewpoints are stored in the cloud.

In the embodiment of the present application, for a viewpoint, because the cloud does not store multiple high-definition images of the viewpoint under the viewpoint, but stores a panoramic image under the viewpoint, or multiple image blocks corresponding to the panoramic image, it can reduce Storage overhead in the cloud. In the following embodiments, a viewpoint is taken as an example to introduce the image processing method provided in the embodiments of the present application.

In an embodiment, the panoramic images under different viewpoints in the embodiment of the present application may be obtained by shooting with a panoramic camera, or spliced from low-overlapping high-definition images under different viewpoints. Wherein, the overlap ratio between the low-overlap high-definition images is smaller than the second overlap ratio. In the following, the panoramic images under different viewpoints obtained by splicing low-overlapping high-definition images under different viewpoints are taken as an example for illustration.

Referring to Figure 4, the process of storing content in the cloud may include the following steps:

S401, the cloud uses panoramic image stitching technology to stitch low-overlap high-definition images under the same viewpoint to obtain panoramic images under different viewpoints.

In one embodiment, low-overlap high-definition images under the same viewpoint can be captured by a camera device capable of capturing high-definition images, such as a single-lens reflex camera. Exemplarily, for example, a single-lens reflex camera may be used in advance to capture high-definition images of different viewing angles at the same viewpoint, and then obtain high-definition images of different viewing angles at different viewpoints. In an embodiment, the high-definition images with low overlap at each viewpoint may be referred to as pre-collected images of multiple viewing angles at each viewpoint.

Wherein, the overlap rate between the high-definition images of adjacent viewing angles collected by the single-lens reflex camera at the same viewpoint is less than the second overlapping ratio, or the high-definition image can be selected from the high-definition images collected by the single-lens reflex camera under the same viewpoint, so that the adjacent viewing angles The overlapping rate between high-definition images is smaller than the second overlapping rate, so as to obtain low-overlapping high-definition images under the same viewpoint. Among them, the purpose of the overlap rate of the low-overlap high-definition image being smaller than the second overlap rate is to reduce the calculation amount of panoramic image stitching in the cloud and improve stitching efficiency. In one embodiment, it can also be said that the overlap rate between the high-definition images of adjacent viewing angles is less than a preset overlap rate, and the preset overlap rate is greater than or equal to the second overlap rate and less than the first overlap rate.

For low-overlap high-definition images at the same viewpoint, the cloud can use panoramic image stitching technology to obtain panoramic images at this viewpoint. According to the panoramic image stitching technology, the cloud can obtain panoramic images from different viewpoints. For the panoramic image stitching technology, please refer to the relevant description in the definition of terms.

Wherein, S401 is shown as S1 in FIG. 5 , and FIG. 5 is a simplified flow diagram of FIG. 4 .

FIG. 6 is a schematic diagram of the cloud acquiring images of different viewing angles under the same viewpoint provided by the embodiment of the present application. Referring to a in FIG. 6 , it is illustrated by taking two low-overlapping high-definition images at a viewpoint as an example. The cloud executes S401 to obtain a panoramic image at the viewpoint, as shown in b in FIG. 6 .

S402. The cloud segmentes the panoramic images under each viewpoint to obtain image blocks corresponding to the panoramic images under each viewpoint.

Taking a viewpoint as an example, the cloud can cut the panoramic image under the viewpoint into image blocks with a preset size to obtain multiple image blocks corresponding to the viewpoint. In one embodiment, the size of each image block is the same, such as 800px*900px, that is, each image block has a first preset size, and the first preset size can be understood as having a first preset width and a first preset size. a preset height. Among them, 1px represents a pixel. In one embodiment, the size of each image block can be different.

In an embodiment, there is no overlap between two adjacent image blocks corresponding to the same viewpoint, that is, they do not contain the same area. In an embodiment, the overlapping ratio between two adjacent image blocks corresponding to the same viewpoint may be smaller than the third overlapping ratio.

In one embodiment, after the cloud divides the panoramic image into image blocks, each image block may be numbered. Exemplarily, it can be numbered according to the row and column of the cut image block in the panoramic image. For example, if an image block is located in the first row and first column of the panoramic image, the image block can be numbered as row 1 and column 1 . Exemplarily, the image blocks can be numbered in the order of 1 to N, for example, the number of the image block in the first row and the first column is 1, the number of the image block in the first row and the second column is 2, and N is greater than 1 integer. In the embodiment of the present application, there is no limitation on the way of numbering the image blocks. In the following embodiments, the numbering of image blocks in rows and columns is used as an example for illustration. In an embodiment, the row, column number or "1-N" number of the image block may be referred to as the identifier of the image block.

In the embodiment of the present application, the purpose of cutting the panoramic images under each viewpoint into image blocks is to facilitate the cloud to load the image blocks, instead of directly loading the entire panoramic image directly from the cloud, because the loading time of the image blocks is shorter than that of the entire panoramic image. Therefore, the loading speed of the cloud can be improved, and the speed of the high-definition image feedback from the cloud to the terminal can be improved. For details, refer to the relevant description in FIG. 7 .

Wherein, S402 is shown as S2 in FIG. 5 .

Referring to FIG. 6 , for example, the cloud executes S402 to cut the panoramic image into 8 image blocks, as shown in c in FIG. 6 .

S403. The cloud adopts back-projection transformation to obtain images of multiple viewing angles corresponding to the panoramic image at each viewing point.

In S401, when the cloud adopts panoramic image mosaic technology to obtain panoramic images under different viewpoints, it can obtain the transformation relationship between the first viewing plane where the low-overlap high-definition image is located at each viewpoint and the second viewing plane where the panoramic image is located, and then the cloud Each part of the panoramic image can be projected onto the first viewing plane by using the transformation relationship at each viewpoint, so as to obtain images of different viewing angles.

In one embodiment, the cloud can sequentially project part of the panoramic images in the sliding window of the second preset size to the second viewing plane according to the order of the panoramic images from left to right and from top to bottom, so as to obtain multiple viewing angles Image. Wherein, the image of each viewing angle is a high-definition image, and the size of the image of each viewing angle is the same, that is, the second preset size, for example, the image of each viewing angle has a second preset width and a second preset height. In one embodiment, the overlapping rate between the images of adjacent viewing angles corresponding to the panoramic image under the same viewpoint is greater than the first overlapping rate, that is to say, the cloud control sliding window can be separated from the previous position of the sliding window every time it slides. The overlap rate between them is kept greater than the first overlap rate, so as to obtain images of adjacent viewing angles corresponding to the panoramic images at each viewing point.

In S403, during the back-projection transformation process, when the partial panoramic image in the sliding window of the second preset size is projected onto the second viewing plane, the distance between each pixel on the partial panoramic image and the image corresponding to the viewing angle can be obtained. The one-to-one mapping relationship of pixels, and in this process, the cloud can obtain the coordinate position of the center point of the image corresponding to the perspective in the panoramic image. In an embodiment, the coordinate position of the center point in the panoramic image may be a latitude and longitude coordinate.

It should be understood that the center point of the image can be understood as: the physical center point of the image.

It should be understood that there is no sequence distinction between S402 and S403, and the two can be executed simultaneously.

Wherein, S403 is shown as S3 in FIG. 5 .

Referring to FIG. 6 , for example, the cloud executes S403 to perform back-projection transformation on the panoramic image to obtain images of four viewing angles corresponding to the viewpoint, as shown in d in FIG. 6 .

S404. The cloud establishes a first index relationship between the center point of the image of each angle of view and the image block according to the coordinate position of the center point of the image of each angle of view under each viewpoint in the panoramic image.

The first index relationship can be understood as: which image blocks in the panoramic image correspond to the images of each viewing angle, that is, to establish an identification mapping relationship between images of each viewing angle and image blocks.

In one embodiment, because the image of each viewing angle has a second preset size, the image of each viewing angle can be acquired on the premise that the coordinate position of the center point of the image of each viewing angle in the panoramic image is known All corresponding image blocks. Exemplarily, based on the second preset size of the image of each viewing angle, and the coordinate position of the center point of the image of each viewing angle in the panoramic image, the cloud can determine the four vertices of the image of each viewing angle in the panoramic image The coordinate position in . Furthermore, for an image of a viewing angle, the cloud can determine the corresponding image block of the image of this viewing angle in the panoramic image according to the four vertices of the image of each viewing angle and the position coordinates of the center point in the panoramic image.

In an embodiment, the cloud may store the first index relationship. In the first index relationship, the image of each perspective can be represented by the coordinate position of the center point of the image of each perspective in the panoramic image, and the image block can be represented by the number of the image block, that is to say, in the first index relationship can Including: the mapping relationship between the coordinate position of the center point of the image of each viewing angle in the panoramic image and the number of the image block.

Exemplarily, as shown in FIG. 6, if the panoramic image corresponds to images of 4 viewing angles, the coordinate positions of the center points of the images of the 4 viewing angles in the panoramic image are (longitude 1, latitude 1), (longitude 2, latitude 2), (longitude 3, latitude 3), and (longitude 4, latitude 4). Among them, the image block corresponding to the image of (longitude 1, latitude 1) is: (row 1, column 1), (row 1, column 2), and the image block corresponding to the image of (longitude 2, latitude 2) is: (row 1. The image block corresponding to the image of column 3), (row 1, column 4), (longitude 3, latitude 3) is: (row 2, column 1), (row 2, column 2), (longitude 4, latitude 4) The image blocks corresponding to the image are: (row 2, column 3), (row 2, column 4). Accordingly, the first index relationship of cloud storage can be shown in Table 1:

Table I

各视角的图像的中心点在全景图像中的坐标位置The coordinate position of the center point of the image of each viewing angle in the panoramic image	图像块image blocks
(经1、纬1)(longitude 1, latitude 1)	(行1、列1)、(行1、列2)(row 1, column 1), (row 1, column 2)
(经2、纬2)(longitude 2, latitude 2)	(行1、列3)、(行1、列4)(row 1, column 3), (row 1, column 4)
(经3、纬3)(longitude 3, latitude 3)	(行2、列1)、(行2、列2)(row 2, column 1), (row 2, column 2)
(经4、纬4)(longitude 4, latitude 4)	(行2、列3)、(行2、列4)(row 2, column 3), (row 2, column 4)

Wherein, S404 is shown as S4 in FIG. 5 .

S405. The cloud acquires the feature of the image of each viewing angle under each viewing point, so as to establish a second index relationship between the feature of the image of each viewing angle and the coordinate position of the center point of the image of each viewing angle in the panoramic image.

For the images of each viewing angle under each viewing point, the cloud can obtain the features of the images of each viewing angle. In an embodiment, the feature of the image of each viewing angle is embodied as a feature vector, for example, the feature vector may be a 2048-dimensional feature vector. That is to say, the cloud can obtain feature vectors of images of each view angle under each view point.

In one embodiment, the cloud may use a neural network model to extract features of images of each viewing angle. Exemplarily, the neural network model may include but not limited to: convolutional neural networks (convolutional neural networks, CNN), recurrent neural networks (recurrent neural network, RNN) and long short-term memory (long short-term memory, LSTM).

In the embodiment of the present application, the cloud can acquire the features of the image of each viewing angle, and then can establish the features of the image of each viewing angle and the coordinate position of the center point of each viewing angle in the panoramic image according to the features of the image of each viewing angle The second index relation of . In one embodiment, the cloud can store a second index relationship, in the second index relationship, the center point of the image of each angle of view can be represented by the coordinate position of the center point of the image of each angle of view in the panoramic image, so as to The feature vector of the image of each viewing angle characterizes the features of the image of each viewing angle, that is, the second index relationship may include: the coordinate position of the center point of each viewing angle in the panoramic image and the features of the image of each viewing angle mapping relationship.

In an embodiment, the cloud can also obtain a third index relationship according to the first index relationship and the second index relationship. For example, the third index relationship is: the mapping relationship between the feature of the image of each viewing angle and the number of the image block. That is to say, the cloud can merge the first index relationship and the second index relationship based on the coordinate position of the center point of each viewing angle in the panoramic image, and combine the features of the center point with the same coordinate position and the image block The number is mapped to obtain the third index relationship.

It should be understood that there is no sequence distinction between S404 and S405, and the two can be executed simultaneously.

Wherein, S405 is shown as S5 in FIG. 5 .

To sum up, in one embodiment, multiple image blocks under each viewpoint, the first index relationship and the second index relationship may be stored in the cloud. Alternatively, in an embodiment, multiple image blocks and the third index relationship under each viewpoint may be stored in the cloud. In this way, compared with the prior art in which high-definition images of different viewing angles are stored in the cloud, storage overhead can be reduced.

Based on the relevant introduction of the content stored in the cloud, the image processing method provided by the embodiment of the present application will be described below in combination with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. FIG. 7 is a schematic flow chart of an embodiment of an image processing method provided in an embodiment of the present application. It should be understood that in FIG. 7, the perspective of interaction between the terminal and the cloud is taken as an example for illustration.

Referring to FIG. 7, the image processing method provided by the embodiment of the present application may include:

S701. The terminal responds to a photographing instruction to obtain a first image by photographing.

In one embodiment, the photographing instruction may be an instruction triggered by the user operating the photographing interface displayed on the terminal. For example, if the photographing interface includes a photographing control, the user's operation of the photographing control may trigger the input of the photographing instruction to the terminal. In one embodiment, the photographing instruction can be triggered by the user's voice, for example, the user says "photographing", which can trigger the input of the photographing instruction to the terminal. In one embodiment, the user can also trigger the input of a photographing instruction to the terminal in a customized manner or by operating other shortcut keys, and the embodiment of the present application does not limit the manner in which the user triggers the photographing instruction.

The terminal may take a photo to obtain the first image in response to the photo-taking instruction.

FIG. 8 is a schematic diagram of a variation of the camera interface provided by the embodiment of the present application. Shown in a in FIG. 8 is the photographing interface, which includes a preview frame 81 , a photographing control 82 and a magnification adjustment bar 83 . The user adjusts the magnification adjustment bar 83 to change the photographing magnification of the terminal. As an example, the user adjusts the photographing magnification to 30 as an example for illustration. The user clicks on the shooting control 82, and the corresponding terminal responds to the shooting instruction and can take a picture at a shooting magnification of 30 to obtain a first image.

S702. The terminal sends the first image to the cloud.

S703. The cloud acquires features of the first image.

For the manner in which the cloud acquires the features of the first image, reference may be made to the description in S405 that the cloud acquires the features of the images of each viewing angle under each viewpoint.

S704, the cloud obtains the similarity between the feature of the image of each viewing angle under each viewing point and the feature of the first image.

In one embodiment, the cloud can obtain the cosine angle or Euclidean distance between the image features of each viewing angle under each viewpoint and the feature of the first image, so as to obtain the features of the image of each viewing angle under each viewing point and the first The similarity of image features. Among them, the smaller the cosine angle and the smaller the Euclidean distance, the greater the representation similarity.

In one embodiment, in order to reduce the amount of similarity calculation in the cloud, when the terminal sends the first image to the cloud, it can upload the location of the terminal, that is, the above S702 can be replaced by: the terminal sends the first image and the location of the terminal to the cloud . Correspondingly, S704 may be replaced by: the cloud obtains the similarity between the feature of the image of each viewing angle and the feature of the first image at a viewpoint within a preset range from the position of the terminal. In an embodiment, a viewpoint within a preset range from the location of the terminal may be referred to as a target viewpoint.

In this embodiment, because the cloud stores the characteristics of images of various angles of view under different viewpoints, the cloud can first determine the viewpoints within a preset distance range of the terminal based on the location of the terminal, wherein the location of the terminal Viewpoints within the preset distance range of can be understood as target viewpoints. Furthermore, the terminal device can obtain the similarity between the feature of the image of each view angle under the target viewpoint and the feature of the first image, which can avoid calculating the similarity of features under all viewpoints, and can improve the computing efficiency of the cloud.

In an embodiment, no matter how much the camera magnification is used by the terminal, the terminal can execute S702-S708. In an embodiment, when the terminal sends the first image and the location of the terminal to the cloud, it may send the first magnification. Correspondingly, the cloud executes S703-S708 in response to the first magnification being greater than or equal to the preset magnification, and the cloud responds to the first magnification being smaller than the preset magnification, because the terminal itself can obtain the first high-definition image, so the cloud does not need to execute S703 -S708 to save computing resources in the cloud.

In one embodiment, because the terminal can obtain high-definition images with low magnification, there is no need for the terminal to interact with the cloud to obtain high-definition images. Therefore, in the scene where the terminal takes pictures at high magnification, the terminal can perform S702. In this embodiment, S701 may be replaced by: in response to the photographing instruction, photographing at a first magnification to obtain a first image, where the first magnification is greater than a preset magnification. Correspondingly, S702 may be replaced by: the terminal sends the first image and the location of the terminal to the cloud in response to the first magnification being greater than the preset magnification.

S705. The cloud determines the position coordinates of the center point of the feature map corresponding to the maximum similarity in the panoramic image according to the feature corresponding to the maximum similarity and the second index relationship.

The second index relationship is: a mapping relationship between the coordinate position of the center point of the image of each viewing angle in the panoramic image and the feature of the image. After the cloud obtains the similarity between the feature of the image of each viewing angle under each viewpoint and the feature of the first image, the maximum similarity can be determined, and then the feature corresponding to the maximum similarity can be determined. In one example, the cloud can determine the maximum similarity after obtaining the similarity between the features of the image of each viewing angle and the features of the first image at the viewpoint within the preset distance range of the location of the terminal, and then determine the maximum similarity corresponding features.

In this way, the cloud can obtain the position coordinates of the center point of the feature map corresponding to the maximum similarity in the panoramic image according to the stored second index relationship and the feature corresponding to the maximum similarity.

S706. The cloud determines the identity of the image block mapped by the center point according to the position coordinates of the center point of the feature map corresponding to the maximum similarity in the panoramic image and the first index relationship.

The first index relationship is: a mapping relationship between the coordinate position of the center point of the image of each viewing angle in the panoramic image and the image block. Therefore, after the position coordinates of the central point corresponding to the feature corresponding to the maximum similarity in the panoramic image are obtained in the cloud, the position coordinates of the central point in the panoramic image and the first index relationship can be used to obtain the position coordinates of the central point in the panoramic image. The identity of the image block mapped to the position coordinates of , that is, the identity of the image block of the feature map corresponding to the maximum similarity. In an embodiment, the identification of the image block of the feature map corresponding to the maximum similarity may be referred to as an object identification.

Exemplarily, if the position coordinates of the center point of the feature map corresponding to the maximum similarity in the panoramic image are (longitude 1, latitude 1), then based on Table 1 (the first index relationship), the (longitude 1, latitude 1) can be obtained 1) The numbers of the mapped image blocks are (row 1, column 1), (row 1, column 2).

In an embodiment, if the third index relationship is stored in the cloud, the third index relationship is: a mapping relationship between image features and image block numbers. In this embodiment, after the cloud obtains the feature corresponding to the maximum similarity, it can obtain the identifier of the image block of the feature map corresponding to the maximum similarity according to the third index relationship. Correspondingly, in this embodiment, S705 and S706 may be replaced by: the cloud determines the identity of the image block of the feature map corresponding to the maximum similarity according to the feature corresponding to the maximum similarity and the third index relationship.

S707. The cloud adopts back-projection transformation, and obtains the second image according to the image block mapped by the center point.

Wherein, the image block mapped by the center point is the image block corresponding to the target identifier. The cloud stores multiple image blocks corresponding to the panorama image at each viewpoint. After the target identifier of the image block mapped by the center point is determined in the cloud, the image blocks corresponding to the target identifier can be spliced, and then back-projection transformation is used to obtain the second image. Wherein, the definition of the second image is higher than that of the first image. In an embodiment, the resolution of the second image is greater than the preset resolution.

Wherein, the cloud can map the image block corresponding to the spliced center point to the first viewing plane according to the transformation relationship between the first viewing plane and the second viewing plane, so as to obtain the second image, which is the image of the viewing plane captured by the terminal .

In an embodiment, the cloud may stitch the image blocks mapped with the center point according to the numbers of the image blocks mapped with the center point. Exemplarily, if the number of image blocks mapped by (longitude 1, latitude 1) is (row 1, column 1), (row 1, column 2), then the cloud can be spliced in the order of rows and columns, and the splicing number is (row 1, column 1), (row 1, column 2) image blocks to obtain the second image.

In one embodiment, when there is an overlapping area between the image blocks, the cloud may cover the overlapping area in the image block (row 1, column 1) with the overlapping area in the image block (row 1, column 2) to Image blocks numbered (row 1, column 1), (row 1, column 2) are spliced. Among them, the cloud can determine the image block of (row 1, column 1) and the image block of (row 1, column 2) according to the similarity of the pixels in the image block of (row 1, column 1) and the image block of (row 1, column 2) ) in the image blocks, for example, an area with a similarity of 100% is used as an overlapping area.

S708. The cloud sends the second image to the terminal.

Correspondingly, the terminal receives the second image from the cloud.

S709. The terminal displays the second image in response to the image display instruction.

After receiving the second image from the cloud, the terminal may display the second image based on the user's operation. Alternatively, the terminal may display the second image after receiving the second image from the cloud.

In an embodiment, the image display instruction may be an instruction triggered by the user operating the camera interface. For example, the camera interface includes an image display control, and the user's operation of the image display control can trigger the input of an image display instruction to the terminal. In one embodiment, the image display instruction may also be triggered by the user's voice, and the embodiment of the present application does not limit the way the terminal receives the photographing instruction.

Exemplarily, referring to a in FIG. 8, after the user clicks the camera control 82, the terminal and the cloud interact to execute S701-S708, and after the terminal receives the second image from the cloud, the second image can be stored in the local image database ( such as a photo album). As shown in b in FIG. 8 , the camera interface includes an image display control 84 , and the user clicks on the image display control 84 , and the terminal can display a high-definition second image, as shown in c in FIG. 8 . The difference from b in Figure 1 above is that when the terminal uses the first magnification (the first magnification is a high magnification) to take pictures, the image obtained by taking pictures has high definition, for example, the user can clearly see the computer screen in the second image on the text.

In an embodiment, steps S701-S709 shown in FIG. 7 can be simplified as shown in FIG. 9 .

In the embodiment of the present application, when the terminal uses a high magnification to take pictures, it can send the first image captured to the cloud, and the cloud determines the maximum similarity based on the characteristics of the first image and the stored similarity of images from multiple perspectives at each viewpoint The corresponding image, and then based on the first index relationship and the second index relationship, the image block corresponding to the first image is obtained, and then the image block is spliced and back-projected to obtain the second image with high definition, so that the terminal can display the image with high The high-definition second image achieves the purpose that the terminal can obtain a high-definition image when taking pictures at a high magnification. On the other hand, it is also because the cloud stores multiple image blocks under each viewpoint, the first index relationship and the second index relationship, or the cloud stores multiple image blocks under each viewpoint and the third index relationship. In this way, compared with the way in the prior art that the cloud stores high-definition images of different viewing angles under various viewpoints, the storage cost of the cloud can be reduced.

In the embodiment shown in FIG. 7 , the image blocks corresponding to the panoramic images under each viewpoint, and the first index relationship and the second index relationship are stored in the cloud, or the image blocks corresponding to the panoramic images under each viewpoint are stored in the cloud, and Third index relationship. In an embodiment, panoramic images at various viewpoints may be stored in the cloud. Exemplarily, the panoramic images at various viewpoints stored in the cloud may be shown in FIG. 10 . It should be understood that in FIG. 10 , the panoramic images include different shapes (such as black rectangles, black triangles, etc.) to represent the panoramic images at different viewpoints.

In such an embodiment, according to the description in S701-S706 above, the cloud can determine the image block corresponding to the first image (that is, the number of the image block of the feature map corresponding to the maximum similarity). Then, according to the number of the image block corresponding to the first image, the cloud can cut out the image block corresponding to the number from the panoramic image under the viewpoint, and perform back projection transformation to obtain the second image. For example, the cloud may first load the panoramic image under the viewpoint, and then cut the image block corresponding to the number of the image block corresponding to the first image in the panoramic image, and perform projection transformation to obtain the second image.

Exemplarily, if the number of the image block corresponding to the first image is (row 1, column 1), (row 1, column 2), then the cloud can convert the panorama according to the image block number and the first preset size of the image block Image blocks numbered (row 1, column 1) and (row 1, column 2) in the image are cut out, and projectively transformed to obtain a second image.

Compared with the above-mentioned embodiment shown in FIG. 7, in the embodiment shown in FIG. 7, because the image block corresponding to the panoramic image is stored in the cloud, when the number of the image block corresponding to the first image is determined in the cloud, it can be directly loaded. The corresponding image blocks are spliced, back-projected, etc., and in the embodiment of the present application, because the cloud is stored as a panoramic image, after the cloud determines the number of the image block corresponding to the first image, it needs to first load the number of the image block. The panoramic image to which the number belongs, and then the corresponding image block is cut out in the panoramic image. However, the speed of loading image blocks in the cloud is much faster than the speed of loading the entire panoramic image. Therefore, in the embodiment shown in FIG. 7 , the loading efficiency of the cloud is high, and the second image can be fed back to the terminal faster.

In the image processing method provided by the embodiment of the present application, the panoramic images under each viewpoint can be stored in the cloud, and after the number of the image block corresponding to the first image is obtained in the cloud, the corresponding image block can be cut in the panoramic image to which the number belongs, and The projective transformation results in the second image. The image processing method provided by the embodiment of the present application can also achieve the purpose of obtaining high-definition images when the terminal uses high-magnification photography. In the embodiment of the present application, because the cloud still needs to load the entire panoramic image, and then cut the image blocks in the panoramic image, the loading time is long, the loading efficiency is low, and the efficiency of feeding back the second image to the terminal is relatively low.

In an embodiment, the terminal may store multiple image blocks under each viewpoint, the first index relationship and the second index relationship, or, the terminal may store multiple image blocks under each viewpoint and the third index relationship, or , the terminal can store panoramic images at various viewpoints, and when the terminal uses a high-magnification camera to obtain the first image, the terminal can execute S703-S707 to obtain a high-definition second image, and then the terminal can respond to the image display command, display second image.

In the above embodiment, the cloud interaction of the terminal is taken as an example to illustrate the scene where the cloud can process the image from the terminal, and the scene where the terminal can process the captured image. In summary, for an electronic device, for example, the electronic device may be a cloud, a terminal, or other devices with processing capabilities. Referring to Figure 11, the image processing method provided by the embodiment of the present application may also include:

S1101. Acquire a first image to be processed.

When the electronic device is a cloud, the way for the cloud to acquire the first image to be processed may be: the terminal sends the first image to the cloud after taking the first image, and reference may be made to related descriptions in S701-S702. In an embodiment, the user may also upload the first image to be processed to the cloud, or the first image is an image locally stored in the cloud.

When the electronic device is a terminal, the terminal may capture the first image, or the first image may be used as an image locally stored in the terminal.

When the electronic device is another device with processing capabilities, the device may capture the first image, or the user may upload the first image to the device, or the first image may be an image stored locally on the device, or the first image may be For images transmitted from other electronic devices.

In this embodiment of the present application, there is no limitation on the manner in which the electronic device acquires the first image to be processed.

S1102. Obtain the similarity between the feature of the image of each viewing angle under each viewpoint and the feature of the first image.

For the electronic device to execute the step of S1102, reference may be made to related descriptions in S703-S704.

S1103. According to the feature of the image of the viewing angle corresponding to the maximum similarity and the mapping relationship, determine the target identifier of the feature map of the image of the viewing angle corresponding to the maximum similarity.

In an embodiment, the mapping relationship may be a third index relationship. The third index relationship is: a mapping relationship between image features and image block numbers. In this embodiment, after the cloud obtains the feature corresponding to the maximum similarity, it can obtain the identifier of the image block of the feature map corresponding to the maximum similarity according to the third index relationship.

In an embodiment, the mapping relationship may include a first index relationship and a second index relationship. The first index relationship is: the coordinate position of the center point of the image of each viewing angle in the panoramic image and the mapping relationship of the image block, and the second index relationship is: the coordinate position and the coordinate position of the center point of the image of each viewing angle in the panoramic image The mapping relationship of image features. In this embodiment, the electronic device obtains the similarity between the feature of the image of each viewing angle under each viewpoint and the feature of the first image (or the similarity of the image of each viewing angle within the preset distance range of the position of the terminal) After the similarity between the feature and the feature of the first image), the maximum similarity can be determined, and then the feature corresponding to the maximum similarity can be determined. Further, the electronic device can obtain the position coordinates of the central point of the feature map corresponding to the maximum similarity in the panoramic image according to the stored second index relationship and the feature corresponding to the maximum similarity, and then according to the position coordinates of the central point in the panoramic image The position coordinates, and the first index relationship, obtain the identity of the image block mapped to the position coordinates of the central point in the panoramic image.

Wherein, the identifier of the image block mapped to the position coordinates of the central point in the panoramic image is the target identifier.

S1104. Acquire a second image according to the image block corresponding to the target identifier, where the definition of the second image is greater than that of the first image.

In an embodiment, the electronic device may stitch the image blocks corresponding to the target identifier to acquire the second image.

Or, in an embodiment, the electronic device may process the image block corresponding to the target identifier in the manner in S707, so as to acquire the second image.

After the electronic device obtains the second image, because the second image is acquired based on the corresponding image block under the viewpoint, the definition of the second image is higher than that of the first image, so the electronic device can realize the accuracy of the first image. processing to obtain a higher-resolution image.

In an embodiment, after the electronic device obtains the second image, the electronic device may store the second image, or transmit the second image to other electronic devices. This embodiment of the present application does not limit the post-processing of the second image. For the scene where the cloud and the terminal interact, the cloud can send the second image to the terminal for display and storage.

In the embodiment of the present application, the electronic device stores the image blocks corresponding to the panoramic images under each viewpoint, and the mapping relationship between the features of the images of each viewpoint under each viewpoint and the identifiers of the image blocks corresponding to the panoramic images under each viewpoint, and each viewpoint The images of each angle of view and the image blocks corresponding to the panoramic images of each viewpoint are obtained based on the panoramic images of each viewpoint, and the panoramic images of each viewpoint are high-definition images. In this way, the storage overhead can be reduced. On this basis, the electronic device can also process the first image with low definition to obtain the second image with higher definition.

FIG. 12 is a schematic structural diagram of an image processing device provided by an embodiment of the present application. The image processing apparatus may be the cloud, terminal, or electronic device in the above embodiments, or a chip in the cloud, or a chip in the terminal, or a chip in the electronic device, and is used to implement the image processing method provided in the embodiment of the present application. Wherein, the electronic device stores the image blocks corresponding to the panoramic images under each viewpoint, and the mapping relationship between the features of the images of each viewpoint under the viewpoints and the identifiers of the image blocks corresponding to the panoramic images under each viewpoint, the The images of each viewing angle at each viewpoint and the image blocks corresponding to the panoramic images at each viewpoint are obtained based on the panoramic images at each viewpoint.

Referring to FIG. 12 , an image processing device 1200 includes: a processing module 1201 , a storage module 1202 , and a transceiver module 1203 .

The processing module 1201 is configured to obtain the first image to be processed, and obtain the similarity between the features of the image of each viewing angle under the various viewpoints and the feature of the first image, and according to the image of the viewing angle corresponding to the maximum similarity features, and the mapping relationship, determine the target identifier of the feature map of the image corresponding to the maximum similarity, and obtain a second image according to the image block corresponding to the target identifier, and the clarity of the second image higher than the resolution of the first image.

In a possible implementation manner, the processing module 1201 is specifically configured to acquire the first image, and a location where the first image is taken, and determine a target viewpoint within a preset range from the location, and acquire the A degree of similarity between the feature of the image of each viewing angle under the target viewpoint and the feature of the first image.

The processing module 1201 is specifically configured to determine the center point of the feature map of the image of the viewing angle corresponding to the maximum similarity according to the feature of the image of the viewing angle corresponding to the maximum similarity and the second index relationship, and according to the The center point of the feature map of the image of the viewing angle corresponding to the maximum similarity and the first index relationship are used to determine the target identifier.

In a possible implementation manner, the processing module 1201 is further configured to acquire, according to the panoramic images under each viewpoint, the feature of the image of each viewpoint under each viewpoint, the first index relationship, and the second index relationship. Two-index relationship.

The storage module 1202 is configured to store the feature of the image of each viewing angle under each viewing point, the first index relationship, and the second index relationship.

In a possible implementation manner, the processing module 1201 is specifically configured to apply back-projection transformation to the panoramic images under each viewpoint to obtain images of multiple viewing angles under each viewpoint, and each viewing angle under each viewpoint The center point of the image is at the coordinate position of the corresponding panoramic image, and the overlapping ratio between images of adjacent viewing angles under each viewpoint is greater than a preset overlapping ratio; extracting the features of the image of each viewing angle under each viewpoint.

In a possible implementation manner, the processing module 1201 is specifically configured to use a sliding window with a second preset size to slide in the panoramic image in the panoramic image at each viewpoint, and use back-projection transformation, sequentially Obtain the image of the angle of view corresponding to the partial panoramic image in the sliding window and the center point of the image of the angle of view corresponding to the partial panoramic image in the coordinate position of the corresponding panoramic image, and the image of each angle of view under each viewpoint has the the second preset size.

In a possible implementation manner, the processing module 1201 is specifically configured to, according to the coordinate position of the center point of the image of each angle of view under each viewpoint in the corresponding panoramic image, and the image of each angle of view under each viewpoint feature to construct the second index relationship.

In a possible implementation manner, the processing module 1201 is specifically configured to cut the panoramic image under each viewpoint to obtain image blocks corresponding to the panoramic image under each viewpoint, and The center point of the image is at the coordinate position of the corresponding panoramic image, and the image blocks corresponding to the panoramic image under each viewpoint are used to construct the first index relationship.

In a possible implementation manner, the processing module 1201 is further configured to use the panoramic image stitching technology to obtain the panoramic images under each viewpoint according to the pre-collected images of multiple viewpoints under each viewpoint, and the The overlapping ratio between the pre-collected images of the lower adjacent viewing angles is smaller than the preset overlapping ratio.

In a possible implementation manner, the processing module 1201 is specifically configured to project the pre-acquired image of each view angle under each view point in the first viewing plane to the second viewing plane to which the panoramic image belongs, so as to obtain The panoramic image under each viewpoint, and the transformation relationship between the first viewing plane and the second viewing plane.

In a possible implementation manner, the processing module 1201 is specifically configured to, according to the transformation relationship, use back projection transformation to project the image block corresponding to the target identifier to the first viewing plane to obtain the second image .

In a possible implementation manner, the electronic device is a cloud, and the transceiver module 1203 is configured to receive the first image from the terminal and the location of the terminal when the terminal captures the first image, and send a message to the The terminal sends the second image.

The image processing device provided in the embodiment of the present application is used to execute the image processing method in the above embodiment, and has the same implementation principle and technical effect as the above embodiment.

In one embodiment, the embodiment of the present application also provides an electronic device. Referring to FIG. 13, the electronic device may be the cloud, terminal or the electronic device described in FIG. Including: a processor (eg CPU) 1301 and a memory 1302 . The memory 1302 may include a high-speed random-access memory (random-access memory, RAM), and may also include a non-volatile memory (non-volatile memory, NVM), such as at least one disk memory, and various instructions may be stored in the memory 1302 , so as to complete various processing functions and realize the method steps of the present application.

In one embodiment, the electronic device may include a screen 1303 for displaying an interface and images of the electronic device.

Optionally, the electronic device involved in this application may further include: a power supply 1304 , a communication bus 1305 and a communication port 1306 . The communication port 1306 is used to realize connection and communication between the electronic device and other peripheral devices. In this embodiment of the present application, the memory 1302 is used to store computer-executable program codes, and the program codes include instructions; when the processor executes the instructions, the instructions cause the processor of the electronic device to perform the actions in the above-mentioned method embodiments, and its implementation principles and The technical effects are similar, and will not be repeated here.

It should be noted that the modules or components described in the above embodiments may be one or more integrated circuits configured to implement the above method, for example: one or more application specific integrated circuits (ASIC), or , one or more microprocessors (digital signal processor, DSP), or, one or more field programmable gate arrays (field programmable gate array, FPGA), etc. For another example, when one of the above modules is implemented in the form of a processing element scheduler code, the processing element can be a general-purpose processor, such as a central processing unit (central processing unit, CPU) or other processors that can call program codes such as control device. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. A computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, a data center, etc. integrated with one or more available media. Available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)).

The term "plurality" herein means two or more. The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the character "/" in this paper generally indicates that the contextual objects are an "or" relationship; in the formula, the character "/" indicates that the contextual objects are a "division" relationship. In addition, it should be understood that in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or imply order.

It can be understood that the various numbers involved in the embodiments of the present application are only for convenience of description, and are not used to limit the scope of the embodiments of the present application.

It can be understood that, in the embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the order of execution of the processes should be determined by their functions and internal logic, and should not be used in the implementation of this application. The implementation of the examples constitutes no limitation.

Claims

An image processing method, which is characterized in that it is applied to an electronic device, and the electronic device stores image blocks corresponding to panoramic images under each viewpoint, and features of images of each viewing angle under each viewpoint and the features of each viewpoint The mapping relationship of the identification of the image block corresponding to the lower panoramic image, the image of each viewing angle under each viewpoint and the image block corresponding to the panoramic image under each viewpoint are obtained based on the panoramic image under each viewpoint, and the method includes :

Obtain the first image to be processed;

Obtaining the similarity between the feature of the image of each viewing angle under each viewpoint and the feature of the first image;

According to the feature of the image of the angle of view corresponding to the maximum similarity, and the mapping relationship, determine the target identifier of the feature map of the image of the angle of view corresponding to the maximum similarity;

A second image is acquired according to the image block corresponding to the target identifier, and the definition of the second image is higher than that of the first image.
The method according to claim 1, wherein said acquiring the first image to be processed comprises:

acquiring the first image, and a location where the first image was taken;

The acquiring the similarity between the feature of the image of each viewing angle under each viewpoint and the feature of the first image includes:

determining a target viewpoint within a preset range from the position;

Obtain the similarity between the feature of the image of each viewing angle under the target viewpoint and the feature of the first image.
The method according to claim 1 or 2, wherein the mapping relationship includes a first index relationship and a second index relationship, and the second index relationship is: the features of images of each view angle under each view point The mapping relationship with the center point of the image of each angle of view under each viewpoint, the first index relationship is: the center point of the image of each angle of view under each viewpoint and the image corresponding to the panoramic image under each viewpoint The mapping relationship of block identification;

According to the feature of the image of the angle of view corresponding to the maximum similarity, and the mapping relationship, determining the target identifier of the feature map of the image of the angle of view corresponding to the maximum similarity includes:

According to the feature of the image of the viewing angle corresponding to the maximum similarity and the second index relationship, determine the center point of the feature map of the image of the viewing angle corresponding to the maximum similarity;

The target identifier is determined according to the center point of the feature map of the image of the viewing angle corresponding to the maximum similarity and the first index relationship.
The method according to claim 3, wherein, before acquiring the first image to be processed, further comprising:

According to the panoramic images under the various viewpoints, acquire the features of the images of each viewpoint under the respective viewpoints, the first index relationship, and the second index relationship;

The feature of the image of each viewing angle under each viewing point, the first index relationship, and the second index relationship are stored.
The method according to claim 4, characterized in that, according to the panoramic image under each viewpoint, acquiring the features of the image of each viewpoint under each viewpoint includes:

Using back-projection transformation on the panoramic images under each viewpoint to obtain images of multiple viewing angles under each viewpoint, and the coordinate position of the center point of the image of each viewing angle under each viewpoint at the corresponding panoramic image, the The overlapping ratio between images of adjacent viewing angles under each viewpoint is greater than the preset overlapping ratio;

Extracting features of images of each viewing angle under each viewing point.
The method according to claim 5, wherein the panoramic images under each viewpoint are back-projected to obtain images of multiple angles of view under each viewpoint, and images of each angle of view under each viewpoint are obtained. The center point of the image is at the coordinate position of the corresponding panoramic image, including:

In the panoramic images at each viewpoint, a sliding window with a second preset size is used to slide in the panoramic image, and back-projection transformation is used to sequentially obtain the images and the corresponding viewing angles of the partial panoramic images in the sliding window. The center point of the image of the viewing angle corresponding to the partial panoramic image is at the coordinate position of the corresponding panoramic image, and the image of each viewing angle under each viewing point has the second preset size.
The method according to claim 5 or 6, wherein obtaining the second index relationship comprises:

The second index relationship is constructed according to the coordinate position of the center point of the image of each angle of view under the various viewpoints in the corresponding panoramic image, and the characteristics of the image of each angle of view under the various viewpoints.
The method according to any one of claims 5-7, wherein obtaining the first index relationship comprises:

Cutting the panoramic image under each viewpoint to obtain image blocks corresponding to the panoramic image under each viewpoint;

The first index relationship is constructed according to the coordinate position of the center point of the image of each viewing angle under each viewpoint in the corresponding panoramic image, and the corresponding image block of the panoramic image under each viewpoint.
The method according to claim 8, wherein the image blocks corresponding to the panoramic images under each viewpoint have a first preset size.
The method according to any one of claims 5-9, characterized in that, according to the panoramic images under each viewpoint, the feature of the image of each viewpoint under each viewpoint and the first index relationship are acquired , and before the second index relation, also include:

Using panoramic image stitching technology, according to the pre-collected images of multiple viewing angles under each viewpoint, the panoramic images under each viewpoint are obtained, and the overlapping ratio between the pre-collected images of adjacent viewing angles under each viewpoint is less than the specified The preset overlap rate described above.
The method according to claim 10, characterized in that, using the panoramic image stitching technology to obtain the panoramic images under the various viewpoints according to the pre-collected images of multiple viewing angles under the respective viewpoints, comprising:

projecting the pre-acquired images of each viewing angle under the viewpoints in the first viewing plane to the second viewing plane to which the panoramic image belongs, so as to obtain the panoramic images under the respective viewpoints, and the first viewing plane and The transformation relationship of the second viewing plane.
The method according to claim 11, wherein said obtaining the second image according to the image block corresponding to the target identifier comprises:

According to the transformation relationship, the image block corresponding to the target identifier is projected to the first viewing plane by using back projection transformation to obtain the second image.
The method according to claim 1, wherein the electronic device is a cloud, and the acquiring the first image and the location where the first image is taken include:

receiving a first image from a terminal and a location of the terminal when the terminal captured the first image;

After said acquiring the second image, it also includes:

sending the second image to the terminal.
An image processing device, characterized in that the electronic device stores image blocks corresponding to panoramic images under each viewpoint, and features of images of each angle of view under each viewpoint and image blocks corresponding to panoramic images under each viewpoint The mapping relationship of the identification, the image of each angle of view under each viewpoint and the image block corresponding to the panoramic image under each viewpoint are all obtained based on the panoramic image under each viewpoint, and the device includes:

Processing modules for:

Obtain the first image to be processed;

Obtaining the similarity between the feature of the image of each viewing angle under each viewpoint and the feature of the first image;

According to the feature of the image of the angle of view corresponding to the maximum similarity, and the mapping relationship, determine the target identifier of the feature map of the image of the angle of view corresponding to the maximum similarity;

A second image is acquired according to the image block corresponding to the target identifier, and the definition of the second image is higher than that of the first image.
An electronic device, characterized in that it includes: a processor and a memory;

the memory stores computer-executable instructions;

The processor executes the computer-implemented instructions stored in the memory, causing the processor to perform the method according to any one of claims 1-13.
A computer-readable storage medium, characterized in that computer programs or instructions are stored in the computer-readable storage medium, and when the computer programs or instructions are run, the computer program or instructions described in any one of claims 1-13 can be realized. described method.
A computer program product, characterized in that it includes a computer program or instruction, and when the computer program or instruction is executed by a processor, the method according to any one of claims 1-13 is realized.