CN114972008A

CN114972008A - Coordinate restoration method and device and related equipment

Info

Publication number: CN114972008A
Application number: CN202111299242.1A
Authority: CN
Inventors: 曾鹏源
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-11-04
Filing date: 2021-11-04
Publication date: 2022-08-30

Abstract

The method obtains and stores transformation parameters of each preprocessing, wherein the transformation parameters can be anchor point coordinates, affine transformation matrixes, transmission transformation matrixes or linear transformation parameters and offset parameters, then obtains the corresponding relation between the coordinates of pixel points in the image subjected to the last preprocessing and the coordinates of corresponding pixel points in the original image through calculation, and further can restore the coordinates of the pixel points in the image subjected to the preprocessing into the coordinates of the original image. Therefore, different coordinate restoration modules do not need to be formulated according to different geometric transformation combinations, and time and labor are saved.

Description

Coordinate restoration method and device and related equipment

Technical Field

The invention relates to the field of computer vision, in particular to a coordinate restoration method, a coordinate restoration device and related equipment.

Background

The computer vision is to carry out model inference after preprocessing the original image to obtain an inference result, finally, mark the model inference result on the original image and output the marked image, or screen the image meeting the conditions according to the model inference result. For example, when computer vision is applied to photographing translation, characters in a photographed image are not horizontal or not clear, the image needs to be preprocessed, including rotation, clipping, image enhancement and the like, to obtain a preprocessed image beneficial to model inference, then the characters are recognized and translated by an inference model, and finally a translated result is superimposed on an original image and output to a user. In the preprocessing process, after the image is subjected to geometric transformation, the coordinates of pixel points in the image can be changed, the inferred result needs to be marked on the original image, the coordinates need to be restored, and the coordinates of the preprocessed image are restored to the coordinates corresponding to the original image.

However, the combination of the geometric transformation in the computer vision preprocessing stage may be different each time, and different coordinate restoration methods need to be formulated each time for the combination of different geometric transformation modes, which consumes human resources and time. Therefore, how to reduce the manpower resources and time consumed by the coordinate restoration is an urgent problem to be solved.

Disclosure of Invention

The application provides a coordinate restoring method, a coordinate restoring device and related equipment.

In a first aspect, the present application provides a coordinate restoring method, including: acquiring and storing transformation parameters, wherein the transformation parameters are used for reflecting the transformation relation between the first image and the second image; and determining the coordinates of pixel points corresponding to part or all of the pixel points in the second image in the first image according to the transformation parameters.

By acquiring and storing the transformation parameters for converting the first image into the second image, the coordinate corresponding relation between the second image and the first image can be determined, and further, the coordinates of pixel points corresponding to part or all of the pixel points in the second image in the first image can be determined. Therefore, different coordinate reduction methods are not needed to be customized manually according to the combination of different geometric transformation modes, and the manpower and time for reducing the coordinates are saved.

With reference to the first aspect, in some implementations, the transformation parameter includes a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the plurality of first pixel points in the first image, and the plurality of second pixel points are obtained according to the rotation amount, the scaling amount, and the translation amount of the second image converted from the plurality of first pixel points and the first image.

The transformation parameter may be coordinates of a plurality of anchor points in the first image and the second image, and the anchor point may be any pixel point in the first image. When the first image is a two-dimensional image, under the condition that the geometrical transformation of converting the first image into the second image comprises transmission transformation, the number of anchor points is more than or equal to 4; in the case where the transmission transform is not included in the geometric transform for converting the first image into the second image, the number of anchor points is greater than or equal to 3. If the dimension of the first image is higher, the number of corresponding anchor points needing to acquire coordinates is increased.

With reference to the first aspect, in some implementations, obtaining and saving the transformation parameters includes: obtaining and storing respective transformation parameters of multiple image transformations, wherein the multiple image transformations are used for transforming a first image into a second image for multiple times, and the respective transformation parameters of the multiple image transformations comprise a plurality of pixel points corresponding to a plurality of first pixel points in the first image in each image transformation.

The first image may be a second image obtained through multiple image transformations, and the coordinates corresponding to the anchor point during each image transformation need to be acquired and stored.

With reference to the first aspect, in some implementations, after obtaining and saving the transformation parameters, the method further includes: determining a transformation matrix according to a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the plurality of first pixel points in the first image, wherein elements in the transformation matrix are used for expressing the rotation amount, the scaling amount and the translation amount of the second image converted into the first image, and the transformation matrix comprises any one of a transmission transformation matrix and an affine transformation matrix.

When the first image is a two-dimensional image and the transmission transformation is included in the geometric transformation for converting the first image into the second image, the number of elements representing the rotation amount and the scaling amount in the transmission transformation matrix is 6 in total, and the number of elements representing the translation amount in the transmission transformation matrix is 3. In the case where the transmission transform is not included in the geometric transform of the first image into the second image, the transform parameter is an affine transform matrix in which the number of elements representing the rotation amount and the zoom amount is 4 in total and the number of elements representing the translation amount is 2. If the dimensionality of the first image is higher, the number of the corresponding anchor point coordinates, the linear transformation parameters and the offset parameters is correspondingly increased, and the number of elements in the transmission transformation matrix and the affine transformation matrix is also correspondingly increased.

With reference to the first aspect, in some implementations, obtaining and saving the transformation parameters includes: and acquiring and storing a plurality of third pixel points in the first image and a plurality of fourth pixel points in the second image corresponding to the plurality of third pixel points in the first image, wherein the third pixel points are determined according to the image area which is interested by a user in the first image, and the fourth pixel points are determined according to the third pixel points, the transformation parameters and the size of the second image.

Besides obtaining the corresponding coordinates of the first pixel points after image transformation, the coordinates of the vertexes of the interested areas of the user after image transformation at each time can be stored, so that the user can pay more convenience to pay attention to the interested areas in the images after image transformation at each time.

With reference to the first aspect, in some implementations, the transformation parameters include a linear transformation parameter for representing an amount of scaling and rotation of the first image into the second image and an offset parameter for representing an amount of translation of the first image into the second image; alternatively, the transformation parameters comprise a transformation matrix.

With reference to the first aspect, in some implementations, the obtaining and saving the transformation parameter includes: acquiring and storing respective transformation parameters of a plurality of times of image transformation, wherein the plurality of times of image transformation is used for transforming a first image into a second image for a plurality of times; determining the coordinates of pixel points corresponding to part or all of the pixel points in the second image in the first image according to the transformation parameters, wherein the coordinate determination method comprises the following steps: and determining the coordinates of partial or all pixel points in the second image in the first image based on the respective transformation parameters of the multiple times of image transformation.

The first image and the second image are obtained through multiple times of geometric transformation, so that transformation parameters during each time of geometric transformation need to be acquired, and the coordinates of the second image can be restored to the coordinates corresponding to the first image through the transformation parameters of the multiple times of image transformation.

With reference to the first aspect, in some implementations, determining coordinates of some or all pixel points in the second image in the first image based on respective transformation parameters of a plurality of image transformations includes: integrating transformation parameters of the multiple image transformations into summary parameters, wherein the image transformations corresponding to the summary parameters are used for transforming the first image into the second image for one time; and determining the coordinates of part or all of the pixel points in the second image in the first image based on the summarizing parameters.

The transformation parameters of the multiple image transformations can be linear transformation parameters and bias parameters, and the linear transformation parameters and the bias parameters of each geometric transformation are integrated to obtain summary parameters capable of directly converting the first image into the second image.

With reference to the first aspect, in some implementation manners, determining coordinates of some or all of the pixel points in the second image in the first image based on the summary parameter includes: determining a reduction parameter according to the summary parameter, wherein the reduction parameter is a parameter of inverse image transformation of the image transformation corresponding to the summary parameter; and determining the coordinates of part or all pixel points in the second image in the first image based on the restoration parameters.

And converting the first image which can be determined by the summary parameters into a transformation function of a second image, and determining an inverse function of the transformation function, wherein the parameters of the inverse function of the transformation function are recovery parameters, and the coordinates in the second image can be recovered to the coordinates corresponding to the coordinates in the first image by the inverse function.

In a second aspect, the present application provides a coordinate restoring apparatus, including an obtaining unit, a determining unit: the acquisition unit is used for acquiring and storing transformation parameters, and the transformation parameters are used for reflecting the transformation relation between the first image and the second image; the determining unit is used for determining the coordinates of pixel points corresponding to part or all of the pixel points in the second image in the first image according to the transformation parameters.

With reference to the second aspect, in some implementations, the transformation parameter includes a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the plurality of first pixel points in the first image, and the plurality of second pixel points are obtained according to the rotation amount, the scaling amount, and the translation amount of the second image converted from the plurality of first pixel points and the first image.

With reference to the second aspect, in some implementations, the determining unit is further configured to acquire and store a transformation parameter of each of multiple image transformations, where the multiple image transformations are used to transform the first image into the second image multiple times, and the transformation parameter of each of the multiple image transformations includes a plurality of pixel points corresponding to a plurality of first pixel points in the first image in each image transformation.

With reference to the second aspect, in some implementations, the determining unit is further configured to determine a transformation matrix according to a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the plurality of first pixel points in the first image, where elements in the transformation matrix are used to represent a rotation amount, a scaling amount, and a translation amount of the second image converted into the first image, and the transformation matrix includes any one of a transmission transformation matrix and an affine transformation matrix.

In some implementations, in combination with the second aspect, the obtaining unit is further configured to obtain and store a plurality of third pixel points in the first image and a plurality of fourth pixel points in the second image corresponding to the plurality of third pixel points in the first image, where the third pixel points are determined according to an image region of the first image in which the user is interested, and the fourth pixel points are determined according to the third pixel points, the transformation parameter, and the size of the second image.

With reference to the second aspect, in some implementations, the transformation parameters include a linear transformation parameter for representing an amount of scaling and an amount of rotation of the first image to the second image and an offset parameter for representing an amount of translation of the first image to the second image; alternatively, the transformation parameters comprise a transformation matrix.

With reference to the second aspect, in some implementations, the obtaining unit is further configured to obtain and store transformation parameters of each of a plurality of image transformations, where the plurality of image transformations are used to transform the first image into the second image a plurality of times; the determining unit is further used for determining the coordinates of part or all of pixel points in the second image in the first image based on the respective transformation parameters of the plurality of times of image transformation.

With reference to the second aspect, in some implementations, the determining unit is further configured to integrate respective transformation parameters of the multiple image transformations into an aggregation parameter, where an image transformation corresponding to the aggregation parameter is used to transform the first image into the second image through one time transformation; the determining unit is further used for determining the coordinates of part or all of the pixel points in the second image in the first image based on the summarizing parameters.

With reference to the second aspect, in some implementations, the determining unit is further configured to determine a reduction parameter according to the summary parameter, where the reduction parameter is a parameter of inverse image transformation of the image transformation corresponding to the summary parameter; the determining unit is further used for determining the coordinates of part or all of the pixel points in the second image in the first image based on the restoration parameters.

In a third aspect, the present application provides a computing device comprising a processor and a memory, the memory being configured to store instructions, the processor being configured to execute the instructions, and the method as described in the first aspect being performed when the processor executes the instructions.

In a fourth aspect, the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computing device, the method as described in the first aspect is performed.

In a fifth aspect, the present application provides a computer program product, characterized in that the computer program product comprises computer instructions which, when executed by a computing device, the computing device performs the method as described in the first aspect.

In summary, the coordinate restoring method, the coordinate restoring device, and the related device provided in the embodiments of the present application restore the coordinates of the pixel points in the preprocessed image to the coordinates of the original image by obtaining and storing the transformation parameters of each preprocessing, and then calculating to obtain the corresponding relationship between the coordinates of the pixel points in the preprocessed image at the last time and the coordinates of the corresponding pixel points in the original image. Therefore, different coordinate reduction modules do not need to be made according to different geometric transformation combinations, and time and labor for reducing the coordinates are saved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below.

FIG. 1 is a schematic diagram of a computer vision system according to an embodiment of the present disclosure;

FIG. 2 is a schematic view of a perspective transformed image provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of another computer vision system according to an embodiment of the present application;

fig. 4A to fig. 4E are schematic diagrams illustrating a preprocessing process in the field of face recognition according to an embodiment of the present application;

5A-5C are schematic diagrams of image coordinate transformation during preprocessing provided by embodiments of the present application;

fig. 6 is a schematic structural diagram of a coordinate restoring system according to an embodiment of the present disclosure;

FIGS. 7A-7C are schematic diagrams of a region of interest parameter provided by an embodiment of the present application;

fig. 8 is a schematic flowchart of a coordinate reduction method provided in an embodiment of the present application;

fig. 9 is a schematic structural diagram of a coordinate restoring apparatus according to an embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

First, a description will be given of a computer vision field to which an embodiment of the present application relates.

Computer Vision (CV) refers to the recognition, tracking and measurement of targets in images by electronic equipment instead of human eyes, and can be applied to scenes such as target recognition and detection, semantic segmentation, motion and tracking, visual question answering and action recognition and the like. As shown in fig. 1, the computer vision system 100 specifically includes a preprocessing unit 110, a coordinate selecting unit 120, and a post-processing unit 130.

The preprocessing unit 110 is configured to preprocess the acquired original image and generate a preprocessed image. The preprocessing can reduce irrelevant information in the original image, enhance useful information in the original image, for example, eliminate noise of the original image, weaken background, strengthen the target in the original image, and the like, and is beneficial for the coordinate selecting unit 120 to identify the target in the preprocessed image.

The preprocessing unit 110 may perform preprocessing on the acquired original image, including any one of or a combination of graying, geometric transformation, and image enhancement. The graying is to represent each pixel value in the original image by a gray value, because the three channels of RGB need to be processed when the color image is processed, the data amount to be processed is large, and the graying of the image is to represent each pixel by only one gray value, which can reduce the data amount when the other subsequent units are processed. Geometric transformations are used to process the acquired raw image into a uniform size, shape and vector space. The geometric transformation may use a processing method including linear transformation, affine transformation, perspective transformation, and the like, where the linear transformation specifically further includes: crop, zoom, mirror, rotate, miscut, and the like. Image enhancement is used to improve the quality of the original image and to enhance useful information in the original image, such as to sharpen unclear portions of the original image, or to enlarge features between different objects in the original image, to suppress features of objects that are not of interest, and so on.

Taking the original image as a two-dimensional image as an example, affine transformation in geometric transformation is a process of converting the original image from one two-dimensional coordinate system to another two-dimensional coordinate system, and only rotation and translation occur in the process. The transmission transformation in the geometric transformation refers to a special case of projecting an image to a new viewing plane, namely affine transformation. As shown in fig. 2, image (a) in fig. 2 is an image before perspective conversion, and image (B) is an image after perspective conversion of image (a). The coordinates of the pixel points in the image before perspective transformation are represented by (u, v), the coordinates of the corresponding pixel points in the image after perspective transformation are (x, y), and the perspective transformation involves conversion between coordinate systems, so that the coordinates of the pixel points in the image before perspective transformation can be represented by three-dimensional coordinates (u, v, w) in the process of performing perspective transformation, and the coordinates of the corresponding pixel points in the image after perspective transformation are also represented by three-dimensional coordinates (x, y, z), and the principle of transmission transformation can refer to the following formula (1):

wherein x is x '/w', y is y '/w',

transform the matrix for transmission, an

Indicates the amount of rotation (a) ₁₃ a ₂₃ ) ^T For indicating the amount of translation.

It should be understood that, when the original image is a two-dimensional image, w is equal to 1, and the calculation modes of x and y can be obtained after transformation by the above formula (1), and the calculation modes of x and y can refer to the following formula (2) and formula (3):

in order to determine the value of each element in the transmission transformation matrix, it is necessary to determine four pixel points in the image as anchor points, obtain coordinates of the four anchor points before and after transmission transformation, and substitute the coordinates of the four anchor points before and after transmission transformation into formula (1), or formula (2) and formula (3), so that the transmission transformation matrix can be obtained by solving.

In some embodiments, if openCV software is used, coordinates of four anchor points before and after transmission transformation may be directly input through an interface of the software, so as to obtain a transmission transformation matrix, where a function of obtaining a matrix parameter may be getperspective transform (points1, points2), where points1 is a coordinate array of the four anchor points in the image before perspective transformation, and points2 is a coordinate array of the corresponding four anchor points in the image after perspective transformation.

Because the affine transformation matrix only rotates and translates, the affine transformation matrix can be used

To illustrate, for ease of computation, the affine transformation can also be expressed as

Similarly, in order to determine the value of each element in the affine transformation matrix, three pixel points need to be determined as anchor points in the image, and the coordinates of the three anchor points before and after the affine transformation are obtained, so that the affine transformation matrix can be obtained by solving.

From affine transformation matrices

The coordinates of the anchor point in the image before affine transformation are represented by (u, v), and the coordinates of the anchor point in the image after perspective transformation are (x, y), so x and y can be represented by u and v, and the specific representation method can refer to the following formula (4) (5):

x＝a ₁₁ u+a ₂₁ v+a ₃₁ (4)

y＝a ₁₂ u+a ₂₂ v+a ₃₂ (5)

wherein, a ₁₁ 、a ₁₂ 、a ₁₂ And a ₂₂ Also referred to as linear transformation parameters and also as bias parameters. The above equations (4) and (5) may be used to represent the correspondence between the coordinates before and after the affine transformation, or may be used to represent the correspondence between the coordinates before and after the linear transformation.

It should be understood that the transmission transformation matrix, the affine transformation matrix, the linear transformation parameters, and the offset parameters are all exemplified by the image being a two-dimensional image. In some embodiments, the image processed by the preprocessing unit 110 may also be three-dimensional, when the image is three-dimensional, the transmission transformation matrix is a matrix with 4 rows and 4 columns, the affine transformation matrix is a matrix with 3 rows and 4 columns, correspondingly, the linear transformation parameters are nine, and the offset parameters are three.

The coordinate selecting unit 120 is configured to perform coordinate selection according to the preprocessed image, and includes: a user selects an interested area in the image; alternatively, the inference model is used to select the target object in the image. In some embodiments, after the coordinate selecting unit 120 selects the coordinates, the inference model may be further used to perform functions such as target detection, image classification, semantic segmentation, instance segmentation, optical character recognition, target tracking, and the like. It should be understood that the inference model used by the inference unit of the present application model may be any image processing model, and the inference model used by the coordinate selection unit 120 is not particularly limited in the present application.

The post-processing unit 130 is configured to further process the image according to the inferred result to obtain an image processing result. The image processing performed by the post-processing unit 130 may be to filter the images according to the inference result of each image, and select the image containing the preset inference result; or, the inferred result is marked on the original image according to the inferred result of each image, and so on. It should be understood that the image processing performed by the post-processing unit 130 is not particularly limited by the present application.

In some embodiments, the operations performed by the preprocessing unit 110, the coordinate selection unit 120, and the post-processing unit 130 in the computer vision system 100 may be repeated multiple times, wherein each time the operations are performed, the specific operations in each unit may be different from the operations performed last time. That is, as shown in fig. 3, after the post-processing unit 130 obtains the image processing result, the image processing result is input to the pre-processing unit 110 for pre-processing, the pre-processing operation of this time may be different from the specific operation of the pre-processing of the previous time, then the coordinate selecting unit 120 obtains the inference result of the image processing result, and finally the post-processing unit 130 performs processing until the preset processing result is satisfied.

For example, when the computer vision system 100 is applied to the field of face recognition, as shown in fig. 4A, the electronic device may acquire an image including a face through a camera, where the camera may be a camera on a terminal such as a mobile phone and a notebook, or a camera of a card punch, an access control system, and the like. The electronic device first preprocesses fig. 4A through the preprocessing unit 110 of the computer vision system 100, cuts fig. 4A to a preset size, and performs image enhancement on the cut image to obtain fig. 4B; the coordinate selecting unit 120 identifies the image in fig. 4B by using the target detection model, and identifies that the detection frame 1 shown in fig. 4C is a human face, and the other images except the human face are backgrounds; the post-processing unit 130 marks and stores the face in the detection frame 1. Further, the post-processing unit 130 inputs the marked image shown in fig. 4C to the pre-processing unit 110, and the pre-processing unit 110 cuts the image in the detection frame 1 to obtain a cut image shown in fig. 4D; then, the coordinate selecting unit 120 extracts local features of the face in fig. 4D, for example, Haar-like features (Haar features for short) are extracted, and the Haar features may reflect gray level change conditions of the face, so as to reflect structural features of the face, for example, some features of the face can be described by gray level change conditions, such as: the eyes are darker than the cheeks, the sides of the bridge of the nose are darker than the bridge of the nose, the mouth is darker than the surroundings, etc. As shown in fig. 4E, the coordinate selecting unit 120 identifies that the inside of the detection frame 2 is an eye, the inside of the detection frame 3 is a nose, and the inside of the detection frame 4 is a mouth; finally, the post-processing unit 130 marks and stores the local features of the human face in the detection frame 2, the detection frame 3, and the detection frame 4, and can be used for other processing units to compare the stored data with the human face information stored in the database, retrieve the user information corresponding to the human face, and output the retrieved result.

In summary, when the computer vision system 100 processes the acquired image, the preprocessed image may be subjected to multiple geometric transformations, including clipping, scaling, mirroring, rotation, miscut, affine transformation, perspective transformation, and the like, and the post-inference result may be obtained by performing model inference on the preprocessed image.

However, in the computer vision system 100, it is often necessary to mark the inferred result on the original image, that is, the post-processing unit 130 also marks the inferred result on the original image, for example, when the computer vision system 100 is applied to the field of photo translation, if the sentences in the image are likely to be distributed in different areas of the image, the pre-processing unit 110 will divide the image into different areas, recognize and translate the sentences in each area, and then mark the translated result on the original image. Alternatively, as shown in fig. 4A to 4E, the results recognized by the detection boxes 2, 3, and 4 in fig. 4E are labeled in fig. 4A. When the electronic device marks an image, a coordinate system is established by using a fixed position of the image as an origin, as shown in fig. 5A, the coordinate system is established by using a lower left corner of the image as the origin, and the coordinates of the original image are obtained according to the coordinate system shown in fig. 5B, at this time, if the detection frame 2 ', the detection frame 3', and the detection frame 4 'are to be found in fig. 5B, and the detection frame 2', the detection frame 3 ', and the detection frame 4' are marked in the original image, the coordinates in the preprocessed image need to be restored to the coordinates in the original image, and the result of restoring the coordinates of the preprocessed image is shown in fig. 5C.

However, the preprocessing unit 110 may perform preprocessing each time, the geometric transformation used for different images or application scenes may be different, and there may be even a combination of different geometric transformations, such as cropping, scaling, and the like. Aiming at the combination of different geometric transformation modes, different coordinate reduction methods need to be formulated each time, and human resources and time are consumed.

In order to solve the problem that human resources and time are consumed when coordinate reduction is performed on combinations of different geometric transformation modes in different scenes, the present application provides a coordinate reduction system 200, as shown in fig. 6, where the coordinate reduction system 200 is applied to a computer vision system 100, and includes a coordinate parameter storage unit 210 and a coordinate reduction calculation unit 220. The coordinate parameter storage unit 210 is configured to store parameter information of geometric transformation in the preprocessing unit 110, and the coordinate restoration calculation unit 220 is configured to calculate a coordinate corresponding relationship between the original image and the preprocessed image according to the parameter information of geometric transformation. Since the coordinate transformation of the image is not involved in the coordinate selecting unit 120 and the post-processing unit 130, the coordinate restoration calculating unit 220 may calculate the coordinate correspondence between the original image and the post-processing unit image.

It should be understood that, if the computer vision system 100 is configured as shown in fig. 3, that is, the preprocessing unit 110 performs the preprocessing operation for multiple times, each time of preprocessing, the coordinate parameter storage unit 210 in the coordinate restoration system 200 acquires the parameter information of the geometric transformation in each preprocessing, and the coordinate restoration calculation unit 220 calculates the coordinate correspondence between the original image before the first preprocessing and the image after the last preprocessing, that is, the coordinate correspondence between the original image before the first preprocessing and the image after the last preprocessing.

The following describes the coordinate parameter storage unit 210 and the coordinate restoration calculation unit 220 in the coordinate restoration system 200 according to the present application in detail.

The coordinate parameter storage unit 210 is configured to obtain and store a transformation parameter of the preprocessing from the preprocessing unit 110, where the transformation parameter may be an anchor coordinate, and the anchor may be any pixel point in the image, including a coordinate of an anchor in the original image and a coordinate of an anchor corresponding to the last preprocessing. The coordinate parameter storage unit 210 is further configured to obtain whether the preprocessing operation in the preprocessing unit 110 includes transmission transformation, where, taking a two-dimensional image as an example, when the preprocessing includes transmission transformation, coordinates of at least four anchor points before and after preprocessing need to be obtained; when the pre-processing does not include the transmission transformation, the coordinates of at least three anchor points before and after the pre-processing need to be acquired.

In some embodiments, the coordinate parameter saving unit 210 is further configured to save coordinates of pixel points determined by a region of interest (ROI), and the coordinate parameter saving unit 210 is further configured to save coordinates of four vertices of the ROI, so that the user can pay attention to a coordinate change of the region of interest after each image transformation in each preprocessing after each image transformation. Wherein, a region in the original image is a region which is interested by a user, the region can be marked by the user or marked by an inference model, and the ROI is a convex quadrangle generally. After image transformation, the coordinates of the region will also change, and in some cases, the ROI region of the original image is cropped in the image transformation, and the region of the ROI will also be correspondingly reduced to the cropped region, that is, after each image transformation, the coordinates of the ROI are determined according to the transformation parameters and the size of the transformed image.

As shown in FIG. 7A, ROI ₁₀ 、ROI ₁₁ 、ROI ₁₂ And ROI ₁₃ An ROI in the image is determined. FIG. 7B is the cropped image, ROI, of the image shown in FIG. 7A ₁₀ 、ROI ₁₁ 、ROI ₁₂ And ROI ₁₃ Corresponding to ROI in FIG. 7B ₂₀ 、ROI ₂₁ 、ROI ₂₂ And ROI ₂₃ According to ROI ₂₀ 、ROI ₂₁ 、ROI ₂₂ And ROI ₂₃ The ROI of fig. 7B can be determined. FIG. 7C is the cropped image, ROI, of the image shown in FIG. 7B ₂₀ 、ROI ₂₁ 、ROI ₂₂ And ROI ₂₃ The corresponding coordinates in FIG. 7C are ROI' ₃₀ 、ROI' ₃₁ 、ROI' ₃₂ And ROI' ₃₃ The area defined by the four points described above, however, is beyond the image, it being understood that,one circle beyond the image range is an area which is not concerned by the user, and the ROI is changed into the ROI according to the current size of the image ₃₀ 、ROI ₃₁ 、ROI ₃₂ And ROI ₃₃ A determined area.

In some embodiments, the transformation parameters acquired by the coordinate parameter holding unit 210 may also be an affine transformation matrix or a transmission transformation matrix. The affine transformation matrix or the transmission transformation matrix is obtained according to the anchor point coordinates in the original image and the anchor point coordinates of the image to be preprocessed at the last time, when the preprocessing comprises the transmission transformation, the transmission transformation matrix needs to be obtained, and when the preprocessing does not comprise the transmission transformation, the affine transformation matrix can be obtained only.

In some embodiments, when the transmission transform is not included in the preprocessing, the transform parameter acquired by the coordinate parameter holding unit 210 may also be a linear transform parameter and a bias parameter for each preprocessing. The linear transformation parameter is used for representing the rotation amount, and the offset parameter represents the translation amount.

The coordinate restoring calculation unit 220 is configured to generate a restoring parameter according to the transformation parameter of each preprocessing, where the restoring parameter is used to restore the coordinates of the preprocessed image to the coordinates before preprocessing.

When the obtained transformation parameters are coordinates of an anchor point in the original image and coordinates of an anchor point in the last preprocessing, if the preprocessing includes transmission transformation, the coordinate reduction calculation unit 220 calculates a transmission transformation matrix according to the coordinates of the anchor point, and then the post-processing unit 130 may substitute the coordinates of the preprocessed image and the affine transformation matrix into formula (1), and convert the coordinates of the preprocessed image into corresponding coordinates in the original image. Similarly, if the pre-processing does not include the transmission transformation, the coordinate restoration calculation unit 220 calculates an affine transformation matrix according to the anchor coordinates, and then the post-processing unit 130 converts the coordinates of the pre-processed image into the corresponding coordinates of the original image.

In some embodiments, when the obtained transformation parameter is an affine transformation matrix or a transmission transformation matrix, the coordinates in the preprocessed image are restored to the corresponding coordinates in the original image according to formula (1) and the coordinates of the preprocessed image directly according to the affine transformation matrix or the transmission transformation matrix.

In some embodiments, when the transformation parameters acquired by the coordinate parameter storage unit 210 are linear transformation parameters and bias parameters during each preprocessing, if geometric transformation of N preprocessing is performed in total, the coordinate reduction calculation unit 220 integrates the linear transformation parameters and the bias parameters during each preprocessing to obtain linear transformation parameters and bias parameters for N preprocessing, the integrated linear transformation parameters and bias parameters are collectively referred to as summary parameters, then determines a transformation function according to the summary parameters, and obtains a coordinate reduction formula by taking an inverse function of the transformation function, where the parameters in the coordinate reduction formula are reduction parameters.

The linear transformation parameter and the offset parameter in each preprocessing refer to formula (4) and formula (5), formula (4) and formula (5) can also be expressed in a vector form, and taking the first preprocessing as an example, the vector expression of the linear transformation parameter and the offset parameter in each preprocessing is exemplified, and specifically, the following formula (6) may be referred to:

wherein, the first and the second end of the pipe are connected with each other,

representing the coordinates of the anchor point after the first preprocessing,

coordinates representing the anchor point before the first preprocessing, A ₁ Representing linear transformation parameters, b ₁ Denotes a bias parameter, it being understood that A ₁ Is based on a first pretreatment ₁₁ 、a ₁₂ 、a ₁₂ And a ₂₂ Obtained by (b) ₁ Is based on a first pretreatment ₃₁ And a ₃₂ And (4) obtaining the product.

After linear transformation parameters and bias parameters of each preprocessing are obtained, linear transformation parameters and bias parameters of N integrated pretreatments are obtained, which may specifically refer to the following formula (7):

wherein the content of the first and second substances,

coordinates representing the anchor point after the last pre-processing, A _N Is based on the linear transformation parameter { A } of each of N pre-processes ₁ ，A ₂ ，…，A _n Obtained of b _N Is based on the bias parameter b for each of the N pretreatments ₁ ，b ₂ ，…，b _N Obtained, i.e. A _N And b _N Are summarized parameters.

And (3) according to the integrated summary parameters and the formula (7), making an inverse function to obtain a coordinate reduction formula, specifically referring to the following formula (8):

by preprocessing the coordinates in the image for the last time according to the above equation (8)

And b _N And then the coordinates of the pixel points in the last preprocessed image in the original image can be obtained

To sum up, the coordinate restoring system 200 provided in the embodiment of the present application obtains and stores the transformation parameters of each preprocessing, and then calculates the correspondence between the coordinates of the pixel points in the image after the last preprocessing and the coordinates of the corresponding pixel points in the original image, thereby restoring the coordinates of the pixel points in the image after the preprocessing to the coordinates of the original image. Therefore, different coordinate restoration modules do not need to be formulated according to different geometric transformation combinations, and time and labor are saved.

In order to solve the problem that the combination of different geometric transformation modes consumes manpower resources and time in different scenes, the application provides a coordinate restoration method, which can restore the coordinates of the preprocessed image into the coordinates of the original image by acquiring and storing transformation parameters of each geometric transformation, wherein the transformation parameters can be anchor coordinates, affine transformation matrixes, transmission transformation matrixes or linear transformation parameters and offset parameters, then calculating the corresponding relation between the coordinates of the pixels in the preprocessed image at the last time and the coordinates of the corresponding pixels in the original image.

As shown in fig. 8, the coordinate restoring method provided in the present application may include the following steps:

and S810, acquiring and storing the transformation parameters for transforming the first image into the second image.

The first image is an original image before preprocessing, and the second image is an image of the first image after one or more times of preprocessing and geometric transformation. The geometric transformation that the first image undergoes when being converted into the second image may be a linear transformation, an affine transformation, a perspective transformation, or the like, as well as any combination of the above geometric transformations. The transformation parameters are used for reflecting the transformation relation between the first image and the second image, and the transformation parameters can be anchor point coordinates, a transmission transformation matrix, an affine transformation matrix, linear transformation parameters and offset parameters.

The following describes in detail the transformation parameters acquired and saved in different cases.

When the geometric transformation of the first image into the second image comprises a perspective transformation, the transformation parameters may be coordinates of an anchor point in the first image and coordinates of a corresponding anchor point in the second image, or a transmission transformation matrix for transforming the first image into the second image.

When the geometric transformation of the first image into the second image does not include the perspective transformation, the transformation parameters may be coordinates of an anchor point in the first image and coordinates of a corresponding anchor point in the second image; or, the first image is converted into an affine transformation matrix of the second image; alternatively, the first image is converted into linear transformation parameters and bias parameters for each geometric transformation in the second image.

In some embodiments, if the first image is a two-dimensional image, when the geometric transformation of the first image into the second image comprises a perspective transformation, at least the coordinates of the four anchor points on the first image and the second image need to be obtained, and the transmission transformation matrix is a 3 × 3 matrix. When the geometric transformation for converting the first image into the second image does not include perspective transformation, at least the coordinates of the three anchor points on the first image and the second image need to be acquired, the affine transformation matrix is a 2 x 3 matrix, the number of linear transformation parameters is 4, and the number of offset parameters is 2. If the dimension of the first image is higher, the number of the corresponding anchor point coordinates, the linear transformation parameters and the offset parameters is correspondingly increased, and the number of elements in the transmission transformation matrix and the affine transformation matrix is also correspondingly increased. It should be understood that the present application does not specifically limit the dimensions of the first image.

For the specific content of the transformation parameters, reference may be made to the related description in the coordinate parameter storage unit 210, and details are not repeated here.

And S820, determining the coordinate corresponding relation between the second image and the first image according to the transformation parameters.

Specifically, the coordinate correspondence may be an affine transformation matrix, a transmission transformation matrix, or a coordinate reduction formula obtained according to a linear transformation parameter and a bias parameter.

The coordinate correspondence determined in different cases will be described in detail below.

When the geometric transformation for converting the first image into the second image includes perspective transformation, if the transformation parameters are coordinates of an anchor point in the first image and coordinates of a corresponding anchor point in the second image, a transmission transformation matrix is obtained according to the anchor point coordinates, and the transmission transformation matrix is used for representing a coordinate corresponding relationship between the second image and the first image, which may specifically refer to the formula (1) and the related description thereof, and is not repeated herein.

When the geometric transformation for converting the first image into the second image does not include perspective transformation, if the transformation parameters are the coordinates of the anchor point in the first image and the coordinates of the corresponding anchor point in the second image, an affine transformation matrix is obtained according to the anchor point coordinates, and the affine transformation matrix is used for representing the coordinate corresponding relation between the first image and the second image. If the conversion parameters are linear conversion parameters and bias parameters of each geometric conversion, firstly, the summary parameters are integrated according to the linear conversion parameters and the bias parameters of each geometric conversion, and the summary parameters are the linear conversion parameters and the bias parameters of the first image directly converted into the second image. The transformation function for converting the first image into the second image can be determined by the summary parameter. The inverse function of the transformation function may be used to obtain a coordinate reduction formula for transforming the coordinates of the second image into the coordinates of the first image, where the parameters in the coordinate reduction formula are reduction parameters, and reference may be made to the above formula (6) -formula (8) and related description thereof, which are not described herein again.

And S830, restoring the coordinates of the second image into the coordinates of the first image according to the coordinate corresponding relation.

Specifically, if the coordinate correspondence is a transmission transformation matrix, the coordinates of the pixel points in the second image are substituted into the relationship shown in formula (1), and the coordinates of the pixel points in the second image corresponding to the coordinates in the first image can be calculated, so that the coordinates of the second image are restored to the coordinates of the first image.

Similarly, if the coordinate correspondence relationship is an affine transformation matrix, the coordinates of the pixel points in the second image corresponding to the coordinates in the first image can also be calculated, and the coordinates of the second image can be restored to the coordinates of the first image.

If the coordinate corresponding relation is a coordinate reduction formula, the coordinates of the pixel points in the second image are brought into the relation shown in the formula (8), the coordinates of the pixel points in the second image corresponding to the first image can be obtained through calculation, and then the coordinates of the second image are reduced into the coordinates of the first image.

In summary, the coordinate restoring method provided in the embodiment of the present application obtains and stores the transformation parameters of each preprocessing, and then calculates the correspondence between the coordinates of the pixel points in the image after the last preprocessing and the coordinates of the corresponding pixel points in the original image, so as to restore the coordinates of the pixel points in the image after the preprocessing to the coordinates of the original image. Therefore, different coordinate restoration modules do not need to be formulated according to different geometric transformation combinations, and time and labor are saved.

In order to solve the problem that the combination of different geometric transformation modes consumes manpower resources and time in different scenes, the application provides a coordinate restoring device 900, which can restore the coordinates of the preprocessed image to the coordinates of the original image by acquiring and storing transformation parameters of each geometric transformation, wherein the transformation parameters can be anchor coordinates, affine transformation matrixes, transmission transformation matrixes or linear transformation parameters and offset parameters, and then calculating the corresponding relation between the coordinates of the pixels in the preprocessed image at the last time and the coordinates of the corresponding pixels in the original image. As shown in fig. 9, the coordinate restoring apparatus 900 may include an obtaining unit 910 and a determining unit 920.

The obtaining unit 910 is configured to obtain and store a transformation parameter, where the transformation parameter is used to reflect a transformation relationship between the first image and the second image.

The determining unit 920 is configured to determine, according to the transformation parameter, coordinates of pixel points in the first image corresponding to part or all of the pixel points in the second image.

In some embodiments, the transformation parameter includes a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the plurality of first pixel points in the first image, and the plurality of second pixel points are obtained according to the plurality of first pixel points and a rotation amount, a scaling amount, and a translation amount of the second image converted from the first image.

In some embodiments, the obtaining unit 910 obtains and stores a transformation parameter of each of a plurality of image transformations, where the plurality of image transformations are used to transform the first image into the second image a plurality of times, and the transformation parameter of each of the plurality of image transformations includes a plurality of pixel points corresponding to a plurality of first pixel points in the first image in each image transformation.

In some embodiments, the determining unit 920 determines a transformation matrix according to a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the plurality of first pixel points in the first image, where elements in the transformation matrix are used to represent a rotation amount, a scaling amount, and a translation amount of the second image converted into the first image, and the transformation matrix includes any one of a transmission transformation matrix and an affine transformation matrix.

In some embodiments, the obtaining unit 910 is further configured to obtain and store a plurality of third pixel points in the first image and a plurality of fourth pixel points in the second image corresponding to the plurality of third pixel points in the first image, where the third pixel points are determined according to an image region of the first image in which a user is interested, and the fourth pixel points are determined according to the third pixel points, the transformation parameter, and the size of the second image.

In some embodiments, the transformation parameters include a linear transformation parameter for representing an amount of scaling and rotation of the first image into the second image and an offset parameter for representing an amount of translation of the first image into the second image; alternatively, the transformation parameters comprise a transformation matrix.

In some embodiments, the obtaining unit 910 is further configured to obtain and store transformation parameters of a plurality of image transformations, wherein the plurality of image transformations are used to transform the first image into the second image a plurality of times; the determining unit 920 is further configured to determine coordinates of some or all pixel points in the second image in the first image based on respective transformation parameters of the plurality of image transformations.

In some embodiments, the determining unit 920 is further configured to integrate transformation parameters of each of the multiple image transformations into an aggregation parameter, where the image transformation corresponding to the aggregation parameter is used to transform the first image into the second image through one transformation; the determining unit 920 is further configured to determine coordinates of some or all of the pixel points in the second image in the first image based on the summary parameter.

In some embodiments, the determining unit 920 is further configured to determine a reduction parameter according to the summarizing parameter, where the reduction parameter is a parameter of an inverse image transformation of the image transformation corresponding to the summarizing parameter; the determining unit 920 is further configured to determine coordinates of some or all pixel points in the second image in the first image based on the restoration parameters.

In some embodiments, when the first image is a two-dimensional image, the number of anchor points is greater than or equal to 4 in the case where the geometric transformation of the first image into the second image comprises a transmission transformation; in the case where the transmission transform is not included in the geometric transform for converting the first image into the second image, the number of anchor points is greater than or equal to 3.

In some embodiments, when the first image is a two-dimensional image, in a case where the transmission transform is not included in the geometric transform of the first image into the second image, the transform parameters are linear transform parameters and offset parameters, the number of the linear transform parameters is 4, and the number of the offset parameters is 2.

In some embodiments, when the first image is a two-dimensional image, in a case where the geometric transformation for converting the first image into the second image includes a transmission transformation, the transformation parameter is a transmission transformation matrix, the number of elements representing the rotation amount and the scaling amount in the transmission transformation matrix is 6 in total, and the number of elements representing the translation amount in the transmission transformation matrix is 3.

In some embodiments, in the case where the transmission transform is not included in the geometric transform of the first image into the second image, the transform parameter is an affine transform matrix in which the number of elements representing the rotation amount and the zoom amount is 4 in total and the number of elements representing the translation amount is 2.

In some embodiments, if the dimension of the first image is higher, the number of corresponding anchor point coordinates, linear transformation parameters, and bias parameters correspondingly increases, and the number of elements in the transmission transformation matrix and the affine transformation matrix also correspondingly increases. It should be understood that the present application does not specifically limit the dimensions of the first image.

To sum up, the coordinate restoring apparatus 900 provided in this embodiment of the present application obtains and stores the transformation parameters of each preprocessing, and then calculates the correspondence between the coordinates of the pixel points in the image after the last preprocessing and the coordinates of the corresponding pixel points in the original image, thereby restoring the coordinates of the pixel points in the image after the preprocessing to the coordinates of the original image. Therefore, different coordinate restoration modules do not need to be formulated according to different geometric transformation combinations, and time and labor are saved.

The method of the embodiment of the present application is explained in detail above, and in order to facilitate better implementation of the above-mentioned solution of the embodiment of the present application, correspondingly, the following also provides related equipment for implementing the above-mentioned solution cooperatively.

Fig. 10 is a schematic structural diagram of a computing device 1000 provided in this application, where the computing device 1000 may be the coordinate restoring apparatus 900 in the foregoing. As shown in fig. 10, computing device 1000 includes: a processor 1010, a communication interface 1020, and a memory 1030. The processor 1010, the communication interface 1020, and the memory 1030 may be connected to each other via an internal bus 1040, or may communicate with each other via other means such as wireless transmission. In the embodiment of the present application, for example, the bus 1040 may be a Peripheral Component Interconnect Express (PCIe) bus, or an Extended Industry Standard Architecture (EISA) bus, a unified bus (UBs or UBs), a computer Express link (CXL), a cache coherent Interconnect protocol (CCIX), and the like. The bus 1040 may be divided into an address bus, a data bus, a control bus, and the like. The bus 1040 may include a power bus, a control bus, a status signal bus, and the like, in addition to a data bus. But for clarity of illustration the various busses are labeled in the drawings as busses 1040.

Processor 1010 may be comprised of at least one general-purpose processor, such as a Central Processing Unit (CPU), or a combination of a CPU and hardware chips. The hardware chip may be an Application-Specific Integrated Circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a Field-Programmable Gate Array (FPGA), General Array Logic (GAL), or any combination thereof. Processor 1010 executes various types of digitally stored instructions, such as software or firmware programs stored in memory 1030, which enable computing device 1000 to provide a variety of services.

The memory 1030 is used for storing program codes and is controlled by the processor 1010 to execute the processing steps of the coordinate restoring method in the above embodiment. The program code may include one or more software modules, and the one or more software modules may be software modules provided in the embodiment of fig. 9, such as the obtaining unit 910, the determining unit 920: the obtaining unit 910 is configured to obtain and store a transformation parameter, where the transformation parameter is used to reflect a transformation relationship between the first image and the second image. The determining unit 920 is configured to determine, according to the transformation parameter, coordinates of pixel points in the first image corresponding to part or all of the pixel points in the second image.

It should be noted that the present embodiment may be implemented by a general physical server, for example, an ARM server or an X86 server, or may also be implemented by a virtual machine implemented based on the general physical server and combining with the NFV technology, where the virtual machine refers to a complete computer system that has a complete hardware system function and is run in a completely isolated environment through software simulation, and the present application is not limited in particular.

Memory 1030 may include Volatile Memory (Volatile Memory), such as Random Access Memory (RAM); the Memory 1030 may also include a Non-Volatile Memory (Non-Volatile Memory), such as a Read-Only Memory (ROM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, HDD), or a Solid-State Drive (SSD); memory 1030 may also include combinations of the above. The memory 1030 may store program codes for performing steps S810 to S830 and optional steps thereof in the embodiment of fig. 8, which are not described herein again.

The communication interface 1020 may be a wired interface (e.g., an ethernet interface), an internal interface (e.g., a Peripheral Component Interconnect express (PCIe) bus interface), a wired interface (e.g., an ethernet interface), or a wireless interface (e.g., a cellular network interface or a wireless lan interface), for communicating with other devices or modules.

It should be noted that fig. 10 is only one possible implementation manner of the embodiment of the present application, and in practical applications, the computing device 1000 may also include more or less components, which is not limited herein. For the content that is not shown or described in the embodiment of the present application, reference may be made to the related explanation in the embodiment of fig. 8, which is not described herein again.

It should be understood that the computing device shown in fig. 10 may also be a computer cluster of at least one server, and the application is not particularly limited.

Embodiments of the present application also provide a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is executed on a processor, the method flow shown in fig. 8 is implemented.

Embodiments of the present application also provide a computer program product, and when the computer program product is run on a processor, the method flow shown in fig. 8 is implemented.

The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded or executed on a computer, cause the flow or functions according to embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in, or transmitted from, a computer-readable storage medium to another computer-readable storage medium, for example, from one website, computer, server, or data center, over a wired (e.g., coaxial cable, fiber optics, Digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.) network, the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more collections of available media, which may be magnetic media (e.g., floppy disks, hard disks, magnetic tape), optical media (e.g., high density Digital Video Disc, DVD), or semiconductor media. The semiconductor medium may be an SSD.

While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A coordinate restoration method, comprising:

acquiring and storing transformation parameters, wherein the transformation parameters are used for reflecting the transformation relation between the first image and the second image;

and determining the coordinates of pixel points corresponding to part or all of the pixel points in the second image in the first image according to the transformation parameters.

2. The method of claim 1, wherein obtaining and saving transformation parameters comprises:

the transformation parameters comprise a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the first pixel points in the first image, and the second pixel points are obtained according to the rotation amount, the zooming amount and the translation amount of the second image converted from the first pixel points and the first image.

3. The method of claim 2,

the obtaining and saving of the transformation parameters comprises: obtaining and storing respective transformation parameters of a plurality of times of image transformation, wherein the plurality of times of image transformation is used for transforming the first image into the second image for a plurality of times, and the respective transformation parameters of the plurality of times of image transformation comprise a plurality of corresponding pixel points of a plurality of first pixel points in the first image in each time of image transformation.

4. The method according to any of claims 2-3, further comprising, after said obtaining and saving transform parameters:

determining the transformation matrix according to a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the plurality of first pixel points in the first image, wherein elements in the transformation matrix are used for representing rotation amount, scaling amount and translation amount of the second image converted into the first image, and the transformation matrix comprises any one of a transmission transformation matrix and an affine transformation matrix.

5. The method of claim 4, wherein obtaining and saving transformation parameters comprises:

and acquiring and storing a plurality of third pixel points in the first image and a plurality of fourth pixel points in the second image corresponding to the plurality of third pixel points in the first image, wherein the third pixel points are determined according to an image area which is interested by a user in the first image, and the fourth pixel points are determined according to the third pixel points, the transformation parameters and the size of the second image.

6. The method of claim 1,

the transformation parameters comprise a linear transformation parameter and an offset parameter, the linear transformation parameter is used for representing the amount of scaling and rotation of the first image converted into the second image, and the offset parameter is used for representing the amount of translation of the first image converted into the second image; alternatively, the first and second electrodes may be,

the transformation parameters include a transformation matrix.

7. The method of claim 6,

the obtaining and saving of the transformation parameters comprises: acquiring and storing respective transformation parameters of a plurality of times of image transformation, wherein the plurality of times of image transformation is used for transforming the first image into the second image for a plurality of times;

the determining, according to the transformation parameter, coordinates of pixel points corresponding to part or all of the pixel points in the second image in the first image includes:

and determining the coordinates of partial or all pixel points in the second image in the first image based on the respective transformation parameters of the multiple times of image transformation.

8. The method of claim 7, wherein the determining coordinates of some or all of the pixels in the second image in the first image based on the transformation parameters of the plurality of image transformations comprises:

integrating transformation parameters of the multiple image transformations into summary parameters, wherein the image transformations corresponding to the summary parameters are used for transforming the first image into the second image for one time;

and determining the coordinates of part or all of the pixel points in the second image in the first image based on the summarizing parameters.

9. The method of claim 8, wherein the determining coordinates of some or all of the pixels in the second image in the first image based on the aggregated parameters comprises:

determining a reduction parameter according to the summary parameter, wherein the reduction parameter is a parameter of inverse image transformation of image transformation corresponding to the summary parameter;

and determining the coordinates of part or all of pixel points in the second image in the first image based on the restoration parameters.

10. A coordinate restoration device is characterized by comprising an acquisition unit and a determination unit:

the acquisition unit is used for acquiring and storing transformation parameters, and the transformation parameters are used for reflecting the transformation relation between the first image and the second image;

the determining unit is used for determining the coordinates of pixel points corresponding to part or all of the pixel points in the second image in the first image according to the transformation parameters.

11. The apparatus of claim 10,

12. The apparatus of claim 11,

the determining unit is further configured to obtain and store respective transformation parameters of multiple image transformations, where the multiple image transformations are used to transform the first image into the second image multiple times, and the respective transformation parameters of the multiple image transformations include multiple pixel points corresponding to multiple first pixel points in the first image in each image transformation.

13. The apparatus according to any one of claims 11-12,

the determining unit is further configured to determine the transformation matrix according to a plurality of first pixel points in the first image and a plurality of second pixel points in the second image corresponding to the plurality of first pixel points in the first image, wherein elements in the transformation matrix are used to represent a rotation amount, a scaling amount, and a translation amount of the second image converted into the first image, and the transformation matrix includes any one of a transmission transformation matrix and an affine transformation matrix.

14. The apparatus of claim 13,

and acquiring and storing a plurality of third pixel points in the first image and a plurality of fourth pixel points in the second image corresponding to the plurality of third pixel points in the first image, wherein the third pixel points are determined according to the image area which is interested by the user in the first image, and the fourth pixel points are determined according to the third pixel points, the transformation parameters and the size of the second image.

15. The apparatus of claim 10,

the transformation parameters include a transformation matrix.

16. The apparatus of claim 15,

the obtaining unit is further configured to obtain and store transformation parameters of a plurality of image transformations, where the plurality of image transformations are used to transform the first image into the second image a plurality of times;

the determining unit is further configured to determine coordinates of some or all pixel points in the second image in the first image based on respective transformation parameters of the plurality of image transformations.

17. The apparatus of claim 16,

the determining unit is further configured to integrate respective transformation parameters of the multiple image transformations into a summary parameter, where the image transformation corresponding to the summary parameter is used to transform the first image into the second image through one-time transformation;

the determining unit is further configured to determine coordinates of some or all pixel points in the second image in the first image based on the summary parameter.

18. The apparatus of claim 17,

the determining unit is further configured to determine a reduction parameter according to the summary parameter, where the reduction parameter is a parameter of inverse image transformation of image transformation corresponding to the summary parameter;

the determining unit is further configured to determine, based on the restoration parameters, coordinates of some or all pixel points in the second image in the first image.

19. A computing system comprising a processor and a memory, the memory for storing instructions, the processor for executing the instructions, the processor when executing the instructions performing the method of any of claims 1 to 9.

20. A computer program product, characterized in that the computer program product comprises computer instructions which, when executed by a computing device, the computing device performs the method according to any one of claims 1 to 9.