CN116934608A

CN116934608A - Image processing method, device, electronic equipment and storage medium

Info

Publication number: CN116934608A
Application number: CN202210369414.6A
Authority: CN
Inventors: 龙良曲; 朱力; 郭士嘉
Original assignee: Insta360 Innovation Technology Co Ltd
Current assignee: Insta360 Innovation Technology Co Ltd
Priority date: 2022-04-08
Filing date: 2022-04-08
Publication date: 2023-10-24

Abstract

The embodiment of the application discloses an image processing method, an image processing device, electronic equipment and a storage medium; the embodiment of the application can acquire the shot image and the size information of the shot image; extracting the shot image according to the shot image and the size information of the shot image to obtain a plurality of sub-images; splicing the local features of all the sub-images; performing feature selection processing on global features of a shot image, and determining an area where target image features are located in the shot image; and taking the region where the target image features are located as a focusing point to obtain a modified shooting image. In the embodiment of the application, the target image characteristics in the photographed image are identified, and the photographed image is modified by taking the area where the target image characteristics are located as the focusing point, so that the modified photographed image is obtained. Therefore, the acquired panoramic image can be processed, and the problem that part of important scenery or figures in the image are distorted greatly is solved.

Description

Image processing method, device, electronic equipment and storage medium

Technical Field

The present application relates to the field of image processing, and in particular, to an image processing method, an image processing device, an electronic device, and a storage medium.

Background

Panoramic images generally refer to photographs taken in line with a normal effective viewing angle of the eyes of a person (about 90 degrees horizontal, 70 degrees vertical) or over a full scene range including both eyes with afterlight viewing angles (about 180 degrees horizontal, 90 degrees vertical), or even 360 degrees.

However, at present, when a panoramic image is photographed, a problem that a part of important scenes or characters are distorted greatly usually occurs in the obtained image, and thus, the photographing requirement of a photographer cannot be met.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, electronic equipment and a storage medium, which can process an acquired panoramic image and solve the problem that part of important scenery or figures in the image are distorted greatly.

In one aspect, an embodiment of the present application provides an image processing method, including:

acquiring a shot image and size information of the shot image;

determining the size, the sliding step length and the sliding direction of a sliding window according to the shot image and the size information of the shot image, and extracting the shot image according to the size, the sliding step length and the sliding direction of the sliding window to obtain a plurality of sub-images, wherein two adjacent sub-images at least comprise an overlapping area where the two sub-images overlap each other;

Performing feature recognition processing on the sub-images to obtain local features, wherein the local features are image features of the sub-images;

all the local features are spliced to obtain global features, wherein the global features are image features of the photographed image;

performing feature selection processing on the global features of the shot image, and determining target image features in the shot image and the region where the target image features are located in the shot image;

and modifying the photographed image by taking the region where the target image features are located as a focusing point to obtain a modified photographed image.

In another aspect, an embodiment of the present application further provides an image processing apparatus, including:

an acquisition unit configured to acquire a captured image and size information of the captured image;

the extraction unit is used for determining the size, the sliding step length and the sliding direction of the sliding window according to the shot image and the size information of the shot image, and extracting the shot image according to the size, the sliding step length and the sliding direction of the sliding window to obtain a plurality of sub-images, wherein two sub-images adjacent to each other in position at least comprise an overlapping area where the two sub-images overlap each other;

The identification unit is used for carrying out feature identification processing on the sub-images to obtain local features, wherein the local features are image features of the sub-images;

the splicing unit is used for carrying out splicing processing on the local features of all the sub-images to obtain global features of the photographed images;

the selection unit is used for carrying out feature selection processing on the global features of the shooting image and determining target image features in the shooting image and the area where the target image features are located in the shooting image;

and the modification unit is used for modifying the shooting image by taking the area where the target image features are located as a focusing point to obtain a modified shooting image.

In another aspect, an embodiment of the present application further provides an electronic device, including a processor and a memory, where the memory stores a plurality of instructions; the processor loads instructions from the memory to perform steps in any of the image processing methods provided by the embodiments of the present application.

In another aspect, an embodiment of the present application further provides a computer readable storage medium, where a plurality of instructions are stored, where the instructions are adapted to be loaded by a processor to perform steps in any one of the image processing methods provided in the embodiments of the present application.

In the application, the image processing device can identify the target image characteristics in the photographed image, and take the area where the target image characteristics are located as the focusing point to modify the photographed image so as to obtain the modified photographed image. Therefore, the acquired panoramic image can be processed, and the problem that part of important scenery or figures in the image are distorted greatly is solved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic view of a scenario of an image processing method according to an embodiment of the present application;

fig. 2 is a schematic flow chart of an image processing method according to an embodiment of the present application;

FIG. 3 is a flowchart of a method for determining a set sliding window size, a sliding step length and a sliding direction according to an embodiment of the present application;

FIG. 4 is a flow chart of a method for obtaining local features by performing feature recognition processing on a sub-image according to an embodiment of the present application;

FIG. 5 is a flowchart of a method for obtaining global features of a captured image according to an embodiment of the present application;

FIG. 6 is a flowchart of a method for determining a target image feature in a captured image and an area where the target image feature is located in the captured image according to an embodiment of the present application;

FIG. 7 is a flowchart of a method for obtaining a modified shot image of a next frame according to an embodiment of the present application;

FIG. 8 is a flowchart of an embodiment of an image processing method according to an embodiment of the present application;

fig. 9 is a schematic structural view of an image processing apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.

The embodiment of the application provides an image processing method, an image processing device, electronic equipment and a storage medium.

The image processing device may be integrated in an electronic device, which may be a terminal, a server, or other devices. The terminal can be a mobile phone, a tablet computer, an intelligent Bluetooth device, a notebook computer, a personal computer (Personal Computer, PC) or the like; the server may be a single server or a server cluster composed of a plurality of servers.

In some embodiments, the image processing apparatus may also be integrated in a plurality of electronic devices, for example, the image processing apparatus may be integrated in a plurality of servers, and the image processing method of the present application is implemented by the plurality of servers.

In some embodiments, the server may also be implemented in the form of a terminal.

For example, referring to fig. 1, the electronic device may be a server, in which an image processing apparatus is integrated, and the server in the embodiment of the present application is configured to obtain a captured image and size information of the captured image; determining the size, sliding step length and sliding direction of a sliding window according to the shot image and the size information of the shot image, and extracting the shot image according to the size, sliding step length and sliding direction of the sliding window to obtain a plurality of sub-images, wherein two sub-images adjacent to each other in position at least comprise an overlapping area where the two sub-images overlap each other; carrying out feature recognition processing on the sub-images to obtain local features, wherein the local features are the image features of the sub-images; the local features of all the sub-images are spliced to obtain global features of the photographed image; performing feature selection processing on global features of a shot image, and determining target image features in the shot image and areas where the target image features are located in the shot image; and modifying the photographed image by taking the region where the target image features are located as a focusing point to obtain a modified photographed image.

The following will describe in detail. The numbers of the following examples are not intended to limit the preferred order of the examples.

In this embodiment, an image processing method is provided, as shown in fig. 2, and a specific flow of the image processing method may be as follows:

s110, acquiring the shot image and the size information of the shot image.

The shot image may be a current frame image obtained when shooting a video, or may be a pre-shot image; the captured image may be a panoramic image or a non-panoramic image, and in some embodiments, the captured image may be a panoramic image that surrounds 0-360 ° in the horizontal direction and 0-180 ° in the vertical direction.

The size information of the photographed image may be the length, width, and angular field of view of the photographed image.

The acquiring of the photographed image and the size information of the photographed image may be directly acquiring the size information of the photographed image by the photographing device during photographing, or may be acquiring the length and the width of the photographed image by processing the photographed image.

S120, determining the size, the sliding step length and the sliding direction of a set sliding window according to the shot image and the size information of the shot image, and extracting the shot image according to the size, the sliding step length and the sliding direction of the sliding window to obtain a plurality of sub-images, wherein two sub-images adjacent to each other in position at least comprise an overlapping area where the two sub-images overlap each other.

The sliding window is a sliding window, in order to improve the accuracy of data, the value of a certain point is expanded to a region containing the point, and the region is used for judging, and the region is the window. The sliding window is capable of framing the time series according to a specified unit length, thereby calculating the statistical index in the frame. The sliding block corresponding to a length designated slides on the graduated scale, and the image data in the sliding block can be fed back every sliding unit.

The size of the sliding window is the range of the sliding window for image extraction on the photographed image, the sliding window may be any shape, and in some embodiments, the sliding window may be a rectangular frame, and the size of the sliding window may include the length and width of the rectangular frame. In some embodiments, the sliding window may be a square frame and the size of the sliding window may include the side length of the square frame.

The sliding direction of the sliding window is a direction in which the sliding window slides when the captured image slides, and the sliding direction of the sliding window may be arbitrary. In some embodiments, the sliding direction of the sliding window is sliding along the length of the captured image.

The sliding step length of the sliding window is the sliding distance of the sliding window when the sliding window slides in the shooting image. The sliding step of the sliding window may be set according to size information of the photographed image, and in some embodiments, when the sliding direction of the sliding window is sliding along the length direction of the photographed image, the sliding step of the sliding window is equal to the length of the photographed image divided by the number of times the sliding window slides.

In some embodiments, as shown in fig. 3, the method for determining the size, the sliding step length and the sliding direction of the sliding window according to the size information of the shot image includes:

s121, determining the angle of view of the shot image in a preset direction, wherein the preset direction is the sliding direction of the sliding window.

The Field of view (FOV), also known as the Field of view in optical engineering, determines the Field of view of an optical instrument. In the embodiment of the application, when the shot image is a panoramic image, the angle of view of the panoramic image in the horizontal direction is 0 to 360 degrees, and the angle of view in the vertical direction is 0 to 180 degrees.

The preset direction refers to a manually preset direction, and in some embodiments, the preset direction is a horizontal direction, and when the sliding window moves, the sliding window moves along the horizontal direction of the photographed image.

S122, selecting a plurality of reference points on the shot image according to the angle of view of the shot image in the preset direction, wherein the angle of view between two adjacent reference points is the same, and the distance between the reference points is equal to the sliding step length of the sliding window.

The reference point is a reference point at which the sliding window moves in position on the captured image when sliding. In some embodiments, the reference point is used to correspond to a datum position on the sliding window, so that the position of the sliding window movement is more accurate. In some embodiments, the reference point is set along a preset direction. In some embodiments, when the preset direction is a horizontal direction, the position of the reference point may be arbitrarily set in the vertical direction, and when the sliding window moves, the reference position is used for aligning with the reference point, and the reference position may be a point or a line.

The same angle of view between two adjacent reference points means that the angle of view between two adjacent reference points is the same along the preset direction, or the distance between two adjacent reference points is the same along the preset direction.

S123, determining an extraction range of the sliding window in the shot image according to the position of the reference points on the shot image, wherein the length of the extraction range in the preset direction is greater than the distance length between the reference points in the preset direction.

The extraction range is a range covered by the sliding window on the captured image, and the extracted image is taken as a sub-image based on the covered range.

The length of the extraction range in the preset direction being greater than the length of the distance between the reference points in the preset direction means that the sliding window has an overlapped part between the positions of the two reference points after the sliding window moves, namely, the two sub-images adjacent to each other in the representation position at least comprise an overlapped area which is overlapped with each other.

The extraction of the photographed image according to the sliding window may be to extract the image in the sliding window as a single image, which is recorded as a sub-image, each time the sliding window slides over a sliding step when the sliding window moves in a preset direction. When the sliding window slides over n sliding steps, n sub-images are acquired.

At least one overlapping area of two adjacent sub-images is that the two adjacent sub-images include the same image. In some embodiments, when the photographed image is a panoramic image surrounding 360 ° in the horizontal direction and 180 ° in the vertical direction, the sliding window extracts the photographed image along the horizontal direction, and each extracted sub-image includes an overlapping region overlapping with other adjacent sub-images at the front and rear.

S130, carrying out feature recognition processing on the sub-images to obtain local features, wherein the local features are the image features of the sub-images.

The feature recognition process may refer to pixel-level image classification of the sub-image, i.e., labeling the object class to which each pixel in the image belongs. The object categories may include scenery, buildings, people. In the embodiment of the application, the scenery, the building and the person in the sub-images can be identified through the feature identification processing, and the mask map covering different image features is determined through dividing the scenery, the building and the person in the sub-images, wherein the mask map refers to the fact that the background and the foreground (the foreground can comprise the building and the person) of the sub-images are presented through images with different gray scales. Local features may refer to global features in a sub-image or to partial image features in a sub-image.

In some embodiments, as shown in fig. 4, the method for performing feature recognition processing on the sub-image and obtaining the local feature includes:

s131, determining pixel types in the sub-images.

The pixel (pixel) class, i.e. the pixel type, is different for different images and for different pixel types the value of the incoming template parameter is also different. For example, the data types of the pixels include cv_32u, cv_32s, cv_32f, cv_8u, cv_8uc3, and the like.

Determining the pixel class in the sub-image refers to determining the pixel class of each pixel point in the sub-image.

S132, dividing the sub-images according to the pixel types in the sub-images to obtain mask images of the sub-images.

The mask map (mask) is an image that reflects the intensity of features in the sub-image by pixel intensity. For example, in some embodiments, the portion belonging to the background may be represented by a black color with a smaller gray value, and the portion belonging to the foreground may be represented by a white or gray color with a larger gray value. The distinction of the foreground portion from the background portion in the sub-image may be reflected by the mask map.

According to the pixel category in the sub-image, the sub-image segmentation processing refers to determining the category of each point (such as belonging to the background, the person or the car) according to the pixel category of each pixel point in the sub-image, so as to divide the region, and finally converting the sub-image in the form of a mask map. For example, in some embodiments, the portion that will belong to the background may be represented in black with a smaller gray value, and the portion that will belong to the person or car may be represented in white or gray with a larger gray value, resulting in a mask map.

S133, according to the mask graph of the sub-image, determining local features in the sub-image.

Determining local features in a sub-image may refer to determining background, character, or object features in the sub-image.

And S140, performing stitching processing on all the local features to obtain global features, wherein the global features are image features of the shot image.

Since the photographed image is composed of sub-images, it is understood that local features of the sub-images may constitute global features of the photographed image.

The local feature stitching processing of the sub-images refers to stitching all the sub-images according to the mapping relation between the sub-images and the photographed image, so as to obtain a mask image of the photographed image. The mask map of the photographed image includes local features of each sub-image, and the local features of all the sub-images form global features of the photographed image. In some embodiments, the photographed image obtained after the stitching process includes a foreground portion and a background portion, where the foreground portion may include a person, a building, and an object in the photographed image, and the background portion may include a landscape in the photographed image, and gray values of the foreground portion and the background portion are different.

In some embodiments, as shown in fig. 5, the method for performing stitching processing on the local features of all the sub-images to obtain the global features of the captured image includes:

S141, determining the position relation between the sub-image and the shooting image.

The positional relationship refers to the position of the sub-image in the captured image, wherein the positional relationship can be determined by the positional relationship between the sliding window and the reference point, or by the coordinate position of the sub-image in the captured image.

S142, mapping the local features of the sub-images to the shooting images according to the position relation between the sub-images and the shooting images, and obtaining global features of the shooting images.

The global feature refers to that mask patterns of a plurality of sub-images are superimposed into a photographed image according to a positional relationship, so as to obtain the photographed image with all local features, wherein all the local features are global features.

When the overlapping area between the first sub-image and the second sub-image which are adjacent in position has the first local feature and the second local feature which are common, in order to avoid the problem of feature recognition repetition, the image processing method further comprises the following steps:

calculating a first feature intensity of the first local feature and a second feature intensity of the second local feature;

comparing the first characteristic intensity with the second characteristic intensity:

when the first characteristic intensity is larger than the second characteristic intensity, selecting the first local characteristic as a target characteristic in the overlapped area;

When the first characteristic intensity is smaller than the second characteristic intensity, selecting the second local characteristic as a target characteristic in the overlapping region;

when the first feature intensity is equal to the second feature intensity, either the first local feature or the second local feature is selected as the target feature within the overlap region.

And S150, performing feature selection processing on the global features of the shot image, and determining target image features in the shot image and the region where the target image features are located in the shot image.

The feature selection processing refers to selecting a feature with higher significance in global features, wherein the feature with higher significance refers to that the pixel intensity of the region where the feature is located is large, and the pixel intensity is large and refers to that the gray value of the pixel is larger than a preset pixel value.

By comparing the pixel value with a preset pixel value, a feature with high significance, that is, a feature with high pixel intensity can be confirmed. Thus, in some embodiments, pixel values of the global feature may be compared to a preset pixel threshold, and image features having pixel values greater than the pixel threshold may be considered target image features.

In some embodiments, as shown in fig. 6, the method for performing feature selection processing on global features of a captured image to determine target image features in the captured image and a region where the target image features are located in the captured image includes:

S151, performing pixel region division processing on the global features of the shot image to obtain a plurality of pixel regions.

The pixel region may refer to a location of each pixel point in the global feature.

Since the mask pattern performs masking processing on the local feature in step S140, that is, the gray value of the pixel of the masked portion is set to zero, in order to reduce the calculation amount, it is unnecessary to calculate the position where the masked pixel is located, and it is only necessary to calculate the position where the unmasked pixel is located. Thus, in some embodiments, since the mask map is used to partition the global features of the captured image into pixel regions, it is sufficient to only confirm where the pixels are located in the portion not masked by the mask map.

S152, determining the pixel value of the pixel area.

Determining the pixel value of the pixel region may refer to determining a gray value of each pixel point.

S153, comparing the pixel value of the pixel region with a preset pixel threshold value, and determining a first pixel region with the pixel value larger than the pixel threshold value and a second pixel region with the pixel value smaller than the pixel threshold value, wherein the image feature at the first pixel region is marked as a target image feature.

The preset pixel threshold may be considered to be set.

The comparison refers to judging the gray level of the pixel value, and the first pixel area with the pixel value larger than the pixel threshold value is marked as a salient area, wherein the salient area is a target image feature, and the second pixel area with the pixel value smaller than the pixel threshold value is marked as a salient area. In some embodiments, the foreground part in the photographed image has larger gray value, the foreground part has larger pixel value intensity, and the foreground part with pixel value larger than the pixel threshold value is taken as a significant region in the photographed image.

And S154, according to the comparison result, performing binarization processing on the pixel value of the first pixel region and the pixel value of the second pixel region to obtain a photographed image after the binarization processing.

The binarization processing means that the pixel value of a first pixel area with the pixel value larger than the pixel threshold value is adjusted to be maximum, the pixel value of a second pixel area with the pixel value smaller than the pixel threshold value is adjusted to be minimum, and then a mask image of the adjusted shooting image is obtained, so that the distinguishing of the target image features is facilitated.

S155, determining the region where the target image features are located in the photographed image according to the photographed image after the binarization processing.

And determining a first pixel region where the target image features are located according to the difference between the pixel value of the first pixel region and the pixel value of the second pixel region in the binarized photographed image, and marking the first pixel region as the region where the target image features are located in the photographed image.

S160, modifying the photographed image by taking the region where the target image features are located as a focusing point, and obtaining a modified photographed image.

Focusing (Focus) refers to making a photographic subject imaged clearly by changing the distance between a lens and an imaging surface when an image is acquired. The focusing point refers to an object to be photographed, wherein the focusing point may be one pixel point or one pixel area.

The region where the target image features are located is taken as a focusing point, namely the region where the target image features are located is taken as the focusing point, so that the region where the target image features are located is the region with the clearest imaging.

Modifying the photographed image according to the focus refers to modifying the target image feature in the photographed image as the target to be presented most, so that the obtained modified target image feature in the photographed image is imaged most clearly and has smaller distortion.

At the time of capturing video, as shown in fig. 7, the method further includes:

s170, acquiring the focusing point of the modified shooting image.

Acquiring the focal point of the modified captured image refers to determining a target feature region in the captured image.

S180, modifying the next frame of shooting image based on the focal point of the modified shooting image, and obtaining the modified next frame of shooting image.

The modification of the next frame of photographed image means that the target feature area of the modified photographed image is taken as a focusing point, the next frame of photographed image is photographed, and the modified photographed image is obtained by modifying the position where the modified photographed image is the sharpest and less distorted.

In some embodiments, the captured image in the modified captured image may be a current frame image in the video capture and the next frame image may be a next frame image in the video capture.

The image processing method in the embodiment of the invention is described below with reference to a specific application scenario.

Referring to fig. 8, a flowchart of an embodiment of an image processing method applied to a server in an experimental scenario according to an embodiment of the present invention is shown, where the image processing method includes:

s201, acquiring a shot image and size information of the shot image.

The shot image is a panoramic image, and the length of the panoramic image is 1000mm and the width of the panoramic image is 500mm.

The horizontal view angle of the panoramic picture is 0-360 degrees, and the vertical view angle is 0-180 degrees.

S202, extracting a shot picture to obtain a sub-image.

The method for extracting the panoramic picture to obtain the sub-image comprises cube projection, multi-view projection and sliding window extraction.

According to the shot image and the size information of the shot image, four reference points are determined, the coordinates are (0, 500), (250, 500), (500 ), (750, 500), the sliding step length of the sliding window is 250, the size of the sliding window is 180 degrees of a horizontal viewing angle, the radius is 1, and the sliding direction of the sliding window is the horizontal direction. There are 4 sub-images extracted through the sliding window.

Since the horizontal interval between the reference points of 4 sub-images is converted to the angle of view of 90 ° and the horizontal distance of each sub-image is converted to the angle of view of 180 °, there is an overlapping area between two adjacent sub-images.

S203, carrying out feature recognition processing on the sub-images to obtain local features, wherein the local features are the image features of the sub-images.

By U-shaped ² And (3) netcarrying out feature recognition processing on the sub-images, wherein the sub-images are input into a network structure model of an encoder-decoder to obtain 6 mask images with the same size as the input sub-images, and the intensities in the 6 mask images are averaged and output to obtain the mask images of the sub-images.

S204, performing stitching processing on the local features of all the sub-images to obtain global features of the shot image.

And mapping the mask image of the sub-image to the shot image according to the position relation between the sub-image and the shot image, wherein when the overlapped feature appears in the overlapped region of the sub-image, the overlapped feature is characterized as a first local feature and a second local feature which are overlapped with each other, and the first local feature or the second local feature with larger intensity is selected by comparing the intensity of the overlapped feature, so that the mask image of the shot image is obtained.

S205, performing feature selection processing on the global features of the shot image, and determining target image features in the shot image and the region where the target image features are located in the shot image.

And carrying out binarization processing on a mask image of the shot image, adjusting the pixel values of the pixel points with the pixel intensity values of all the pixel points being larger than a preset threshold value in the image characteristics to 255, and adjusting the pixel values of the pixel points with the pixel intensity values being smaller than the preset threshold value to 0. Thus, a binary image of the shot image is obtained, and further, the target image features are conveniently identified and selected, and the area where the target image features are located in the shot image is obtained.

S206, modifying the photographed image by taking the region where the target image features are located as a focusing point, and obtaining the modified photographed image.

In the embodiment of the application, the image features in the panoramic image are identified, the target image features with larger intensity and the salient regions of the target image features in the shot image are determined, and the shot image or the subsequent shot video image is modified by taking the salient regions of the target image features as focusing points, so that the modified shot image is obtained. Therefore, the acquired panoramic image can be processed, and the problem that part of important scenery or figures in the image are distorted greatly is solved.

In order to better implement the method, the embodiment of the application also provides an image processing device which can be integrated in an electronic device, wherein the electronic device can be a terminal, a server and the like. The terminal can be a mobile phone, a tablet computer, a notebook computer, a personal computer and other devices; the server may be a single server or a server cluster composed of a plurality of servers.

For example, in the present embodiment, a method according to an embodiment of the present application will be described in detail by taking an example in which an image processing apparatus is specifically integrated in a server.

For example, as shown in fig. 9, the image processing apparatus may include:

An acquisition unit 301 for acquiring a captured image and size information of the captured image;

the extracting unit 302 is configured to determine a size, a sliding step length, and a sliding direction of a sliding window according to the captured image and size information of the captured image, and extract the captured image according to the size, the sliding step length, and the sliding direction of the sliding window to obtain a plurality of sub-images, where two sub-images adjacent to each other in position at least include an overlapping region where one sub-image overlaps with another;

the identifying unit 303 is configured to perform feature identification processing on the sub-image, and obtain a local feature, where the local feature is an image feature of the sub-image;

the stitching unit 304 is configured to stitch the local features of all the sub-images to obtain global features of the photographed image;

a selection unit 305, configured to perform feature selection processing on global features of the captured image, and determine a target image feature in the captured image and an area where the target image feature is located in the captured image;

and the modifying unit 306 is configured to modify the captured image with the region where the target image feature is located as a focusing point, so as to obtain a modified captured image.

In some embodiments of the present application, the extracting unit 302 is specifically configured to:

Determining the angle of view of a shot image in a preset direction, wherein the preset direction is the sliding direction of a sliding window;

selecting a plurality of reference points on the shot image according to the angle of view of the shot image in the preset direction, wherein the angle of view between two adjacent reference points is the same, and the distance between the reference points is equal to the sliding step length of the sliding window;

according to the position of the reference points on the shot image, determining the extraction range of the sliding window in the shot image, wherein the length of the extraction range in the preset direction is larger than the distance length between the reference points in the preset direction.

In some embodiments of the present application, the identifying unit 303 is specifically configured to:

determining a pixel class in the sub-image;

dividing the sub-images according to the pixel types in the sub-images to obtain mask images of the sub-images;

and determining local features in the sub-images according to the mask graph of the sub-images.

In some embodiments of the present application, the stitching unit 304 is specifically configured to:

determining the position relationship between the sub-image and the shot image;

and mapping the local features of the sub-images to the shooting images according to the position relation between the sub-images and the shooting images to obtain global features of the shooting images.

The sub-images comprise a first sub-image and a second sub-image which are adjacent in position, wherein the overlapping area of the first sub-image comprises a first local feature, and the overlapping area of the second sub-image comprises a second local feature;

when the first local feature overlaps the second local feature, the method includes:

selecting the first local feature or the second local feature as the target feature in the overlapping region when the first feature intensity is equal to the second feature intensity

In some embodiments of the present application, the selection unit 305 is specifically configured to:

carrying out pixel region division processing on the global features of the shot image to obtain a plurality of pixel regions;

determining a pixel value of the pixel region;

comparing the pixel value of the pixel region with a preset pixel threshold value, and determining a first pixel region with the pixel value larger than the pixel threshold value and a second pixel region with the pixel value smaller than the pixel threshold value, wherein the image feature at the first pixel region is marked as a target image feature;

According to the comparison result, carrying out binarization processing on the pixel value of the first pixel area and the pixel value of the second pixel area to obtain a photographed image after the binarization processing;

and determining the region where the target image features are located in the photographed image according to the binarized photographed image.

In some embodiments of the present application, the modification unit 306 is specifically configured to:

acquiring a focusing point of the modified shooting image;

and modifying the next frame of shooting image based on the focal point of the modified shooting image to obtain the modified next frame of shooting image.

In the implementation, each unit may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.

As can be seen from the above, the image processing apparatus of the present embodiment acquires the captured image and the size information of the captured image by the acquisition unit 301; the extraction unit 302 determines the size, the sliding step length and the sliding direction of the sliding window according to the shot image and the size information of the shot image, and extracts the shot image according to the size, the sliding step length and the sliding direction of the sliding window to obtain a plurality of sub-images, wherein two adjacent sub-images at least comprise an overlapping area where the two sub-images overlap each other; the recognition unit 303 performs feature recognition processing on the sub-image to acquire local features, wherein the local features are image features of the sub-image; the stitching unit 304 performs stitching processing on the local features of all the sub-images to obtain global features of the photographed image; the selection unit 305 performs feature selection processing on the global features of the captured image, and determines the target image features in the captured image and the region where the target image features are located in the captured image; the modification unit 306 modifies the captured image with the region where the target image feature is located as a focusing point, and obtains a modified captured image. Therefore, the embodiment of the application can process the acquired panoramic image and solve the problem that part of important scenery or figures in the image are distorted greatly.

The embodiment of the application also provides electronic equipment which can be a terminal, a server and other equipment. The terminal can be a mobile phone, a tablet computer, an intelligent Bluetooth device, a notebook computer, a personal computer and the like; the server may be a single server, a server cluster composed of a plurality of servers, or the like.

In this embodiment, a detailed description will be given taking an example in which the electronic apparatus of this embodiment is an image processing apparatus, for example, as shown in fig. 10, which shows a schematic configuration diagram of the image processing apparatus according to an embodiment of the present application, specifically:

the image processing may include one or more processor cores 'processors 401, one or more computer-readable storage media's memory 402, power supply 403, input module 404, and communication module 405, among other components. It will be appreciated by those skilled in the art that the configuration of the image processing apparatus shown in fig. 10 does not constitute a limitation of the image processing apparatus, and may include more or less components than illustrated, or may combine certain components, or may be arranged in different components. Wherein:

The processor 401 is a control center of the image processing, connects respective portions of the entire image processing using various interfaces and lines, and performs various functions of SSS and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402. In some embodiments, processor 401 may include one or more processing cores; in some embodiments, processor 401 may integrate an application processor that primarily processes operating systems, user interfaces, applications, and the like, with a modem processor that primarily processes wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the image processing apparatus, and the like. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

The image processing apparatus further includes a power supply 403 for supplying power to the respective components, and in some embodiments, the power supply 403 may be logically connected to the processor 401 through a power management system, so that functions of managing charging, discharging, and power consumption management are implemented through the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The image processing may also include an input module 404, which input module 404 may be used to receive entered numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

The image processing apparatus may further comprise a communication module 405, and in some embodiments the communication module 405 may comprise a wireless module, and the image processing may be performed by a short-range wireless transmission via the wireless module of the communication module 405, thereby providing wireless broadband internet access to the user. For example, the communication module 405 may be used to assist a user in e-mail, browsing web pages, accessing streaming media, and so forth.

The image processing apparatus may further comprise a sensor module, which in some embodiments may comprise an image sensor, which may acquire an image.

Although not shown, the image processing may further include a display unit or the like, which is not described here. In particular, in this embodiment, the processor 401 in image processing loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, thereby implementing various functions.

In some embodiments, a computer program product is also proposed, comprising a computer program or instructions which, when executed by a processor, implement the steps of any of the above-mentioned image processing methods.

The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.

Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a computer readable storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform the steps of any one of the image processing methods provided by the embodiments of the present application.

Wherein the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.

The instructions stored in the storage medium may perform steps in any image processing method provided by the embodiments of the present application, so that the beneficial effects that any image processing method provided by the embodiments of the present application can be achieved, which are detailed in the previous embodiments and are not described herein.

The foregoing has described in detail the image processing method, apparatus, electronic device and storage medium provided by the embodiments of the present application, and specific examples have been applied to illustrate the principles and embodiments of the present application, where the foregoing examples are only used to help understand the method and core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, the present description should not be construed as limiting the present application.

Claims

1. An image processing method, comprising:

acquiring a shot image and size information of the shot image;

2. An image processing method according to claim 1, wherein after said obtaining a modified captured image, the method comprises:

Acquiring a focusing point of the modified shooting image;

3. The method according to claim 1, wherein the method of determining the size, the sliding step length, and the sliding direction of the set sliding window based on the captured image and the size information of the captured image comprises:

determining a field angle of the shot image in a preset direction, wherein the preset direction is the sliding direction of the sliding window;

selecting a plurality of reference points on the shot image according to the angle of view of the shot image in a preset direction, wherein the angles of view between two adjacent reference points are the same, and the distance between the reference points is equal to the sliding step length of the sliding window;

and determining an extraction range of the sliding window in the shot image according to the position of the reference point on the shot image, wherein the length of the extraction range in the preset direction is larger than the distance length between the reference points in the preset direction.

4. The image processing method according to claim 1, wherein the method for performing feature recognition processing on the sub-image to acquire the local feature includes:

Determining a pixel class in the sub-image;

dividing the sub-image according to the pixel types in the sub-image to obtain a mask image of the sub-image;

and determining local features in the sub-images according to the mask map of the sub-images.

5. The image processing method according to claim 1, wherein the method for stitching the local features of all the sub-images to obtain the global feature of the captured image includes:

determining the position relationship between the sub-image and the photographed image;

6. The image processing method according to claim 5, wherein the sub-image includes a first sub-image and a second sub-image that are adjacent to each other, the first sub-image including a first local feature in an overlapping region, and the second sub-image including a second local feature in an overlapping region;

when the first local feature overlaps with the second local feature, the method includes:

when the first characteristic intensity is larger than the second characteristic intensity, selecting the first local characteristic as a target characteristic in an overlapping area;

selecting the second local feature as a target feature in an overlapping region when the first feature intensity is less than the second feature intensity;

and selecting the first local feature or the second local feature as a target feature in an overlapping region when the first feature intensity is equal to the second feature intensity.

7. The image processing method according to claim 1, wherein the method for performing feature selection processing on the global feature of the captured image and determining the target image feature in the captured image and the region where the target image feature is located in the captured image includes:

determining a pixel value of the pixel region;

According to the comparison result, binarizing the pixel value of the first pixel area and the pixel value of the second pixel area to obtain a binarized shooting image;

8. An image processing apparatus, comprising:

9. An electronic device comprising a processor and a memory, the memory storing a plurality of instructions; the processor loads instructions from the memory to perform the steps in the image processing method according to any one of claims 1 to 8.

10. A computer readable storage medium, characterized in that it stores a plurality of instructions adapted to be loaded by a processor for performing the steps in the image processing method according to any one of claims 1 to 7.