CN116258846A

CN116258846A - Image correction method, apparatus, computer device, and storage medium

Info

Publication number: CN116258846A
Application number: CN202211089455.6A
Authority: CN
Inventors: 张伟俊; 姜文杰; 贾顺; 蔡智
Original assignee: Insta360 Innovation Technology Co Ltd
Current assignee: Insta360 Innovation Technology Co Ltd
Priority date: 2022-09-07
Filing date: 2022-09-07
Publication date: 2023-06-13

Abstract

The present application relates to an image correction method, apparatus, computer device, storage medium and computer program product. The method comprises the following steps: detecting a positioning symbol of an image to be processed to obtain the position of the positioning symbol in the image and the direction represented by the positioning symbol; when the number of the positioning symbols is detected to meet a preset condition and the positions correspond to the orientations, connecting the positioning symbols to obtain a target area of the positioning symbols; and correcting the target area to obtain a corrected target image. By adopting the method, the identification accuracy of the target area can be improved.

Description

Image correction method, apparatus, computer device, and storage medium

Technical Field

The present application relates to the field of image processing technology, and in particular, to an image correction method, an image correction apparatus, a computer device, a storage medium, and a computer program product.

Background

In application scenarios such as video conference, live course broadcast, recorded course broadcast, etc., there is a class of user requirements that ensure the display effect of whiteboard content during display/recording. In the conventional art, a dedicated camera is installed above a whiteboard, edge detection is performed in a photographed picture, and the whiteboard in the photographed picture is identified by an edge detection result.

However, this method can only probabilistically identify objects with clear boundaries (such as white boards), and the detection result is strongly related to the edge definition and the edge detection algorithm, and when there is no boundary or the boundary is not clear enough or is partially blocked, the identified object area is incomplete and not clear, and the problem of inaccurate image correction exists when the object area is corrected.

Disclosure of Invention

Based on this, it is necessary to provide an image correction method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product capable of determining a target area more accurately, in view of the above-described technical problems.

In a first aspect, the present application provides an image correction method. The method comprises the following steps:

detecting a positioning symbol of an image to be processed to obtain the position of the positioning symbol in the image and the direction represented by the positioning symbol;

when the number of the positioning symbols is detected to meet a preset condition and the positions correspond to the orientations, connecting the positioning symbols to obtain a target area of the positioning symbols;

and correcting the target area to obtain a corrected target image.

In one embodiment, the detecting the positioning symbol of the acquired image to obtain the position of the positioning symbol in the image and the position represented by the positioning symbol includes:

responding to a target event, and acquiring a detection model of a positioning symbol;

detecting a positioning symbol in the image according to the detection model to obtain the position of the positioning symbol in the image and the characteristic of the positioning symbol;

and acquiring the direction characterized by the positioning symbol based on the characteristics of the positioning symbol.

In one embodiment, the connecting the positioning symbol to obtain the target area of the positioning symbol includes:

connecting the positioning symbols to obtain an initial area;

judging whether the boundary of the initial area is matched with the boundary of the preset shape or not;

if not, filtering the initial area;

if yes, filtering the initial area according to the boundary included angle, and judging whether the initial area is a target area or not.

In one embodiment, the filtering the initial area according to the boundary angle, and determining whether the initial area is a target area includes:

Calculating the boundary included angle of the initial area;

filtering the initial region according to whether the boundary included angle is in the included angle range of the preset shape or not;

and taking the filtered initial area as the target area.

In one embodiment, the positioning symbols include a first positioning symbol and a second positioning symbol, the first positioning symbol is a detected positioning symbol, the second positioning symbol is an undetected positioning symbol, and the position and the orientation are a position and an orientation of the first positioning symbol; the method further comprises the steps of:

when the number of the first positioning symbols does not meet the preset condition, determining a detection area of the second positioning symbols according to the azimuth;

detecting the second positioning symbol in the detection area to obtain the updated number, the updated position and the updated azimuth of the first positioning symbol;

and when the updated quantity meets the preset condition and the updated position is consistent with the updated azimuth, connecting the updated positioning symbols to obtain a target area of the positioning symbols.

In one embodiment, the determining the detection area of the second positioning symbol according to the position characterized by the first positioning symbol includes:

Adjusting acquisition parameters of shooting equipment according to the azimuth represented by the first positioning symbol;

re-acquiring the image according to the adjusted acquisition parameters;

in the re-acquired image, a detection area of the second positioning symbol is determined according to the orientation.

In one embodiment, the correcting the target area to obtain a target image includes:

generating a corresponding perspective change matrix according to the vertex position of the preset shape;

and correcting the target area according to the perspective change matrix to obtain a corrected target image.

In a second aspect, the present application also provides an image correction apparatus. The device comprises:

the detection module is used for detecting a positioning symbol of an image to be processed to obtain the position of the positioning symbol in the image and the direction represented by the positioning symbol;

the target area determining module is used for connecting the positioning symbols to obtain a target area of the positioning symbols when the number of the positioning symbols is detected to meet a preset condition and the positions correspond to the orientations;

and the image correction module is used for correcting the target area to obtain a corrected target image.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of image correction in any of the embodiments described above when the processor executes the computer program.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of image correction in any of the embodiments described above.

In a fifth aspect, the present application also provides a computer program product. The computer program product comprising a computer program which, when executed by a processor, carries out the steps of image correction in any of the embodiments described above.

The image correction method, the image correction device, the computer equipment, the storage medium and the computer program product perform positioning symbol detection on the image to be processed to obtain the position of the positioning symbol in the image and the direction represented by the positioning symbol; when the number of the positioning symbols is detected to meet the preset condition and the positions correspond to the directions, the positioning symbols are connected to obtain a target area of the positioning symbols, the target area determined based on the positioning symbols is an area of interest, the target area is not required to be identified through an edge detection algorithm, the possibility of false identification is reduced, the accuracy is high, and a corrected target image is obtained through correcting the target area.

Drawings

FIG. 1 is a diagram of an application environment for an image correction method in one embodiment;

FIG. 2 is a flow chart of an image correction method according to an embodiment;

FIG. 3 is a schematic diagram of a positioning symbol in one embodiment;

FIG. 4 is a schematic diagram of a positioning symbol in one embodiment;

FIG. 5 is a schematic diagram of an application scenario of a conventional image correction method;

FIG. 6 is a schematic diagram of another conventional image correction method;

FIG. 7 is a schematic diagram of another conventional image correction method;

FIG. 8 is a diagram of an application environment for an image correction method in one embodiment;

FIG. 9 is a block diagram showing the configuration of an image correction apparatus in one embodiment;

fig. 10 is an internal structural view of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

The image correction method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The image correction method provided in this embodiment may be applied only to the terminal 102, and in this integrated embedded device, one or more of the processes, control and storage related to the AI algorithm are performed by the AI algorithm, and these processes are related to the scheme of the present application.

The terminal 102 may be, but not limited to, various cameras, sensors, radars, personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, etc. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.

In one embodiment, as shown in fig. 2, an image correction method is provided, which is illustrated by taking the application of the method to the terminal 102 in fig. 1 as an example, and includes the following steps:

step 202, detecting a positioning symbol of an image to be processed to obtain the position of the positioning symbol in the image and the direction represented by the positioning symbol;

in one embodiment, step 202 is a step performed after the terminal receives or detects a target event. The target event may be a preset event or an event that changes dynamically based on a user, so as to control the terminal to perform positioning symbol detection on the acquired image, and determine the next image processing based on the acquired current image. The target events can be of various types, each target event has the corresponding advantages, and the corresponding starting strategy is set according to the various target events, so that the execution of the method is facilitated. For example: when the target event is triggered based on a specific gesture, the user can operate more conveniently; when the target event is triggered according to a certain button, the information displayed by each button is clear; when the target event is a terminal start event, the terminal can comprehensively record the image of the corresponding object; when the target event is a timing event, the corresponding process can be automatically carried out, and the method is particularly suitable for modes such as lesson taking or lecturing; and multiple target events are selected for combination, so that the method has the advantages of various types.

The positioning symbol is a symbol having an azimuth indicator for determining a region of interest resulting from the preset shape setting. The positioning symbol is a part preset at a certain position, and the part can be made of paper, plastic, metal, magnet and other arbitrary materials; regardless of the material of the positioning symbol, the orientation may be indicated by the features therein. For example: the predetermined shape determined by the positioning symbol is a rectangle, and the positioning symbol is an edge corner point of the rectangle, and the positioning symbol characterizing the upper left corner of the rectangle is shown in any one of fig. 3 (a) -3 (d).

In one embodiment, the positioning symbol is a "positioning sticker" having features by which the direction indicated by the direction is determined, and by rotating by 90 degrees clockwise, the "positioning sticker" indicating four angles can be obtained from a certain "positioning sticker" of any angle, respectively. The four angle positioning stickers are the upper left corner, the upper right corner, the lower right corner and the lower left corner in sequence, and are shown in figure 4.

Because the image is acquired by the shooting equipment, distortion is generated due to acquisition parameters such as an image acquisition angle and the like, and the position of the positioning symbol in the image is further caused to be different from the pre-placed actual position. Because of the direction indirection of the positioning symbol, the image is subjected to positioning symbol detection, and the target area is determined according to the positioning symbol detection result, so that the identification accuracy of the target area is higher and the false detection rate is lower. The shooting device may be the terminal 102 itself, or may be various electronic devices with shooting functions, where the various electronic devices include devices such as a pan-tilt camera, a video conference machine, a motion camera, a panoramic camera, a pan-tilt camera, and an unmanned aerial vehicle, which is not limited in this case.

The terminal can take the acquired image as an image to be processed in the positioning symbol detection process, can also perform certain preprocessing on the acquired image, and takes the preprocessed image as the image to be processed; the preprocessing includes, but is not limited to, one or more of image structure change processing, image smoothing processing, restoration processing, and picture enhancement processing. Alternatively, when image correction is performed a plurality of times, the target image after the previous image correction may be used as the current image to be processed until the target image after the image correction satisfies a certain specific condition.

In one embodiment, the positioning symbol detection of the image to be processed is performed by a template matching method. The template matching method is to search the region with the highest similarity with the positioning symbol in the image to be processed, and then obtain the positioning symbol. Illustratively, the method of template matching includes: in the image to be processed, the areas similar to the positioning symbols are sequentially calculated according to a preset sequence, and the positions of the positioning symbols in the image and the directions represented by the positions are identified based on the similarity of the areas. The template matching method is combined with other methods, so that a better detection effect can be achieved, and the method combined with the template matching method can be one or more of a threshold segmentation method, a color segmentation method and a detection method based on a convolutional neural network.

In one embodiment, the positioning symbol detection of the image is performed according to a combination method of threshold segmentation and template matching. The image is segmented by a threshold segmentation method to obtain candidate areas, and a template matching method is used for searching positioning symbols in the candidate areas, so that the detection process can be faster. Illustratively, the threshold segmentation method includes: calculating the image value of each unit area in the image, calculating whether the combined area formed by each unit area is a candidate area through the comparison process of the image value of each unit area and the corresponding threshold value, processing the candidate area according to a template matching method, and determining one or more positioning symbols corresponding to the candidate area to identify, so as to obtain the position of the positioning symbol in the image and the represented azimuth thereof.

Wherein, a unit area can be formed by one or a plurality of pixels, and a combined area can be formed by one or a plurality of unit areas; the corresponding threshold value of the image value may be any type of value, such as a gray value, an RGB value, and combinations thereof. When all pixel values in the set threshold range are set to be 1, and the rest all pixel values are set to be 0, the set threshold range is binarized; when the corresponding threshold value of the image value may be divided into a plurality of threshold values by size, a complex image having a plurality of objects may be processed, and the image is divided into a plurality of different portions using a plurality of different threshold values.

In one embodiment, the positioning symbol detection of the image is performed according to a combination method of color segmentation and template matching. The manner of color segmentation may be the same as the manner of threshold segmentation, and other means may exist. Illustratively, prior to the color segmentation, the method includes: filtering the image through a filter to obtain a blurred image, retaining the characteristic with larger difference in the blurred image, and filtering the characteristic with smaller difference; next, the image is converted from BGR (blue, green, red) to HSV (hue saturation value) in order to distinguish the values of pixels B, G, and R from the light falling on the object, and to reduce the coupling between these pixels in order to more accurately acquire the brightness, saturation, and chromaticity of the pixels. In the process of color segmentation, the maximum value and the minimum value of a positioning symbol to be extracted are firstly determined, the area without the positioning symbol in the image is removed based on the maximum value and the minimum value, the candidate area of the positioning symbol is obtained, template matching is carried out on the candidate area, and the position of the positioning symbol and the represented azimuth thereof are identified.

In one embodiment, performing positioning symbol detection on an image to be processed to obtain a position of the positioning symbol in the image and a position represented by the positioning symbol, including: responding to a target event, and acquiring a detection model of a positioning symbol; detecting the positioning symbol in the image according to the detection model to obtain the position of the positioning symbol in the image; based on the characteristics of the positioning symbol, the orientation characterized by the positioning symbol is obtained.

The detection model of the positioning symbol is a detection model trained in advance for the positioning symbol, and the detection model can comprise a model such as YOLO (You Only Look Once, you only see once), SSD (Single Shot MultiBox, single shot multi-frame), R-CNN (Region-based Convolutional Neural Networks, region-based convolutional neural network) or Mask R-CNN (Mask Region-based Convolutional Neural Networks, mask-based convolutional neural network), and the like, and can detect the positioning symbol, judge the position of the positioning symbol in an image, and acquire the represented azimuth according to the characteristic of the positioning symbol.

The positioning symbol is characterized by being a structural characteristic of the positioning symbol, and the structural characteristic can be set according to one or more of characteristic dimensions such as color, gray value, positioning symbol shape and the like; when the feature dimension comprises the predefined positioning symbol shape and color, various changes of the target can be detected by combining an advanced target detection model, and the method has higher robustness, better stability and higher performance; moreover, accurate positioning results are easy to obtain, false detection of positioning symbol shape analogues can be effectively avoided while the detection rate is ensured, accurate identification of positioning symbols is realized, and further, the region of interest in the image is accurately determined.

The areas obtained by connecting the positions of the positioning symbols are different from the preset shapes, the image distortion is caused by factors such as an image acquisition angle, an alternating coefficient and the like, and the distortion degree is different to be larger or smaller, so that the areas determined according to the positioning symbols are necessary to be screened. The screening steps comprise: detecting based on the number of the positioning symbols, and judging whether the number of the positioning symbols meets a preset condition; and matching the position and the azimuth of the positioning symbol, and judging whether the position and the azimuth correspond or not.

And 204, when the number of the positioning symbols is detected to meet the preset condition and the positions correspond to the directions, connecting the positioning symbols to obtain a target area of the positioning symbols.

When the number of the positioning symbols is detected to meet the preset condition, the image is judged to be capable of being corrected based on the number dimension of the positioning symbols, the target area of the positioning symbols can be analyzed, and the difference between the target area and the preset shape is within a certain range. Illustratively, the number of positioning symbols is four, and the preset shape is rectangular, the number of positioning symbols satisfies the preset condition; when the number of the positioning symbols is three and the preset shape is a triangle, the number of the positioning symbols meets the preset condition.

The determination that the position corresponds to the orientation is made by determining from the position dimension of the positioning symbol that the image is correctable. A positioning symbol is abnormal when it characterizes an orientation that differs from the position it is in. For example: when the preset shape is a rectangle, the positioning symbols of the rectangle respectively represent four directions of an upper left corner, an upper right corner, a lower right corner and a lower left corner, and when the positions of the positioning symbols of the rectangle in the image correspond to the four directions one by one, the image which can be corrected can be screened. When the user misplaces a certain positioning symbol, if the upper left symbol is at the lower right corner of the image, an abnormality occurs, and no legal rectangle is detected.

In one embodiment, in the process of connecting the positioning symbols, the terminal uses the positioning symbols as edge corner points of the region of interest, and obtains the target region by using the edge corner points as vertices of the region of interest, for example: the positions of the interested areas can be determined by using the vertexes of the rectangle as the directions characterized by the positioning symbols.

In one embodiment, the connecting the positioning symbols to obtain the target area of the positioning symbols includes: connecting the positioning symbols to obtain an initial area; judging whether the boundary of the initial area is matched with the boundary of the preset shape; if not, filtering the initial area; if yes, filtering the initial area according to the boundary included angle, and judging whether the initial area is a target area or not.

The initial area is an area obtained through preliminary screening, the boundary number of the initial area is detected according to a preset shape, the initial area with large deformation can be filtered, then the initial area with small deformation can be filtered again according to the boundary included angle, and the area obtained through the filtering again is determined as a target area.

As for the method for judging whether the boundary of the initial region is matched with the boundary of the preset shape, the method can be carried out by judging whether the number of the boundaries is matched, so that the initial region with large deformation can be filtered out quickly and accurately. In one embodiment, when the number of boundaries in the region exceeds the number of boundaries of the preset shape, the terminal may erroneously detect some symbols similar to the positioning symbols as positioning symbols, resulting in the occurrence of misrecognized positioning symbols; when the number of boundaries in the region is less than the number of boundaries of the preset shape, a certain positioning symbol is not connected with an adjacent positioning symbol.

The method for judging whether the boundaries are matched can also be based on the area mode for screening, and the area of the area surrounded by the boundary of the preset shape can be preset by comparing the area of the initial area with the area of the area surrounded by the boundary of the preset shape. The mode of carrying out area calculation on the initial area can be an area growing algorithm or other area calculation methods; the region growing algorithm takes a positioning symbol or any coordinate point as a seed, takes adjacent pixels with similar properties as a growing region of the seed, and takes edge points of the growing region as seeds for the next region growth; when the seeds grown in the next region are positioned at the boundary of the initial region, calculating the area of the initial region; wherein, adjacent pixels with similar properties to seeds refer to the characteristic difference of gray level or color dimension is smaller than a certain range relative to seeds.

The method for judging whether the boundaries are matched can be based on a region shape mode for screening, and based on a region shape mode for screening, the method can be based on a shape obtained by matching the boundary of the initial region with the boundary of the preset shape, and can also use the difference of gradient vectors of the pixel characteristics as a similarity measurement standard between two images, so that whether the boundary of the initial region is matched with the boundary of the preset shape or not can be obtained.

In one embodiment, filtering the initial region according to the boundary angle, and determining whether the initial region is a target region includes: calculating the boundary included angle of the initial area; filtering the initial area according to whether the boundary included angle is in the included angle range of the preset shape or not; and taking the filtered initial area as a target area.

The boundary angle of the initial region is the angle between adjacent boundaries of the initial region. When the boundary included angle is in the included angle range of the preset shape, the distortion degree of the image is small, and the correction can be performed; when the boundary angle is outside the angle range, the distortion degree of the image is large, which is difficult to correct, and the initial region is not the target region.

In one embodiment, the preset shape is a rectangle and the positioning symbol is an edge corner of the rectangle. In the process of filtering the 4 detected positioning symbols according to the rectangle, when the positions of the positioning stickers installed by the user just form four vertexes of the rectangle, the initial area in the image is less distorted, and the initial area can be used as a target area; when the area determined by the position of the positioning sticker installed by the user is a very irregular quadrangle, it is also difficult to ensure a superior correction effect after the target area is projected. Based on this design algorithm, very irregular quadrilaterals are filtered out.

Correspondingly, one feasible filtering mode is to detect 4 positioning symbols to obtain a quadrangle according to the clockwise sequence of two connecting lines (namely, connecting an upper left corner, an upper right corner, a lower left corner and an upper left corner); calculating boundary included angles of the quadrangles, and when the angle of each boundary included angle meets the included angle range of more than or equal to 60 degrees and less than or equal to 120 degrees, judging that the initial area is a legal rectangle, judging that the detection is successful, and determining that the initial area is a target area; otherwise, judging that the detection fails, and feeding back the detection failure by increasing the detection time and continuing the detection; the feedback mode can also be to send out certain information to remind the user that the positioning symbol is not pasted normally.

In one embodiment, the positioning symbols include a first positioning symbol and a second positioning symbol, the first positioning symbol being a detected positioning symbol, the position and the orientation being a position and an orientation of the first positioning symbol; correspondingly, the method further comprises the steps of: when the number of the first positioning symbols does not meet the preset condition, determining a detection area of the second positioning symbols according to the direction; in the detection area, detecting the second positioning symbols to obtain updated quantity, updated positions and updated positions of the first positioning symbols; and when the updated quantity meets the preset condition and the updated position is consistent with the updated azimuth, connecting the updated positioning symbols to obtain a target area of the positioning symbols. The detection area is determined according to the direction characterized by the detected positioning symbol, the undetected positioning symbol is searched in the detection area, the required detection area is further determined, and the shot object is further changed so as to acquire an image in the target area of the positioning symbol.

In one embodiment, determining the detection region of the second positioning symbol based on the position characterized by the first positioning symbol comprises: adjusting acquisition parameters of shooting equipment according to the azimuth represented by the first positioning symbol; re-acquiring the image according to the adjusted acquisition parameters; in the re-acquired image, a detection area of the second positioning symbol is determined according to the orientation.

The acquisition parameters of the shooting equipment are adjusted, the acquisition parameters can be the acquisition direction or the acquisition direction and the focal length, the first image is the currently acquired image, the first positioning symbol is the positioning symbol detected in the first image, and the second positioning symbol is the positioning symbol which is not detected.

In one embodiment, the first positioning symbol and the second positioning symbol in each image are described from different stages of the implementation of the image correction method. Before the stage of positioning symbol detection of the image to be processed, a first positioning symbol does not exist in the image to be processed; in the stage of positioning symbol detection of the image to be processed, if the number of positioning symbols in the image does not meet the preset condition, dividing the positioning symbols in the image into a first positioning symbol and a second positioning symbol, and adjusting acquisition parameters of shooting equipment according to the direction represented by the first positioning symbol so as to acquire the image again; before the image to be processed in the image correction process, the acquired image is taken as the image to be processed, and the acquired image is not provided with the first positioning symbol, so that the detection area of the second positioning symbol is determined according to the azimuth, and the detection area of each positioning symbol in the acquired image is actually determined. The re-acquired image can be used as an image to be processed in the next execution process of the image correction method until the execution process of the image correction method is completed or terminated.

In one embodiment, after at least one of the acquisition direction and the focal length of the photographing device is adjusted, the second positioning symbols in the acquired images can identify the corresponding positions and the represented directions thereof in the positioning symbol detection process, so as to obtain the target area of the positioning symbols.

The position characterized by the first positioning symbol is determined based on at least one first positioning symbol. When the position represented by a certain positioning symbol is the upper left, the acquisition direction is to be adjusted to the lower right, and the focal length can be adjusted at the same time; when the position represented by one first positioning symbol is the upper left and the position represented by the other first positioning symbol is the lower left, the acquisition direction is determined and adjusted to the right according to the horizontal direction based on the two positions of the upper left and the lower left. And after the acquisition parameters are adjusted, re-acquiring the image according to the adjusted acquisition parameters, and determining a detection area of the second symbol in the re-acquired image. Through the mode, the region of interest can be automatically searched under the condition that only part of the region of interest enters the mirror, the accuracy is high, the intelligent degree is high, and the interaction experience is good.

And 206, correcting the target area to obtain a corrected target image.

The target image is an image obtained by correcting the target region. The image is a map of the image of the target area to another plane in order to correct for perspective distortion of the target area. In the process, correction realized by perspective transformation of the image can be one or more of rigid transformation, similarity transformation and affine transformation, and specific correction means can be one or more of digital correction and optical correction.

In one embodiment, the correction is a quadrilateral correction process. The quadrilateral correction process can be digital correction, and the principle is that the original image is geometrically transformed according to an interpolation algorithm aiming at rectangular distortion generated by an imaging light path, so that a rectangular target area presents a regular rectangle in the processed output image. The process of quadrilateral correction may also be optical correction. The optical correction can be achieved by adjusting the physical position of the lens to achieve the purpose of adjusting the quadrilateral shape in the imaging picture.

In one embodiment, correcting the target area to obtain a corrected target image includes: generating a corresponding perspective change matrix according to the vertex position of the preset shape; and correcting the target area according to the perspective change matrix to obtain a corrected target image.

The vertex positions of the preset shape are a plurality of non-collinear positions which are preset, and the positions are calculated according to projection to obtain a perspective change matrix; and then calculating the positions of the positioning symbols of the target area according to the perspective change matrix to realize the correction of the target area and obtain a corrected target image.

The method further comprises the steps of: and displaying the target image.

The target image may be the first target image of the corrected target region directly, or may be the second target image obtained by performing some display enhancement processing on the first target image. One way of display enhancement processing is to sharpen the target image by blurring the boundary and increasing the sharpness of the region of interest and the focal length of the image, thereby improving the sharpness; another way of displaying enhancement processing is to increase the difference in gray values of the region of interest to enhance the contrast effect of the target image and improve sharpness. The second target image may also be magnified or otherwise processed.

In the image correction method, the image is subjected to positioning symbol detection to obtain the position of the positioning symbol in the image and the direction represented by the positioning symbol; when the number of the positioning symbols is detected to meet the preset condition and the positions and the directions are corresponding, the positioning symbols are connected to obtain a target area of the positioning symbols, so that the region of interest is determined based on the positioning symbols, the region of interest is not required to be identified through an edge detection algorithm, the possibility of false identification is reduced, the accuracy is high, the correction of the image is realized through the target image obtained through correcting the target area, and after the corrected target image is obtained, the target image can be further subjected to enhancement processing or amplification processing or other processing modes by adopting other display enhancement processing modes, so that the displayed image can be clearer or can meet other requirements of users.

In one embodiment, an application scenario of the present solution is described. In application scenarios such as video conference, live course broadcast, recorded course broadcast, etc., there is a class of user requirements to ensure the correction effect of whiteboard content during display/recording. The correction effect generally includes the following demand points: 1. because the camera cannot normally align the whiteboard, the picture content of the whiteboard is deformed to a certain extent, and a user has the need of correcting the whiteboard area into a rectangle so as to be convenient for reading; 2. there is a need for visual enhancement, such as a desire to zoom in on the whiteboard area, to make details more clear, etc. Whichever is required, its precondition is to find the positions of the 4 corner points of the whiteboard in the picture (automatically or manually).

Where "whiteboard" is a whiteboard in a broad sense, it is intended to refer broadly to a plane that can be used to correct information, including but not limited to a whiteboard, semi-transparent whiteboard, blackboard, whitewall, electronic whiteboard, wallpaper, glass plane where writing can be done, television screen, display, projection screen, electronic whiteboard, etc.

Compared with a certain traditional scheme in the industry, in the scheme of the application, a special camera is not required to be installed for the whiteboard, various objects can be shot by the camera, the whiteboard without boundaries can be obtained, and the accuracy is high. The traditional scheme is that a special camera is arranged above a whiteboard, whiteboard detection is carried out in a shot picture, positions of 4 corner points of the whiteboard are obtained (if a user feels inaccurate, correction can be carried out manually in self-contained software), and then a corrected rectangular picture is generated according to the positions of the 4 corner points. However, the camera needs to be installed in advance, and the installation and the disassembly are inconvenient; the camera can only be used for shooting whiteboard pictures after being installed, and a speaker or other objects cannot be shot; more importantly, the method can only probabilistically identify the white board with the boundary, can not identify the white board without the boundary or with the boundary not being clear enough, is usually not accurately identified enough, and usually requires a user to manually adjust the identification result on the client after installing the camera and the white board and automatically identifying, and the application scene is shown in fig. 5.

Compared with another traditional scheme in the industry, the scheme has higher accuracy, and the condition of missed detection or misunderstanding is not easy to occur. The other traditional scheme is that a user fixes a camera on a table, a chair or a bracket in front of the whiteboard, then uses an algorithm to detect the whiteboard, carries out picture correction and enhanced display according to the positions of 4 corner points of the whiteboard, easily misdetects some white areas or closed rectangular areas into the whiteboard, and simultaneously easily causes missed detection due to insufficient definition of the edges of the whiteboard, and has low identification accuracy and poor usability. There are two specific embodiments of this approach: one embodiment is based on edge detection, detects line segments in a picture, and then forms a rectangle, and the other specific embodiment is to perform threshold segmentation according to colors, and then fit a rectangular area; the application scenarios of the two embodiments are shown in fig. 6 and fig. 7 in sequence.

In one embodiment, the overall flow of the scheme is described through an application scenario.

Firstly, after a user reads the requirements of the specification, the user knows that the user needs to install the positioning sticker on four edge corner points on the region of interest according to the direction indicated by the positioning sticker, and starts a camera to record. The region of interest is the user's whiteboard, as shown in fig. 8, with a locating sticker.

Then, when responding to the target event, after initiating the whiteboard mode, the terminal automatically detects the positioning symbol in the acquired picture, and the method for detecting the positioning symbol is to calculate by using a detector based on a convolutional neural network, so as to obtain the position of the positioning symbol in the picture and the represented azimuth thereof.

When four positioning symbols are detected at one time and the positions of the four positioning symbols are respectively corresponding to the represented directions, the position of the whiteboard is determined to be a target area, and the image acquisition direction can be adjusted according to the target area to acquire images, and projection mapping can also be directly performed.

When the number of the detected positioning symbols is less than four, determining the already detected positioning symbols as first positioning symbols and determining the undetected positioning symbols as second positioning symbols; determining a detection area of a second positioning symbol according to the position and the azimuth of the first positioning symbol so as to convert the second positioning symbol into the first positioning symbol as much as possible, thereby obtaining the updated number, the updated position and the updated azimuth of the first positioning symbol; and when the number of the updated positioning symbols is four and the updated positions are consistent with the updated positions, connecting the updated positioning symbols to obtain a target area of the positioning symbols. Wherein, for the detection area, detection is carried out once every 0.5 seconds, and the whiteboard mode is exited if continuous detection is unsuccessful for 10 seconds.

After the target area is detected, the correction of the quadrilateral area is realized through a projection image obtained by projecting the target area, and the processing such as image amplification, sharpening, contrast enhancement and the like can be performed to enhance the image definition.

In order to correct the image more accurately, the method further comprises the step of filtering the target area according to a rectangle before the correction is performed. It should be appreciated that if the four positioning stickers to which the user sticks form a perfect rectangle, or the error is small, then there is a good visual effect after correction. If the display itself is quite different from the standard rectangle, poor post-display effects may result. We can design an algorithm to filter this very irregular quadrilateral. The designed filtering algorithm is that 4 positioning symbols are detected to be connected in pairs according to a clockwise sequence (namely, the upper left corner, the upper right corner, the lower left corner and the upper left corner are connected) to determine a quadrilateral region; and judging each boundary included angle of the quadrangular region, if each boundary included angle is more than or equal to 60 degrees and less than or equal to 120 degrees, judging that the rectangle is legal, judging that the detection is successful, entering the subsequent step, otherwise, judging that the detection is failed, and continuing to detect.

Therefore, the embodiment provides a whiteboard detection and correction scheme based on positioning sticker, a user attaches the positioning sticker to 4 corner points of a region of interest to be corrected of a whiteboard according to a specified form, and a terminal automatically identifies the 4 corner points through an image algorithm and carries out definition processing such as rectangle correction, amplification, display effect enhancement and the like on the region of interest of the user. The method and the device have the advantages that the region of interest is not required to have obvious boundaries, the identification accuracy is extremely high, the false detection rate is extremely low, and the like. In addition, to the camera of taking the cloud platform, the scheme of this embodiment can be in the whiteboard under the circumstances that only part goes into the mirror automatic search whiteboard, and intelligent degree is high, and interactive experience is good.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiments of the present application also provide an image correction apparatus for implementing the above-mentioned image correction method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the image correction device or devices provided below may be referred to the limitation of the image correction method hereinabove, and will not be repeated here.

In one embodiment, as shown in fig. 9, there is provided an image correction apparatus including: a detection module 902, a target area determination module 904, an image correction module 906, wherein:

the detection module 902 is configured to perform positioning symbol detection on an image to be processed, so as to obtain a position of the positioning symbol in the image and a direction represented by the positioning symbol;

the target area determining module 904 is configured to connect the positioning symbols to obtain a target area of the positioning symbols when it is detected that the number of the positioning symbols meets a preset condition and the position corresponds to the azimuth;

the image correction module 906 is configured to correct the target area to obtain a corrected target image.

In one embodiment, the image acquisition module 902 is configured to:

In one embodiment, the target area determining module 904 is configured to:

connecting the positioning symbols to obtain an initial area;

if not, filtering the initial area;

In one embodiment, the target area determining module 904 is specifically configured to:

calculating the boundary included angle of the initial area;

and taking the filtered initial area as the target area.

In one embodiment, the positioning symbols include a first positioning symbol and a second positioning symbol, the first positioning symbol is a detected positioning symbol, the second positioning symbol is an undetected positioning symbol, and the position and the orientation are a position and an orientation of the first positioning symbol;

Correspondingly, the target area determining module 904 is further configured to determine, when the number of the first positioning symbols does not meet the preset condition, a detection area of the second positioning symbol according to the direction;

the image acquisition module 902 is further configured to detect the second positioning symbol in the detection area, to obtain an updated number, an updated position, and an updated azimuth of the first positioning symbol;

the target area determining module 904 is further configured to connect the updated positioning symbol to obtain a target area of the positioning symbol when the updated number meets the preset condition and the updated position is consistent with the updated azimuth.

In one embodiment, the image acquisition module 902 is further configured to: adjusting the acquisition direction of the image according to the direction represented by the first positioning symbol; re-acquiring the image according to the adjusted acquisition direction;

correspondingly, the target area determining module 904 is further configured to determine, in the re-acquired image, a detection area of the second positioning symbol according to the position.

In one embodiment, the image correction module 906 is configured to:

and carrying out projection correction on the target area according to the perspective change matrix to obtain a corrected projection image of the target area.

By executing the method of any embodiment, the identification accuracy of the region of interest can be improved.

The respective modules in the above-described image correction apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a terminal, and an internal structure diagram thereof may be as shown in fig. 10. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement an image correction method. The display unit of the computer equipment is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device, wherein the display screen can be a liquid crystal display screen or an electronic ink display screen, the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on a shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 10 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims

1. An image correction method, the method comprising:

And correcting the target area to obtain a corrected target image.

2. The method according to claim 1, wherein the performing positioning symbol detection on the image to be processed to obtain a position of the positioning symbol in the image and an orientation characterized by the positioning symbol includes:

3. The method of claim 1, wherein the concatenating the positioning symbol to obtain the target region of the positioning symbol comprises:

connecting the positioning symbols to obtain an initial area;

if not, filtering the initial area;

4. The method of claim 3, wherein the filtering the initial region according to the boundary angle, and determining whether the initial region is a target region, comprises:

Calculating the boundary included angle of the initial area;

and taking the filtered initial area as the target area.

5. The method of claim 1, wherein the positioning symbols comprise a first positioning symbol and a second positioning symbol, the first positioning symbol being a detected positioning symbol and the second positioning symbol being an undetected positioning symbol, the position and the orientation being a position and an orientation of the first positioning symbol; the method further comprises the steps of:

6. The method of claim 5, wherein the determining the detection area of the second positioning symbol based on the position characterized by the first positioning symbol comprises:

re-acquiring the image according to the adjusted acquisition parameters;

7. The method of claim 1, wherein correcting the target region to obtain a corrected target image comprises:

8. An image correction apparatus, characterized in that the apparatus comprises:

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.