CN110930336B

CN110930336B - Image processing method and device, electronic equipment and storage medium

Info

Publication number: CN110930336B
Application number: CN201911199655.5A
Authority: CN
Inventors: 高哲峰; 李若岱; 马堃; 庄南庆
Original assignee: Shenzhen Sensetime Technology Co Ltd
Current assignee: Shenzhen Sensetime Technology Co Ltd
Priority date: 2019-11-29
Filing date: 2019-11-29
Publication date: 2023-11-28
Anticipated expiration: 2039-11-29
Also published as: CN110930336A

Abstract

The present disclosure relates to an image processing method and apparatus, an electronic device, and a storage medium, the method including: acquiring an image to be processed; and carrying out distortion correction processing on a plurality of rectangular grids in the image to be processed according to the calibration parameters of the image acquisition device to obtain a target image. According to the image processing method disclosed by the embodiment of the invention, the batch images can be automatically distorted and corrected through the calibration parameters respectively corresponding to the rectangular grids, each image is not required to be calibrated, the manual participation can be reduced, and the correction accuracy and the working efficiency are improved.

Description

Image processing method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of computer vision, and in particular relates to an image processing method and device, electronic equipment and a storage medium.

Background

The image captured by the image capturing device may be distorted (e.g., projection distortion, lens distortion, etc.), for example, the image capturing device of the OCR (Optical Character Recognition ) apparatus is located at the side of the plane to be processed, and the captured image of the plane to be processed is distorted, which may affect the recognition effect.

In the related art, image distortion correction requires a lot of manpower to perform four-corner calibration and camera calibration to solve both geometric distortion and lens distortion. Because of involving much labor, the working efficiency is low, the operation error rate is high, and batch operation is not possible.

Disclosure of Invention

The disclosure provides an image processing method and device, electronic equipment and a storage medium.

According to an aspect of the present disclosure, there is provided an image processing method including:

acquiring an image to be processed, wherein the image to be processed is an image of a plane to be processed, which is acquired by an image acquisition device and is divided into a plurality of rectangular lattices, an included angle between the optical axis direction of the image acquisition device and the normal direction of the plane to be processed is the preset angle, and the plurality of rectangular lattices in the image to be processed are distorted;

and carrying out distortion correction processing on a plurality of rectangular grids in the image to be processed according to the calibration parameters of the image acquisition device to obtain a target image, wherein the calibration parameters comprise homography transformation parameters respectively corresponding to the plurality of rectangular grids in a calibration plane, and the dividing mode of dividing the rectangular grids by the calibration plane is the same as the dividing mode of dividing the plane to be processed.

According to the image processing method disclosed by the embodiment of the invention, the batch images can be automatically distorted and corrected through the calibration parameters respectively corresponding to the rectangular grids, each image is not required to be calibrated, the manual participation can be reduced, and the correction accuracy and the working efficiency are improved.

In one possible implementation manner, the pixel points in the rectangular grid in the image to be processed correspond to homography conversion parameters of the rectangular grid in the same position in the calibration plane.

In one possible implementation manner, according to calibration parameters of the image acquisition device, distortion correction processing is performed on a plurality of rectangular lattices in an image to be processed to obtain a target image, including:

respectively obtaining homogeneous coordinates of the pixel points in each rectangular grid according to the position coordinates of the pixel points in the rectangular grid and homography transformation parameters corresponding to the rectangular grid;

obtaining target coordinates of a plurality of pixel points in each rectangular grid according to the homogeneous coordinates;

and obtaining the target image according to the target coordinates of the pixel points in the rectangular grids.

In one possible implementation, the method includes:

obtaining a calibration image, wherein the calibration image is an image of a calibration plane obtained through an image obtaining device, an included angle between the optical axis direction of the image obtaining device and the normal direction of the calibration plane is a preset angle, and a plurality of rectangular grids in the calibration image are distorted;

Acquiring first vertex coordinates of a plurality of distorted rectangular grids in the calibration image;

and determining calibration parameters of the image acquisition device according to the first vertex coordinates and second vertex coordinates of a plurality of rectangular grids on the calibration plane.

In this way, homography transformation parameters (for example, homography matrix) of each rectangular grid can be obtained respectively, the homography transformation parameters can be used for determining undistorted coordinates of each pixel point in the rectangular grid, and inherent parameters of the image acquisition device can be calibrated parameters Gu Huawei, so that any image acquired by the image acquisition device can be subjected to distortion correction, each image needs to be calibrated, manual participation can be reduced, and correction accuracy and working efficiency can be improved.

In one possible implementation manner, determining the calibration parameter according to the first vertex coordinate and the second vertex coordinates of the plurality of rectangular lattices on the calibration plane includes:

determining homography transformation parameters of the plurality of rectangular grids according to the first vertex coordinates and the second vertex coordinates of the plurality of rectangular grids;

and obtaining the calibration parameters according to homography transformation parameters of the rectangular grids.

In one possible implementation, the method further includes:

And storing the calibration parameters in a solid-state storage unit of the image acquisition device.

According to an aspect of the present disclosure, there is provided an image processing apparatus including:

the first acquisition module is used for acquiring an image to be processed, wherein the image to be processed is an image of a plane to be processed, which is acquired through an image acquisition device and is divided into a plurality of rectangular lattices, an included angle between the optical axis direction of the image acquisition device and the normal direction of the plane to be processed is the preset angle, and the plurality of rectangular lattices in the image to be processed are distorted;

the correction module is used for carrying out distortion correction processing on a plurality of rectangular grids in the image to be processed according to the calibration parameters of the image acquisition device to obtain a target image, wherein the calibration parameters comprise homography transformation parameters respectively corresponding to the plurality of rectangular grids in a calibration plane, and the division mode of dividing the rectangular grids by the calibration plane is the same as the division mode of dividing the plane to be processed.

In one possible implementation, the correction module is further configured to:

In one possible implementation, the apparatus further includes:

the second acquisition module is used for acquiring a calibration image, wherein the calibration image is an image of a calibration plane acquired by an image acquisition device, an included angle between the optical axis direction of the image acquisition device and the normal direction of the calibration plane is a preset angle, and a plurality of rectangular grids in the calibration image are distorted;

the third acquisition module is used for acquiring first vertex coordinates of a plurality of distorted rectangular grids in the calibration image;

and the determining module is used for determining the calibration parameters of the image acquisition device according to the first vertex coordinates and the second vertex coordinates of the plurality of rectangular lattices on the calibration plane.

In one possible implementation, the determining module is further configured to:

In one possible implementation, the apparatus further includes:

According to an aspect of the present disclosure, there is provided an electronic apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: the above image processing method is performed.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described image processing method.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the technical aspects of the disclosure.

FIG. 1 shows a flow chart of an image processing method according to an embodiment of the present disclosure;

FIGS. 2A and 2B illustrate schematic diagrams of calibration planes according to embodiments of the present disclosure;

fig. 3 shows an application schematic of an image processing method according to an embodiment of the present disclosure;

fig. 4 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure;

fig. 5 shows a block diagram of an electronic device according to an embodiment of the disclosure;

fig. 6 shows a block diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.

Fig. 1 shows a flowchart of an image processing method according to an embodiment of the present disclosure, as shown in fig. 1, the method including:

in step S11, an image to be processed is acquired, wherein the image to be processed is an image of a plane to be processed, which is acquired by an image acquisition device and is divided into a plurality of rectangular lattices, an included angle between an optical axis direction of the image acquisition device and a normal direction of the plane to be processed is the preset angle, and the plurality of rectangular lattices in the image to be processed are distorted;

In step S12, distortion correction is performed on a plurality of rectangular lattices in the image to be processed according to calibration parameters of the image acquisition device, so as to obtain a target image, where the calibration parameters include homography transformation parameters corresponding to the plurality of rectangular lattices in a calibration plane, and a division manner of dividing the rectangular lattices by the calibration plane is the same as a division manner of dividing the plane to be processed.

In one possible implementation, the image processing method may be performed by a terminal device or other processing device, where the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, etc. The other processing device may be a server or cloud server, etc. In some possible implementations, the image processing method may be implemented by way of a processor invoking computer readable instructions stored in a memory.

In one possible implementation, the image processing method may be used to correct distortion of an image captured by an image capturing device such as a camera. In an example, the image to be processed acquired by the image acquisition device of the OCR (Optical Character Recognition ) apparatus may be subjected to distortion correction, the image acquisition device of the OCR apparatus is located at a side of a plane to be processed (for example, a plane printed with characters), the captured image of the plane to be processed is distorted (for example, projection distortion, lens distortion, etc.), and the calibration method may be used to acquire calibration parameters of the image acquisition device of the OCR apparatus and to perform distortion correction on the image acquired by the image acquisition device, so as to improve recognition accuracy.

In one possible implementation, the image acquisition device may be calibrated first, and the method further includes: obtaining a calibration image, wherein the calibration image is an image of a calibration plane which is divided into a plurality of rectangular lattices and is obtained by an image obtaining device, wherein an included angle between the optical axis direction of the image obtaining device and the normal direction of the calibration plane is a preset angle, and the plurality of rectangular lattices in the calibration image are distorted; acquiring first vertex coordinates of a plurality of distorted rectangular grids in the calibration image; and determining calibration parameters of the image acquisition device according to the first vertex coordinates and second vertex coordinates of a plurality of rectangular lattices on a calibration plane, wherein the calibration parameters comprise homography transformation parameters respectively corresponding to the plurality of rectangular lattices.

In one possible implementation, the calibration plane may be a plane divided into a plurality of rectangular cells, such as a checkerboard. In an example, the checkerboard may be a 60×40 checkerboard. The present disclosure does not limit the size of the checkerboard.

Fig. 2A and 2B show schematic diagrams of calibration planes according to embodiments of the present disclosure. As shown in fig. 2A, the calibration plane may be a plane divided into checkerboards, the calibration plane including a plurality of rectangular cells, and coordinates of vertices of each rectangular cell being identifiable in a calibration image of the calibration plane. As shown in FIG. 2B, the checkerboard can be black-white crossed checkerboard, and the contrast ratio between adjacent checkerboards is large, so that the recognition accuracy of the vertices of the rectangular checkerboard can be improved.

In one possible implementation, the calibration plane (e.g., the calibration plane divided into checkers) may be disposed on the imaging device of the OCR device such that the image acquisition means of the OCR device may capture an image of the calibration plane, i.e., a calibration image, from the side. In an example, the image capturing device of the OCR device may capture the calibration image at a side of the calibration plane, i.e. the optical axis of the image capturing device is not parallel to the normal direction of the calibration plane, e.g. the angle between the optical axis direction of the image capturing device and the normal direction of the calibration plane is a preset angle.

In one possible implementation, in the calibration image acquired from the side by the image acquisition device, the rectangular grid may be distorted, for example, due to a photographing angle problem, the rectangular grid may be distorted in projection (for example, the rectangular grid may be distorted into a trapezoid shape), and due to a lens problem, the rectangular grid may be distorted in lens (for example, a side of the rectangular grid is distorted from a straight line into a curved shape). Distortion such as projection distortion and lens distortion may cause distortion of an image acquired by the image acquisition device, and in OCR devices, the photographed image distortion may reduce the accuracy of character recognition.

In one possible implementation, the first vertex coordinates of a plurality of distorted rectangular lattices may be obtained in the calibration image, for example, if the rectangular lattice is not distorted, the vertex coordinates of the rectangular lattice are known (i.e. the second vertex coordinates), but if the rectangular lattice is distorted, the vertex coordinates are obtained by image detection or the like.

In an example, the first vertex coordinates of the rectangular grid may be detected by a neural network or the like, or may be detected by a findchessboard corners algorithm in the openCV. The present disclosure does not limit the detection method of the first vertex coordinates.

In one possible implementation, the calibration parameters may be determined from the first vertex coordinates and the second vertex coordinates. For example, the homography transformation parameters of each rectangular grid may be determined according to the first vertex coordinates and the second vertex coordinates of each rectangular grid, and the calibration parameters may include homography transformation parameters of a plurality of rectangular grids.

In one possible implementation manner, determining the calibration parameters of the image acquisition device according to the first vertex coordinates and the second vertex coordinates of the plurality of rectangular lattices on the calibration plane may include: determining homography transformation parameters of the plurality of rectangular grids according to the first vertex coordinates and the second vertex coordinates of the plurality of rectangular grids; and obtaining the calibration parameters according to homography transformation parameters of the rectangular grids.

In one possible implementation manner, for any one rectangular grid, due to distortion, there is a certain difference between the first vertex coordinate and the second vertex coordinate of the rectangular grid, where the first vertex coordinate is a coordinate detected on the calibration image, and the second vertex coordinate is an actual coordinate of the rectangular grid, that is, a coordinate obtained according to a division manner of the calibration plane.

In one possible implementation, the homography matrix of the rectangular lattice may be determined from the first vertex coordinates and the second vertex coordinates. For example, the homogeneous coordinates of the first vertex coordinates and the homogeneous coordinates of the second vertex coordinates may be calculated, respectively, and for example, the homogeneous coordinates of the first vertex coordinates in the form of a vector and the homogeneous coordinates of the second vertex coordinates in the form of a vector may be obtained. And obtaining a homography matrix (namely homography transformation parameters) between the first vertex coordinates and the second vertex coordinates according to the homogeneous coordinates in the vector form, namely the homography matrix of the rectangular grid. Further, a homography matrix (e.g., a 3×3 matrix) of each rectangular lattice may be obtained in the manner described above, i.e., the calibration parameters are obtained.

In one possible implementation, although the homography matrix of the rectangular grid may be determined according to the vertex coordinates (i.e., the first vertex coordinates and the second vertex coordinates) of the rectangular grid, when the rectangular grid divided by the calibration plane is more and each rectangular grid area is smaller, the homography matrix of the vertex may be approximately used as the homography matrix of any pixel point in the rectangular grid, and may be used to correct distortion of any pixel point in the rectangular grid in the calibration image, so as to determine the undistorted coordinates of the pixel point.

In one possible implementation, the calibration parameter may be stored in a solid state memory unit of the image capturing device as an intrinsic parameter of the image capturing device for correcting the distortion of any image obtained by the image capturing device. For example, the calibration parameters may be stored in binary form in a flash of the image acquisition device as intrinsic parameters of the image acquisition device.

In a possible implementation manner, in step S11, the calibration parameter may be used to correct distortion of the image to be processed captured by the image capturing device. For example, an image acquisition device in an OCR apparatus may be used to capture a plane to be processed (e.g., a plane in which a print including characters is located, etc.) on an imaging device, to obtain an image to be processed. The included angle between the optical axis direction of the image acquisition device and the normal direction of the plane to be processed is the same as the included angle between the optical axis direction of the image acquisition device and the normal direction of the calibration plane, and is the preset included angle, that is, the angle of the image acquisition device for shooting the plane to be processed is consistent with the angle of the image for shooting the calibration plane, and the distortion of the image to be processed is consistent with the calibration image. Therefore, the distortion correction processing can be performed on the image to be processed using the calibration parameters obtained from the calibration image.

In one possible implementation, the image to be processed is divided in a manner consistent with the division of the calibration image, for example, the image to be processed may be also divided into 60×40 checkerboards. The pixel points in each rectangular grid can use homography transformation parameters of the corresponding rectangular grid in the calibration parameters to correct distortion.

In one possible implementation, step S12 may include: respectively obtaining homogeneous coordinates of the pixel points in each rectangular grid according to the position coordinates of the pixel points in the rectangular grid and homography transformation parameters corresponding to the rectangular grid; obtaining target coordinates of a plurality of pixel points in each rectangular grid according to the homogeneous coordinates; and obtaining the target image according to the target coordinates of the pixel points in the rectangular grids.

In one possible implementation manner, the pixel points in the rectangular grid in the image to be processed correspond to homography conversion parameters of the rectangular grid in the same position in the calibration plane, that is, the plurality of pixel points in the matrix grid of the image to be processed can use homography conversion parameters of the rectangular grid in the same position in the calibration plane to correct distortion. Although the homography matrix of the rectangular lattice can be determined according to the vertex coordinates of the rectangular lattice, when the rectangular lattices divided by the plane to be processed are more and each rectangular lattice has smaller area, the homography matrix of the vertex can be approximately used as the homography matrix of any pixel point in the rectangular lattice and can be used for carrying out distortion correction on any pixel point in the rectangular lattice so as to determine the undistorted coordinates of the pixel point.

In one possible implementation, the coordinates of the pixel points in the rectangular grid with distortion in the image to be processed may be represented as a vector form, and the homogeneous coordinates of the pixel points are obtained according to the coordinates in the vector form and the homography transformation parameters (for example, homography matrix) corresponding to the rectangular grid, that is, the coordinates after homography transformation of the homography matrix (the coordinates in the vector form). For example, the homography matrix may be used to multiply the coordinates of the pixel in vector form to obtain homogeneous coordinates of the pixel.

In one possible implementation, the target coordinates of the pixel points, i.e., the undistorted coordinates, may be obtained from the homogeneous coordinates of the pixel points. In an example, the homogeneous coordinates of the pixel point may be coordinates in the form of a vector, the target coordinates of the pixel point may be determined according to the coordinates in the form of a vector, for example, the coordinates in the form of a vector may be three-dimensional coordinates (x, y, z), and the target coordinates of the pixel point may be obtained according to the three-dimensional coordinates as (x/z, y/z).

In one possible implementation, the target coordinates of each pixel point in the image to be processed may be determined according to the above manner. And the target coordinates may be used to obtain a target image. For example. Each pixel point in the image to be processed can be translated to the position of the target coordinate to obtain the target image, and the mode of obtaining the target image is not limited in the present disclosure.

In one possible implementation, the distortion correction process may include a correction process for projection distortion and lens distortion. Under the condition that the rectangular grids are small in area and large in number, lens distortion in each rectangular grid is negligible, and in the plurality of rectangular grids, the lens distortion can have a certain influence, namely, the influence generated after the lens distortion of each rectangular grid is overlapped. When the pixel points in the plurality of rectangular grids of the image to be processed are subjected to distortion correction at the same time, projection distortion can be corrected through homography matrixes of the rectangular grids, and meanwhile, lens distortion of each rectangular grid is corrected in the correction process of the projection distortion, so that the lens distortion and the projection distortion of the plurality of rectangular grids of the image to be processed are corrected.

Fig. 3 shows an application schematic of an image processing method according to an embodiment of the present disclosure. As shown in fig. 3, the OCR device may be provided with a reflective optical path to reduce the thickness of the OCR device, and the image capturing device may capture a virtual image of the calibration plane at the side. In an example, the calibration plane is divided into a plurality of rectangular lattices, for example, into a 60×40 checkerboard.

In one possible implementation, the image capturing device may capture a virtual image of the calibration plane, and obtain a calibration image in which each rectangular grid may be distorted. In an example, a findchessboard corners algorithm in the neural network or openCV may be employed to detect the first vertex coordinates of the rectangular grid. And obtaining the second vertex coordinates of the rectangular grid according to the dividing mode of the calibration plane.

In one possible implementation manner, for any rectangular lattice, the homogeneous coordinates of the first vertex coordinate and the homogeneous coordinates of the second vertex coordinate may be calculated respectively, and the homography matrix between the first vertex coordinate and the second vertex coordinate may be determined according to the homogeneous coordinates. Further, the homography matrix of each rectangular lattice can be obtained in the above manner, that is, the calibration parameters of the image acquisition device are obtained. And saving the calibration parameters in a flash of the image acquisition device to serve as inherent parameters of the image acquisition device for correcting the to-be-processed image of any to-be-processed plane of the OCR equipment.

In one possible implementation manner, a printed matter optionally including characters may be disposed on an imaging device where the calibration plane is located, to obtain a plane to be processed, and the plane to be processed is photographed by an image obtaining device, to obtain an image to be processed. The plane to be processed is also divided into 60 x 40 checkerboards. And correcting the distortion of each rectangular grid in the image to be processed through the homography matrix of each rectangular grid in the calibration parameters.

In one possible implementation manner, the coordinates of the pixel point in the rectangular grid with distortion in the image to be processed can be expressed as a vector form, and matrix multiplication is performed on the vector form and the homography matrix of the corresponding rectangular grid to obtain homogeneous coordinates of the pixel point, so as to obtain the target coordinates of the pixel point, namely, the undistorted coordinates. The method can obtain the target coordinates of each pixel point, and translate the pixel points in the image to be processed to the positions of the target coordinates, so that the target image with smaller distortion can be obtained.

In one possible implementation manner, for the OCR device, the image acquiring device in the OCR device is a macro camera, and the calibration plane set on the imaging device can be shot at a fixed angle, so that images at a plurality of angles are not required to be shot, calibration parameters can be obtained only by shooting one calibration image, excessive manual calibration is avoided, the calibration stability can be improved, the method can be used in distortion correction of other images shot at the same angle, the method is suitable for batch correction of a plurality of images, the processing stability is high, and the recognition accuracy of the OCR device can be improved.

In one possible implementation, the calibration method according to embodiments of the present disclosure may be used in distortion correction of OCR devices, human verification devices, scanning or imaging devices. Aiming at the condition that the image acquisition device shoots images at a fixed angle, the calibration parameters of the image acquisition device can be obtained through one calibration image, the image to be processed is subjected to distortion correction by utilizing the calibration parameters, the image with smaller distortion is obtained, the shooting of each image is not required to be subjected to manual calibration, correction and other treatments, and the processing efficiency and the accuracy are improved. The application field of the calibration mode is not limited by the present disclosure.

Fig. 4 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure, as shown in fig. 4, the apparatus including:

a first obtaining module 11, configured to obtain an image to be processed, where the image to be processed is an image of a plane to be processed, which is obtained by an image obtaining device and is divided into a plurality of rectangular lattices, an included angle between an optical axis direction of the image obtaining device and a normal direction of the plane to be processed is the preset angle, and distortion exists in the plurality of rectangular lattices in the image to be processed;

and the correction module 12 is configured to perform distortion correction processing on a plurality of rectangular lattices in the image to be processed according to calibration parameters of the image acquisition device, so as to obtain a target image, where the calibration parameters include homography transformation parameters corresponding to the plurality of rectangular lattices in a calibration plane, and a division manner of dividing the rectangular lattices by the calibration plane is the same as a division manner of dividing the plane to be processed.

In one possible implementation, the correction module is further configured to:

In one possible implementation, the apparatus further includes:

It will be appreciated that the above-mentioned method embodiments of the present disclosure may be combined with each other to form a combined embodiment without departing from the principle logic, and are limited to the description of the present disclosure.

In addition, the disclosure further provides an image processing apparatus, an electronic device, a computer readable storage medium, and a program, where the foregoing may be used to implement any one of the image processing methods provided in the disclosure, and corresponding technical schemes and descriptions and corresponding descriptions referring to method parts are not repeated.

It will be appreciated by those skilled in the art that in the above-described method of the specific embodiments, the written order of steps is not meant to imply a strict order of execution but rather should be construed according to the function and possibly inherent logic of the steps.

In some embodiments, a function or a module included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and a specific implementation thereof may refer to the description of the foregoing method embodiments, which is not repeated herein for brevity

The disclosed embodiments also provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method. The computer readable storage medium may be a non-volatile computer readable storage medium.

The embodiment of the disclosure also provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured as the method described above.

The electronic device may be provided as a terminal, server or other form of device.

Fig. 5 is a block diagram of an electronic device 800, according to an example embodiment. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.

Referring to fig. 5, an electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen between the electronic device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operational mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the electronic device 800. For example, the sensor assembly 814 may detect an on/off state of the electronic device 800, a relative positioning of the components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of a user's contact with the electronic device 800, an orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices, either wired or wireless. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi,2G, or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 804 including computer program instructions executable by processor 820 of electronic device 800 to perform the above-described methods.

Fig. 6 is a block diagram illustrating an electronic device 1900 according to an example embodiment. For example, electronic device 1900 may be provided as a server. Referring to FIG. 6, electronic device 1900 includes a processing component 1922 that further includes one or more processors and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by processing component 1922. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, processing component 1922 is configured to execute instructions to perform the methods described above.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 1932, including computer program instructions executable by processing component 1922 of electronic device 1900 to perform the methods described above.

The present disclosure may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.

Computer program instructions for performing the operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized in that the method further comprises:

acquiring an image to be processed, wherein the image to be processed is an image of a plane to be processed, which is divided into a plurality of rectangular lattices and is acquired by an image acquisition device, an included angle between the optical axis direction of the image acquisition device and the normal direction of the plane to be processed is a preset angle, and the plurality of rectangular lattices in the image to be processed are distorted;

performing distortion correction processing on a plurality of rectangular grids in the image to be processed according to calibration parameters of the image acquisition device to obtain a target image, wherein the calibration parameters comprise homography conversion parameters respectively corresponding to the plurality of rectangular grids in a calibration plane, the division mode of dividing the rectangular grids by the calibration plane is the same as the division mode of dividing the plane to be processed, and the distortion correction processing comprises correction processing on projection distortion and lens distortion;

The distortion correction processing for the plurality of rectangular grids in the image to be processed comprises the following steps:

according to homography transformation parameters respectively corresponding to the rectangular lattices in the calibration plane, correcting the projection distortion of the rectangular lattices in the image to be processed;

and in the process of correcting the projection distortion, correcting the lens distortion of each rectangular grid.

2. The method according to claim 1, wherein the pixel points in the rectangular grid in the image to be processed correspond to homography transformation parameters of the rectangular grid in the same position in the calibration plane.

3. The method according to claim 2, wherein performing distortion correction processing on a plurality of rectangular lattices in the image to be processed according to the calibration parameters of the image acquisition device to obtain the target image comprises:

4. The method according to claim 1, characterized in that the method comprises:

5. The method of claim 4, wherein determining the calibration parameters based on the first vertex coordinates and second vertex coordinates of a plurality of rectangular cells on a calibration plane comprises:

6. The method according to claim 4, wherein the method further comprises:

7. An image processing apparatus, comprising:

the image processing device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring an image to be processed, the image to be processed is an image of a plane to be processed, the plane to be processed is divided into a plurality of rectangular lattices, the included angle between the optical axis direction of the image acquisition device and the normal direction of the plane to be processed is a preset angle, and the plurality of rectangular lattices in the image to be processed are distorted;

the correction module is used for carrying out distortion correction processing on a plurality of rectangular grids in the image to be processed according to the calibration parameters of the image acquisition device to obtain a target image, wherein the calibration parameters comprise homography transformation parameters respectively corresponding to the plurality of rectangular grids in a calibration plane, the division mode of dividing the rectangular grids by the calibration plane is the same as the division mode of dividing the plane to be processed, and the distortion correction processing comprises correction processing on projection distortion and lens distortion;

wherein, the correction module is specifically used for:

8. The apparatus of claim 7, wherein the apparatus further comprises:

9. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

Wherein the processor is configured to: performing the method of any one of claims 1 to 6.

10. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 6.