WO2022027191A1 - 平面矫正方法及装置、计算机可读介质和电子设备 - Google Patents

平面矫正方法及装置、计算机可读介质和电子设备 Download PDF

Info

Publication number
WO2022027191A1
WO2022027191A1 PCT/CN2020/106624 CN2020106624W WO2022027191A1 WO 2022027191 A1 WO2022027191 A1 WO 2022027191A1 CN 2020106624 W CN2020106624 W CN 2020106624W WO 2022027191 A1 WO2022027191 A1 WO 2022027191A1
Authority
WO
WIPO (PCT)
Prior art keywords
plane
image
area
processed
correction
Prior art date
Application number
PCT/CN2020/106624
Other languages
English (en)
French (fr)
Inventor
顾磊
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to CN202080104505.6A priority Critical patent/CN116134476A/zh
Priority to PCT/CN2020/106624 priority patent/WO2022027191A1/zh
Publication of WO2022027191A1 publication Critical patent/WO2022027191A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to a plane correction method, a plane correction device, a computer-readable medium, and an electronic device.
  • the Manhattan World Hypothesis believes that most artificially constructed objects, such as buildings and roads, have strong orthogonal properties, and there is a Cartesian coordinate system that makes the surface of the object parallel or perpendicular to the coordinate axis, as shown in Figure 1.
  • this hypothesis can be used to construct a plane model, and the acquired image to be processed can be processed such as plane detection and correction, so that the to-be-processed image can be converted into a correction containing the front view of the object surface. image, in order to obtain information on the surface of the object.
  • plane detection and correction processing can be performed on the captured image to be processed, so that the to-be-processed image is converted into a corrected image containing the front view of the paper document, so as to facilitate reading according to the corrected image Information such as text in paper documents.
  • the purpose of the present disclosure is to provide a plane correction method, a plane correction device, a computer-readable medium and an electronic device, thereby at least to a certain extent avoid the related art, when the image to be processed is corrected directly by the correction matrix, because the image to be processed is directly corrected. It contains too much non-target plane information, which leads to the problem of distortion in the correction result.
  • a plane correction method comprising: acquiring plane parameters of a target plane in an image to be processed, and calculating a position identification line and a correction matrix corresponding to the to-be-processed image according to the plane parameters; wherein the position identification line is in the On the image plane of the image to be processed; when there is an intersection on the image plane between the position identification line and the frame of the image to be processed, the correction area is determined in the image to be processed according to the plane parameters, and the correction area is corrected according to the correction matrix, and the correction is obtained.
  • a plane correction device comprising: a parameter calculation module for acquiring plane parameters of a target plane in an image to be processed, and calculating a position identification line and a correction matrix corresponding to the image to be processed according to the plane parameters; Wherein, the position identification line is on the image plane of the image to be processed; the plane correction module is used to determine the correction area in the image to be processed according to the plane parameters when there is an intersection point between the position identification line and the frame of the image to be processed on the image plane, And correct the correction area according to the correction matrix to obtain a corrected image.
  • a computer-readable medium on which a computer program is stored, and when the computer program is executed by a processor, implements the above-mentioned method.
  • an electronic device characterized by comprising:
  • a memory for storing one or more programs, which, when executed by one or more processors, enables the one or more processors to implement the above-mentioned method.
  • the plane parameters of the target plane in the to-be-processed image are obtained, and the position identification line and the correction matrix corresponding to the to-be-processed image are calculated according to the plane parameters;
  • a correction area is determined in the image to be processed according to the plane parameters, and the correction area is corrected to obtain a corrected image.
  • the position identification line it can be determined by the position identification line whether there will be an error in directly correcting the image to be processed through the correction matrix;
  • the correction area is determined in the image to be processed, and then the correction area is corrected by the correction matrix, so as to avoid the problem of incorrect correction results caused by too much non-target plane information contained in the image to be processed.
  • Fig. 1 shows a schematic diagram of the orthogonal properties that most objects in a three-dimensional control have
  • FIG. 2 shows a schematic diagram of an exemplary system architecture to which embodiments of the present disclosure may be applied;
  • FIG. 3 shows a schematic diagram of an electronic device to which an embodiment of the present disclosure can be applied
  • Fig. 4 shows a kind of image to be processed
  • Fig. 5 shows that the to-be-processed image of Fig. 4 is corrected by the related art to obtain a corrected image
  • Figure 6 shows another image to be processed and its corresponding corrected image
  • Fig. 7 schematically shows another image to be processed and its corresponding corrected image
  • Figure 8 schematically shows yet another image to be processed and its corresponding corrected image
  • FIG. 9 schematically shows a flow chart of a plane correction method in an exemplary embodiment of the present disclosure.
  • FIG. 10 schematically shows a principle diagram of determining a position identification line in an exemplary embodiment of the present disclosure
  • FIG. 11 schematically shows another principle diagram of determining a position identification line in an exemplary embodiment of the present disclosure
  • FIG. 12 schematically shows a schematic diagram of a frame enhancement method in an exemplary embodiment of the present disclosure
  • FIG. 13 schematically shows a schematic diagram of another frame enhancement method in an exemplary embodiment of the present disclosure
  • FIG. 15 schematically shows a schematic diagram of another method for determining a correction area in an exemplary embodiment of the present disclosure
  • FIG. 16 schematically shows a schematic diagram of still another method for determining a correction area in an exemplary embodiment of the present disclosure
  • FIG. 17 schematically shows a schematic diagram of offsetting a position identification line in an exemplary embodiment of the present disclosure
  • FIG. 18 schematically shows a flowchart of another plane correction method in an exemplary embodiment of the present disclosure
  • FIG. 19 schematically shows a schematic diagram of the composition of a plane correction device in an exemplary embodiment of the present disclosure
  • FIG. 20 is a schematic diagram showing the composition of another plane correction device in an exemplary embodiment of the present disclosure.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
  • the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
  • FIG. 2 shows a schematic diagram of a system architecture of an exemplary application environment to which a plane correction method and apparatus according to embodiments of the present disclosure can be applied.
  • the system architecture 200 may include one or more of terminal devices 201 , 202 , and 203 , a network 204 and a server 205 .
  • the network 204 is a medium used to provide a communication link between the terminal devices 201 , 202 , 203 and the server 205 .
  • the network 204 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
  • the terminal devices 201, 202, and 203 may be various electronic devices with image processing functions, including but not limited to desktop computers, portable computers, smart phones, and tablet computers. It should be understood that the numbers of terminal devices, networks and servers in FIG. 1 are merely illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.
  • the server 205 may be a server cluster composed of multiple servers, or the like.
  • the plane correction method provided by the embodiment of the present disclosure is generally performed by the terminal devices 201 , 202 , and 203 , and correspondingly, the plane correction device is generally provided in the terminal devices 201 , 202 , and 203 .
  • the plane correction method provided by the embodiment of the present disclosure may also be executed by the server 205, and correspondingly, the plane correction device may also be provided in the server 205, which is not required in this exemplary embodiment. Make special restrictions.
  • the user can obtain the plane parameters of the target plane in the image to be processed through the terminal devices 201, 202, 203, calculate the position identification line and the correction matrix, and compare the position identification line with the to-be-processed image.
  • a correction area is determined in the image to be processed according to the plane parameters, and then the correction area is corrected according to the calculated correction matrix to obtain a corrected image corresponding to the target plane; in another embodiment , the server 205 may also acquire the relevant data provided by the terminal devices 201 , 202 , and 203 , and after processing through the plane correction method, feed back the processing result, ie, the corrected image, to the terminal devices 201 , 202 , and 203 .
  • An exemplary embodiment of the present disclosure provides an electronic device for implementing a plane correction method, which may be the terminal devices 201 , 202 , 203 or the server 205 in FIG. 1 .
  • the electronic device includes at least a processor and a memory for storing executable instructions of the processor, and the processor is configured to perform the plane correction method by executing the executable instructions.
  • the mobile terminal 300 in FIG. 3 takes the mobile terminal 300 in FIG. 3 as an example to illustrate the structure of the electronic device. It will be understood by those skilled in the art that the configuration in Figure 3 can also be applied to stationary type devices, in addition to components specifically for mobile purposes.
  • the mobile terminal 300 may include more or less components than shown, or combine some components, or separate some components, or different component arrangements.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the interface connection relationship between the components is only schematically shown, and does not constitute a structural limitation of the mobile terminal 300 .
  • the mobile terminal 300 may also adopt an interface connection manner different from that in FIG. 3 , or a combination of multiple interface connection manners.
  • the mobile terminal 300 may specifically include: a processor 310, an internal memory 321, an external memory interface 322, a Universal Serial Bus (USB) interface 330, a charging management module 340, a power management module 341, Battery 342, Antenna 1, Antenna 2, Mobile Communication Module 350, Wireless Communication Module 360, Audio Module 370, Speaker 371, Receiver 372, Microphone 373, Headphone Interface 374, Sensor Module 380, Display Screen 390, Camera Module 391, Indication 392, a motor 393, a key 394, a subscriber identification module (SIM) card interface 395, and the like.
  • the sensor module 380 may include a depth sensor 3801, a pressure sensor 3802, a gyroscope sensor 3803, and the like.
  • the processor 310 may include one or more processing units, for example, the processor 310 may include an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), controller, video codec, digital signal processor (Digital Signal Processor, DSP), baseband processor and/or Neural-Network Processing Unit (NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • an application processor Application Processor, AP
  • modem processor e.g., GPU
  • ISP image signal processor
  • ISP image Signal Processor
  • controller e.g., video codec
  • DSP Digital Signal Processor
  • NPU Neural-Network Processing Unit
  • NPU is a neural network (Neural-Network, NN) computing processor.
  • NN neural network
  • Applications such as intelligent cognition of the mobile terminal 300 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
  • a memory is provided in the processor 310 .
  • the memory can store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and the execution is controlled by the processor 310 .
  • the charging management module 340 is used to receive charging input from the charger.
  • the power management module 341 is used for connecting the battery 342 , the charging management module 340 and the processor 310 .
  • the power management module 341 receives input from the battery 342 and/or the charging management module 340, and supplies power to the processor 310, the internal memory 321, the display screen 390, the camera module 391, the wireless communication module 360, and the like.
  • the wireless communication function of the mobile terminal 300 may be implemented by the antenna 1, the antenna 2, the mobile communication module 350, the wireless communication module 360, the modulation and demodulation processor, the baseband processor, and the like.
  • the antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals;
  • the mobile communication module 350 can provide a wireless communication solution including 2G/3G/4G/5G applied on the mobile terminal 300;
  • the modulation and demodulation processor can include Modulator and demodulator;
  • the wireless communication module 360 can provide applications on the mobile terminal 300 including wireless local area networks (Wireless Local Area Networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) networks), Bluetooth (Bluetooth (Bluetooth) , BT) and other wireless communication solutions.
  • the antenna 1 of the mobile terminal 300 is coupled with the mobile communication module 350, and the antenna 2 is coupled with the wireless communication module 360, so that the mobile terminal 300 can communicate with the network and other devices through wireless communication technology.
  • the mobile terminal 300 implements a display function through a GPU, a display screen 390, an application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 390 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 310 may include one or more GPUs that execute program instructions to generate or alter display information.
  • the mobile terminal 300 can realize the shooting function through the ISP, the camera module 391, the video codec, the GPU, the display screen 390, the application processor, and the like.
  • the ISP is used to process the data fed back by the camera module 391;
  • the camera module 391 is used to capture still images or videos;
  • the digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals;
  • video The codec is used to compress or decompress the digital video, and the mobile terminal 300 may also support one or more video codecs.
  • the external memory interface 322 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile terminal 300.
  • the external memory card communicates with the processor 310 through the external memory interface 322 to realize the data storage function. For example to save files like music, video etc in external memory card.
  • Internal memory 321 may be used to store computer executable program code, which includes instructions.
  • the internal memory 321 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like.
  • the storage data area may store data (such as audio data, phone book, etc.) created during the use of the mobile terminal 300 and the like.
  • the internal memory 321 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, Universal Flash Storage (Universal Flash Storage, UFS), and the like.
  • the processor 310 executes various functional applications and data processing of the mobile terminal 300 by executing instructions stored in the internal memory 321 and/or instructions stored in a memory provided in the processor.
  • the mobile terminal 300 may implement audio functions through an audio module 370, a speaker 371, a receiver 372, a microphone 373, an earphone interface 374, an application processor, and the like. Such as music playback, recording, etc.
  • the depth sensor 3801 is used to acquire depth information of the scene.
  • the depth sensor may be disposed in the camera module 391, and the camera module 391 and the depth sensor 3801 simultaneously collect data on the scene, and the color data and depth data of the image to be processed can be obtained, that is, the corresponding image data of the image to be processed can be obtained. point cloud data.
  • the pressure sensor 3802 is used to sense pressure signals, and can convert the pressure signals into electrical signals.
  • the pressure sensor 3802 may be provided on the display screen 390 .
  • the gyroscope sensor 3803 may be used to determine the motion attitude of the mobile terminal 300 .
  • the angular velocity of the mobile terminal 300 about three axes ie, x, y and z axes
  • the gyroscope sensor 3803 can be used for image stabilization, navigation, and somatosensory game scenes.
  • sensors with other functions can also be set in the sensor module 380 according to actual needs, such as an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor. sensors, etc.
  • the mobile terminal 300 may further include other devices providing auxiliary functions.
  • the keys 394 include a power-on key, a volume key, etc., and the user can input key signals related to user settings and function control of the mobile terminal 300 through key input.
  • Another example is the indicator 392, the motor 393, the SIM card interface 395, and the like.
  • the developer in order to realize the correction of the target plane in the image to be processed, the developer directly corrects the image to be processed through the correction matrix calculated by the plane parameters.
  • the rectification result may be distorted.
  • the target plane 410 in the image to be processed as shown in FIG. 4 .
  • a corrected image as shown in FIG. 5 may be obtained.
  • the target plane 410 is corrected as the target plane 510 , which is obviously different from user requirements; another example, the correction process shown in Figure 6 also has obvious errors.
  • the above distortion is mainly caused by the existence of a large amount of information outside the target plane in the image to be processed.
  • Correcting the target plane in the two-dimensional image to be processed is actually performing perspective transformation on the target plane in the two-dimensional image to be processed, and this perspective transformation is not applicable to points beyond the vanishing line range corresponding to the target plane. If there are points beyond the vanishing line in the correction range, meaningless points will appear in the corrected image, so correction errors are likely to occur.
  • the vanishing line refers to the image formed in the camera when the plane extends to infinity
  • the vanishing line range refers to the image formed in the camera at the part of the plane whose distance is less than infinity.
  • one is to use the boundary information of the image to be processed to determine the target plane range of the image to be processed, and then correct the range; The cropped part is corrected.
  • the boundary information in the image to be processed is insufficient or the boundary is not clear, it is likely that the range of the target plane cannot be accurately determined, resulting in errors in correction; while in the second method, due to cropping Usually, it is also based on the image information of the image to be processed, so the cropped target plane may also not meet the user's requirements. In this case, some parts other than the target plane also need to be corrected, so that the user can manually correct the corrected image. Since the part outside the target plane needs to be corrected during the correction, there is still the possibility of errors in the correction.
  • the target plane may be corrected as a small part of the corrected image.
  • the resolution of the corrected image is also limited. Therefore, when manually correcting the corrected image, select the area corresponding to the target plane in the corrected image for After cropping, the corrected picture corresponding to the target plane is likely to have low image quality; as shown in Figure 8, in the corrected area including the target plane, there may still be points beyond the range of the vanishing line, so correction may occur. error, which makes the corrected image distorted.
  • the present disclosure provides a plane correction method and a plane correction device, which will be described in detail below.
  • FIG. 9 shows the flow of a plane correction method in this exemplary embodiment, including the following steps S910 and S920:
  • step S910 the plane parameters of the target plane in the image to be processed are acquired, and the position identification line and the correction matrix corresponding to the image to be processed are calculated according to the plane parameters.
  • the above-mentioned plane parameters are algebraic representation parameters of the target plane in three-dimensional space, which are used to determine a unique infinite plane in three-dimensional space, for example, P(a, b, c, d);
  • the matrix is a matrix calculated based on the plane parameters and used to perform perspective transformation on the two-dimensional image and correct the image.
  • the algebraic representation parameters in the three-dimensional space represent an infinite plane in the space, that is, P(a , b, c, d) represent a fixed infinite plane in space.
  • the position identification line is a specific line or surface in the three-dimensional space, and the imaging on the image plane where the image to be processed is located can be based on the principle of perspective transformation, and the specific position in the three-dimensional space can be identified in the two-dimensional image to be processed.
  • the image plane is further divided into near-scene imaging, which is closer than a specific line or plane distance, and far-scene imaging, which is farther than a specific line or plane distance.
  • the position identification line is always located on the image plane corresponding to the image to be processed, and the image plane is the imaging plane of the shooting camera.
  • the image plane is an infinite plane and is not limited by the size of the camera sensor.
  • the camera may collect point cloud data corresponding to the image to be processed by means of a photosensitive element and a depth sensor. Therefore, the plane parameters of the target plane in the image to be processed can be obtained by performing plane detection on the point cloud data.
  • the process of plane detection may be performed by methods such as RANSAC, which is not specifically limited in the present disclosure.
  • the image to be processed may include multiple planes at the same time, so the target plane in the plane to be processed needs to be determined first.
  • the area occupied by the surface of the object in the corresponding image to be processed is the largest. Therefore, plane detection can be performed on the point cloud data of the image to be processed, and among all the planes that can be detected, the plane that occupies the largest area on the image to be processed is determined as the target plane, and then the plane parameters corresponding to the target plane are obtained.
  • an error may occur after transforming points outside the range of the target plane corresponding to the vanishing line. Therefore, the vanishing line corresponding to the target plane can be used as the position identification line to ensure that no error occurs in the perspective transformation.
  • the camera internal parameters corresponding to the camera that shoots the image to be processed can be obtained, and then the vanishing line can be calculated according to the camera internal parameters and the plane parameters, and the vanishing line can be used as the position identification line.
  • the vanishing line refers to the imaging result on the image plane when the target plane extends to infinity.
  • the normal vector n of the target plane can be calculated through the plane parameters, and then the vanishing line can be calculated according to the following formula (1):
  • l 1 is the vanishing line
  • K is the camera internal parameter corresponding to the camera that shoots the image to be processed
  • n is the normal vector of the target plane.
  • a preset plane may be set, and the position identification line may be determined according to the plane parameters of the target plane and the preset plane.
  • the preset plane is parallel to the image plane of the camera, and the distance between the preset plane and the image plane is the preset distance.
  • the intersection of the target plane and the preset plane in three-dimensional space can be determined according to the plane parameters, and the imaging of the intersection in the image plane corresponding to the image to be processed is calculated, and the imaging is used as the position identification line.
  • the distance that the depth sensor of the camera can actually measure is limited. Therefore, in order to obtain effective target plane information, usually the distance between the target plane and the camera should be smaller than the distance that the depth sensor can effectively measure. Conversely, if the target plane is outside the distance that can be effectively measured, the obtained depth data is likely to be wrong, so the surface information of the target plane cannot be obtained. Based on the above reasons, preset planes with different distances can be set to facilitate the determination of the position identification line due to the different properties of the depth sensor of the camera itself.
  • the paper document when scanning a paper document, the paper document needs to be placed within a distance range that the camera can effectively measure. Assuming that the measurement range of the depth sensor of a camera is within 2 meters, at this time, the paper document needs to be placed within the range of 2 meters, and the distance between the preset plane and the image plane of the image to be processed is set to 2 rice.
  • the preset plane can be represented by an algebraic representation parameter in three-dimensional space, eg, Q(0, 0, 1, -2).
  • the preset plane makes the target plane unable to extend to infinity, so the imaging of the above-mentioned intersection line on the image plane can further reduce the area within the range of the vanishing line to ensure that the distance between the intersection lines in the image plane is greater than that of the intersection line. There are no points in the near scene imaging that would make the perspective transformation erroneous.
  • intersection line L 2 can be calculated by the following formula (2):
  • [l 2 ]x represents the cross-product antisymmetric matrix of the vector l 2
  • P represents the target plane
  • L 2 represents the intersection of the target plane P and the preset plane Q in the three-dimensional space.
  • step S920 when there is an intersection on the image plane between the position identification line and the frame of the image to be processed, a correction area is determined in the image to be processed according to the plane parameters, and the correction area is corrected according to the correction matrix to obtain a corrected image.
  • the size of the image to be processed on the image plane is limited by the photosensitive element.
  • the position identification line calculated by the plane parameters is not limited by the photosensitive element, so the position of the position identification line on the image plane may have an intersection with the image to be processed, or it may be completely outside the image to be processed. For example, as shown in FIG. 10 and FIG. 11 , l 2 may or may not have an intersection with the image to be processed.
  • the correction area can be determined in the image to be processed according to the plane parameters, so as to avoid the influence on the correction result caused by the part that is not suitable for correction by the correction matrix corresponding to the target plane.
  • the border refers to the edge line of the outermost circle of the image to be processed.
  • the data collected by the photosensitive element can be imaged in a certain area of the image plane to form an image to be processed.
  • the boundary line of this area is the frame of the image to be processed.
  • image enhancement processing may be performed on the frame of the image to be processed to obtain an enhanced frame.
  • the frame of the to-be-processed image may be enhanced through a preset enhancement rule, so as to increase the area of the area enclosed by the frame on the image plane.
  • the preset enhancement rule may include a preset value, and the frame of the image to be processed is enlarged according to the preset value to form an enhanced frame.
  • preset values e and f can be set, and the original frame 1210 can be translated and extended to form an enhanced frame 1220, so that the enclosed area is increased from W*H to (W+2e)*(H +2f).
  • the preset enhancement rule may further include calculating the circumscribed ellipse of the original frame 1310, and then using the circumscribed ellipse as the enhancement frame 1320, as shown in FIG. 13 .
  • a correction region where correction errors will not occur may be determined in the image to be processed according to plane parameters.
  • the plane point cloud data corresponding to the target plane can be directly determined from the point cloud data corresponding to the image to be processed according to the plane parameters, and then the correction area is determined in the image to be processed according to the plane point cloud data.
  • the imaging area 1420 of the plane point cloud data corresponding to the target plane on the to-be-processed image 1410 may be calculated, and the imaging area 1420 may be determined as the correction area, as shown in FIG. 14 . Since the point cloud data of the target plane can accurately determine the area in the image to be processed when the target plane is imaged, the imaging area of the plane point cloud data on the image to be processed is directly used as the correction area, which can avoid the appearance outside the vanishing line range. point, thereby avoiding the problem of distortion in correction.
  • the position identification line can divide the to-be-processed image into near-scene imaging that is closer than a specific line or surface distance based on the principle of perspective transformation. , and images of distant scenes that are farther than a specific line or area distance, thereby determining the rectified area in the image to be processed.
  • the target area can be determined in the first area and the second area according to the plane point cloud data and the position identification line, Further, a correction area is determined based on the target area.
  • the target plane in order to obtain the surface information on the target plane, the target plane can be located closer to the camera, so the target plane area can be determined according to the projection of the plane point cloud data of the target plane in the image to be processed. Whether the first area or the second area is determined, the first area or the second area is further determined as the target area. For example, for the image to be processed 1510 and the position identification line 1520 shown in FIG. 15 , it can be determined that the left side of the position identification line 1520 is the target area 1530 through the plane point cloud data.
  • the target area may be directly used as a correction area for correction.
  • the integrity of the correction area can be ensured to the greatest extent under the premise of avoiding data points that cause distortion.
  • other ways can also be used to determine the correction area based on the target area. For example, in the scenario of scanning paper documents, the paper documents are generally rectangular. Therefore, referring to FIG. 16 , an inscribed rectangle can be made in the target area 1610 , and the to-be-processed image area surrounded by the inscribed rectangle with the largest area among all the obtained inscribed rectangles can be used as the correction area 1620 .
  • the position identification line can also be offset first, so as to avoid that the position identification line cannot divide all the points that may have transformation errors in the position identification line. within the same area.
  • the positional relationship between the target plane and the position identification line 1710 in the image to be processed 1720 can be determined according to the plane point cloud data, and then the position identification line 1710 is shifted by a preset distance to the side where the target plane is located. , and the offset position identification line 1730 is obtained. Then, the image to be processed may be divided into regions according to the shifted position identification line 1730 .
  • Step S1810 collecting point cloud data corresponding to the image to be processed by the camera; the image to be processed includes the target plane;
  • Step S1820 performing plane detection on the point cloud data to determine plane parameters corresponding to the target plane
  • Step S1830 calculating the position identification line and the correction matrix corresponding to the image to be processed based on the plane parameters
  • Step S1840 judging whether there is an intersection between the position identification line and the image to be processed on the image plane;
  • Step S1850 when there is an intersection on the image plane between the position identification line and the image to be processed, determine a correction area in the image to be processed according to the plane parameters, and correct the correction area according to the correction matrix to obtain a corrected image;
  • Step S1860 when there is no intersection between the position identification line and the image to be processed on the image plane, correct the image to be processed according to the correction matrix to obtain a corrected image.
  • the distance in the three-dimensional space can be reflected on the image plane of the camera through the position identification line, and then according to whether there is an intersection between the position identification line on the image plane and the image to be processed, it is determined whether the distance in the image to be processed exists. Are there any points that could lead to correction errors. When there is an intersection between the position identification line and the image to be processed, it means that there are points in the image to be processed that may cause correction errors.
  • the correction area can be determined in the image to be processed according to the plane point cloud data, and then the correction corresponding to the target plane can be used. The matrix corrects the correction area. In this way, when performing perspective transformation through the correction matrix, due to the existence of points other than the vanishing line, the problem of correction errors can be avoided.
  • the embodiment of this example further provides a plane correction device 1900 , which includes a parameter calculation module 1910 and a plane correction module 1920 . in:
  • the parameter calculation module 1910 can be used to obtain the plane parameters of the target plane in the image to be processed, and calculate the position identification line and the correction matrix corresponding to the to-be-processed image according to the plane parameters; wherein, the position identification line is on the image plane of the to-be-processed image.
  • the plane correction module 1920 can be used to determine the correction area in the to-be-processed image according to the plane parameters when the position identification line and the frame of the image to be processed have an intersection on the image plane, and correct the correction area according to the correction matrix to obtain a corrected image .
  • the plane correction module 1920 may be configured to correct the image to be processed according to the correction matrix to obtain a corrected image when there is no intersection between the position identification line and the frame of the image to be processed on the image plane.
  • the parameter calculation module 1910 can be used to obtain the camera internal parameters corresponding to the camera that captures the image to be processed; according to the camera internal parameters and the plane parameters, calculate the vanishing line formed on the image plane when the target plane extends to infinity , and use the vanishing line as the position marker.
  • the parameter calculation module 1910 can be used to determine the intersection of the target plane and the preset plane according to the plane parameters; wherein, the preset plane is parallel to the image plane, and the distance between the preset plane and the image plane is a preset value. Set the distance; calculate the section line formed by the intersection line on the image plane, and use the section line as the position identification line.
  • the plane correction module 1920 can be used to obtain the plane point cloud data corresponding to the target plane in the point cloud data corresponding to the image to be processed according to the plane parameters; Correction area.
  • the plane correction module 1920 may be used to calculate the imaging area of the plane point cloud data on the image to be processed, and determine the imaging area as the correction area.
  • the plane correction module 1920 can be used to divide the image to be processed into a first area and a second area according to the position identification line; according to the plane point cloud data and the position identification line, in the first area and the second area A target area is determined in the second area, and a correction area is determined based on the target area.
  • the plane correction module 1920 can be used to calculate the imaging area of the plane point cloud data on the image to be processed; when the imaging area is located in the first area, the first area is determined as the target area; when the imaging area is located in the When the second area is used, the second area is determined as the target area.
  • the plane correction module 1920 may be configured to generate an inscribed rectangle in the target area, and determine the area enclosed by the inscribed rectangle with the largest area among all the inscribed rectangles as the correction area.
  • the plane correction module 1920 can be used to determine the positional relationship between the target plane and the position marker line in the image to be processed according to the plane point cloud data; Offset the preset distance to obtain the offset position marker line.
  • the plane correction apparatus 2000 further includes a frame enhancement module 2010 for performing image enhancement processing on the frame of the to-be-processed image to obtain the enhanced frame of the to-be-processed image.
  • a frame enhancement module 2010 for performing image enhancement processing on the frame of the to-be-processed image to obtain the enhanced frame of the to-be-processed image.
  • the frame enhancement module 2010 may be configured to perform enhancement processing on the frame of the image to be processed according to a preset enhancement rule, so as to increase the area of the area enclosed by the frame on the image plane.
  • the parameter calculation module 1910 can be used to perform plane detection based on the point cloud data of the image to be processed, and obtain plane parameters corresponding to the target plane; The plane with the largest area to process the image.
  • aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be embodied in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein as implementations "circuit", “module” or "system”.
  • Exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-described method of the present specification is stored.
  • various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code, when the program product runs on a terminal device, the program code is used to cause the terminal device to execute the above-mentioned procedures in this specification.
  • the steps described in the "Exemplary Methods" section according to various exemplary embodiments of the present disclosure for example, any one or more of the steps in FIGS. 9 and 18 may be performed.
  • the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
  • Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., as well as conventional procedural Programming Language - such as the "C" language or similar programming language.
  • the program code may execute entirely on the user computing device, partly on the user device, as a stand-alone software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on.
  • the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (eg, using an Internet service provider business via an Internet connection).
  • LAN local area network
  • WAN wide area network

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

一种平面矫正方法、平面矫正装置、计算机可读介质和电子设备，涉及图像处理技术领域。该方法包括：获取待处理图像中目标平面的平面参数，根据平面参数计算待处理图像对应的位置标识线和矫正矩阵；在位置标识线与待处理图像的边框在像平面上存在交点时，根据平面参数在待处理图像中确定矫正区域，并根据矫正矩阵针对矫正区域进行矫正，得到矫正图像。可以避免相关技术中，直接通过矫正矩阵对待处理图像进行矫正时，由于待处理图像中包含过多非目标平面信息，导致矫正结果出现畸变的问题。

Description

平面矫正方法及装置、计算机可读介质和电子设备 技术领域
本公开涉及图像处理技术领域，具体涉及一种平面矫正方法、平面矫正装置、计算机可读介质和电子设备。
背景技术
曼哈顿世界假说认为，大部分人工建造的物体，例如楼房，道路等具有很强的正交属性，都存在一个直角坐标系使得该物体的表面平行或垂直于该坐标轴，如图1所示。在对三维世界中的物体表面进行拍摄扫描时，可以利用这一假说构建平面模型，对拍摄得到的待处理图像进行平面检测和矫正等处理，使待处理图像转换为包含物体表面正视图的矫正图像，以便于获取物体表面的信息。例如，在对纸质文档进行扫描时，可以对拍摄得到的待处理图像进行平面检测和矫正处理，使待处理图像转换为包含纸质文档的正视图的矫正图像，以便于根据矫正图像读取纸质文档中的文字等信息。
发明内容
本公开的目的在于提供一种平面矫正方法、平面矫正装置、计算机可读介质和电子设备，进而至少在一定程度上避免相关技术中，直接通过矫正矩阵对待处理图像进行矫正时，由于待处理图像中包含过多非目标平面信息，导致矫正结果出现畸变的问题。
根据本公开的第一方面，提供一种平面矫正方法，包括：获取待处理图像中目标平面的平面参数，根据平面参数计算待处理图像对应的位置标识线和矫正矩阵；其中，位置标识线在待处理图像的像平面上；在位置标识线与待处理图像的边框在像平面上存在交点时，根据平面参数在待处理图像中确定矫正区域，并根据矫正矩阵针对矫正区域进行矫正，得到矫正图像。
根据本公开的第二方面，提供一种平面矫正装置，包括：参数计算模块，用于获取待处理图像中目标平面的平面参数，根据平面参数计算待处理图像对应的位置标识线和矫正矩阵；其中，位置标识线在待处理图像的像平面上；平面矫正模块，用于在位置标识线与待处理图像的边框在像平面上存在交点时，根据平面参数在待处理图像中确定矫正区域，并根据矫正矩阵针对矫正区域进行矫正，得到矫正图像。
根据本公开的第三方面，提供一种计算机可读介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述的方法。
根据本公开的第四方面，提供一种电子设备，其特征在于，包括：
处理器；以及
存储器，用于存储一个或多个程序，当一个或多个程序被一个或多个处理器执行时，使得一个或多个处理器实现上述的方法。
本公开的一种实施例所提供的平面矫正方法，通过获取待处理图像中目标平面的平面参数，并根据该平面参数计算待处理图像对应的位置标识线和矫正矩阵；然后在位置标识线与待处理图像的边框在像平面上存在交点时，根据平面参数在待处理图像中确定矫正区域，并对矫正区域进行矫正，得到矫正图像。一方面，可以通过位置标识线确定通过矫正矩阵直接对待处理图像进行矫正是否会出现错误；另一方面，在位置标识线与待处理图像的边框在像平面上存在交点时，可以根据平面参数在待处理图像中确定矫正区域，进而通过矫正矩阵对矫正区域进行矫正，避免由于待处理图像中包含过多非目标平面信息，导致矫正结果出现错误的问题。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：
图1示出了三维控件中大部分物体具有的正交属性的示意图；
图2示出了可以应用本公开实施例的一种示例性系统架构的示意图；
图3示出了可以应用本公开实施例的一种电子设备的示意图；
图4示出了一种待处理图像；
图5示出了通过相关技术对图4的待处理图像进行矫正，得到矫正图像；
图6示出了另一待处理图像以及其对应的矫正图像；
图7示意性示出了又一待处理图像以及其对应的矫正图像；
图8示意性示出了再一待处理图像以及其对应的矫正图像；
图9示意性示出本公开示例性实施例中一种平面矫正方法的流程图；
图10示意性示出本公开示例性实施例中一种确定位置标识线的原理图；
图11示意性示出本公开示例性实施例中另一确定位置标识线的原理图；
图12示意性示出本公开示例性实施例中一种边框增强方法的示意图；
图13示意性示出本公开示例性实施例中另一种边框增强方法的示意图；
图14示意性示出本公开示例性实施例中一种确定矫正区域方法的示意图；
图15示意性示出本公开示例性实施例中另一种确定矫正区域方法的示意图；
图16示意性示出本公开示例性实施例中再一种确定矫正区域方法的示意图；
图17示意性示出本公开示例性实施例中对位置标识线进行偏移的示意图；
图18示意性示出本公开示例性实施例中另一种平面矫正方法的流程图；
图19示意性示出本公开示例性实施例中一种平面矫正装置的组成示意图；
图20示意性示出本公开示例性实施例中另一种平面矫正装置的组成示意图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本公开将更加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施方式中。
此外，附图仅为本公开的示意性图解，并非一定是按比例绘制。图中相同的附图标记表示相同或类似的部分，因而将省略对它们的重复描述。附图中所示的一些方框图是功能实体，不一定必须与物理或逻辑上独立的实体相对应。可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
图2示出了可以应用本公开实施例的一种平面矫正方法及装置的示例性应用环境的系统架构的示意图。
如图2所示，系统架构200可以包括终端设备201、202、203中的一个或多个，网络204和服务器205。网络204用以在终端设备201、202、203和服务器205之间提供通信链路的介质。网络204可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。终端设备201、202、203可以是各种具有图像处理功能的电子设备，包括但不限于台式计算机、便携式计算机、智能手机和平板电脑等等。应 该理解，图1中的终端设备、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备、网络和服务器。比如服务器205可以是多个服务器组成的服务器集群等。
本公开实施例所提供的平面矫正方法一般由终端设备201、202、203中执行，相应地，平面矫正装置一般设置于终端设备201、202、203中。但本领域技术人员容易理解的是，本公开实施例所提供的平面矫正方法也可以由服务器205执行，相应的，平面矫正装置也可以设置于服务器205中，本示例性实施例中对此不做特殊限定。举例而言，在一种示例性实施例中，可以是用户通过终端设备201、202、203获取待处理图像中目标平面的平面参数，计算位置标识线和矫正矩阵，在位置标识线与待处理图像的边框在像平面上存在交点时，根据平面参数在待处理图像中确定矫正区域，进而根据计算得到的矫正矩阵对矫正区域进行矫正，得到目标平面对应的矫正图像；在另一种实施例中，也可以是服务器205获取终端设备201、202、203提供的相关数据，通过平面矫正方法进行处理后，将处理结果即矫正图像反馈至终端设备201、202、203等。
本公开的示例性实施方式提供一种用于实现平面矫正方法的电子设备，其可以是图1中的终端设备201、202、203或服务器205。该电子设备至少包括处理器和存储器，存储器用于存储处理器的可执行指令，处理器配置为经由执行可执行指令来执行平面矫正方法。
下面以图3中的移动终端300为例，对电子设备的构造进行示例性说明。本领域技术人员应当理解，除了特别用于移动目的的部件之外，图3中的构造也能够应用于固定类型的设备。在另一些实施方式中，移动终端300可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件、软件或软件和硬件的组合实现。各部件间的接口连接关系只是示意性示出，并不构成对移动终端300的结构限定。在另一些实施方式中，移动终端300也可以采用与图3不同的接口连接方式，或多种接口连接方式的组合。
如图3所示，移动终端300具体可以包括：处理器310、内部存储器321、外部存储器接口322、通用串行总线(Universal Serial Bus，USB)接口330、充电管理模块340、电源管理模块341、电池342、天线1、天线2、移动通信模块350、无线通信模块360、音频模块370、扬声器371、受话器372、麦克风373、耳机接口374、传感器模块380、显示屏390、摄像模组391、指示器392、马达393、按键394以及用户标识模块(subscriber identification module，SIM)卡接口395等。其中传感器模块380可以包括深度传感器3801、压力传感器3802、陀螺仪传感器3803等。
处理器310可以包括一个或多个处理单元，例如：处理器310可以包括应用处理器(Application Processor，AP)、调制解调处理器、图形处理器(Graphics Processing Unit，GPU)、图像信号处理器(Image Signal Processor，ISP)、控制器、视频编解码器、数字信号处理器(Digital Signal Processor，DSP)、基带处理器和/或神经网络处理器(Neural-Network Processing Unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
NPU为神经网络(Neural-Network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现移动终端300的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。
处理器310中设置有存储器。存储器可以存储用于实现六个模块化功能的指令：检测指令、连接指令、信息管理指令、分析指令、数据传输指令和通知指令，并由处理器310来控制执行。
充电管理模块340用于从充电器接收充电输入。电源管理模块341用于连接电池342、充电管理模块340与处理器310。电源管理模块341接收电池342和/或充电管理模块340的输入，为处理器310、内部存储器321、显示屏390、摄像模组391和无线通信模块360等供电。
移动终端300的无线通信功能可以通过天线1、天线2、移动通信模块350、无线通信模块360、调制解调处理器以及基带处理器等实现。其中，天线1和天线2用于发射和接收电磁波信号；移动通信模块350可以提供应用在移动终端300上的包括2G/3G/4G/5G等无线通信的解决方案；调制解调处理器可以包括调制器和解调器；无线通信模块360可以提供应用在移动终端300上的包括无线局域网(Wireless Local Area Networks，WLAN)(如无线保真(Wireless Fidelity，Wi-Fi)网络)、蓝牙(Bluetooth，BT)等无线通信的解决方案。在一些实施例中，移动终端300的天线1和移动通信模块350耦合，天线2和无线通信模块360耦合，使得移动终端300可以通过无线通信技术与网络以及其他设备通信。
移动终端300通过GPU、显示屏390及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏390和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器310可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
移动终端300可以通过ISP、摄像模组391、视频编解码器、GPU、显示屏390及应用处理器等实现拍摄功能。其中，ISP用于处理摄像模组391反馈的数据；摄像模组391用于捕获静态图像或视频；数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号；视频编解码器用于对数字视频压缩或解压缩，移动终端300还可以支持一种或多种视频编解码器。
外部存储器接口322可以用于连接外部存储卡，例如Micro SD卡，实现扩展移动终端300的存储能力。外部存储卡通过外部存储器接口322与处理器310通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器321可以用于存储计算机可执行程序代码，可执行程序代码包括指令。内部存储器321可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储移动终端300使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器321可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(Universal Flash Storage，UFS)等。处理器310通过运行存储在内部存储器321的指令和/或存储在设置于处理器中的存储器的指令，执行移动终端300的各种功能应用以及数据处理。
移动终端300可以通过音频模块370、扬声器371、受话器372、麦克风373、耳机接口374及应用处理器等实现音频功能。例如音乐播放、录音等。
深度传感器3801用于获取景物的深度信息。在一些实施例中，深度传感器可以设置于摄像模组391，通过摄像模组391和深度传感器3801同时对场景进行数据采集，可以获取待处理图像的颜色数据和深度数据，即待处理图像对应的点云数据。
压力传感器3802用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器3802可以设置于显示屏390。压力传感器3802的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。
陀螺仪传感器3803可以用于确定移动终端300的运动姿态。在一些实施方式中，可以通过陀螺仪传感器3803确定移动终端300围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传感器3803可以用于拍摄防抖、导航、体感游戏场景等。
此外，还可以根据实际需要在传感器模块380中设置其他功能的传感器，例如 气压传感器、磁传感器、加速度传感器、距离传感器、接近光传感器、指纹传感器、温度传感器、触摸传感器、环境光传感器、骨传导传感器等。
移动终端300中还可包括其它提供辅助功能的设备。例如，按键394包括开机键，音量键等，用户可以通过按键输入，产生与移动终端300的用户设置以及功能控制有关的键信号输入。再如，指示器392、马达393、SIM卡接口395等。
在相关技术中，为了实现对待处理图像中目标平面的矫正，开发人员通过平面参数计算得到的矫正矩阵直接对待处理图像进行矫正。但是，在通过这种方式进行矫正时，由于待处理图像中包含了目标平面以外的其它信息，因此矫正结果可能出现畸变的情况。例如，图4所示的待处理图像中存在目标平面410，如果直接对待处理图像进行矫正，可能得到如图5所示的矫正图像，在该矫正图像中，目标平面410被矫正为目标平面510，这明显与用户需求不同；再如，图6所示矫正过程也出现了明显的错误。
上述畸变是主要是由于待处理图像中存在大量目标平面以外的信息所导致的。对二维待处理图像中的目标平面进行矫正，实际是对二维待处理图像中的目标平面进行透视变换，而这种透视变换并不适用于超出目标平面对应的消失线范围的点。如果矫正范围内出现超出消失线范围的点，矫正图像中会出现无意义的点，因此很可能出现矫正错误的情况。其中，消失线是指平面延伸至无穷远时在相机中所成的像，而消失线范围内是指平面在距离小于无穷远的部分在相机中所成的像。
为了解决上述问题，通常采用以下两种方式：一是借助待处理图像的边界信息在待处理图像的确定目标平面的范围，进而对该范围进行矫正；二是通过对待处理图像进行裁剪，并对裁剪得到的部分进行矫正处理。
然而，上述第一种方法，如果待处理图像中的边界信息不足，或者边界不清晰时，很可能无法准确确定目标平面的范围，进而导致矫正出现错误；而在第二种方法中，由于裁剪通常情况下也是基于待处理图像的图像信息进行的，因此裁剪的目标平面也可能出现不符合用户要求的情况。这种情况下，需要对一些目标平面以外的部分也需要进行矫正，以便于用户可以对矫正后的图像进行手动修正。由于在矫正时需要对目标平面以外的部分进行矫正，因此矫正仍然会存在出现错误的可能性。
在第二种方法中，参照图7所示，对包括目标平面的矫正区域进行矫正后，可能出现目标平面被矫正为矫正图像中很小一部分的情况。这时，即使通过人工进行手动裁剪，由于矫正区域在待处理图像中的分辨率有限，矫正图像的分辨率也有限，因此在对矫正图像进行手动修正，选择矫正图像中目标平面对应的区域进行裁剪，最终得到目标平面对应的矫正画面很可能出现图像画质低的问题；参照图8所示，包括目标平面的矫正区域中，仍然可能存在超出消失线范围的点，因此可能会出现矫正发生错误，使得矫正图像出现畸变的情况。
基于上述问题，本公开提供了一种平面矫正方法和平面矫正装置，以下进行具体说明。
图9示出了本示例性实施方式中一种平面矫正方法的流程，包括以下步骤S910和S920：
在步骤S910中，获取待处理图像中目标平面的平面参数，根据平面参数计算待处理图像对应的位置标识线和矫正矩阵。
在一示例性实施例中，上述平面参数为目标平面在三维空间中的代数表示参数，用于在三维空间中确定唯一的无限大平面，例如，P(a，b，c，d)；矫正矩阵为基于平面参数计算得到的用于对二维图像进行透视变换，对图像进行矫正的矩阵。需要说明的是，虽然在待处理图像中目标平面为有限大的平面，但是在对目标平面进行描述时，三维空间中的代数表示参数表示的是空间中一个无限大的平面，即P(a， b，c，d)代表空间中一个固定的无限大平面。
其中，位置标识线是三维空间中特定的线或面，在待处理图像所在的像平面上的成像，可以基于透视转换原理，将三维空间中的特定位置标识在二维的待处理图像中，进而将像平面划分为比特定的线或面距离更近的近场景成像，和比特定线或面距离更远的远场景成像。
需要说明的是，位置标识线始终位于待处理图像对应的像平面上，该像平面为拍摄相机成像平面。从理论上讲，该像平面为无穷大的平面，并不受限于相机感光元件的大小。
在一示例性实施例中，相机可以通过感光元件和深度传感器等方式采集待处理图像对应的点云数据。因此，可以通过对点云数据进行平面检测的方式获取待处理图像中目标平面的平面参数。其中，平面检测的过程可以通过RANSAC等方法进行，本公开对此不做特殊限定。
需要说明的是，待处理图像可能同时包括多个平面，因此需要先确定待处理平面中的目标平面。通常情况下，在需要获取某一物体表面的信息时，对应的待处理图像中该物体表面占据的面积是最大的。因此，可以通过对待处理图像的点云数据进行平面检测，并将可以检测到的所有平面中，在待处理图像上占据面积最大的平面确定为目标平面，然后获取目标平面对应的平面参数。
在一示例性实施例中，由于针对目标平面的透视变换，在对目标平面对应消失线范围之外的点进行变换后，可能会发生错误。因此，可以以目标平面对应的消失线作为位置标识线，以保证透视变换不会发生错误。
举例而言，可以获取拍摄待处理图像的相机对应的相机内参，然后根据相机内参和平面参数计算消失线，并将消失线作为位置标识线。其中，消失线是指与目标平面延伸至无穷远时，在像平面上的成像结果。在具体计算时，可以通过平面参数计算目标平面的法向量n，然后根据以下公式(1)计算消失线：
l 1＝K-1n                     公式(1)
其中，l 1为消失线，K为拍摄待处理图像的相机对应的相机内参，n为目标平面的法向量。
在另一示例性实施例中，可以通过设置预设平面，并根据目标平面的平面参数与预设平面确定位置标识线。其中，预设平面与相机的像平面平行，且预设平面与像平面之间的距离为预设距离。具体的，可以根据平面参数确定目标平面与预设平面在三维空间中的交线，并计算该交线在待处理图像对应的像平面的成像，将该成像作为位置标识线。
需要说明的是，相机的深度传感器实际能有效测量的距离有限，因此为了能够得到有效的目标平面的信息，通常情况下目标平面与相机之间的距离应该小于深度传感器能够有效测量的距离。反之，如果目标平面在可以有效测量的距离之外，得到的深度数据很可能出现错误，因此也无法获取目标平面的表面信息。基于上述原因，通过相机本身的深度传感器的属性不同，可以设置不同距离的预设平面，以便于确定位置标识线。
例如，在对纸质文档进行扫描时，需要将纸质文档放置于相机可以有效测量的距离范围内。假定某相机的深度传感器的测量范围为2米以内，此时，需要将纸质文档放置于2米以内的范围内，并将预设平面与待处理图像的像平面之间的距离设置为2米。预设平面可以通过三维空间中的代数表示参数表示，例如，Q(0，0，1，-2)。这种情况下，预设平面使得目标平面无法延伸至无穷远，因此上述交线在像平面上的成像可以对消失线范围内的区域进行进一步缩小，以确保像平面中比该交线距离更近的近场景成像中不存在会使透视变换发生错误的点。
在具体计算时，参照图10所示，交线L 2可以通过以下公式(2)计算得到：
L 2＝PQT-QPT                     公式(2)
其中，P表示目标平面，Q表示预设平面，L 2表示目标平面P与预设平面Q在三维空间中的交线。在得到交线L 2后，可以通过叉乘反对称矩阵[l 2]x表示交线L 2在像平面上的像l 2，具体计算如下公式(3)：
[l 2]x＝PL 2PT                     公式(3)
其中，[l 2]x表示向量l 2的叉乘反对称矩阵，P表示目标平面，L 2表示目标平面P与预设平面Q在三维空间中的交线。
在步骤S920中，在位置标识线与待处理图像的边框在像平面上存在交点时，根据平面参数在待处理图像中确定矫正区域，并根据矫正矩阵针对矫正区域进行矫正，得到矫正图像。
由于待处理图像是通过感光元件得到的，因此待处理图像在像平面上的大小受到感光元件的限制。而通过平面参数计算得到的位置标识线则不受感光元件的限制，因此位置标识线在像平面上的位置可能与待处理图像存在交点，也可能完全位于待处理图像之外。例如，如图10和图11中所示，l 2和待处理图像可能存在交点，也可能和待处理图像不存在交点。
在一示例性实施例中，参照图11所示，在位置标识线与待处理图像的边框在像平面上存在交点时，可以说明待处理图像中存在不适宜通过目标平面对应的矫正矩阵进行矫正的部分。此时，可以根据平面参数在待处理图像中确定矫正区域，以避免不适宜通过目标平面对应的矫正矩阵进行矫正的部分对矫正结果造成影响。
其中，边框是指待处理图像的最外圈的边缘线。在无限大的像平面中，感光元件采集到的数据可以在像平面的一定区域内成像，形成待处理图像，此时该区域的边界线即是待处理图像的边框。
在一示例性实施例中，在确定位置标识线与待处理图像的边框在像平面上有没有交点之前，可以先对待处理图像的边框进行图像增强处理，以获取增强后的边框。具体的，可以通过预设增强规则对待处理图像的边框进行增强处理，以增大边框在像平面上包围区域的面积。通过对边框进行增强，增大了边框在像平面上包围区域的面积，可以更加鲁棒的确定边框与位置标识线是否存在交点。
在一示例性实施例中，预设增强规则可以包括预设值，根据预设值对待处理图像的边框进行扩大，形成增强后的边框。例如，参照图12所示，可以设置预设值e和f，对原边框1210进行平移和延长形成增强边框1220，使其包围的面积由W*H增大至(W+2e)*(H+2f)。
在另一示例性实施例中，预设增强规则还可以包括计算原边框1310的外接椭圆，然后以外接椭圆作为增强边框1320，参照图13所示。
需要说明的是，通过上述增强处理，可以提高确定位置标识线与边框是否存在交点的鲁棒性。此外，还可以使用其他的增强处理方式对边框进行增强，本公开对此不做特殊限定。
在一示例性实施例中，在确定位置标识线与待处理图像的边框在像平面上存在交点时，可以根据平面参数在待处理图像中确定不会出现矫正错误的矫正区域。
在一示例性实施例中，可以直接根据平面参数在待处理图像对应的点云数据中确定目标平面对应的平面点云数据，然后根据这些平面点云数据在待处理图像中确定矫正区域。
可选的，可以计算目标平面对应的平面点云数据在待处理图像1410上的成像区域1420，并将该成像区域1420确定为矫正区域，如图14所示。由于目标平面的点云数据可以准确的确定目标平面成像时，在待处理图像中的区域，因此直接以平 面点云数据在待处理图像上的成像区域作为矫正区域，能够避免出现消失线范围外的点，进而避免矫正出现畸变的问题。
可选的，在位置标识线与待处理图像的边框在像平面上存在交点时，位置标识线可以基于透视转换原理，将待处理图像划分为比特定的线或面距离更近的近场景成像，和比特定线或面距离更远的远场景成像，进而确定待处理图像中的矫正区域。
在一示例性实施例中，在根据位置标识线将待处理图像分为第一区域和第二区域后，可以根据平面点云数据和位置标识线在第一区域和第二区域确定目标区域，进而基于该目标区域确定矫正区域。
在一示例性实施例中，为了获取目标平面上的表面信息，目标平面可以处于距离相机更近的位置，因此可以根据目标平面的平面点云数据在待处理图像中的投影，确定目标平面区域第一区域还是第二区域，进而确定第一区域或者第二区域为目标区域。例如，如图15所示的待处理图像1510和位置标识线1520，通过平面点云数据可以确定位置标识线1520左侧为目标区域1530。
可选的，在确定了目标区域以后，可以直接将目标区域作为矫正区域进行矫正。通过直接对目标区域进行矫正，可以在避免出现导致畸变的数据点的前提下，最大程度上保证矫正区域的完整性。此外，还可以采用其他方式基于目标区域确定矫正区域。例如，在对纸质文档进行扫描的场景中，通常情况下纸质文档均为矩形。因此参照图16所示，可以在目标区域1610作内接矩形，并将得到的所有内接矩形中，面积最大的内接矩形包围的待处理图像区域作为矫正区域1620。
进一步地，在根据位置标识线将待处理图像分为第一区域和第二区域之前，还可以先对位置标识线进行偏移，以避免位置标识线无法将所有可能出现变换错误的点划分在同一区域内。具体的，参照图17所示，可以根据平面点云数据确定目标平面与位置标识线1710在待处理图像1720中的位置关系，然后将位置标识线1710向目标平面所在一侧偏移预设距离，得到偏移后的位置标识线1730。然后可以根据偏移后的位置标识线1730对待处理图像进行区域划分。
同时，在位置标识线与待处理图像的边框在像平面上不存在交点时，说明待处理图像中不存在可能导致矫正错误的点，因此可以根据矫正矩阵直接针对待处理图像进行矫正，得到矫正图像。
以下参照图18，对本公开的一种示例性实施例进行详细阐述。
步骤S1810，通过相机采集待处理图像对应的点云数据；待处理图像中包含目标平面；
步骤S1820，对点云数据进行平面检测，确定目标平面对应的平面参数；
步骤S1830，基于平面参数计算待处理图像对应的位置标识线和矫正矩阵；
步骤S1840，判断位置标识线与待处理图像在像平面上是否存在交点；
步骤S1850，在位置标识线与待处理图像在像平面上存在交点时，根据平面参数在待处理图像中确定矫正区域，根据矫正矩阵针对矫正区域进行矫正，得到矫正图像；
步骤S1860，在位置标识线与待处理图像在像平面上不存在交点时，根据矫正矩阵针对待处理图像进行矫正，得到矫正图像。
综上，本示例性实施方式中，可以通过位置标识线将三维空间中的距离反映在相机的像平面上，然后根据像平面上位置标识线和待处理图像是否存在交点，判断待处理图像中是否存在可能导致矫正错误的点。在位置标识线和待处理图像是否存在交点时，说明待处理图像中存在可能导致矫正错误的点，此时可以根据平面点云数据在待处理图像中确定矫正区域，进而根据目标平面对应的矫正矩阵对矫正区域进行矫正。通过这种方式可以避免通过矫正矩阵进行透视变换时，由于存在消失线 以外的点，导致矫正发生错误的问题。
需要注意的是，上述附图仅是根据本公开示例性实施例的方法所包括的处理的示意性说明，而不是限制目的。易于理解，上述附图所示的处理并不表明或限制这些处理的时间顺序。另外，也易于理解，这些处理可以是例如在多个模块中同步或异步执行的。
进一步的，参考图19所示，本示例的实施方式中还提供一种平面矫正装置1900，包括参数计算模块1910和平面矫正模块1920。其中：
参数计算模块1910可以用于获取待处理图像中目标平面的平面参数，根据平面参数计算待处理图像对应的位置标识线和矫正矩阵；其中，位置标识线在待处理图像的像平面上。
平面矫正模块1920可以用于在位置标识线与待处理图像的边框在像平面上存在交点时，根据平面参数在待处理图像中确定矫正区域，并根据矫正矩阵针对矫正区域进行矫正，得到矫正图像。
在一示例性实施例中，平面矫正模块1920可以用于在位置标识线与待处理图像的边框在像平面上不存在交点时，根据矫正矩阵针对待处理图像进行矫正，得到矫正图像。
在一示例性实施例中，参数计算模块1910可以用于获取拍摄待处理图像的相机对应的相机内参；根据相机内参和平面参数，计算目标平面延伸至无穷远时在像平面上形成的消失线，并将消失线作为位置标识线。
在一示例性实施例中，参数计算模块1910可以用于根据平面参数确定目标平面与预设平面的交线；其中，预设平面与像平面平行，且预设平面与像平面的距离为预设距离；计算交线在像平面形成的截面线，并将截面线作为位置标识线。
在一示例性实施例中，平面矫正模块1920可以用于根据平面参数在待处理图像对应的点云数据中，获取目标平面对应的平面点云数据；根据平面点云数据在待处理图像中确定矫正区域。
在一示例性实施例中，平面矫正模块1920可以用于计算平面点云数据在待处理图像上的成像区域，并将成像区域确定为矫正区域。
在一示例性实施例中，平面矫正模块1920可以用于根据位置标识线将待处理图像分为第一区域和第二区域；根据所平面点云数据和位置标识线，在第一区域和第二区域中确定目标区域，并基于目标区域确定矫正区域。
在一示例性实施例中，平面矫正模块1920可以用于计算平面点云数据在待处理图像上的成像区域；在成像区域位于第一区域时，确定第一区域为目标区域；在成像区域位于第二区域时，确定第二区域为目标区域。
在一示例性实施例中，平面矫正模块1920可以用于在目标区域中生成内接矩形，并将所有内接矩形中，面积最大的内接矩形所包围的区域确定为矫正区域。
在一示例性实施例中，平面矫正模块1920可以用于根据平面点云数据确定目标平面与位置标识线在待处理图像中的位置关系；根据位置关系将位置标识线向目标平面所在的一侧偏移预设距离，以获取偏移后的位置标识线。
在一示例性实施例中，参照图20所示，平面矫正装置2000还包括边框增强模块2010，用于对待处理图像的边框进行图像增强处理，以获取增强后的待处理图像的边框。
在一示例性实施例中，边框增强模块2010可以用于根据预设增强规则对待处理图像的边框进行增强处理，以增大边框在像平面上包围区域的面积。
在一示例性实施例中，参数计算模块1910可以用于基于待处理图像的点云数据进行平面检测，并获取目标平面对应的平面参数；其中，目标平面为检测到的所 有平面中，占据待处理图像的面积最大的平面。
上述装置中各模块的具体细节在方法部分实施方式中已经详细说明，未披露的细节内容可以参见方法部分的实施方式内容，因而不再赘述。
所属技术领域的技术人员能够理解，本公开的各个方面可以实现为系统、方法或程序产品。因此，本公开的各个方面可以具体实现为以下形式，即：完全的硬件实施方式、完全的软件实施方式(包括固件、微代码等)，或硬件和软件方面结合的实施方式，这里可以统称为“电路”、“模块”或“系统”。
本公开的示例性实施方式还提供了一种计算机可读存储介质，其上存储有能够实现本说明书上述方法的程序产品。在一些可能的实施方式中，本公开的各个方面还可以实现为一种程序产品的形式，其包括程序代码，当程序产品在终端设备上运行时，程序代码用于使终端设备执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤，例如可以执行图9和图18中任意一个或多个步骤。
需要说明的是，本公开所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、RF等等，或者上述的任意合适的组合。
此外，可以以一种或多种程序设计语言的任意组合来编写用于执行本公开操作的程序代码，程序设计语言包括面向对象的程序设计语言—诸如Java、C++等，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。在涉及远程计算设备的情形中，远程计算设备可以通过任意种类的网络，包括局域网(LAN)或广域网(WAN)，连接到用户计算设备，或者，可以连接到外部计算设备(例如利用因特网服务提供商来通过因特网连接)。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其他实施例。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由权利要求指出。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求 来限。

Claims (15)

  1. 一种平面矫正方法，其特征在于，包括：
    获取待处理图像中目标平面的平面参数，根据所述平面参数计算所述待处理图像对应的位置标识线和矫正矩阵；其中，所述位置标识线在所述待处理图像的像平面上；
    在所述位置标识线与所述待处理图像的边框在所述像平面上存在交点时，根据所述平面参数在所述待处理图像中确定矫正区域，并根据所述矫正矩阵针对所述矫正区域进行矫正，得到矫正图像。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    在所述位置标识线与所述待处理图像的边框在所述像平面上不存在交点时，根据所述矫正矩阵针对所述待处理图像进行矫正，得到矫正图像。
  3. 根据权利要求1所述的方法，其特征在于，所述根据所述平面参数计算所述待处理图像对应的位置标识线，包括：
    获取拍摄所述待处理图像的相机对应的相机内参；
    根据所述相机内参和所述平面参数，计算所述目标平面延伸至无穷远时在所述像平面上形成的消失线，并将所述消失线作为位置标识线。
  4. 根据权利要求1所述的方法，其特征在于，根据所述平面参数计算所述待处理图像对应的位置标识线，包括：
    根据平面参数确定所述目标平面与预设平面的交线；其中，所述预设平面与所述像平面平行，且所述预设平面与所述像平面的距离为预设距离；
    计算所述交线在所述像平面形成的截面线，并将所述截面线作为位置标识线。
  5. 根据权利要求1所述的方法，其特征在于，根据所述平面参数在所述待处理图像中确定矫正区域，包括：
    根据所述平面参数在所述待处理图像对应的点云数据中，获取所述目标平面对应的平面点云数据；
    根据所述平面点云数据在所述待处理图像中确定矫正区域。
  6. 根据权利要求5所述的方法，其特征在于，根据所述平面点云数据在所述待处理图像中确定矫正区域，包括：
    计算所述平面点云数据在所述待处理图像上的成像区域，并将所述成像区域确定为矫正区域。
  7. 根据权利要求5所述的方法，其特征在于，根据所述平面点云数据在所述待处理图像中确定矫正区域，包括：
    根据所述位置标识线将所述待处理图像分为第一区域和第二区域；
    根据所平面点云数据和所述位置标识线，在所述第一区域和所述第二区域中确定目标区域，并基于所述目标区域确定矫正区域。
  8. 根据权利要求7所述的方法，其特征在于，所述根据所平面点云数据和所述位置标识线，在所述第一区域和所述第二区域中确定目标区域，包括：
    计算所述平面点云数据在所述待处理图像上的成像区域；
    在所述成像区域位于第一区域时，确定所述第一区域为目标区域；
    在所述成像区域位于第二区域时，确定所述第二区域为目标区域。
  9. 根据权利要求7所述的方法，其特征在于，所述基于所述目标区域确定矫正区域，包括：
    在所述目标区域中生成内接矩形，并将所有所述内接矩形中，面积最大的内接矩形所包围的区域确定为矫正区域。
  10. 根据权利要求7所述的方法，其特征在于，在根据所述位置标识线将所述 待处理图像分为第一区域和第二区域之前，所述方法还包括：
    根据平面点云数据确定目标平面与所述位置标识线在所述待处理图像中的位置关系；
    根据所述位置关系将所述位置标识线向所述目标平面所在的一侧偏移预设距离，以获取偏移后的位置标识线。
  11. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    对所述待处理图像的边框进行图像增强处理，以获取增强后的所述待处理图像的边框。
  12. 根据权利要求11所述的方法，其特征在于，所述对所述待处理图像的边框进行图像增强处理，包括：
    根据预设增强规则对所述待处理图像的边框进行增强处理，以增大所述边框在所述像平面上包围区域的面积。
  13. 根据权利要求1所述的方法，其特征在于，获取待处理图像中目标平面的平面参数，包括：
    基于所述待处理图像的点云数据进行平面检测，并获取目标平面对应的平面参数；其中，所述目标平面为检测到的所有平面中，占据所述待处理图像的面积最大的平面。
  14. 一种平面矫正装置，其特征在于，包括：
    参数计算模块，用于获取待处理图像中目标平面的平面参数，根据所述平面参数计算所述待处理图像对应的位置标识线和矫正矩阵；其中，所述位置标识线在所述待处理图像的像平面上；
    平面矫正模块，用于在所述位置标识线与所述待处理图像的边框在所述像平面上存在交点时，根据所述平面参数在所述待处理图像中确定矫正区域，并根据所述矫正矩阵针对所述矫正区域进行矫正，得到矫正图像。
  15. 一种电子设备，其特征在于，包括：
    处理器；以及
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1至12任一项所述的方法。
PCT/CN2020/106624 2020-08-03 2020-08-03 平面矫正方法及装置、计算机可读介质和电子设备 WO2022027191A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080104505.6A CN116134476A (zh) 2020-08-03 2020-08-03 平面矫正方法及装置、计算机可读介质和电子设备
PCT/CN2020/106624 WO2022027191A1 (zh) 2020-08-03 2020-08-03 平面矫正方法及装置、计算机可读介质和电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/106624 WO2022027191A1 (zh) 2020-08-03 2020-08-03 平面矫正方法及装置、计算机可读介质和电子设备

Publications (1)

Publication Number Publication Date
WO2022027191A1 true WO2022027191A1 (zh) 2022-02-10

Family

ID=80119283

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/106624 WO2022027191A1 (zh) 2020-08-03 2020-08-03 平面矫正方法及装置、计算机可读介质和电子设备

Country Status (2)

Country Link
CN (1) CN116134476A (zh)
WO (1) WO2022027191A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114596346A (zh) * 2022-03-15 2022-06-07 北京达佳互联信息技术有限公司 图像处理方法、视频处理方法、装置以及电子设备
CN117853382A (zh) * 2024-03-04 2024-04-09 武汉人工智能研究院 基于稀疏标记物的图像矫正方法、装置及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101561924A (zh) * 2008-04-18 2009-10-21 株式会社Pfu 笔记本信息处理器和投影变换参数计算方法
US20100128972A1 (en) * 2008-11-25 2010-05-27 Nec System Technologies, Ltd. Stereo matching processing system, stereo matching processing method and recording medium
CN102521806A (zh) * 2011-12-04 2012-06-27 山东大学 基于消失线的长方体表面透视变形的矫正方法
CN103177253A (zh) * 2011-12-22 2013-06-26 北大方正集团有限公司 一种透视变形平面几何形状特征信息提取方法及系统

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3723111B1 (en) * 2012-10-02 2021-09-08 Nikon Corporation Exposure apparatus, exposure method and device manufacturing method
CN106447730B (zh) * 2016-09-14 2020-02-28 深圳地平线机器人科技有限公司 参数估计方法、装置和电子设备
CN110068270B (zh) * 2019-04-18 2021-04-02 上海拓今智能科技有限公司 一种基于多线结构光图像识别的单目视觉箱体体积测量方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101561924A (zh) * 2008-04-18 2009-10-21 株式会社Pfu 笔记本信息处理器和投影变换参数计算方法
US20100128972A1 (en) * 2008-11-25 2010-05-27 Nec System Technologies, Ltd. Stereo matching processing system, stereo matching processing method and recording medium
CN102521806A (zh) * 2011-12-04 2012-06-27 山东大学 基于消失线的长方体表面透视变形的矫正方法
CN103177253A (zh) * 2011-12-22 2013-06-26 北大方正集团有限公司 一种透视变形平面几何形状特征信息提取方法及系统

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114596346A (zh) * 2022-03-15 2022-06-07 北京达佳互联信息技术有限公司 图像处理方法、视频处理方法、装置以及电子设备
CN117853382A (zh) * 2024-03-04 2024-04-09 武汉人工智能研究院 基于稀疏标记物的图像矫正方法、装置及存储介质
CN117853382B (zh) * 2024-03-04 2024-05-28 武汉人工智能研究院 基于稀疏标记物的图像矫正方法、装置及存储介质

Also Published As

Publication number Publication date
CN116134476A (zh) 2023-05-16

Similar Documents

Publication Publication Date Title
CN112927362B (zh) 地图重建方法及装置、计算机可读介质和电子设备
US10915998B2 (en) Image processing method and device
US9135678B2 (en) Methods and apparatus for interfacing panoramic image stitching with post-processors
US20120321193A1 (en) Method, apparatus, and computer program product for image clustering
CN112085775B (zh) 图像处理的方法、装置、终端和存储介质
CN108830186B (zh) 文本图像的内容提取方法、装置、设备及存储介质
KR102402148B1 (ko) 전자 장치 및 그의 문자 인식 방법
CN110781823B (zh) 录屏检测方法、装置、可读介质及电子设备
WO2022027191A1 (zh) 平面矫正方法及装置、计算机可读介质和电子设备
WO2022042120A1 (zh) 目标图像提取方法、神经网络训练方法及装置
WO2021129345A1 (zh) 场景地图建立方法、设备及存储介质
CN114049674A (zh) 一种三维人脸重建方法、装置及存储介质
CN113569052A (zh) 知识图谱的表示学习方法及装置
CN106548117A (zh) 一种人脸图像处理方法和装置
CN110765926A (zh) 图画书识别方法、装置、电子设备和存储介质
JP7003617B2 (ja) 推定装置、推定方法、及び推定プログラム
WO2023061195A1 (zh) 图像获取模型的训练方法、图像检测方法、装置及设备
CN111402391A (zh) 用户面部图像展示方法、展示装置及对应的存储介质
CN110163192B (zh) 字符识别方法、装置及可读介质
KR102534449B1 (ko) 이미지 처리 방법, 장치, 전자 장치 및 컴퓨터 판독 가능 저장 매체
CN112291445B (zh) 图像处理的方法、装置、设备和存储介质
WO2022011560A1 (zh) 图像裁剪方法与装置、电子设备及存储介质
CN116137025A (zh) 视频图像矫正方法及装置、计算机可读介质和电子设备
CN112348069B (zh) 数据增强方法、装置、计算机可读存储介质及终端设备
WO2021073204A1 (zh) 对象的显示方法、装置、电子设备及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20948576

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20948576

Country of ref document: EP

Kind code of ref document: A1