CN114510192A

CN114510192A - Image processing method and related device

Info

Publication number: CN114510192A
Application number: CN202210113847.5A
Authority: CN
Inventors: 顾磊
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2022-01-30
Filing date: 2022-01-30
Publication date: 2022-05-17
Anticipated expiration: 2042-01-30
Also published as: CN114510192B

Abstract

The application discloses an image processing method and a related device, which are applied to electronic equipment, wherein the method comprises the following steps: acquiring a target image, wherein the target image comprises at least one target object with a quadrilateral peripheral outline; performing line segment detection on the target image to obtain a plurality of line segments; determining at least one candidate quadrangle according to the line segments; acquiring a first touch parameter aiming at the target image; performing confidence calculation on each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence, wherein each confidence is used for representing the strength of will of the corresponding candidate quadrangle for being selected; ranking the at least one candidate quadrilateral according to the at least one confidence; and displaying the image of the area where the at least one candidate quadrangle is located according to the sorting result. By adopting the method and the device, the quadrangle needed by the user can be quickly identified.

Description

Image processing method and related device

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image processing method and a related apparatus.

Background

In life, for convenience of work, when files are stored, a user sometimes scans a large number of documents, business cards, posters and the like, and stores the scanned files in electronic equipment (such as a mobile phone, a tablet personal computer and the like).

In general, when scanning documents, business cards, posters, etc., due to the fact that these objects usually have quadrangles, the photographed images may be interfered by the lines of the objects themselves or the background, and therefore, false detection may occur, and a wrong quadrangle is detected, and therefore, how to quickly identify the quadrangles required by the user is a problem to be solved.

Disclosure of Invention

The embodiment of the application provides an image processing method and a related device, which can quickly identify a quadrangle required by a user.

In a first aspect, an embodiment of the present application provides an image processing method, where the method includes:

acquiring a target image, wherein the target image comprises at least one target object with a quadrilateral peripheral outline;

performing line segment detection on the target image to obtain a plurality of line segments;

determining at least one candidate quadrangle according to the line segments;

acquiring a first touch parameter aiming at the target image;

performing confidence calculation on each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence, wherein each confidence is used for representing the strength of will of the corresponding candidate quadrangle for being selected;

ranking the at least one candidate quadrilateral according to the at least one confidence;

and displaying the image of the area where the at least one candidate quadrangle is located according to the sorting result.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including: a first acquisition unit, a detection unit, a determination unit, a second acquisition unit, a calculation unit, a sorting unit and a presentation unit, wherein,

the first acquisition unit is used for acquiring a target image, and the target image comprises at least one target object with a quadrilateral peripheral outline;

the detection unit is used for carrying out line segment detection on the target image to obtain a plurality of line segments;

the determining unit is used for determining at least one candidate quadrangle according to the line segments;

the second obtaining unit is used for obtaining a first touch parameter aiming at the target image;

the calculation unit is configured to perform confidence calculation on each candidate quadrangle of the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence, where each confidence is used to indicate an intensity of will of the corresponding candidate quadrangle being selected;

the ordering unit is used for ordering the at least one candidate quadrangle according to the at least one confidence level;

the display unit is used for displaying the image of the area where the at least one candidate quadrangle is located according to the sorting result.

In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, and a memory, where the memory is configured to store one or more programs and is configured to be executed by the processor, and the program includes instructions for performing some or all of the steps described in the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, where the computer program enables a computer to perform some or all of the steps described in the first aspect of the embodiment of the present application.

In a fifth aspect, embodiments of the present application provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, where the computer program is operable to cause a computer to perform some or all of the steps as described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

The embodiment of the application has the following beneficial effects:

it can be seen that, in the image processing method and the related apparatus described in the embodiments of the present application, a target image is obtained, the target image includes at least one target object whose peripheral contour is a quadrangle, a line segment detection is performed on the target image to obtain a plurality of line segments, at least one candidate quadrangle is determined according to the plurality of line segments, a first touch parameter for the target image is obtained, a confidence level calculation is performed on each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence level, each confidence level is used to indicate a strength of will of selecting the corresponding candidate quadrangle, the at least one candidate quadrangle is ranked according to the at least one confidence level, an image of a region where the at least one candidate quadrangle is located is displayed according to a ranking result, so that the candidate quadrangle can be quickly determined through the line segment detection, and then determining the reliability between the touch parameter and the candidate quadrangle, reflecting the selection intention of the user through the reliability, sequencing based on the reliability, quickly identifying the quadrangle really required by the user, and displaying the image of the area where the quadrangle is located.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;

fig. 2 is a schematic diagram of a software structure of an electronic device according to an embodiment of the present application;

fig. 3A is a schematic flowchart of an image processing method according to an embodiment of the present application;

fig. 3B is a schematic diagram illustrating a user touch operation provided in an embodiment of the present application;

fig. 3C is a schematic diagram illustrating a touch point provided in the embodiment of the present application;

fig. 3D is a schematic diagram illustrating a touch trajectory provided in an embodiment of the present application;

fig. 3E is a schematic illustration showing a touch area provided in an embodiment of the present application;

fig. 3F is a schematic diagram illustrating a relationship between a touch parameter and a quadrilateral according to an embodiment of the present disclosure;

fig. 3G is a schematic diagram illustrating another relationship between a touch parameter and a quadrilateral provided in the embodiment of the present application;

fig. 3H is a schematic diagram illustrating another relationship between a touch parameter and a quadrilateral provided in the embodiment of the present application;

FIG. 3I is a schematic diagram illustrating a confidence level calculation according to an embodiment of the present disclosure;

fig. 3J is a schematic flowchart of another image processing method provided in the embodiment of the present application;

FIG. 3K is a schematic flowchart of another image processing method provided in the embodiments of the present application;

fig. 3L is a schematic flowchart of another image processing method provided in the embodiment of the present application;

fig. 4 is a schematic structural diagram of another electronic device provided in an embodiment of the present application;

fig. 5 is a block diagram of functional units of an image processing apparatus according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

In order to better understand the scheme of the embodiments of the present application, the following first introduces the related terms and concepts that may be involved in the embodiments of the present application.

The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

In specific implementation, in this embodiment of the application, the electronic device may include various devices having a computer function, for example, a handheld device (a smart phone, a tablet computer, etc.), a vehicle-mounted device (a navigator, an auxiliary backing system, a car recorder, a vehicle-mounted refrigerator, etc.), a wearable device (a smart bracelet, a wireless headset, a smart watch, smart glasses, etc.), a computing device or other processing devices connected to a wireless modem, and various forms of User Equipment (UE), a Mobile Station (MS), a virtual reality/augmented reality device, a terminal device (terminal device), etc., where the electronic device may also be a base Station or a server.

The electronic device may further include an intelligent home device, and the intelligent home device may be at least one of: intelligent audio amplifier, intelligent camera, intelligent electric rice cooker, intelligent wheelchair, intelligent massage armchair, intelligent furniture, intelligent dish washer, intelligent TV set, intelligent refrigerator, intelligent electric fan, intelligent room heater, intelligent clothes hanger that dries in the air, intelligent lamp, intelligent router, intelligent switch, intelligent flush mounting plate, intelligent humidifier, intelligent air conditioner, intelligent door, intelligent window, intelligent top of a kitchen range, intelligent sterilizer, intelligent closestool, the robot etc. of sweeping the floor do not restrict here.

In a first section, the software and hardware operating environment of the technical solution disclosed in the present application is described as follows.

As shown, fig. 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a compass 190, a motor 191, a pointer 192, a camera 193, a display screen 194, a Subscriber Identification Module (SIM) card interface 195, and the like.

It is to be understood that the illustrated structure of the embodiment of the present application does not specifically limit the electronic device 100. In other embodiments of the present application, electronic device 100 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units, such as: the processor 110 may include an application processor AP, a modem processor, a graphics processor GPU, an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural network processor NPU, among others. Wherein the different processing units may be separate components or may be integrated in one or more processors. In some embodiments, the electronic device 101 may also include one or more processors 110. The controller can generate an operation control signal according to the instruction operation code and the time sequence signal to complete the control of instruction fetching and instruction execution. In other embodiments, a memory may also be provided in processor 110 for storing instructions and data. Illustratively, the memory in the processor 110 may be a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to reuse the instruction or data, it can be called directly from memory. This avoids repeated accesses and reduces the latency of the processor 110, thereby increasing the efficiency with which the electronic device 101 processes data or executes instructions. The processor may also include an image processor, which may be an image Pre-processor (Pre-ISP), which may be understood as a simplified ISP, which may also perform some image processing operations, e.g. may obtain image statistics.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit audio (I2S) interface, a Pulse Code Modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a SIM card interface, a USB interface, and/or the like. The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the electronic device 101, and may also be used to transmit data between the electronic device 101 and peripheral devices. The USB interface 130 may also be used to connect to a headset to play audio through the headset.

It should be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only an illustration, and does not limit the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners or a combination of multiple interface connection manners in the above embodiments.

The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 may also supply power to the electronic device through the power management module 141 while charging the battery 142.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be used to monitor parameters such as battery capacity, battery cycle count, battery state of health (leakage, impedance), etc. In some other embodiments, the power management module 141 may also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be disposed in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution including wireless communication of 2G/3G/4G/5G/6G, etc. applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 150 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules of the processor 110.

The wireless communication module 160 may provide a solution for wireless communication applied to the electronic device 100, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (blue tooth, BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into electromagnetic waves through the antenna 2 to radiate the electromagnetic waves.

The electronic device 100 implements display functions via the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used to display images, videos, and the like. The display screen 194 includes a display panel. The display panel may be a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (mini-led), a Micro led, a Micro-o led, a quantum dot light-emitting diode (QLED), or the like. In some embodiments, the electronic device 100 may include 1 or more display screens 194.

The electronic device 100 may implement a photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, the application processor, and the like.

The ISP is used to process the data fed back by the camera 193. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, the electronic device 100 may include 1 or more cameras 193.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to perform fourier transform or the like on the frequency bin energy.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record video in a variety of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor that processes input information quickly by using a biological neural network structure, for example, by using a transfer mode between neurons of a human brain, and can also learn by itself continuously. Applications such as intelligent recognition of the electronic device 100 can be realized through the NPU, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to extend the memory capability of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

Internal memory 121 may be used to store one or more computer programs, including instructions. The processor 110 may execute the above-mentioned instructions stored in the internal memory 121, so as to enable the electronic device 101 to execute the method for displaying page elements provided in some embodiments of the present application, and various applications and data processing. The internal memory 121 may include a program storage area and a data storage area. Wherein, the storage program area can store an operating system; the storage program area may also store one or more applications (e.g., gallery, contacts, etc.), and the like. The storage data area may store data (such as photos, contacts, etc.) created during use of the electronic device 101, and the like. Further, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic disk storage components, flash memory components, Universal Flash Storage (UFS), and the like. In some embodiments, the processor 110 may cause the electronic device 101 to execute the method for displaying page elements provided in the embodiments of the present application, and other applications and data processing by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor 110. The electronic device 100 may implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. Such as music playing, recording, etc.

The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

The pressure sensor 180A is used for sensing a pressure signal, and converting the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A can be of a wide variety, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a sensor comprising at least two parallel plates having an electrically conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the strength of the pressure from the change in capacitance. When a touch operation is applied to the display screen 194, the electronic apparatus 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic apparatus 100 may also calculate the touched position from the detection signal of the pressure sensor 180A. In some embodiments, the touch operations that are applied to the same touch position but different touch operation intensities may correspond to different operation instructions. For example: and when the touch operation with the touch operation intensity smaller than the first pressure threshold value acts on the short message application icon, executing an instruction for viewing the short message. And when the touch operation with the touch operation intensity larger than or equal to the first pressure threshold value acts on the short message application icon, executing an instruction of newly building the short message.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of electronic device 100 about three axes (i.e., X, Y and the Z axis) may be determined by gyroscope sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. For example, when the shutter is pressed, the gyro sensor 180B detects a shake angle of the electronic device 100, calculates a distance to be compensated for by the lens module according to the shake angle, and allows the lens to counteract the shake of the electronic device 100 through a reverse movement, thereby achieving anti-shake. The gyroscope sensor 180B may also be used for navigation, somatosensory gaming scenes.

The acceleration sensor 180E may detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. The method can also be used for recognizing the posture of the electronic equipment, and is applied to horizontal and vertical screen switching, pedometers and other applications.

The ambient light sensor 180L is used to sense the ambient light level. Electronic device 100 may adaptively adjust the brightness of display screen 194 based on the perceived ambient light level. The ambient light sensor 180L may also be used to automatically adjust the white balance when taking a picture. The ambient light sensor 180L may also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

The fingerprint sensor 180H is used to collect a fingerprint. The electronic device 100 can utilize the collected fingerprint characteristics to unlock the fingerprint, access the application lock, photograph the fingerprint, answer an incoming call with the fingerprint, and so on.

The temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 implements a temperature processing strategy using the temperature detected by temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 performs a reduction in performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, the electronic device 100 heats the battery 142 when the temperature is below another threshold to avoid the low temperature causing the electronic device 100 to shut down abnormally. In other embodiments, when the temperature is lower than a further threshold, the electronic device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

The touch sensor 180K is also referred to as a "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation applied thereto or nearby. The touch sensor can communicate the detected touch operation to the application processor to determine the touch event type. Visual output associated with the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may be disposed on the surface of the electronic device 100 at a different position than the display screen 194.

Fig. 2 shows a block diagram of a software structure of the electronic device 100. The layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, an application layer, an application framework layer, an Android runtime (Android runtime) and system library, and a kernel layer from top to bottom. The application layer may include a series of application packages.

As shown in fig. 2, the application layer may include applications such as camera, gallery, calendar, phone call, map, navigation, WLAN, bluetooth, music, video, short message, etc.

The application framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions.

As shown in FIG. 2, the application framework layers may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and the like.

The window manager is used for managing window programs. The window manager can obtain the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide communication functions of the electronic device 100. Such as management of call status (including on, off, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.

The notification manager enables the application to display notification information in the status bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text at the top status bar of the system, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, prompting text information in the status bar, sounding a prompt tone, vibrating the electronic device, flashing an indicator light, etc.

The Android Runtime comprises a core library and a virtual machine. The Android runtime is responsible for scheduling and managing an Android system.

The core library comprises two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. And executing java files of the application program layer and the application program framework layer into a binary file by the virtual machine. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface managers (surface managers), media libraries (media libraries), three-dimensional graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), and the like.

The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files, among others. The media library may support a variety of audio-video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

In the second section, the image processing method and the related apparatus disclosed in the embodiments of the present application are described as follows.

Referring to fig. 3A, fig. 3A is a schematic flowchart of an image processing method according to an embodiment of the present application, where as shown in the figure, the image processing method includes:

301. and acquiring a target image, wherein the target image comprises at least one target object with a quadrilateral peripheral outline.

In this embodiment of the application, the target object may be an object whose peripheral contour is a quadrangle or an approximate quadrangle, for example, the target object may include at least one of the following: business cards, posters, books, packaging boxes, documents (e.g., identification cards, driver's licenses, social security cards, etc.), wallets, transit cards, bank cards, greeting cards, toy cards, playing cards, magic prop cards, memorial cards, amulet cards, graffiti, hand paintings, photographs, boxes, bread, and the like, without limitation. In a specific implementation, the target image may be a scanned image, a pre-stored image, or a currently captured image. Of course, the quadrilateral peripheral outline of the target object in the target image may be a complete quadrilateral, or may also be a partially incomplete quadrilateral, for example, an identity card with one corner cut off. The number of the target objects can be one or more, and the peripheral outline of each target object is a quadrangle.

In a specific implementation, a target object may be photographed, and a target image may be obtained, and since other backgrounds may exist in the target image, more than one quadrilateral target object may exist, and of course, a quadrilateral pattern may exist on the wallpaper, and may also be used as the target object. As shown in fig. 3B, in the target image, the user realizes a touch function through a gesture to select a quadrangle required by the user, where the quadrangle selected by the user may be a target quadrangle, and the quadrangle not selected by the user may be a non-target quadrangle.

The target image can be one or more frames of images, and when the target image is multiple frames, the functions of batch scanning and storage of objects can be realized.

302. And carrying out line segment detection on the target image to obtain a plurality of line segments.

In specific implementation, a line segment detection algorithm may be used to perform line segment detection on the target image to obtain a plurality of line segments, and the line segment detection algorithm may include at least one of the following: hough transform, fast line detection, etc., and are not limited herein.

In specific implementation, a line segment detection algorithm can be adopted to perform line segment detection on the target image, and then the detected line segments are preprocessed, so that some line segments with poor quality can be screened out through preprocessing. For example, the following functions may be implemented by preprocessing, such as filtering out some line segments with widths lower than a preset width, or filtering out some line segments with lengths smaller than a preset length, or removing noise line segments from the detected line segments, or performing overlapping line segment fusion processing on the detected line segments, and then obtaining a plurality of line segments after preprocessing. Both the preset width and the preset length may be preset or default to the system.

303. At least one candidate quadrilateral is determined from the plurality of line segments.

In a specific implementation, at least one candidate quadrangle may be obtained by performing permutation and combination according to a plurality of line segments, for example, four line segments may form a closed region, and if the closed region is a quadrangle, the closed region may be regarded as a candidate quadrangle.

Specifically, different manners may be adopted according to practical applications, for example, four line segments may be included, and the four line segments may be arranged and combined to obtain all possible candidate quadrangles.

Of course, if some line segments cannot form a closed quadrangle in the arranging and combining process, the sides (line segments) of the unclosed quadrangle may be extended to obtain a closed quadrangle. In a specific implementation, a length ratio between the length of the front line segment to the length of the rear line segment is in a set range interval, and the set range interval may be preset or default by a system.

304. And acquiring a first touch parameter aiming at the target image.

In a specific implementation, the first touch parameter may include at least one of the following: the touch position of the touch display screen, the touch area of the touch display screen, the number of touch points of the touch display screen, the touch strength of the touch display screen, the touch duration of the touch display screen, the touch frequency of the touch display screen, the touch trajectory of the touch display screen, the touch pattern of the touch display screen, and the like, which are not limited herein. In this embodiment, the touch operation may be completed before step 304 or step 304, the first touch parameter may be a touch parameter within a preset time range, and the preset time range may be preset or default by the system, for example, the preset time range may be a touch parameter within the latest 1 second. As shown in fig. 3C, after the user touches, at least one touch point may be generated, and each touch point may correspond to one coordinate point. As shown in fig. 3D, after the user touches the target image, a corresponding touch track may be generated, where the touch track may be one or more, for example, multiple fingers touch, and one finger may correspond to one touch track. As shown in fig. 3E, in the target image, a touch area may be formed by a touch operation of the user, and the touch area may be a closed area.

In specific implementation, a target image can be displayed on a touch display screen of the electronic device, and a user can perform touch operation on an area of the target image in the touch display screen, so that a first touch parameter can be obtained.

305. And calculating the reliability of each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one reliability, wherein each reliability is used for representing the strength of will of the corresponding candidate quadrangle for being selected.

In a specific implementation, the candidate quadrangles may be a contour of the target object to a large extent, and the first touch parameter reflects a desire of the user to select the target object to a certain extent, and further, the first touch parameter may determine each candidate quadrangle of the at least one candidate quadrangle to perform confidence calculation, so as to obtain at least one confidence, for example, the touch parameter may include a touch position, and the touch position may correspond to a coordinate, and then may determine an average distance between the coordinate and four vertices of each candidate quadrangle of the at least one candidate quadrangle, and according to a mapping relationship between a preset distance and the confidence, may determine the confidence corresponding to each average distance, that is, determine the confidence corresponding to each candidate quadrangle. The reliability reflects the strength of the intention of the user to select the candidate quadrangle, and for example, the stronger the intention, the greater the reliability.

For example, as shown in fig. 3F, in the target image, the touch points generated by the touch operation may fall into different candidate quadrangles, and the corresponding confidence level may be determined according to how many touch points in each candidate quadrangle, for example, the confidence level is greater as the number of touch points is greater. As shown in fig. 3G, in the target image, the touch tracks generated by the touch operation may fall into different candidate quadrangles, and the total length of the track length in each candidate quadrangle may be counted, for example, the longer the total length is, the greater the reliability is. As shown in fig. 3H, in the target image, the touch area generated by the touch operation may also fall into different candidate quadrangles, and the area of the touch area in each candidate quadrangle may be counted. Of course, it is also possible to detect whether the four deformations are the quadrangles required by the user with respect to the deviation degrees between the four corners and the right angle of the quadrangle, and in general, the real object selected by the user is a rectangle with the four corners thereof being 90 degrees, and if the deviation between the angle in the candidate quadrangle and 90 degrees is larger, it is described that the candidate quadrangle is not the quadrangle selected by the user, and the reliability thereof is lower. In addition, the confidence may also be determined by determining the distance between the touch center and the center (middle position) of the candidate quadrangle, for example, the confidence may be greater as the distance is smaller.

Specifically, for the calculation of the reliability, the calculation may be performed by integrating information of each dimension to calculate the integrated reliability of the quadrangle. The comprehensive strategy may be to add the evaluation functions, to set weights for the functions to weight the cameras, or to set normalization processing for the functions and then to add the functions. As shown in fig. 3I, each dimension may correspond to an evaluation parameter, an area ratio may be understood as a ratio between an area of a touch region falling in a quadrangle and a total touch area, a vertical angle may be understood as a correlation between an angle of the quadrangle and a vertical angle (90 degrees), a center position may be understood as a position relationship between a touch center and a center of the quadrangle, and further, based on each evaluation function, the input coordinate information and the quadrangle information may be combined to calculate an influence of gesture information (touch parameter) given by a user on evaluation of the quadrangle reliability. Generally, if the more user gesture information appears in the range of the quadrangle, the greater the confidence given by the evaluation function, so that the overall confidence of the quadrangle increases.

Further, the calculation of the merit function, such as: the number of times that user's touch-control point appears in the quadrangle can be calculated, or, also can calculate the total length that user's touch-control orbit is in the quadrangle inside, or, also can calculate the total area that user's gesture region is in the quadrangle inside, or, also can calculate the ratio that the total area that user's gesture region is in the quadrangle inside accounts for the quadrangle area, and then, can carry out weighted operation with the evaluation function of these dimensions, finally, obtain the evaluation function after synthesizing, because the user's wish has been fully expressed to a plurality of dimensions, and then, can accurately grasp user's intention, promote user experience.

Optionally, in the step 305, performing confidence calculation on each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence, which may include the following steps:

51. determining a touch parameter of at least one dimension according to the first touch parameter;

52. acquiring a weight corresponding to the touch parameter of each dimension in the touch parameters of the at least one dimension;

53. and determining the confidence corresponding to each candidate quadrangle in the at least one candidate quadrangle according to the touch parameter of the at least one dimension and the weight corresponding to the touch parameter of each dimension to obtain the at least one confidence.

In a specific implementation, the touch parameter of at least one dimension may be determined according to the first touch parameter, the at least one dimension may include one or more touch parameters, and the touch parameter of one dimension may include any one of: the touch position of the touch display screen, the touch area of the touch display screen, the number of touch points of the touch display screen, the touch strength of the touch display screen, the touch duration of the touch display screen, the touch frequency of the touch display screen, the touch trajectory of the touch display screen, the touch pattern of the touch display screen, and the like, which are not limited herein.

Furthermore, a weight corresponding to the touch parameter of each dimension in the touch parameter of at least one dimension may be obtained to obtain at least one weight, the weight may be preset or default in a system, different dimensions may correspond to different weights, a sum of the at least one weight may be less than or equal to 1, and further, a confidence corresponding to each candidate quadrangle in the at least one candidate quadrangle may be determined according to the touch parameter of at least one dimension and the at least one weight to obtain at least one confidence, wherein for any candidate quadrangle, each dimension may correspond to a reference confidence, and then the reference reliabilities of all dimensions are weighted to obtain a final confidence of the candidate quadrangle, since the selection intention of the user is reflected by a plurality of dimensions, further, the selection of the user may be accurately grasped, and the user experience is improved.

Further, optionally, in the step 53, determining a confidence level corresponding to each candidate quadrangle in the at least one candidate quadrangle according to the touch parameter of the at least one dimension and the weight corresponding to the touch parameter of each dimension, to obtain the at least one confidence level, includes:

531. determining a reference confidence level between each touch parameter in the touch parameters of the at least one dimension and a candidate quadrangle i to obtain at least one reference confidence level, wherein the candidate quadrangle i is any quadrangle in the at least one candidate quadrangle;

532. and performing weighting operation according to the at least one reference reliability and the at least one weight to obtain the reliability corresponding to the candidate quadrangle i.

For any candidate quadrangle, it may correspond to a confidence level, and taking the candidate quadrangle i as an example, the candidate quadrangle i is any quadrangle in at least one candidate quadrangle. The reference reliability between each touch parameter in the touch parameters of at least one dimension and the candidate quadrangle i can be determined to obtain at least one reference reliability, then, weighting operation can be performed according to at least one reference reliability and at least one weight, namely, multiplication operation is performed on each reference reliability and the corresponding weight, and then, the results of all the multiplication operations are summed, so that the reliability corresponding to the candidate quadrangle i can be obtained.

Optionally, in step 531, determining a reference reliability between each of the at least one dimension of the touch parameters and the candidate quadrangle i to obtain at least one reference reliability, which may be implemented as follows:

when the touch parameter is a touch point, determining the number of touch points of the touch point falling into the candidate quadrilateral i, and determining the reference reliability according to the number of the touch points;

alternatively, the first and second electrodes may be,

when the touch parameter is a touch track, determining the track length of the touch track falling into the candidate quadrangle i, and determining the reference reliability according to the track length;

alternatively, the first and second electrodes may be,

and when the touch parameter is a touch area, determining the touch area of the touch area falling into the candidate quadrangle i, and determining the reference reliability according to the touch area.

In a specific implementation, when the touch parameter is a touch point, the number of touch points falling into the candidate quadrilateral i may be determined, a mapping relationship between the number of touch points and the reliability may be preset, and then a reference reliability corresponding to the number of touch points may be determined based on the mapping relationship. For another example, when the touch parameter is a touch trajectory, the trajectory length of the touch trajectory falling into the candidate quadrilateral i may be determined, and the reference confidence corresponding to the trajectory length may be determined according to a mapping relationship between a preset length and the confidence. For another example, when the touch parameter is a touch area, the touch area of the touch area falling into the candidate quadrilateral i may be determined, and the reference confidence level corresponding to the touch area may be determined according to a mapping relationship between a preset area and the confidence level.

306. Ranking the at least one candidate quadrilateral according to the at least one confidence.

In a specific implementation, the confidence degrees reflect the user's selection intention, and further, the different confidence degrees reflect different intentions of the user, so that the at least one candidate quadrangle can be sorted according to the at least one confidence degree, for example, the at least one candidate quadrangle can be sorted according to a descending order of the confidence degrees.

307. And displaying the image of the area where the at least one candidate quadrangle is located according to the sorting result.

In specific implementation, in the embodiment of the application, the candidate quadrangle corresponding to the maximum confidence level may be displayed according to the sorting result, and certainly, the region image corresponding to the candidate quadrangle may be subjected to perspective transformation, and then the region image after the perspective transformation is displayed.

In a specific implementation, as shown in fig. 3J, segment detection is performed on a target image, candidate quadrangles are found based on a detection result, confidence calculation can be performed on all candidate quadrangles based on a first touch parameter, all candidate quadrangles are sorted according to the confidence, and information of a quadrangle with the highest confidence is selected from the sorted quadrangles and output. Namely, the user interface can display a preview picture currently shot by the camera, display the best quadrangle detected in the current picture, respond to the interaction of the user and update the displayed contact or contact track, and further, can process the area image corresponding to the best quadrangle and output the processed image.

In the embodiment of the application, a multi-dimensional touch parameter input by a user, such as a multi-point touch coordinate, a touch track, a touch area and the like, can be provided in a preview user interface of document scanning, a position or a range of a quadrangle expected by the user is indicated based on the multi-dimensional touch parameter, information such as the coordinate of an input contact or track is incorporated into the calculation of the sum reliability of the quadrangles when the optimal quadrangle is found, the intention of the user can be more accurately reflected in the result of quadrangle detection, so that the automatic detection of the quadrangle is assisted to obtain a better effect, and false detection in ambiguous or more complex scenes is avoided.

Optionally, in the step 307, displaying the image of the area where the at least one candidate quadrangle is located according to the sorting result, the method may include the following steps:

71. selecting a preset number of candidate quadrangles from the at least one candidate quadrangle according to the sorting result to obtain at least one reference candidate quadrangle;

72. screening the at least one reference candidate quadrangle to obtain a target quadrangle;

73. carrying out perspective transformation on the image of the region where the target quadrangle is located to obtain a target region image;

74. and displaying the target area image.

In a specific implementation, the preset number may be preset or default to the system, for example, the preset number may be a fixed value, or may be a specified percentage of the total number of the at least one candidate quadrangle, and the specified percentage may be preset or default to the system.

Specifically, a preset number of candidate quadrangles ranked in front in the at least one candidate quadrangle can be selected according to the ranking result to obtain at least one reference candidate quadrangle, the at least one reference candidate quadrangle can be screened to obtain a target quadrangle, the quadrangle most desired by the user is screened out, then the image of the region where the target quadrangle is located is subjected to perspective transformation to obtain a target region image, namely the region image of the original quadrangle is converted into a rectangular region image, and finally the target region image can be displayed.

Optionally, in the step 72, the step of screening the at least one reference candidate quadrangle to obtain the target quadrangle may include the following steps:

a721, acquiring a second touch parameter;

and A722, selecting the target quadrangle from the at least one reference candidate quadrangle according to the second touch parameter.

The second touch parameter may be the same as the first touch parameter, or may be different from the first touch parameter. The second touch parameter may include at least one of: the touch position of the touch display screen, the touch area of the touch display screen, the number of touch points of the touch display screen, the touch strength of the touch display screen, the touch duration of the touch display screen, the touch frequency of the touch display screen, the touch trajectory of the touch display screen, the touch pattern of the touch display screen, and the like, which are not limited herein. In the step 72, a candidate quadrangle with the highest reliability may be selected as the target quadrangle based on the reliability calculation principle. As shown in fig. 3K, the user touch operation may generate a second touch parameter, and the quadrilateral filtering is implemented by using the second touch parameter.

Of course, the target quadrangle may also be determined by, for example, displaying all candidate quadrangles in the at least one reference candidate quadrangle on the display screen, and selecting the candidate quadrangle that needs to be retained by the user through a touch operation, for example, retaining the candidate quadrangle when the user touches which candidate quadrangle.

b721, obtaining eyeball gazing parameters of the user;

and B722, selecting the target quadrangle from the at least one reference candidate quadrangle according to the user eyeball gaze parameter.

Wherein the user eye gaze parameters may comprise at least one of: the gazing position, the gazing duration, and the like, which are not limited herein, may correspond to a coordinate, and the closer the coordinate is to the reference candidate quadrangle, the stronger the user will select the reference candidate quadrangle. In the embodiment of the present application, as shown in fig. 3L, a gaze heat map may also be constructed based on the eyeball position, the gaze position, and the center of each reference candidate quadrangle, that is, a body coordinate system may be constructed, the points are mapped to the body coordinate system, the first distance between the eyeball position and the gaze position is determined, and the second distance between the eyeball position and the center of each reference candidate quadrangle is determined, so that the absolute value of the difference between the first distance and each second distance may be determined, the reference candidate quadrangle corresponding to the minimum value of the absolute value is selected as the target quadrangle, and further, the user selection intention may be identified based on an eyeball tracking technology, so as to implement quick identification of the quadrangle required by the user.

It can be seen that, in the image processing method described in the embodiment of the present application, a target image is obtained, the target image includes at least one target object whose peripheral contour is a quadrangle, line segment detection is performed on the target image to obtain a plurality of line segments, at least one candidate quadrangle is determined according to the plurality of line segments, a first touch parameter for the target image is obtained, confidence calculation is performed on each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence, each confidence is used to represent the strength of will of the corresponding candidate quadrangle being selected, the at least one candidate quadrangle is ranked according to the at least one confidence, an image of a region where the at least one candidate quadrangle is located is displayed according to the ranking result, so that candidates can be quickly determined through line segment detection, and then the confidence between the touch parameter and the candidate quadrangle is determined, the selection intention of the user is reflected through the reliability, and the quadrangles really needed by the user are quickly identified and displayed on the image of the area where the quadrangles are located by sequencing based on the reliability.

In accordance with the foregoing embodiments, please refer to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and as shown in the drawing, the electronic device includes a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and in an embodiment of the present application, the programs include instructions for performing the following steps:

determining at least one candidate quadrangle according to the line segments;

acquiring a first touch parameter aiming at the target image;

Optionally, in the aspect that the confidence level is calculated for each candidate quadrangle of the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence level, the program includes instructions for performing the following steps:

determining a touch parameter of at least one dimension according to the first touch parameter;

acquiring a weight corresponding to the touch parameter of each dimension in the touch parameters of the at least one dimension;

and determining the confidence corresponding to each candidate quadrangle in the at least one candidate quadrangle according to the touch parameter of the at least one dimension and the weight corresponding to the touch parameter of each dimension to obtain the at least one confidence.

Optionally, in the aspect that the confidence level corresponding to each candidate quadrangle in the at least one candidate quadrangle is determined according to the touch parameter of the at least one dimension and the weight corresponding to the touch parameter of each dimension, so as to obtain the at least one confidence level, the program includes instructions for performing the following steps:

determining a reference confidence level between each touch parameter in the touch parameters of the at least one dimension and a candidate quadrangle i to obtain at least one reference confidence level, wherein the candidate quadrangle i is any quadrangle in the at least one candidate quadrangle;

and performing weighting operation according to the at least one reference reliability and the at least one weight to obtain the reliability corresponding to the candidate quadrangle i.

Optionally, in terms of determining a reference confidence between each of the touch parameters of the at least one dimension and the candidate quadrilateral i to obtain at least one reference confidence, the program includes instructions for:

alternatively, the first and second electrodes may be,

Optionally, in the aspect of displaying the image of the region where the at least one candidate quadrangle is located according to the sorting result, the program includes instructions for performing the following steps:

selecting a preset number of candidate quadrangles from the at least one candidate quadrangle according to the sorting result to obtain at least one reference candidate quadrangle;

screening the at least one reference candidate quadrangle to obtain a target quadrangle;

carrying out perspective transformation on the image of the region where the target quadrangle is located to obtain a target region image;

and displaying the target area image.

Optionally, in the aspect of screening the at least one reference candidate quadrangle to obtain the target quadrangle, the program includes instructions for performing the following steps:

acquiring a second touch parameter;

and selecting the target quadrangle from the at least one reference candidate quadrangle according to the second touch parameter.

acquiring eyeball watching parameters of a user;

and selecting the target quadrangle from the at least one reference candidate quadrangle according to the user eyeball gazing parameter.

It can be seen that, in the electronic device described in this embodiment of the present application, a target image is obtained, where the target image includes at least one target object whose peripheral contour is a quadrangle, a line segment detection is performed on the target image to obtain a plurality of line segments, at least one candidate quadrangle is determined according to the plurality of line segments, a first touch parameter for the target image is obtained, a confidence level calculation is performed on each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence level, where each confidence level is used to indicate the strength of will of the corresponding candidate quadrangle being selected, the at least one candidate quadrangle is ranked according to the at least one confidence level, an image of a region where the at least one candidate quadrangle is located is displayed according to a ranking result, and thus, the candidate quadrangles can be quickly determined through the line segment detection, and then the confidence level between the touch parameter and the candidate quadrangles is determined, the selection intention of the user is reflected through the reliability, and the quadrangles really needed by the user are quickly identified and displayed on the image of the area where the quadrangles are located by sequencing based on the reliability.

The above description has introduced the solution of the embodiment of the present application mainly from the perspective of the method-side implementation process. It is understood that the electronic device comprises corresponding hardware structures and/or software modules for performing the respective functions in order to realize the above-mentioned functions. Those of skill in the art will readily appreciate that the present application is capable of hardware or a combination of hardware and computer software implementing the various illustrative elements and algorithm steps described in connection with the embodiments provided herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiment of the present application, the electronic device may be divided into the functional units according to the method example, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit. It should be noted that the division of the unit in the embodiment of the present application is schematic, and is only a logic function division, and there may be another division manner in actual implementation.

Fig. 5 is a block diagram showing functional units of an image processing apparatus 500 according to an embodiment of the present application. The image processing apparatus 500 is applied to an electronic device, and the apparatus 500 includes: a first obtaining unit 501, a detecting unit 502, a determining unit 503, a second obtaining unit 504, a calculating unit 505, a sorting unit 506 and a presenting unit 507, wherein,

the first obtaining unit 501 is configured to obtain a target image, where the target image includes at least one target object whose peripheral outline is a quadrilateral;

the detection unit 502 is configured to perform line segment detection on the target image to obtain a plurality of line segments;

the determining unit 503 is configured to determine at least one candidate quadrangle according to the plurality of line segments;

the second obtaining unit 504 is configured to obtain a first touch parameter for the target image;

the calculating unit 505 is configured to perform confidence calculation on each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence, where each confidence is used to indicate an intensity of will of the corresponding candidate quadrangle being selected;

the sorting unit 506 is configured to sort the at least one candidate quadrangle according to the at least one confidence level;

the presentation unit 507 is configured to present an image of an area where the at least one candidate quadrangle is located according to the sorting result.

Optionally, in the aspect that the confidence level is calculated for each candidate quadrangle of the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence level, the calculating unit 505 is specifically configured to:

Optionally, in the aspect that the confidence level corresponding to each candidate quadrangle in the at least one candidate quadrangle is determined according to the touch parameter of the at least one dimension and the weight corresponding to the touch parameter of each dimension, so as to obtain the at least one confidence level, the calculating unit 505 is specifically configured to:

Optionally, in terms of determining a reference reliability between each of the touch parameters of the at least one dimension and the candidate quadrangle i to obtain at least one reference reliability, the calculating unit 505 is specifically configured to:

alternatively, the first and second electrodes may be,

Optionally, in the aspect of displaying the image of the area where the at least one candidate quadrangle is located according to the sorting result, the displaying unit 507 is specifically configured to:

and displaying the target area image.

Optionally, in the aspect of screening the at least one reference candidate quadrangle to obtain a target quadrangle, the display unit 507 is specifically configured to:

acquiring a second touch parameter;

acquiring eyeball watching parameters of a user;

It can be seen that, the image processing apparatus described in the embodiment of the present application obtains a target image, where the target image includes at least one target object whose peripheral contour is a quadrangle, performs line segment detection on the target image to obtain a plurality of line segments, determines at least one candidate quadrangle according to the plurality of line segments, obtains a first touch parameter for the target image, performs confidence calculation on each candidate quadrangle in the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence, where each confidence is used to indicate the strength of will of the corresponding candidate quadrangle being selected, sorts the at least one candidate quadrangle according to the at least one confidence, and displays an image of a region where the at least one candidate quadrangle is located according to the sorting result, so that, through line segment detection, candidates can be quickly determined, and then determines the confidence between the touch parameter and the candidate quadrangle, the selection intention of the user is reflected through the reliability, and the quadrangles really needed by the user are quickly identified and displayed on the image of the area where the quadrangles are located by sequencing based on the reliability.

It should be noted that the electronic device described in the embodiments of the present application is presented in the form of a functional unit. The term "unit" as used herein should be understood in its broadest possible sense, and objects used to implement the functionality described in each "unit" may be, for example, an integrated circuit ASIC, a single circuit, a processor (shared, dedicated, or group) and memory that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality.

The first obtaining unit 501, the detecting unit 502, the determining unit 503, the second obtaining unit 504, the calculating unit 505, the sorting unit 506, and the displaying unit 507 may be processors, which may be artificial intelligence chips, NPUs, CPUs, GPUs, etc., and are not limited herein, and the second obtaining unit 504 and the displaying unit 507 may include touch display screens. The functions or steps of any of the above methods can be implemented based on the above unit modules.

The present embodiment also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the method according to the present embodiment.

The present embodiment also provides a computer program product, which when run on a computer causes the computer to execute the relevant steps described above to implement any of the methods in the above embodiments.

In addition, embodiments of the present application further provide an image processing apparatus, which may specifically be a chip, a component or a module, and the apparatus may include a processor and a memory connected to each other; the memory is used for storing computer execution instructions, and when the device runs, the processor can execute the computer execution instructions stored in the memory, so that the chip can execute any one of the methods in the above method embodiments.

The electronic device, the computer storage medium, the computer program product, or the chip provided in this embodiment are all configured to execute the corresponding method provided above, so that the beneficial effects achieved by the electronic device, the computer storage medium, the computer program product, or the chip may refer to the beneficial effects in the corresponding method provided above, and are not described herein again.

Through the description of the above embodiments, those skilled in the art will understand that, for convenience and simplicity of description, only the division of the above functional modules is used as an example, and in practical applications, the above function distribution may be completed by different functional modules as needed, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, a module or a unit may be divided into only one logic function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another apparatus, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed to a plurality of different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or partially contributed to by the prior art, or all or part of the technical solutions may be embodied in the form of a software product, where the software product is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that the method comprises:

determining at least one candidate quadrangle according to the line segments;

acquiring a first touch parameter aiming at the target image;

2. The method of claim 1, wherein the performing confidence calculation on each of the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence comprises:

3. The method according to claim 2, wherein the determining the confidence level corresponding to each candidate quadrangle of the at least one candidate quadrangle according to the touch parameter of the at least one dimension and the weight corresponding to the touch parameter of each dimension to obtain the at least one confidence level comprises:

4. The method according to claim 3, wherein the determining a reference confidence between each of the touch parameters of the at least one dimension and the candidate quadrilateral i to obtain at least one reference confidence comprises:

alternatively, the first and second electrodes may be,

5. The method according to any one of claims 1 to 4, wherein the displaying the image of the area where the at least one candidate quadrangle is located according to the sorting result comprises:

and displaying the target area image.

6. The method according to claim 5, wherein the filtering the at least one reference candidate quadrangle to obtain a target quadrangle comprises:

acquiring a second touch parameter;

7. The method according to claim 5, wherein the filtering the at least one reference candidate quadrangle to obtain a target quadrangle comprises:

acquiring eyeball watching parameters of a user;

8. An image processing apparatus, characterized in that the apparatus comprises: a first acquisition unit, a detection unit, a determination unit, a second acquisition unit, a calculation unit, a sorting unit and a presentation unit, wherein,

the calculating unit is configured to perform confidence calculation on each candidate quadrangle of the at least one candidate quadrangle according to the first touch parameter to obtain at least one confidence, where each confidence is used to indicate an intention strength of the corresponding candidate quadrangle to be selected;

9. An electronic device, comprising a processor, a memory for storing one or more programs and configured to be executed by the processor, the programs comprising instructions for performing the steps of the method of any of claims 1-7.

10. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-7.