CN116686281A

CN116686281A - Image display method, terminal, chip and storage medium

Info

Publication number: CN116686281A
Application number: CN202180084568.4A
Authority: CN
Inventors: 顾磊
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-02-10
Filing date: 2021-02-10
Publication date: 2023-09-01
Also published as: WO2022170554A1

Abstract

The embodiment of the application discloses an image display method, a terminal and a storage medium, wherein the method comprises the following steps: acquiring an ith frame preview image corresponding to a target object, and performing frame detection processing on the ith frame preview image to acquire an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0; performing similarity clustering processing based on the first quadrilateral frame to the ith quadrilateral frame corresponding to the target object to obtain at least one frame group; selecting a target frame group from at least one frame group, and determining an initial stable frame from the target frame group; determining an ith stable frame based on the initial stable frame and the (i-1) th stable frame; and displaying the ith frame preview image according to the ith stable frame.

Description

Image display method, terminal, chip and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image display method, a terminal, and a storage medium.

Background

With the development of internet technology, more and more businesses, such as the communication industry, the financial industry, the entry and exit fields, etc., need to collect and register the certificate information of the user to perform real-name management. In order to improve the collection and registration efficiency of certificate information, related technologies propose a document picture scanning technology based on pictures, and automatic identification of information can be realized by scanning the pictures.

Before information identification is performed by scanning, the terminal needs to search for a quadrilateral frame containing a target object from the shot images by using the detection method, so as to preview the current shot picture and the found quadrilateral frame in real time, and further obtain the information of the target object in the quadrilateral frame.

However, due to the influence of various abnormal factors, the accuracy of the detected quadrilateral frame cannot be ensured, so that the problem of unstable display such as shaking and jumping of the quadrilateral frame may exist in the preview picture, and the defect of unsmooth display of the preview picture is caused.

Disclosure of Invention

The embodiment of the application provides an image display method, a terminal, a chip and a storage medium, solves the problem of unstable display of a quadrangular frame in a preview picture, and overcomes the defect of unsmooth display of the preview picture.

The technical scheme of the embodiment of the application is realized as follows:

in a first aspect, an embodiment of the present application provides an image display method, including:

acquiring an ith frame preview image corresponding to a target object, and performing frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0;

Performing similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame to obtain at least one frame group;

selecting a target frame group from the at least one frame group, and determining an initial stable frame from the target frame group;

determining an ith stable frame based on the initial stable frame and an (i-1) th stable frame;

and displaying the ith frame preview image according to the ith stable frame.

In a second aspect, an embodiment of the present application provides a terminal, including: an acquisition section, a detection section, a clustering section, a selection section, a determination section, and a display section,

the acquisition part is configured to acquire an ith frame preview image corresponding to the target object;

the detection part is configured to perform frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0;

the clustering part is configured to perform similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame to obtain at least one frame group;

The selecting part is configured to select a target frame group from the at least one frame group;

the determining part is configured to determine an initial stable frame from the target frame group; and determining an ith stable frame based on the initial stable frame and the (i-1) th stable frame;

the display part is configured to display the ith frame preview image according to the ith stable frame.

In a third aspect, an embodiment of the present application provides a terminal, including: a quadrilateral detection module, a time sequence stabilization module, a denoising stabilization module and a preview module,

the quadrilateral detection module is configured to acquire an ith frame preview image corresponding to the target object; performing frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0;

the time sequence stabilizing module is configured to perform similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame to obtain at least one frame group; selecting a target frame group from the at least one frame group; and determining an initial stable frame from the set of target frames;

The denoising stabilization module is configured to determine an ith stabilization frame based on the initial stabilization frame and an (i-1) th stabilization frame;

and the preview module is configured to display the ith frame of preview image according to the ith stable frame.

In a fourth aspect, an embodiment of the present application provides a terminal, where the terminal includes a quadrilateral detection module, a timing stabilization module, a denoising stabilization module, a preview module, a processor, and a memory storing instructions executable by the processor, and when the instructions are executed by the processor, the image display method described above is implemented.

In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and an interface, where the processor obtains program instructions through the interface, and the processor is configured to execute the program instructions to perform the image display method as described above.

In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having stored thereon a program for use in a terminal, the program, when executed by a processor, implementing an image display method as described above.

The embodiment of the application provides an image display method, a terminal, a chip and a storage medium, wherein the terminal acquires an ith frame preview image corresponding to a target object, and performs frame detection processing on the ith frame preview image to acquire an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0; performing similarity clustering processing based on the first quadrilateral frame to the ith quadrilateral frame corresponding to the target object to obtain at least one frame group; selecting a target frame group from at least one frame group, and determining an initial stable frame from the target frame group; determining an ith stable frame based on the initial stable frame and the (i-1) th stable frame; and displaying the ith frame preview image according to the ith stable frame. That is, in the embodiment of the present application, after performing a frame detection process on a current preview image including a target object to obtain a quadrilateral frame corresponding to the target object, the terminal may perform a clustering process based on a frame similarity on the quadrilateral frame, select a target frame group from at least one obtained frame group, further determine an initial stable frame from the target frame group, and further determine a current stable frame based on a comparison between the initial stable frame and a historical stable frame, so that the current preview image is to be displayed according to the current stable frame. Therefore, in the application, the terminal does not directly preview the image based on the quadrangular frames obtained by frame detection, but removes abnormal frame operations such as similarity clustering, target frame group selection, initial stable frame determination and the like on the quadrangular frames obtained by detection, and noise-removing stable operation for comparison with the historical stable frames, and after the current stable quadrangular frames are obtained, the image is previewed based on the stable quadrangular frames, thereby solving the problem of unstable display of the quadrangular frames in the preview picture and overcoming the defect of unsmooth display of the preview picture.

Drawings

Fig. 1 is a schematic diagram of an implementation flow of an image display method according to an embodiment of the present application;

fig. 2 is a schematic diagram of a second implementation flow of an image display method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a frame group smoothing filter according to an embodiment of the present application;

fig. 4 is a schematic diagram of an implementation flow chart of an image display method according to an embodiment of the present application;

FIG. 5 is a schematic view of a scene of initial stable frame smoothing filtering according to an embodiment of the present application;

fig. 6 is a schematic diagram of an implementation flow of an image display method according to an embodiment of the present application;

fig. 7 is a schematic diagram of an implementation flow of an image display method according to an embodiment of the present application;

fig. 8 is a schematic diagram of an implementation flow of an image display method according to an embodiment of the present application;

fig. 9 is a schematic diagram seventh of an implementation flow of an image display method according to an embodiment of the present application;

fig. 10 is a schematic diagram illustrating an implementation flow of an image display method according to an embodiment of the present application;

fig. 11 is a schematic diagram of an implementation flow chart of an image display method according to an embodiment of the present application;

fig. 12 is a schematic diagram showing an implementation flow of an image display method according to an embodiment of the present application;

FIG. 13A is a schematic diagram of a scene I of target stability frame determination according to an embodiment of the present application;

Fig. 13B is a schematic diagram of a second scenario of target stability frame determination according to an embodiment of the present application;

fig. 14 is a schematic diagram of an execution flow of image processing according to an embodiment of the present application;

fig. 15 is a schematic diagram of a composition structure of a terminal according to an embodiment of the present application;

fig. 16 is a schematic diagram of a second component structure of the terminal according to the embodiment of the present application;

fig. 17 is a schematic diagram of a composition structure of a terminal according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It is to be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to be limiting. It should be noted that, for convenience of description, only a portion related to the related application is shown in the drawings.

With the development of internet technology, more and more businesses, such as the communication industry, the financial industry, the entry and exit fields, etc., need to collect and register the certificate information of the user to perform real-name management. In order to improve the collection and registration efficiency of certificate information, a document picture scanning technology based on pictures is generated, and automatic information identification can be realized by scanning the pictures.

The scanning technology depends on a quadrilateral detection method of an image, and the terminal can firstly search a quadrilateral frame containing a target object from the shot image by applying the detection method, so that a current shot picture and the searched quadrilateral frame are previewed in real time, and finally information acquisition of the target object in the quadrilateral frame is realized.

However, the result obtained by quadrilateral detection is often affected by various factors, so that the complete correct running cannot be ensured, and the problem of unstable display such as shaking and jumping of the quadrilateral frame can occur when the preview picture is displayed in real time, thereby causing the defect of unsmooth display of the preview picture and being incapable of efficiently realizing picture scanning.

In the related art, the field adopts a direct time sequence filtering mode, such as Kalman filtering, average filtering and the like, to reduce the negative influence of the instability of the quadrilateral display. However, directly applying the temporal filtering, while making the filtered quadrilateral frame result appear smoother in time sequence, still cannot exclude the effects of partial outliers. The presence of the partial outlier directly causes deviation of the output result of the quadrangle under the influence of the partial outlier, and particularly when the occurrence frequency of the partial outlier is high, the deviation of the result is large. That is, the direct filtering cannot obtain accurate and stable quadrilateral output results, and cannot meet the existing scene requirements.

In order to solve the problems of the existing quadrilateral output result, the embodiment of the application provides an image display method, a terminal, a chip and a storage medium. Specifically, after performing frame detection processing on a current preview image including a target object to obtain a quadrilateral frame corresponding to the target object, the terminal may perform clustering processing based on frame similarity on the quadrilateral frame first, select a target frame group from at least one obtained frame group, further determine an initial stable frame from the target frame group, and further determine a current stable frame based on a comparison between the initial stable frame and a historical stable frame, so that display processing is performed on the current preview image according to the current stable frame. Therefore, in the application, the terminal does not directly preview the image based on the quadrangular frames obtained by frame detection, but removes abnormal frame operations such as similarity clustering, target frame group selection, initial stable frame determination and the like on the quadrangular frames obtained by detection, and noise-removing stable operation for comparison with the historical stable frames, and after the current stable quadrangular frames are obtained, the image is previewed based on the stable quadrangular frames, thereby solving the problem of unstable display of the quadrangular frames in the preview picture and overcoming the defect of unsmooth display of the preview picture.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

Fig. 1 is a schematic implementation flow diagram of an image display method according to an embodiment of the present application, as shown in fig. 1, in an embodiment of the present application, a method for performing image processing by a terminal may include the following steps:

step 101, acquiring an ith frame preview image corresponding to a target object, and performing frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0.

In the embodiment of the application, the terminal can acquire the preview image containing the target object, i.e. the ith frame preview image in real time, and perform frame detection processing on the ith frame preview image, so as to obtain the real-time quadrilateral frame corresponding to the target object, i.e. the ith quadrilateral frame.

It should be noted that, in the embodiment of the present application, the terminal may be any electronic device having a text scanning function. Specifically, the terminal may have a camera, and image frames are acquired through the camera.

Alternatively, the terminal is not limited to electronic devices such as a smart phone, a tablet computer, a personal computer (Personal Computer, PC), a notebook computer, and the like.

It should be understood that, when the terminal shoots the document picture through the camera, the i-th frame preview image refers to a frame of preview image of the document picture acquired by the terminal at the i-th moment. Correspondingly, the target object refers to a target object specified in the preview image, such as a rectangular object with a rectangular frame.

For example, a document picture may include various rectangular objects such as a document, paper, business card, photograph, whiteboard, screen, etc., and a target object may be a person photograph, an identification card, a passport, a driver's license, a ticket, a business card, a work card, etc. in the document picture.

It should be noted that, in the embodiment of the present application, after the terminal acquires the preview image including the target object in real time, the terminal may acquire the quadrilateral frame corresponding to the target object by performing a frame detection process, such as quadrilateral detection, on the preview image.

Alternatively, since the quadrilateral detection is directed to a rectangular object, the outline of the rectangular object is composed of straight line segments, and therefore, the terminal can determine the outline of the rectangular object, namely, the quadrilateral frame by using a characteristic line detection method.

Optionally, the terminal may also establish a quadrilateral detection model based on a deep learning mode, and after acquiring the preview image in real time, input the preview image into a pre-trained model to perform quadrilateral detection processing on the image frame to be detected, and further output a quadrilateral frame.

Further, in the embodiment of the present application, after acquiring the ith frame preview image including the target object and performing frame detection processing on the ith frame preview image, the terminal may further perform clustering processing based on frame similarity based on the quadrilateral frame after acquiring the ith quadrilateral frame corresponding to the target object.

Step 102, performing similarity clustering processing based on the first to the ith quadrangular frames corresponding to the target object to obtain at least one frame group.

In the embodiment of the application, after performing frame detection processing on the terminal to obtain the ith quadrangular frame, the terminal may further perform similarity clustering processing based on the first quadrangular frame to the ith quadrangular frame, so as to obtain at least one frame group.

It will be appreciated that clustering, i.e., unsupervised machine learning, classifies similar objects. In the embodiment of the application, the terminal can cluster the quadrangular frames based on the frame similarity, and the quadrangular frames with high similarity are classified into one type.

Specifically, in the embodiment of the application, the terminal can acquire vertex coordinate data corresponding to the quadrilateral frame, and perform similarity calculation based on the vertex coordinate data, so that classification of the quadrilateral frame is realized based on the similarity result.

Here, the terminal may acquire vertex coordinate data corresponding to the first to the i-th quadrilateral frames, and further perform similarity clustering processing based on the vertex coordinate data, so as to construct at least one frame group.

Further, in the embodiment of the present application, after performing clustering processing based on the frame similarity based on the first to the i-th quadrilateral frames to obtain at least one frame group, the terminal may further perform selection processing of the target frame group based on the at least one frame group.

And 103, selecting a target frame group from at least one frame group, and determining an initial stable frame from the target frame group.

In the embodiment of the present application, after performing similarity clustering processing on the terminal to obtain at least one frame group, the terminal may first select one frame group from the at least one frame group as a target frame group (step 103 a), and then determine one frame from the target frame group as an initial stable frame (step 103 b).

It can be understood that, in at least one frame group obtained based on similarity clustering, the number of quadrilateral frame samples in each frame group may be different, the larger the number of quadrilateral frame samples is, the larger the corresponding quadrilateral frame of the frame group with larger number of quadrilateral frame samples occupies all quadrilateral frame samples, the smaller the corresponding quadrilateral frame of the frame group with smaller number of quadrilateral frame samples occupies all quadrilateral frame samples, the smaller the occupied ratio is, which indicates that the probability that the quadrilateral frame is dissimilar from other quadrilaterals is larger, and the probability that the quadrilateral frame in the corresponding frame group belongs to an abnormal frame is larger.

Therefore, in the embodiment of the application, in order to ensure the stability of the quadrangular frames, abnormal quadrangular frames are filtered out and the more stable quadrangular frames are reserved, and the terminal can select one quadrangular frame group with a more stable sample from at least one frame group as a target frame group.

Specifically, fig. 2 is a schematic diagram of a second implementation flow chart of an image display method according to an embodiment of the present application, and as shown in fig. 2, a method for selecting a target frame group from at least one frame group by a terminal (step 103 a) includes the following steps:

step 103a1, obtaining the number of quadrilateral frames contained in each frame group in at least one frame group.

Step 103a2, determining the frame group corresponding to the maximum quadrilateral frame number as the target frame group.

Optionally, in an embodiment of the present application, the terminal may directly determine, as the target frame group, the frame group having the largest number of quadrangular frame samples in the at least one frame group.

Optionally, because the number of the quadrilateral frame samples in the frame group is easy to generate jump fluctuation at a certain moment, and then returns to normal, misoperation may occur to the selection of the target frame group. Therefore, for this case, the terminal may perform a certain smoothing filtering process, such as average filtering, on each frame group, so as to reduce the jump when selecting the target frame group.

The terminal can track each frame group in time sequence, smooth filter the quadrilateral frame sample number in the frame group, and then select one frame group with the largest quadrilateral frame sample number from at least one filtered frame group as a target frame group.

Fig. 3 is a schematic diagram of a frame group smoothing filter according to an embodiment of the present application, where, as shown in fig. 3, an abscissa of the schematic diagram indicates different time sequences, and an ordinate indicates a change of the number of samples in the frame group; wherein, the thick solid line represents the number curve of the quadrilateral frame samples in the original frame group 1, and the thin solid line represents the number curve of the quadrilateral frame samples in the original frame group 2; the thick dotted line represents the number curve of the quadrangular frame samples in the filtered frame set 1, and the thin dotted line represents the number curve of the quadrangular frame samples in the filtered frame set 2. It can be seen that, before filtering, the number of quadrilaterals in the original frame set 2 is greater than the number of quadrilaterals in the original frame set 1 for a period of time, the number of quadrilaterals in the original frame set 1 is less than the number of quadrilaterals in the original frame set 1 for a period of time after filtering, the terminal cannot accurately select the target frame set, at this time, the number of quadrilaterals in the frame set 2 after filtering is always greater than the number of quadrilaterals in the frame set 1 after filtering in the t1 period of time, at this time, the target frame set is determined to be the frame set 2. Similarly, in the period from t1 to t2, after smoothing filtering, the number of quadrilateral samples in the filtered frame group 1 is always greater than that of the filtered frame group 2, and at this time, the terminal can select the frame group 1 as a target frame group; similarly, in the period from t2 to t3, after smoothing filtering, the number of quadrilateral samples in the filtered frame group 2 is always greater than that of the filtered frame group 1, and at this time, the terminal can select the frame group 2 as the target frame group.

Further, in the embodiment of the present application, after the terminal selects the target frame group from the at least one frame group, a frame may be further determined from the target frame group as the initial stable frame.

Specifically, fig. 4 is a schematic diagram of a third implementation flow chart of an image display method according to an embodiment of the present application, as shown in fig. 4, in an embodiment of the present application, a method for determining an initial stable frame (step 103 b) from a target frame group by a terminal includes the following steps:

step 103b1, arranging the quadrilateral frames in the target frame group according to the time sequence, and obtaining a frame list.

Step 103b2, determining the last quadrangular frame in the frame list as an initial stable frame.

It can be understood that, because each quadrilateral frame in the target frame group corresponds to a preview image, and based on the time sequence of the preview image, the quadrilateral frames also correspond to the time sequence thereof, in the embodiment of the present application, the terminal may perform the arrangement processing on all quadrilateral frames in the target frame group according to the time sequence from first to last, so as to obtain the frame time sequence list. Further, the terminal may determine the last quadrangular frame in the list as an initial stable frame, that is, the quadrangular frame corresponding to the latest preview image in the target frame group is determined as an initial stable frame.

Optionally, in the embodiment of the present application, the terminal may also perform mean filtering processing, such as kalman filtering, on all quadrilateral frames in the target frame group, where the object of filtering is quadrilateral frame vertex coordinate data or center point coordinate data, so as to obtain an initial stable frame.

For example, fig. 5 is a schematic view of a scene of initial stable frame smoothing filtering according to an embodiment of the present application, and it is assumed that a target frame group includes a quadrilateral frame a, a quadrilateral frame B and a quadrilateral frame C, as shown in fig. 5, although three frames A, B, C belong to one frame group and are similar frames, there are gaps between the three frames actually, and vertex coordinate data and center point coordinate data are different, so that a terminal may perform mean filtering on the three frames according to a time sequence, thereby obtaining a more stable quadrilateral frame D, and determining the frame D as an initial stable frame.

Further, after successfully selecting the target frame group and successfully determining the initial stable frame, the terminal may further perform a determination process of the target stable frame.

Step 104, determining the ith stable frame based on the initial stable frame and the (i-1) th stable frame.

In the embodiment of the application, after the terminal selects the target frame group from at least one frame group and determines the initial stable frame from the target frame group, the terminal can further determine the quadrilateral frame which is finally subjected to preview output, namely the ith stable frame, based on the initial stable frame and the historical stable reference frame, namely the (i-1) th stable frame.

It should be noted that, in the embodiment of the present application, the (i-1) th stable frame refers to the stable frame of the last preview image that finally outputs the preview.

Specifically, after the terminal performs the processing flows of similarity clustering processing, target frame group selection, initial stable frame determination, stable frame determination and the like corresponding to each frame of preview image, the terminal stores stable frame information corresponding to the current frame of preview image and uses the stable frame information as a historical reference stable frame when the next frame of preview image performs stable frame determination.

It should be understood that, in the embodiment of the present application, in order to reduce the jitter of the quadrilateral frame during the display of the preview interface, the terminal does not directly determine the currently obtained initial stable frame as the ith stable frame corresponding to the preview image of the current ith frame, but compares the similarity between the currently obtained initial stable frame and the prestored historical (i-1) th stable frame, so as to determine the ith stable frame that is finally output for preview based on the comparison result.

Further, in the embodiment of the present application, after the terminal successfully determines the ith stable frame, the terminal may further perform display processing on the ith preview image according to the stable frame.

And 105, displaying the ith preview image according to the ith stable frame.

In the embodiment of the application, after the terminal successfully determines the ith stable frame, the terminal can further perform display processing on the ith preview image according to the stable frame.

Specifically, in the embodiment of the present application, the terminal may perform rendering processing on the ith preview image based on the ith stable frame, so as to obtain a rendered preview image, and further display the rendered preview image.

In detail, the terminal renders the ith stable frame in the ith preview image to obtain a rendered stable frame, and then generates a rendered preview image based on the rendered stable frame and the ith preview image, so that the rendered preview image is displayed in a preview picture.

Further, in the embodiment of the present application, the terminal may perform real-time scanning processing on the rendered preview image, so as to obtain specific parameters of the target object. Specifically, the terminal may perform real-time scanning processing on only the target object in the target stable frame, so as to perform automatic information identification.

The embodiment of the application provides an image display method, which comprises the steps that after a current preview image containing a target object is subjected to frame detection processing to obtain a quadrangular frame corresponding to the target object, a terminal can firstly perform clustering processing based on frame similarity on the quadrangular frame, select a target frame group from at least one obtained frame group, further determine an initial stable frame from the target frame group, further determine the current stable frame based on the comparison of the initial stable frame and a historical stable frame, and further display the current preview image according to the current stable frame. Therefore, in the application, the terminal does not directly preview the image based on the quadrangular frames obtained by frame detection, but removes abnormal frame operations such as similarity clustering, target frame group selection, initial stable frame determination and the like on the quadrangular frames obtained by detection, and noise-removing stable operation for comparison with the historical stable frames, and after the current stable quadrangular frames are obtained, the image is previewed based on the stable quadrangular frames, thereby solving the problem of unstable display of the quadrangular frames in the preview picture and overcoming the defect of unsmooth display of the preview picture.

Based on the above embodiment, in another embodiment of the present application, fig. 6 is a schematic diagram showing a realization flow of the image display method according to the embodiment of the present application, as shown in fig. 6, after the terminal performs a frame detection process on the i-th frame preview image, to obtain the i-th quadrilateral frame corresponding to the target object, that is, after step 101, and based on the first quadrilateral frame corresponding to the target object to perform a similarity clustering process, before at least one frame group is obtained, that is, before step 102, the method for performing image processing by the terminal includes:

step 106, storing the ith quadrangular frame to the nth bit of the first-in first-out queue (First Input First Output, FIFO); where N is an integer greater than 2, N characterizes the maximum storage capacity of the FIFO.

It should be noted that, in the embodiment of the present application, the terminal performs the quadrilateral detection processing on each frame of preview image, and after obtaining each quadrilateral frame, stores the quadrilateral frame corresponding to the current preview image to the tail of the FIFO queue, that is, the last bit of the queue.

Specifically, the number of FIFO queues is determined by the maximum storage capacity, i.e. how many image frames can be stored in the FIFO queues. If the current FIFO queue has N bits, the maximum storage number of the FIFO queue is represented as N.

It should be understood that the FIFO follows the principle of "first in first out", and in the embodiment of the present application, the terminal always stores the ith quadrangular frame obtained by detecting the preview image of the current ith frame to the tail of the FIFO queue, i.e. the nth bit. At the moment, the historical quadrilateral frame obtained by detecting the historical preview image sequentially translates forward for one bit in the FIFO queue; wherein the history square frame originally located at the first bit in the FIFO queue is shifted out of the queue, and the (i-1) th history square frame is shifted to the (N-1) th bit.

Based on the above embodiment, in another embodiment of the present application, fig. 7 is a schematic diagram of a fifth implementation flow of the image display method according to the embodiment of the present application, as shown in fig. 7, after the terminal stores the ith quadrangular frame in the nth bit of the FIFO, i.e. after step 106, if i is smaller than N, the terminal performs similarity clustering processing based on the first quadrangular frame to the ith quadrangular frame corresponding to the target object, and the method for obtaining at least one frame group includes the following steps:

step 102a, reading the first to the ith quadrangular frames from the FIFO.

Step 102b, performing similarity clustering processing based on the first to the ith quadrangular frames to obtain at least one frame group.

It should be noted that, in the embodiment of the present application, the number of quadrilateral frames stored in the FIFO is associated with the maximum storage number N of the FIFO queue.

Specifically, if i is smaller than N, after the terminal stores the ith quadrangular frame in the nth bit of the FIFO queue, the FIFO queue includes the first to the ith quadrangular frames, that is, the FIFO queue space is large enough, and there is no removed quadrangular frame.

Further, the terminal may read the first to i-th quadrilateral frames corresponding to the first to i-th frame preview images from the FIFO queue, and perform clustering based on the frame similarity based on the i-th quadrilateral frames.

Specifically, fig. 8 is a schematic diagram of an implementation flow chart of an image display method according to an embodiment of the present application, as shown in fig. 8, a method for obtaining at least one frame group by performing similarity clustering processing on a terminal based on a first quadrilateral frame to an i-th quadrilateral frame includes:

102b1, acquiring kth vertex coordinate data corresponding to the kth quadrilateral frame and front (k-1) vertex coordinate data corresponding to the front (k-1) quadrilateral frame; wherein k is an integer greater than 1 and less than or equal to i.

Step 102b2, calculating the previous (k-1) distance difference value between the kth vertex coordinate data and the previous (k-1) vertex coordinate data according to the preset similarity function.

Step 102b3, determining a minimum distance difference from the previous (k-1) distance differences.

Step 102b4, constructing at least one frame group based on the minimum distance difference and the first history frame group corresponding to the previous (k-1) quadrilateral frames.

It should be noted that, in the embodiment of the present application, the terminal performs the clustering processing of the first quadrangular frame in the first to the i th quadrangular frames in the FIFO queue, and since there is no quadrangular frame sample that has been clustered before the first quadrangular frame, that is, no frame group, at this time, the terminal may first establish a new frame group for the first quadrangular frame.

Further, when the clustering process of the second quadrangular frame in the FIFO queue is performed, that is, when k is equal to 2, the terminal may first perform similarity comparison on the second quadrangular frame and the clustered first quadrangular frame, and determine, based on the comparison result, the frame group to which the second quadrangular frame belongs.

Specifically, the terminal may obtain vertex coordinate data of the second quadrangular frame and vertex coordinate data of the first quadrangular frame, and then calculate a distance difference value capable of representing similarity based on a preset similarity function and the two vertex coordinate data.

Here, the terminal may calculate the distance difference value based on the formulas (1) to (3) to determine the similarity of the quadrangular frames.

Specifically, it is assumed that the quadrangle information Q is coordinate positions of four vertexes of each quadrangle frame.

Q＝{p _i |p _i ＝(x _i ,y _i ),i＝{0,1,2,3}} (1)

Wherein when i=1, p ₁ ＝(x ₁ ,y ₁ ) Namely, the coordinate data of the first vertex of the quadrangle; similarly, when i=2, p ₂ ＝(x ₂ ,y ₂ ) I.e., second vertex coordinate data; when i=3, p ₃ ＝(x ₃ ,y ₃ ) Namely third vertex coordinate data; when i=4, p ₄ ＝(x ₄ ,y ₄ ) I.e. fourth vertex coordinate data.

At this time, a similarity function is preset to obtain the distance difference between the two quadrilaterals.

Distance(A,B)＝|M(A)-M(B)| _p (2)

Wherein I _p Is L ^p The spatial norms, more commonly Manhattan distance when p takes 1, euclidean distance when p takes 2, and maximum absolute value when p takes +.. M (Q) is a mapping function of quadrilateral information for mapping the original quadrilateral informationThe information Q maps to a space of distance calculations.

Here, the determination method of M (Q) in the formula (2) is as follows.

M(Q)＝(k ₀ (Q),k ₁ (Q),k ₂ (Q),…) (3)

Wherein k is _i (Q) is a specific mapping function. For example, the number of the cells to be processed,i.e. calculating the location, area, etc. of the center point of the quadrilateral as a mapping term.

In detail, the terminal may map the two quadrilateral frames to the distance space based on the formula (1) and the formula (3) in the preset similarity function and the vertex coordinate data of the first quadrilateral frame and the vertex coordinate data of the second quadrilateral frame, so as to obtain the distances respectively corresponding to the two quadrilateral frames, and further calculate the distance difference based on the formula (3), so as to determine the similarity comparison result of the first quadrilateral frame and the second quadrilateral frame.

Further, the terminal may preset a preset distance threshold capable of representing the similarity result, and the terminal may compare the distance difference with the preset distance threshold, so as to determine the similarity result of the first quadrilateral frame and the second quadrilateral frame based on the comparison result.

If the distance difference is smaller than the preset distance threshold, the terminal may determine that the first square frame is similar to the second square frame, and determine that the second square frame is classified as the frame group to which the first square frame belongs. If the distance difference is greater than or equal to the preset distance threshold, the terminal may determine that the first square frame is dissimilar to the second square frame, and reestablish a new frame set and classify the second square frame in the new frame set.

Further, repeating the steps, continuing to execute judging processing of whether the kth quadrangular frame is similar to the previous (k-1) quadrangular frame, and executing grouping processing based on a similarity judging result on the kth quadrangular frame until finishing the grouping processing of the ith quadrangular frame, thereby obtaining at least one frame group; where k is an integer less than i.

It should be noted that, when the judgment processing of whether the kth quadrangular frame is similar to the previous (k-1) quadrangular frame is continuously performed, the terminal may calculate a (k-1) distance difference value between the kth quadrangular frame and the previous (k-1) quadrangular frame, determine a minimum difference value from the (k-1) distance difference values, and further construct at least one frame group based on the frame group corresponding to the minimum difference value and the previous (k-1) quadrangular frame.

Specifically, if the minimum distance difference is greater than or equal to the preset distance threshold, that is, there is no frame group corresponding to the kth quadrilateral frame, and the kth quadrilateral frame cannot be classified into the frame groups corresponding to the previous (k-1) quadrilateral frames, the terminal may establish a newly added frame group corresponding to the kth quadrilateral frame, and construct at least one frame group based on the newly added frame group and the first historical frame group.

Specifically, if the minimum distance difference is smaller than the preset distance threshold, that is, there is a frame group corresponding to the kth quadrilateral frame, the kth quadrilateral frame may be classified in the frame group corresponding to the previous (k-1) quadrilateral frame, and then the terminal may classify the kth quadrilateral frame in the target frame group corresponding to the minimum distance difference in the frame group corresponding to the previous (k-1) quadrilateral frame, and construct at least one frame group based on the frame group updated by the quadrilateral frame sample number.

For example, when the terminal performs clustering processing on the third quadrangular frame in the FIFO queue, the terminal calculates the distance difference between the third quadrangular frame and the first quadrangular frame and the second quadrangular frame respectively by using formulas (1) to (3) based on the vertex coordinate data. If the distance difference between the third quadrilateral and the first quadrilateral frame is smaller than the preset distance threshold value and the distance difference between the third quadrilateral and the second quadrilateral frame is larger than the preset distance threshold value, the terminal can determine that the third quadrilateral frame is classified into the frame group to which the first quadrilateral frame belongs; if the distance difference value between the third quadrilateral and the first quadrilateral frame is smaller than the preset distance threshold value, and meanwhile, the distance difference value between the third quadrilateral and the second quadrilateral frame is also smaller than the preset distance threshold value, the terminal classifies the third quadrilateral frame into a frame group to which the first quadrilateral frame with smaller distance difference value belongs; if the distance differences are all larger than the preset distance threshold, the terminal reestablishes the frame group and classifies the third quadrilateral frame into the new frame group.

Repeating the steps until the clustering processing of the first to the ith quadrangular frames in the FIFO queue is completed, and further obtaining at least one quadrangular frame group.

Based on the foregoing embodiment, in another embodiment of the present application, fig. 9 is a schematic diagram of a seventh implementation flow of the image display method according to the embodiment of the present application, as shown in fig. 9, after the terminal stores the ith quadrangular frame to the nth bit of the FIFO, that is, after step 106, if i is greater than or equal to N, the terminal performs similarity clustering processing based on the first quadrangular frame to the ith quadrangular frame corresponding to the target object, and the method for obtaining at least one frame group may further include the following steps:

step 102c, reading the (i-n+1) th quadrilateral frame to the i-th quadrilateral frame from the FIFO.

102d, performing similarity clustering processing based on the (i-N+1) th quadrilateral frame to the i-th quadrilateral frame to obtain at least one frame group.

Specifically, if i is equal to or greater than N, after the terminal stores the ith quadrangular frame in the nth bit of the FIFO queue, the FIFO queue contains the (i-n+1) th to the ith quadrangular frames, that is, the FIFO queue space is insufficient, and the previous (i-n+2) th quadrangular frame has been removed from the FIFO queue.

Further, the terminal can read the (i-n+1) th to the (i-n+1) th deformed frames corresponding to the (i-n+1) th frame preview image from the FIFO queue, and perform clustering processing based on the frame similarity.

Specifically, fig. 10 is a schematic diagram eighth implementation flow chart of an image display method according to an embodiment of the present application, as shown in fig. 10, a method for performing similarity clustering processing by a terminal based on (i-n+1) th to i-th quadrilateral frames to obtain at least one frame group includes:

102d1, acquiring (i-N+k) th vertex coordinate data corresponding to the (i-N+k) th quadrilateral frame and front (i-N+k-1) vertex coordinate data corresponding to the front (i-N+k-1) quadrilateral frame; wherein k is an integer greater than 1 and less than or equal to N.

Step 102d2, calculating the difference value of the previous (i-n+k-1) distances between the (i-n+k) th vertex coordinate data and the previous (i-n+k-1) vertex coordinate data according to a preset similarity function.

Step 102d3, determining a minimum distance difference from the previous (i-n+k-1) distance differences.

Step 102d4, constructing at least one frame group based on the minimum distance difference and the second history frame group corresponding to the previous (i-n+k-1) quadrilateral frames.

It should be noted that, in the embodiment of the present application, the terminal always performs clustering processing only for all the quadrilateral frames currently existing in the FIFO sequence, and does not save the clustering result of the quadrilateral frames of the history FIFO sequence.

It should be noted that, in the embodiment of the present application, the terminal performs the clustering processing of the (i-n+1) -th quadrilateral frame in the FIFO queue from the (i-n+1) -th quadrilateral frame first, and since there is no quadrilateral frame sample that has completed clustering before the (i-n+1) -th quadrilateral frame, that is, no frame group, at this time, the terminal may first establish a new frame group for the (i-n+1) -th quadrilateral frame.

Further, when the clustering process of the (i-n+2) th quadrangular frame in the FIFO queue is performed, that is, when k is equal to 2, the terminal may first perform similarity comparison on the (i-n+2) th quadrangular frame and the clustered (i-n+1) th quadrangular frame, and determine, based on the comparison result, the frame group to which the (i-n+2) th quadrangular frame belongs.

Specifically, the terminal may obtain vertex coordinate data of the (i-n+2) th quadrilateral frame and vertex coordinate data of the (i-n+1) th quadrilateral frame respectively, then calculate a distance difference value based on formulas (1) to (3), compare the distance difference value with a preset distance threshold representing a similarity result, and if the distance difference value is smaller than or equal to the preset distance threshold, determine that the (i-n+1) th quadrilateral frame is similar to the (i-n+2) th quadrilateral frame, and determine that the (i-n+2) th quadrilateral frame is classified as a frame group to which the (i-n+1) th quadrilateral frame belongs. If the distance difference is greater than the preset distance threshold, the terminal may determine that the (i-n+1) th quadrilateral frame is dissimilar to the (i-n+2) th quadrilateral frame, and reestablish a new frame set and classify the (i-n+2) th quadrilateral frame into the new frame set.

Further, repeating the steps, continuing to execute the judgment processing of whether the (i-N+k) th quadrangular frame is similar to the previous (i-N+k-1) th quadrangular frame, and executing grouping processing based on the similarity judgment result on the (i-N+k) th quadrangular frame until the grouping processing of the (i) th quadrangular frame is completed, so as to obtain at least one frame group; where k is an integer less than i.

It should be noted that, when the determination process of whether the (i-n+k) th quadrangular frame is similar to the previous (i-n+k-1) quadrangular frame is continuously performed, the terminal may calculate the (i-n+k-1) distance difference between the (i-n+k) th quadrangular frame and the previous (i-n+k-1) quadrangular frame, determine the minimum difference from the (i-n+k-1) distance differences, and construct at least one frame group based on the frame group corresponding to the minimum difference and the previous (i-n+k-1) quadrangular frames.

Specifically, if the minimum distance difference is greater than or equal to the preset distance threshold, that is, there is no frame group corresponding to the (i-n+k) th quadrilateral frame, and the (i-n+k) th quadrilateral frame cannot be classified into the frame group corresponding to the previous (i-n+k-1) quadrilateral frame, the terminal may establish a new frame group corresponding to the (i-n+k) th quadrilateral frame, and establish at least one frame group based on the new frame group and the first historical frame group.

Specifically, if the minimum distance difference is smaller than the preset distance threshold, that is, the frame group corresponding to the (i-n+k) th quadrilateral frame exists, the (i-n+k) th quadrilateral frame may be classified into the frame group corresponding to the previous (i-n+k-1) quadrilateral frame, and the terminal may classify the (i-n+k) th quadrilateral frame into the target frame group corresponding to the minimum distance difference in the frame group corresponding to the previous (i-n+k-1) quadrilateral frame, and construct at least one frame group based on the updated frame group of the quadrilateral frame sample number.

For example, when the terminal performs clustering processing on the (i-n+3) th quadrilateral frame in the FIFO queue, the terminal calculates the distance difference between the (i-n+3) th quadrilateral frame and the (i-n+1) th quadrilateral frame and the (i-n+2) th quadrilateral frame by using formulas (1) to (3) based on vertex coordinate data. If the difference between the distances between the (i-N+3) th quadrilateral and the (i-N+1) th quadrilateral frame is smaller than a preset distance threshold value and the difference between the distances between the (i-N+2) th quadrilateral frame and the (i-N+2) th quadrilateral frame is larger than the preset distance threshold value, the terminal can determine that the (i-N+3) th quadrilateral frame classifies the frame group to which the (i-N+1) th quadrilateral frame belongs; if the distance difference between the (i-N+3) th quadrilateral and the (i-N+1) th quadrilateral frame is smaller than a preset distance threshold value, and the distance difference between the (i-N+2) th quadrilateral frame and the preset distance threshold value, the terminal classifies the (i-N+1) th quadrilateral frame with smaller distance difference into a frame group to which the (i-N+3) th quadrilateral frame belongs; if the distance differences are all greater than the preset distance threshold, the terminal reestablishes the frame group and classifies the (i-N+3) th quadrilateral frame into a new frame group.

Repeating the steps until the clustering treatment of the (i-N+1) th to the i th quadrangular frames in the FIFO queue is completed, and further obtaining at least one quadrangular frame group.

Based on the foregoing embodiments, in another embodiment of the present application, fig. 11 is a schematic diagram of an implementation flow chart of an image display method according to an embodiment of the present application, as shown in fig. 11, after step 101, a method for obtaining at least one frame group by performing similarity clustering processing by a terminal based on a first quadrilateral frame to an i-th quadrilateral frame corresponding to a target object may further include the following steps:

102e, acquiring the ith vertex coordinate data corresponding to the ith quadrilateral frame and the previous (i-1) vertex coordinate data corresponding to the previous (i-1) quadrilateral frames grouped by history.

Step 102f, respectively calculating (i-1) distance differences corresponding to the ith vertex coordinate data and the (i-1) vertex coordinate data before history according to a preset similarity function.

Step 102g, determining a minimum distance difference value from the (i-1) distance difference values.

Step 102h, constructing at least one frame group based on the minimum distance difference and the third historical frame group corresponding to the previous (i-1) quadrilateral frames.

In the embodiment of the application, the terminal does not need to store the detected quadrilateral frame into the FIFO queue, but directly compares the similarity between the ith quadrilateral frame corresponding to the detected latest frame, namely the current ith preview image, and the classified quadrilateral frame sample to realize clustering of the quadrilateral frames.

Specifically, the terminal may calculate the distance difference value of each historical quadrangular frame in the ith quadrangular frame and the previous (i-1) quadrangular frames by using formulas (1) to (3), that is, (i-1) distance difference values, and compare the distance difference values with a preset distance threshold value, thereby determining a frame similarity result according to the comparison result, and implementing clustering of the quadrangular frames.

Specifically, the terminal may determine a minimum distance difference value from the (i-1) distance difference values, and construct at least one frame group based on a history frame group corresponding to the previous (i-1) quadrangular frames based on the minimum distance difference value.

Here, if the minimum distance difference is greater than or equal to the preset distance threshold, that is, there is no frame group corresponding to the ith quadrangular frame, and the ith quadrangular frame cannot be classified into the frame groups corresponding to the previous (i-1) quadrangular frames, the terminal may establish a newly added frame group corresponding to the ith quadrangular frame, and construct at least one frame group based on the newly added frame group and the first history frame group.

Here, if the minimum distance difference is smaller than the preset distance threshold, that is, there is a frame group corresponding to the ith quadrangular frame, the ith quadrangular frame may be classified into a frame group corresponding to the previous (i-1) quadrangular frame, and then the terminal may classify the ith quadrangular frame into a target frame group corresponding to the minimum distance difference among frame groups corresponding to the previous (i-1) quadrangular frames, and construct at least one frame group based on the frame groups updated by the number of quadrangular frame samples.

The embodiment of the application provides an image display method, wherein a terminal can not perform similarity clustering on a quadrangular frame obtained by detection, selecting a target frame group, determining an initial stable frame and the like to remove abnormal frame operation, so that the problem of unstable display of the quadrangular frame in a preview picture is solved, and the defect of unsmooth display of the preview picture is overcome.

Based on the foregoing embodiments, in another embodiment of the present application, fig. 12 is a schematic diagram showing an implementation flow chart of an image display method according to an embodiment of the present application, and as shown in fig. 12, a method for determining an ith stable frame by a terminal based on an initial stable frame and an (i-1) th stable frame may include the following steps:

step 104a, acquiring first vertex coordinate data corresponding to the initial stable frame and second vertex coordinate data corresponding to the (i-1) th stable frame.

Step 104b, calculating the distance difference between the first vertex coordinate data and the second vertex coordinate data according to a preset similarity function.

And 104c, if the distance difference value is smaller than the preset distance threshold value, determining the (i-1) th stable frame as the i-th stable frame.

And 104d, if the distance difference value is greater than or equal to a preset distance threshold value, determining the initial stable frame as the ith stable frame.

Specifically, in the embodiment of the present application, in the process of determining the ith stable frame based on the initial stable frame and the (i-1) th stable frame, the terminal may first obtain first vertex coordinate data corresponding to the initial stable frame and second vertex coordinate data corresponding to the (i-1) th stable frame, and then calculate the similarity between the initial stable quadrilateral frame and the (i-1) th stable frame based on the two coordinate data and a preset similarity function, i.e., formulas (1) to (3).

In detail, the terminal may map the first vertex coordinate data of the initial stable quadrilateral frame and the second vertex coordinate data of the (i-1) th stable frame to the distance space based on the formula (1) and the formula (3) in the preset similarity function, so as to obtain a first distance corresponding to the initial stable quadrilateral frame and a second distance corresponding to the (i-1) th stable frame, and further calculate a distance difference value based on the formula (3).

Further, the terminal may preset a preset distance threshold representing the similarity result, and the terminal may compare the distance difference value with the preset distance threshold, so as to determine the similarity result of the initial stable quadrilateral frame and the (i-1) th stable frame based on the comparison result.

On the one hand, if the distance difference is smaller than or equal to the preset distance threshold, the terminal may determine that the initial stable quadrilateral frame is similar to the (i-1) th stable frame, and in order to ensure smoothness of the preview image, the terminal adopts the stable quadrilateral frame identical to the previous frame image, that is, continuously determines the (i-1) th stable frame as the i-th stable frame corresponding to the current i-th preview image.

It should be noted that, since the ith stable frame is not changed, the terminal does not update the (i-1) th stable frame stored in advance for stable frame comparison, and continues to serve as the reference stable quadrilateral frame when the (i+1) th stable quadrilateral frame determination is performed.

For example, fig. 13A is a schematic diagram of a scenario for determining a stable frame according to an embodiment of the present application, assuming that a dotted line is the (i-1) th stable frame and a solid line is the initial stable frame, as shown in fig. 13A, the initial stable frame has a higher similarity with the (i-1) th stable frame, and then the terminal may reserve the (i-1) th stable frame as the i-th stable frame of the current image frame.

On the other hand, if the distance difference is greater than the preset distance threshold, the terminal may determine that the initial stable frame is not similar to the (i-1) th stable frame, that is, the quadrilateral frame corresponding to the target object in the preview image changes, and in order to ensure accuracy of the preview image, the terminal determines the initial stable quadrilateral frame that is currently determined as the stable quadrilateral frame corresponding to the preview image of the current i frame.

It should be noted that, because the target stable quadrilateral frame changes, the terminal needs to update the (i-1) th stable frame stored in advance for comparing the stable frames at the same time, and continue to use the initial stable frame corresponding to the current i-th preview image as the reference stable quadrilateral frame when the (i+1) th stable quadrilateral frame is determined.

For example, fig. 13B is a schematic diagram of a second scenario of determining a stable frame according to the embodiment of the present application, assuming that a dotted line is the (i-1) th stable frame and a solid line is the initial stable frame, as shown in fig. 13B, the similarity between the initial stable frame and the (i-1) th stable frame is poor, and then the terminal may update the stored (i-1) th stable frame and use the initial stable frame as the i-th stable frame of the i-th frame image.

The embodiment of the application provides an image display method, wherein a terminal can execute different determination of a current stable quadrilateral frame according to different similarity results by comparing the similarity of the quadrilateral frame of a current latest frame and a reference stable quadrilateral frame stored in history, so that the problem of unstable display of the quadrilateral frame in a preview picture is solved, the defect of unsmooth display of the preview picture is overcome, and further high-efficiency picture scanning is realized.

Based on the above embodiment, in still another embodiment of the present application, fig. 14 is a schematic diagram of an execution flow of the image processing according to the embodiment of the present application, as shown in fig. 14, in the embodiment of the present application, the terminal acquires the preview image (step S01), and then the terminal performs a frame detection, such as a quadrilateral detection process, on the preview image (step S02); and the obtained quadrangular frame is stored to the tail of the FIFO, i.e. the last bit of the queue (step S03).

Further, the terminal may sequentially select unclassified quadrilateral frame samples from the quadrilateral frame samples in the current FIFO queue according to the sequence of entering the FIFO queue (step S03), and perform distance calculation according to the preset similarity function (step S04), so as to determine whether a classifiable frame group corresponding to the unclassified quadrilateral frame sample exists in the clustered frame groups based on the distance difference (step S05); on the one hand, if the classified frame groups have frame groups with similar distances, the unclassified quadrangular frame sample can be judged to belong to the frame groups with similar distances, and the quadrangular frame can be directly added into the frame groups (step S06); on the other hand, if there are multiple border groups with similar distances meeting the conditions in the border groups after classification, the distances can be sorted, and the quadrangular border is added into the border group with the nearest distance; in still another aspect, if there is no frame group close in distance among the classified frame groups, the terminal may establish a new frame group and add the quadrangular frame to the new frame group (step S07).

Then, the terminal may determine whether all the uncategorized quadrilateral frame samples in the FIFO sequence complete clustering, that is, whether there is an uncategorized quadrilateral frame sample in the current FIFO queue (step S08, if yes, the terminal jumps to step S03 to repeat the above steps, if no, the terminal may select a target frame group from at least one frame group obtained by clustering, for example, a frame group with the largest quadrilateral frame sample number in the at least one frame group is used as a target frame group (step S09), and select a quadrilateral frame sample corresponding to the latest frame from the target frame group based on the time sequence to determine as an initial stable quadrilateral frame (step S010).

Further, the terminal may perform similarity-based distance calculation on the initial stable quadrilateral frame and the historically stored reference stable quadrilateral frame (step S011). And judges whether the distance is smaller than a preset distance threshold (step S012). If the reference frame is smaller than the target frame, the terminal does not need to update the historical reference stable quadrilateral frame, but directly takes the historical reference stable quadrilateral frame as the target stable quadrilateral frame corresponding to the current preview image and outputs the target stable quadrilateral frame (step S013); if not, the terminal may update the historical reference stabilized quadrilateral frame with the initial stabilized quadrilateral frame (step S014), and determine the current determined new historical reference stabilized quadrilateral frame as the target stabilized quadrilateral frame corresponding to the current preview image and output. Further, the terminal may render the obtained stable quadrilateral frame, and generate and display a rendered preview image based on the rendered quadrilateral frame and the current preview image (step S015).

Based on the above steps S01 to S015, the terminal removes abnormal frame operations through similarity clustering, selection of the target frame group, determination of the target frame, and the like, and performs a denoising stabilization operation compared with the historical reference stabilization frame, the terminal does not directly preview an image frame based on the detected quadrilateral frame, but performs image preview based on the stabilized quadrilateral frame after obtaining the stabilized quadrilateral frame, thereby solving the problem that the display of the quadrilateral frame in the preview frame is unstable, and overcoming the defect that the display of the preview frame is not smooth.

Based on the above embodiment, in another embodiment of the present application, fig. 15 is a schematic diagram of the composition structure of a terminal according to the present application, as shown in fig. 15, a terminal 10 according to an embodiment of the present application may include a quadrilateral detecting module 11, a timing stabilization module 12, a denoising stabilization module 13 and a preview module 14,

the quadrilateral detection module 11 is configured to acquire an ith frame preview image corresponding to a target object; performing frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0;

The timing stabilization module 12 is configured to perform similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame, so as to obtain at least one frame group; selecting a target frame group from the at least one frame group; and determining an initial stable frame from the set of target frames;

the denoising stabilization module 13 is configured to determine an ith stabilization frame based on the initial stabilization frame and an (i-1) th stabilization frame;

the preview module 14 is configured to perform display processing on the ith frame preview image according to the ith stable frame.

Based on the above embodiment, in another embodiment of the present application, fig. 16 is a schematic diagram of a composition structure of a terminal according to the present application, and as shown in fig. 16, a terminal 10 according to an embodiment of the present application may include an acquisition portion 15, a detection portion 16, a clustering portion 17, a selection portion 18, a determination portion 19, a display portion 110, and a storage portion 111

The acquisition part 15 is configured to acquire an ith frame preview image corresponding to a target object;

the detecting part 16 is configured to perform frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0;

The clustering portion 17 is configured to perform similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame, so as to obtain at least one frame group;

the selecting part 18 is configured to select a target frame group from the at least one frame group;

the determining part 19 is configured to determine an initial stable frame from the target frame group; and determining an ith stable frame based on the initial stable frame and the (i-1) th stable frame;

the display section 110 is configured to perform display processing on the i-th frame preview image in accordance with the i-th stable frame.

Further, in an embodiment of the present application, the storage portion 111 is configured to store, after obtaining an ith quadrangular frame corresponding to the target object and before performing similarity clustering based on the ith quadrangular frame to obtain at least one frame group, the ith quadrangular frame to an nth bit of the FIFO; wherein N is an integer greater than 2, said N characterizing the maximum storage capacity of the FIFO.

Further, in the embodiment of the present application, when the i is smaller than the N, the clustering section 17 is specifically configured to read the first to the i-th quadrangular frames from the FIFO; and performing similarity clustering processing based on the first to the ith quadrangular frames to obtain the at least one frame group.

Further, in an embodiment of the present application, when the i is greater than or equal to the N, the clustering section 17 is further specifically configured to read an (i-n+1) -th quadrangular frame to the i-th quadrangular frame from the FIFO; and performing the similarity clustering processing based on the (i-n+1) th quadrilateral frame to the i-th quadrilateral frame to obtain the at least one frame group.

Further, in the embodiment of the present application, the clustering section 17 is further specifically configured to obtain the kth vertex coordinate data corresponding to the kth quadrilateral frame, and the front (k-1) vertex coordinate data corresponding to the front (k-1) quadrilateral frame; wherein k is an integer greater than 1 and less than or equal to i; calculating a front (k-1) distance difference value corresponding to the kth vertex coordinate data and the front (k-1) vertex coordinate data according to a preset similarity function; and determining a minimum distance difference from the first (k-1) distance differences; and constructing the at least one frame group based on the minimum distance difference and a first historical frame group corresponding to the front (k-1) quadrilateral frames.

Further, in the embodiment of the present application, the clustering portion 17 is further specifically configured to establish a new frame group corresponding to the kth quadrilateral frame if the minimum distance difference is greater than or equal to a preset distance threshold, and construct the at least one frame group based on the new frame group and the first historical frame group; and if the minimum distance difference value is smaller than a preset distance threshold value, classifying the kth quadrilateral frame into the first historical frame group, and constructing at least one frame group based on the first historical frame group.

Further, in the embodiment of the present application, the clustering section 17 is further specifically configured to obtain (i-n+k) th vertex coordinate data corresponding to the (i-n+k) th quadrilateral frame and (i-n+k-1) front vertex coordinate data corresponding to the (i-n+k-1) front quadrilateral frame; wherein k is an integer greater than 1 and less than or equal to N; calculating a front (i-N+k-1) distance difference value corresponding to the (i-N+k-1) th vertex coordinate data and the front (i-N+k-1) vertex coordinate data according to a preset similarity function; and determining a minimum distance difference from the previous (i-n+k-1) distance differences; and constructing the at least one frame group based on the minimum distance difference and a second historical frame group corresponding to the previous (i-n+k-1) quadrilateral frames.

Further, in the embodiment of the present application, the clustering section 17 is further specifically configured to obtain the ith vertex coordinate data corresponding to the ith quadrangular frame and the previous (i-1) vertex coordinate data corresponding to the previous (i-1) quadrangular frames grouped according to history; respectively calculating (i-1) distance differences corresponding to the ith vertex coordinate data and the (i-1) vertex coordinate data before history according to a preset similarity function; and determining a minimum distance difference from the (i-1) distance differences; and constructing the at least one frame group based on the minimum distance difference and a third historical frame group corresponding to the previous (i-1) quadrilateral frames.

Further, in the embodiment of the present application, the selecting portion 18 is specifically configured to obtain the number of quadrilateral frames included in each frame group in the at least one frame group; and determining a frame group corresponding to the maximum quadrilateral frame number as the target frame group.

Further, in the embodiment of the present application, the determining portion 19 is specifically configured to perform a permutation process on the quadrilateral frames in the target frame group according to a time sequence, so as to obtain a frame list; and determining the last quadrilateral frame in the frame list as an initial stable frame.

Further, in the embodiment of the present application, the determining portion 19 is further specifically configured to perform a mean filtering process on the quadrilateral frames in the target frame group, so as to obtain an initial stable frame.

Further, in the embodiment of the present application, the determining part 19 is further specifically configured to obtain first vertex coordinate data corresponding to the initial stable frame and second vertex coordinate data corresponding to the (i-1) th stable frame; calculating a distance difference value between the first vertex coordinate data and the second vertex coordinate data according to a preset similarity function; and if the distance difference is smaller than a preset distance threshold, determining the (i-1) th stable frame as the i-th stable frame; and if the distance difference value is greater than or equal to the preset distance threshold value, determining the initial stable frame as the ith stable frame.

Further, in the embodiment of the present application, the display portion 110 is specifically configured to perform rendering processing on the ith stable frame to obtain a rendered stable frame; and generating a rendered preview image based on the rendered stable border and the ith frame preview image; and displaying the rendered preview image.

In an embodiment of the present application, further, fig. 17 is a schematic diagram of a third component structure of the terminal according to the embodiment of the present application, and as shown in fig. 17, the terminal 10 according to the embodiment of the present application may further include a processor 112, a memory 113 storing instructions executable by the processor 112, further, the terminal 10 may further include a communication interface 114, and a bus 115 for connecting the processor 112, the memory 113 and the communication interface 114.

In an embodiment of the present application, the processor 112 may be at least one of an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (ProgRAMmable Logic Device, PLD), a field programmable gate array (Field ProgRAMmable Gate Array, FPGA), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, and a microprocessor. It will be appreciated that the electronics for implementing the above-described processor functions may be other for different devices, and embodiments of the present application are not particularly limited. The terminal 10 may further comprise a memory 113, which memory 113 may be connected to the processor 112, wherein the memory 113 is adapted to store executable program code comprising computer operation instructions, the memory 113 may comprise a high speed RAM memory, and may further comprise a non-volatile memory, e.g. at least two disk memories.

In an embodiment of the present application, bus 115 is used to connect communication interface 114, processor 112, and memory 113, as well as the intercommunication among these devices.

In an embodiment of the application, memory 113 is used for storing instructions and data.

Further, in the embodiment of the present application, the processor 112 is configured to obtain an ith frame preview image corresponding to a target object, and perform frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0; performing similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame to obtain at least one frame group; selecting a target frame group from the at least one frame group, and determining an initial stable frame from the target frame group; determining an ith stable frame based on the initial stable frame and an (i-1) th stable frame; and displaying the ith frame preview image according to the ith stable frame.

In practical applications, the Memory 113 may be a volatile Memory (RAM), such as a Random-Access Memory (RAM); or a nonvolatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard Disk (HDD) or a Solid State Drive (SSD); or a combination of the above types of memory and provides instructions and data to the processor 112.

In addition, each functional module in the present embodiment may be integrated in one file restoring unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional modules.

The integrated units, if implemented in the form of software functional modules, may be stored in a computer-readable storage medium, if not sold or used as separate products, and based on this understanding, the technical solution of the present embodiment may be embodied essentially or partly in the form of a software product, or all or part of the technical solution may be embodied in a storage medium, which includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or processor (processor) to perform all or part of the steps of the method of the present embodiment. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The embodiment of the application provides a terminal, which is used for carrying out frame detection processing on a current preview image containing a target object, after a quadrilateral frame corresponding to the target object is obtained, the terminal can firstly carry out clustering processing based on frame similarity on the quadrilateral frame, select a target frame group from at least one obtained frame group, further determine an initial stable frame from the target frame group, further determine the current stable frame based on the comparison of the initial stable frame and a historical stable frame, and further carry out display processing on the current preview image according to the current stable frame. Therefore, in the application, the terminal does not directly preview the image based on the quadrangular frames obtained by frame detection, but removes abnormal frame operations such as similarity clustering, target frame group selection, initial stable frame determination and the like on the quadrangular frames obtained by detection, and noise-removing stable operation for comparison with the historical stable frames, and after the current stable quadrangular frames are obtained, the image is previewed based on the stable quadrangular frames, thereby solving the problem of unstable display of the quadrangular frames in the preview picture and overcoming the defect of unsmooth display of the preview picture.

An embodiment of the present application provides a computer-readable storage medium having stored thereon a program which, when executed by a processor, implements the image display method as described above.

Specifically, the program instructions corresponding to one image display method in the present embodiment may be stored on a storage medium such as an optical disc, a hard disk, or a usb disk, and when the program instructions corresponding to one image display method in the storage medium are read or executed by an electronic device, the method includes the following steps:

and displaying the ith frame preview image according to the ith stable frame.

The embodiment of the application provides a chip, which comprises a processor and an interface, wherein the processor acquires program instructions through the interface, and the processor is used for running the program instructions to realize the image display method. Specifically, the image display method includes the steps of:

and displaying the ith frame preview image according to the ith stable frame.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of implementations of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block and/or flow of the flowchart illustrations and/or block diagrams, and combinations of blocks and/or flow diagrams in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block or blocks and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.

The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the present application.

Industrial applicability

The embodiment of the application discloses an image display method, a terminal and a storage medium, wherein the method comprises the following steps: acquiring an ith frame preview image corresponding to a target object, and performing frame detection processing on the ith frame preview image to acquire an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0; performing similarity clustering processing based on the first quadrilateral frame to the ith quadrilateral frame corresponding to the target object to obtain at least one frame group; selecting a target frame group from at least one frame group, and determining an initial stable frame from the target frame group; determining an ith stable frame based on the initial stable frame and the (i-1) th stable frame; and displaying the ith frame preview image according to the ith stable frame. That is, in the embodiment of the application, the terminal does not directly preview the image based on the quadrangular frames obtained by frame detection, but performs similarity clustering on the quadrangular frames obtained by detection, selection of the target frame group, determination of the initial stable frame and the like, and performs denoising and stabilization operation compared with the historical stable frame, so that after the current stable quadrangular frame is obtained, the image is previewed based on the stable quadrangular frame, the problem of unstable display of the quadrangular frame in the preview picture is solved, and the defect of unsmooth display of the preview picture is overcome.

Claims

An image display method, the method comprising:

acquiring an ith frame preview image corresponding to a target object, and performing frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0;

performing similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame to obtain at least one frame group;

selecting a target frame group from the at least one frame group, and determining an initial stable frame from the target frame group;

determining an ith stable frame based on the initial stable frame and an (i-1) th stable frame;

and displaying the ith frame preview image according to the ith stable frame.
The method of claim 1, wherein after the obtaining the ith quadrangular frame corresponding to the target object and before the performing similarity clustering based on the ith quadrangular frame to obtain at least one frame group, the method further comprises:

storing the ith quadrilateral frame to an nth bit of a first-in first-out queue FIFO; wherein N is an integer greater than 2, said N characterizing the maximum storage capacity of the FIFO.
The method of claim 2, wherein when the i is smaller than the N, the performing similarity clustering based on the first to the i-th quadrilateral frames corresponding to the target object to obtain at least one frame group includes:

reading the first quadrilateral frame to the ith quadrilateral frame from the FIFO;

and carrying out similarity clustering processing based on the first quadrilateral frame to the ith quadrilateral frame to obtain at least one frame group.
The method of claim 2, wherein when the i is greater than or equal to the N, the performing similarity clustering based on the first to the i-th quadrilateral frames corresponding to the target object to obtain at least one frame group includes:

reading an (i-n+1) th quadrilateral frame from the FIFO to the i th quadrilateral frame;

and carrying out similarity clustering processing on the basis of the (i-N+1) th quadrilateral frame to the i-th quadrilateral frame to obtain at least one frame group.
The method of claim 3, wherein the performing the similarity clustering based on the first to the i-th quadrilateral bounding box to obtain the at least one bounding box group comprises:

Acquiring kth vertex coordinate data corresponding to the kth quadrilateral frame and front (k-1) vertex coordinate data corresponding to the front (k-1) quadrilateral frame; wherein k is an integer greater than 1 and less than or equal to i;

calculating a front (k-1) distance difference value corresponding to the kth vertex coordinate data and the front (k-1) vertex coordinate data according to a preset similarity function;

determining a minimum distance difference from the first (k-1) distance differences;

and constructing the at least one frame group based on the minimum distance difference and the first historical frame group corresponding to the front (k-1) quadrilateral frames.
The method of claim 5, wherein the constructing the at least one bounding box group based on the minimum distance difference and the first historical bounding box group corresponding to the first (k-1) quadrilateral bounding boxes comprises:

if the minimum distance difference value is greater than or equal to a preset distance threshold value, a new frame group corresponding to the kth quadrilateral frame is established, and the at least one frame group is established based on the new frame group and the first historical frame group;

and if the minimum distance difference value is smaller than a preset distance threshold value, classifying the kth quadrilateral frame into the first historical frame group, and constructing at least one frame group based on the first historical frame group.
The method of claim 4, wherein the performing the similarity clustering based on the (i-n+1) th to the i-th quadrilateral frames to obtain the at least one frame group includes;

acquiring (i-N+k) th vertex coordinate data corresponding to the (i-N+k) th quadrilateral frame and front (i-N+k-1) vertex coordinate data corresponding to the front (i-N+k-1) quadrilateral frame; wherein k is an integer greater than 1 and less than or equal to N;

calculating a front (i-N+k-1) distance difference value corresponding to the (i-N+k-1) th vertex coordinate data and the front (i-N+k-1) vertex coordinate data according to a preset similarity function;

determining a minimum distance difference from the first (i-n+k-1) distance differences;

and constructing the at least one frame group based on the minimum distance difference and a second historical frame group corresponding to the front (i-N+k-1) quadrilateral frames.
The method of claim 1, wherein the performing similarity clustering based on the first to the ith quadrangular frames corresponding to the target object to obtain at least one frame group includes:

acquiring the ith vertex coordinate data corresponding to the ith quadrilateral frame and the previous (i-1) vertex coordinate data corresponding to the previous (i-1) quadrilateral frames grouped by history;

Respectively calculating (i-1) distance differences corresponding to the ith vertex coordinate data and the historic previous (i-1) vertex coordinate data according to a preset similarity function;

determining a minimum distance difference from the (i-1) distance differences;

and constructing the at least one frame group based on the minimum distance difference and a third historical frame group corresponding to the previous (i-1) quadrilateral frames.
The method of claim 1, wherein selecting the target frame group from the at least one frame group comprises:

acquiring the number of quadrilateral frames contained in each frame group in the at least one frame group;

and determining the frame group corresponding to the maximum quadrilateral frame number as the target frame group.
The method of claim 1, wherein the determining an initial stable bounding box from the set of target bounding boxes comprises:

arranging the quadrilateral frames in the target frame group according to the time sequence to obtain a frame list;

and determining the last quadrilateral frame in the frame list as an initial stable frame.
The method of claim 1, wherein the selecting an initial stable bounding box from the set of target bounding boxes comprises:

And carrying out mean value filtering treatment on the quadrilateral frames in the target frame group to obtain an initial stable frame.
The method of claim 1, wherein the determining an ith stable bounding box based on the initial stable bounding box and an (i-1) th stable bounding box comprises:

acquiring first vertex coordinate data corresponding to the initial stable frame and second vertex coordinate data corresponding to the (i-1) th stable frame;

calculating a distance difference value between the first vertex coordinate data and the second vertex coordinate data according to a preset similarity function;

if the distance difference value is smaller than a preset distance threshold value, determining the (i-1) th stable frame as the i-th stable frame;

and if the distance difference value is greater than or equal to the preset distance threshold value, determining the initial stable frame as the ith stable frame.
The method of claim 1, wherein the displaying the ith frame preview image according to the ith stable frame comprises:

rendering the ith stable frame to obtain a rendered stable frame;

generating a rendered preview image based on the rendered stable border and the ith frame preview image;

Displaying the rendered preview image.
A terminal, the terminal comprising: an acquisition section, a detection section, a clustering section, a selection section, a determination section, and a display section,

the acquisition part is configured to acquire an ith frame preview image corresponding to the target object;

the detection part is configured to perform frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0;

the clustering part is configured to perform similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame to obtain at least one frame group;

the selecting part is configured to select a target frame group from the at least one frame group;

the determining part is configured to determine an initial stable frame from the target frame group; and determining an ith stable frame based on the initial stable frame and the (i-1) th stable frame;

the display part is configured to display the ith frame preview image according to the ith stable frame.
A terminal, the terminal comprising: a quadrilateral detection module, a time sequence stabilization module, a denoising stabilization module and a preview module,

The quadrilateral detection module is configured to acquire an ith frame preview image corresponding to the target object; performing frame detection processing on the ith frame preview image to obtain an ith quadrilateral frame corresponding to the target object; wherein i is an integer greater than 0;

the time sequence stabilizing module is configured to perform similarity clustering processing based on a first quadrilateral frame corresponding to the target object to the ith quadrilateral frame to obtain at least one frame group; selecting a target frame group from the at least one frame group; and determining an initial stable frame from the set of target frames;

the denoising stabilization module is configured to determine an ith stabilization frame based on the initial stabilization frame and an (i-1) th stabilization frame;

and the preview module is configured to display the ith frame of preview image according to the ith stable frame.
A terminal comprising a quadrilateral detection module, a timing stabilization module, a denoising stabilization module, a preview module, a processor, a memory storing instructions executable by the processor, which when executed by the processor, implement the method of any of claims 1-13.
A chip comprising a processor and an interface through which the processor obtains program instructions, the processor being configured to execute the program instructions to perform the method of any of claims 1-13.
A computer readable storage medium having stored thereon a program for use in a terminal, the program, when executed by a processor, implementing the method according to any of claims 1-13.