CN116071220A - Image window data processing method, device, equipment and medium - Google Patents

Image window data processing method, device, equipment and medium Download PDF

Info

Publication number
CN116071220A
CN116071220A CN202310201079.3A CN202310201079A CN116071220A CN 116071220 A CN116071220 A CN 116071220A CN 202310201079 A CN202310201079 A CN 202310201079A CN 116071220 A CN116071220 A CN 116071220A
Authority
CN
China
Prior art keywords
sub
window
windows
data
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310201079.3A
Other languages
Chinese (zh)
Other versions
CN116071220B (en
Inventor
官乐乐
彭雪峰
卢伍平
朱亮亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Huagan Technology Co ltd
Original Assignee
Zhejiang Huagan Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Huagan Technology Co ltd filed Critical Zhejiang Huagan Technology Co ltd
Priority to CN202310201079.3A priority Critical patent/CN116071220B/en
Publication of CN116071220A publication Critical patent/CN116071220A/en
Application granted granted Critical
Publication of CN116071220B publication Critical patent/CN116071220B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20064—Wavelet transform [DWT]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the application provides an image window data processing method, device, equipment and medium, wherein in the method, if the acquired image window is determined to be divided, the image window is divided into a plurality of sub-windows; determining a reference sub-window among the plurality of sub-windows; rearranging data in other sub-windows except the reference sub-window in the plurality of sub-windows according to the arrangement rule, wherein the rearranged data in the other sub-windows are identical to the arrangement rule of the data in the reference sub-window; and processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm. In the method, the data arrangement rules in the sub-windows are the same after rearrangement, the data can be used as a data source for subsequent processing, the operation can be realized by using a processing resource, and the individual processing resource is not required to be spent for each sub-window, so that the consumption of the resource can be reduced.

Description

Image window data processing method, device, equipment and medium
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for processing image window data.
Background
Video stream image processing in real time using field-programmable gate arrays (FPGAs) is a common approach. When implementing an image algorithm in an FPGA, it is often necessary to solve a pixel window with a certain size for an input video stream, and then perform certain processing on data in the window to implement a specific function. When the divided window is larger, the pixels in the window are often required to be divided into small windows again for multi-stage processing, and parallel operation is required among a plurality of small windows.
In the conventional scheme, a parallel operation mode is generally adopted for data of a plurality of small windows in a large window, and as the positions of each small window relative to a central pixel point of the window are different, the operation rules of the data in each small window are often different, so that each small window needs to be operated by independently spending a part of processing resources, but the resource consumption is overlarge.
Disclosure of Invention
The embodiment of the application provides a method, a device, equipment and a medium for processing image window data, which are used for solving the problem of overlarge resource consumption in the prior art when multi-level operation is performed in a large window.
In a first aspect, an embodiment of the present application provides a method for processing image window data, where the method includes:
if the obtained image window is determined to be divided, dividing the image window into a plurality of sub-windows;
determining a reference sub-window among the plurality of sub-windows;
rearranging data in other sub-windows except the reference sub-window in the plurality of sub-windows according to an arrangement rule, wherein the rearranged data in the other sub-windows are identical to the arrangement rule of the data in the reference sub-window;
and processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm.
In a second aspect, an embodiment of the present application provides an image window data processing apparatus, including:
the dividing module is used for dividing the plurality of image windows into a plurality of sub-windows if the obtained image windows are determined to be divided;
a determining module, configured to determine a reference sub-window from the plurality of sub-windows;
the arrangement module is used for rearranging the data in the rest of the sub-windows except the reference sub-window according to the arrangement rule, wherein the data in the rest of the rearranged sub-windows are identical to the arrangement rule of the data in the reference sub-window;
and the processing module is used for processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm.
In a third aspect, an embodiment of the present application provides an electronic device, where the electronic device includes at least a processor and a memory, where the processor is configured to implement the steps of the image window data processing method according to any one of the preceding claims when executing a computer program stored in the memory.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the image window data processing method according to any one of the preceding claims.
In the embodiment of the application, if the obtained image window is determined to be divided, dividing the image window into a plurality of sub-windows; determining a reference sub-window among the plurality of sub-windows; rearranging data in other sub-windows except the reference sub-window in the plurality of sub-windows according to the arrangement rule, wherein the rearranged data in the other sub-windows are identical to the arrangement rule of the data in the reference sub-window; and processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm. In the method, the data in the other sub-windows except the reference sub-window are rearranged, so that the data arrangement rules in the sub-windows are the same, the subsequent processing can be carried out as a data source, the operation can be realized by using a processing resource, and the consumption of the resource can be reduced without spending a separate processing resource for each sub-window.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an image window data processing procedure according to some embodiments of the present application;
FIG. 2 is a schematic diagram of a divided image window according to some embodiments of the present application;
FIG. 3 is a schematic diagram of a wavelet decomposition process provided by some embodiments of the present application;
FIG. 4 is a schematic diagram of a wavelet decomposition process provided by some embodiments of the present application;
FIG. 5 is a schematic diagram of data transformation provided in some embodiments of the present application;
FIG. 6 is a schematic diagram of data transformation provided in some embodiments of the present application;
FIG. 7 is a schematic diagram of data transformation provided in some embodiments of the present application;
FIG. 8 is a schematic diagram of data transformation provided in some embodiments of the present application;
FIG. 9 is a schematic diagram of data transformation provided in some embodiments of the present application;
FIG. 10 is a schematic diagram of an image window data processing flow provided in some embodiments of the present application;
FIG. 11 is a schematic diagram of an image window data processing flow according to some embodiments of the present application;
FIG. 12 is a schematic diagram of an image window data processing apparatus according to some embodiments of the present application;
fig. 13 is a schematic structural diagram of an electronic device according to some embodiments of the present application.
Detailed Description
For purposes of clarity and implementation of the present application, the following description will make clear and complete descriptions of exemplary implementations of the present application with reference to the accompanying drawings in which exemplary implementations of the present application are illustrated, it being apparent that the exemplary implementations described are only some, but not all, of the examples of the present application.
It should be noted that the brief description of the terms in the present application is only for convenience in understanding the embodiments described below, and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.
The terms "first," second, "" third and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar or similar objects or entities and not necessarily for limiting a particular order or sequence, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.
The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.
The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that is capable of performing the function associated with that element.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.
Fig. 1 is a schematic diagram of an image window data processing procedure according to some embodiments of the present application, where the procedure includes:
s101: and if the acquired image window is determined to be divided, dividing the image window into a plurality of sub-windows.
The image window data processing method provided by the embodiment of the application is applied to electronic equipment, and the electronic equipment is not limited to image acquisition equipment (such as a camera), user equipment (such as a mobile phone, a tablet personal computer, wearing equipment and the like), a server and the like.
For example, the acquired image window may include an image window corresponding to the acquired video stream image.
After the electronic device acquires the image window, it may first determine whether to divide the image window. For example, the electronic device may determine whether to divide the image window according to stored image window division rules, which may be related to, but not limited to, one or more of the following: the size of the image window, the selected image processing algorithm, the amount of data in the image window, or the type of data in the image window, etc. When the image window dividing rule is met, the electronic device determines to divide the image window, and when the image window dividing rule is not met, the electronic device determines not to divide the image window. In yet another example, the electronic device may determine whether to divide the image window based on whether an image window division instruction is received, which may be triggered based on an operation of a related person. When an image window dividing instruction is received, the electronic device determines to divide the image window, and when the image window dividing instruction is not received or an instruction for directly processing the image window is received, the electronic device determines not to divide the image window.
In order to enable the plurality of sub-windows to use one processing resource, in general, the plurality of sub-windows divided into the image window include the same amount of data, that is, the same size of each sub-window. Of course, in some cases, the amount of data included in the multiple sub-windows may also be different, in which case the electronic device may not be limited to sub-windows that include sub-windows/sub-windows that are not the largest in size and that include sub-windows that are not the largest in amount of data, in which sub-windows meaningless data is supplemented, and after the supplementing process, each sub-window includes the same amount of data.
In the embodiment of the application, the division mode of the sub-windows is not limited, and the number of the sub-windows is not limited. Overlapping data (meaning that adjacent sub-windows include pixel points at the same position) may or may not exist between the plurality of sub-windows. For example, in some image processing algorithms (including but not limited to wavelet denoising algorithms), the image processing process is related to the image window center pixel, so the plurality of sub-windows may each include the image window center pixel.
S102: a reference sub-window is determined among the plurality of sub-windows.
For example, the electronic device may randomly select one sub-window among the plurality of sub-windows as the reference sub-window. In yet another example, the electronic device can determine the reference sub-window based on the location of each sub-window in the image window. In yet another example, the electronic device may determine a reference sub-window based on the processing resources used by each sub-window, e.g., select the sub-window with the smallest processing resources used as the reference sub-window.
S103: and rearranging the data in the rest of the sub-windows except the reference sub-window according to the arrangement rule, wherein the data in the rest of the rearranged sub-windows are identical to the data arrangement rule in the reference sub-window.
The electronic device stores an arrangement rule, which may be related to the data operation requirements of the selected image processing algorithm. That is, the electronic device rearranges the data in the multiple sub-windows in the same rule according to the data operation requirement of the analysis image processing algorithm, the multiple sub-windows in the same arrangement mode can be completed by using a single processing resource, and the processing of the image window data can be completed by converting the multiple resource operations required by the multiple sub-windows into the single resource operations required by the single sub-window, so that the resource consumption can be minimized. Especially, the more sub-windows are divided, the more complex the calculation is, the more resources can be saved by the embodiment of the application.
For example, although the positions of each sub-window relative to the center pixel point of the image window are different, in some image processing algorithms, each sub-window is divided by the center point of the image window, so that the data arrangement rule of each sub-window presents a certain rule relative to one or more of the center pixel point, the center row or the center column of the image window, and based on this rule, the data rule in each sub-window can be transformed to the same arrangement mode, thereby reducing the resources.
Some implementations of this step can be seen in the description of the embodiments below.
S104: and processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm.
For example, the electronic device may use a processing resource corresponding to the image processing algorithm to process the data in the reference sub-window and the rearranged data in the remaining sub-windows. The same processing resources when processing the reference sub-window and the remaining sub-windows after rearrangement include, but are not limited to, logic and/or dedicated digital processing signal (digital signal processing, DSP) resources.
In this embodiment, by rearranging the data in the remaining sub-windows except the reference sub-window, the data arrangement rules in the sub-windows are the same, and the subsequent processing can be performed as a data source, and the operation can be realized by using one processing resource, so that it is unnecessary to spend separate processing resources for each sub-window, and therefore the consumption of resources can be reduced.
On the basis of the above embodiment, in the embodiment of the present application, dividing the image window into a plurality of sub-windows includes:
determining the number of sub-windows to be divided according to the size of the image window and/or an image processing algorithm;
the image window is divided into a number of sub-windows.
The electronic device stores the corresponding relation between the size of the image window and/or the image processing algorithm and the number of the sub-windows, so that the electronic device can determine the number of the sub-windows to be divided according to the size of the image window and/or the image processing algorithm. In one implementation, when the size of the image window exceeds the set size of the image window, the electronic device determines that the size of the image window accords with the image window division rule, that is, determines to divide the image window, and then determines the number of sub-windows corresponding to the image window. In another implementation, the electronic device determines, according to the complexity of the image processing algorithm, the number of sub-windows corresponding to the image processing algorithm.
For example, as shown in fig. 2, the size of the image window obtained by the electronic device is 15×15 (accords with the image window division rule), the center pixel point of the image window is a88, and the image processing algorithm selected by the electronic device is a wavelet denoising algorithm, which generally has a large calculation amount and needs a large amount of logic and special DSP resources. After the electronic device acquires the image window, the image window of 15×15 is divided into 4 sub-windows of 8×8, which are LU (corresponding to the upper left sub-window), RU (corresponding to the upper right sub-window), LD (corresponding to the lower left sub-window), and RD (corresponding to the lower right sub-window).
In the embodiment of the application, the process of dividing the sub-window by the image window considers the size of the image window and/or an image processing algorithm, so that the flexibility of dividing the sub-window is improved.
Also taking a wavelet denoising algorithm as an example, when the wavelet denoising algorithm is adopted for processing, the wavelet coefficients can be obtained by layering for the pixel values in each sub-window (namely, the data in the sub-window).
Taking LU as an example, as shown in fig. 3, a represents different pixels in an 8×8 window, b, c, d represent the accumulated sums of corresponding covered pixels (e.g. b11=a11+a12+a21+a22, c11=b11+b12+b21+b22), it is understood that a, b, c, and d represent only pixels at corresponding positions and do not represent specific values of pixel values/data. In some implementations, the pixel value corresponding to the d-position is used for wavelet reconstruction.
The calculation process of LU third-order wavelet decomposition is represented as follows, where L1, L2, L3 respectively represent three-layer wavelet orders, HL, LH, HH respectively represent high-frequency components in three directions of horizontal, vertical, diagonal:
l1 order wavelet order:
LU.L1.HL = a77 + a78 – a87 – a88
LU.L1.LH = a77 – a78 + a87 – a88
LU.L1.HH = a77 – a78 - a87 + a88
l2 order wavelet order:
LU.L2.HL = b33 + b34 – b43 –b44
LU.L2.LH = b33 - b34 + b43 – b44
LU.L2.HH = b33 - b34 – b43 + b44
l3 order wavelet order:
LU.L3.HL = c11 + c12 – c21 – c22
LU.L3.HL = c11 - c12 + c21 – c22
LU.L3.HH = c11 - c12 – c21 + c22
the decomposition process of RU third-order wavelet decomposition is shown in fig. 4, and the calculation process is shown as follows:
l1 order wavelet order:
RU.L1.HL = a79 + a78 – a88 – a89
RU.L1.LH = a79 – a78 + a88 – a89
RU.L1.HH = a79 – a78 - a88 + a89
l2 order wavelet order:
RU.L2.HL = b31 + b32 – b41 –b42
RU.L2.LH = b31 - b32 + b41 – b42
RU.L2.HH = b31 - b32 – b41 + b42
l3 order wavelet order:
RU.L3.HL = c12 + c11 – c22 – c21
RU.L3.HL = c12 - c11 + c22 – c21
RU.L3.HH = c12 - c11 – c22 + c21
similarly, formulas of the decomposition processes of the LD and the RD sub-windows can be calculated, and are not listed here. As can be seen from the above formula, since the decomposition formulas for 4 different wavelet windows (i.e., 4 different sub-windows) within the same 15×15 image window are different, at least 4 logic resources are required to implement wavelet decomposition in the related art. Similarly, for the wavelet denoising process after wavelet decomposition, 4 computing resources are still needed to respectively compute the 4 wavelet windows. That is, the logical and operation resources consumed by each wavelet window need to be multiplied by 4 to be the resources consumed by the wavelet denoising algorithm, which leads to the problem of excessive resource consumption.
Based on the foregoing embodiments, in the embodiments of the present application, rearranging the data in the remaining sub-windows of the plurality of sub-windows except the reference sub-window according to the arrangement rule includes:
and according to the position relation between the first sub-window in the reference sub-window and the rest sub-windows and the central pixel point of the image window, taking the reference sub-window as a reference, and symmetrically rotating each data in the first sub-window relative to the central column of the image window.
In some scenes, the sub-windows are divided by the values of the central pixel points of the image window, and the arrangement rule of the data in the sub-windows may show a certain rule relative to the central column of the image, so that the data in the first sub-window can be symmetrically rotated relative to the central column of the image window by taking the reference sub-window as a reference, so that the rotated data in the first sub-window and the data in the reference sub-window have the same arrangement rule, and therefore the same processing resource can be adopted for processing.
In the embodiment of the application, the implementation process of symmetrically rotating the data in the sub-window is not limited, so long as the data in the sub-window has the same arrangement rule after the data are symmetrically rotated.
Taking a wavelet denoising algorithm as an example, the following can be known according to the wavelet decomposition calculation formulas of LU and RU: as shown in fig. 5, the 4 a values of the L1-order component calculated in LU and RU are symmetrically distributed on both sides of the center column of the image window (i.e. the columns where a18, a28, a 38-a 88 are located) (e.g. a77 is symmetrical to a79, a87 is symmetrical to a 89); as shown in fig. 6, the 4 b values for calculating the L2 order component in LU and RU are also symmetrically distributed on both sides of the center column of the image window; as shown in fig. 7, the 4 c values for the calculated L3 order components in LU and RU are also symmetrically distributed on both sides of the center column of the image window.
Therefore, as shown in fig. 5 to 7, with LU as a reference (i.e., LU is a reference sub-window), each pixel point in the RU sub-window is symmetrically rotated to LU direction according to the central column, then each pixel point required in the 3-order wavelet decomposition of the L1 order, the L2 order and the L3 order corresponds exactly one to one, the arrangement modes of the LU sub-window and the RU sub-window after rotation are the same, and the same processing resource can be adopted for processing, so as to reduce the consumed resource.
Based on the above embodiments, in the embodiments of the present application, rearranging the data in the remaining sub-windows except the reference sub-window in the plurality of sub-windows according to the arrangement rule includes:
and according to the position relation between the second sub-window in the reference sub-window and the rest sub-windows and the central pixel point of the image window, taking the reference sub-window as a reference, and symmetrically rotating each data in the second sub-window relative to the central line of the image window.
In some scenes, the sub-windows are divided by the value of the central pixel point of the image window, and the arrangement rule of the data in the sub-windows may show a certain rule relative to the central line of the image, so that the data in the second sub-window can be symmetrically rotated relative to the central line of the image window by using the reference sub-window without reference, so that the rotated data in the second sub-window and the rotated data in the reference sub-window have the same arrangement rule, and therefore the same processing resource can be adopted for processing.
In the embodiment of the application, the implementation process of symmetrically rotating the data in the sub-window is not limited, so long as the data in the sub-window has the same arrangement rule after the data are symmetrically rotated.
Taking a wavelet denoising algorithm as an example for illustration, the wavelet decomposition calculation formulas of LU and LD can be known: as shown in fig. 8, the 4 a values of the L1-order component calculated in LU and LD are symmetrically distributed on both sides (e.g. a77 and a79 are symmetrical) of the central line (line where a81, a82, a 83-a 88 are located) of the image window; the 4 b values of the L2 order component calculated in LU and RU are also symmetrically distributed on both sides of the center line of the image window (not shown in the figure); the 4 c values for the L3 order components calculated in LU and RU are also symmetrically distributed on both sides of the center line of the image window (not shown in the figure).
Therefore, with LU as a reference, each pixel in the LD sub-window rotates to LU direction along the central line, then each pixel required in the 3-order wavelet decomposition of L1 order, L2 order and L3 order corresponds exactly one by one, the LU sub-window and the rotated LD sub-window are arranged in the same way, and the same processing resource can be adopted for processing, so as to reduce the consumed resource.
Based on the above embodiments, in the embodiments of the present application, rearranging the data in the remaining sub-windows except the reference sub-window in the plurality of sub-windows according to the arrangement rule includes:
and rotating each data in the third sub-window relative to the central pixel point by a set angle by taking the reference sub-window as a reference according to the position relation between the reference sub-window and the central pixel point of the image window and the third sub-window in the rest sub-windows.
In some scenes, the sub-windows are divided by the value of the central pixel point of the image window, and the arrangement rule of the data in the sub-windows may show a certain rule relative to the central pixel point of the image, so that the data in the third sub-window can be symmetrically rotated relative to the central pixel point of the image window by taking the reference sub-window as a reference, so that the data in the rotated third sub-window and the data in the reference sub-window have the same arrangement rule, and the same processing resource can be adopted for processing.
Optionally, each data in the third sub-window may be rotated by a set angle with respect to the center pixel point according to a set direction.
In the embodiment of the application, the value of the set angle and the optional set direction are not limited, so long as the data in the sub-window can have the same arrangement rule after symmetrical rotation. For example, the set angle includes, but is not limited to, 180 degrees.
Taking a wavelet denoising algorithm as an example, the wavelet decomposition calculation formula of LU and RD can be known: as shown in fig. 9, the 4 a values of the L1-order component calculated in LU and RD are distributed on both sides of the center pixel point (a 88) of the image window in a 180 degree relationship (e.g. a78 is symmetrical with a98, a87 is symmetrical with a 89); the 4 b values for calculating the L2-order component and the 4 c values for calculating the L3-order component in LU and RD are also distributed on both sides of the center pixel (not shown in the figure) with the same rule.
Therefore, with LU as a reference, each pixel in the RD sub-window is rotated 180 degrees to LU direction according to the central pixel, so that each pixel required in the 3 RD order wavelet decomposition of L1 order, L2 order and L3 order corresponds exactly one by one, the arrangement modes of the LU sub-window and the rotated RD sub-window are the same, and the same processing resource can be adopted for processing, thereby reducing consumed resources.
On the basis of the above embodiments, fig. 10 provides a schematic flow chart of image window data processing, which includes the following steps:
s1001: an image window is acquired.
In this step, the electronic device may acquire an image window in the video stream.
S1002: it is determined whether to divide the image window into multiple levels of sub-windows. If yes, S1003 is performed, and if no, the process ends.
For example, the electronic device may determine whether to divide the image window into multiple levels of sub-windows based on the selected image processing algorithm. Specifically, the electronic device may determine according to the operation requirement of the image processing algorithm, for example, the above-mentioned wavelet denoising algorithm, which generally has a large calculation amount and needs to spend a large amount of logic and special DSP resources.
In this step, the electronic device may also determine the number of multi-level sub-windows and divide the image window into the multi-level sub-windows.
S1003: and determining the corresponding relation between the multi-level sub-windows according to the operation requirement of the image processing algorithm.
For example, in a wavelet denoising algorithm, the electronic device can analyze the position relationship between each sub-window and the central pixel point of the image window by reasoning the decomposition formula of each sub-window, so as to determine the corresponding relationship between the multi-level sub-windows.
S1004: and judging whether the data of the multi-level sub-window can be arranged to the same rule. If yes, S1005 is executed, and if no, the process ends.
The electronic device can judge whether the data of the multi-level sub-windows can be arranged to the same rule through the analyzed position relation between each sub-window and the central pixel point of the image window.
S1005: the data of the sub-windows are rearranged in a rotationally symmetrical manner or the like, and then image processing is performed.
In the scheme of the related art without data transformation, as shown in fig. 11, the video stream is input into the electronic device, the electronic device determines that the image window is a 15×15 pixel window, and divides the image window into 8×8 small windows to obtain 4 sub-windows, and for each sub-window, 4 parts of wavelet decomposition processing, 4 parts of wavelet noise reduction processing and wavelet reconstruction processing are respectively performed, and the video stream obtained after the processing is output, so that the resource consumption is excessive. In the scheme of performing data transformation in the embodiment of the application, the video stream is input into the electronic device, the electronic device determines that the image window is a 15 x 15 pixel window, the image window is divided into 8 x 8 small windows to obtain 4 sub-windows, before the data is subjected to wavelet decomposition in a multi-level window, the data in the 4 sub-windows can be symmetrically rotated to the LU direction by inserting a data rule transformation module, the data is transformed into the same arrangement mode, then 1 part of wavelet decomposition processing, 1 part of wavelet noise reduction processing and wavelet reconstruction processing are performed, and the video stream obtained after the processing is output. In the scheme provided by the embodiment of the application, only the same data is needed to be operated, the consumed resources are reduced to 1/4 of the original resources, the resources are greatly reduced, the more the window is divided, the more complex the subsequent calculation is, and the more the resources are saved.
Based on the same technical concept, on the basis of the above embodiments, the present application provides an image window data processing apparatus, and fig. 12 is a schematic structural diagram of an image window data processing apparatus provided in some embodiments of the present application, as shown in fig. 12, where the apparatus includes:
the dividing module 1201 is configured to divide the plurality of image windows into a plurality of sub-windows if it is determined to divide the acquired image windows;
a determining module 1202 for determining a reference sub-window among the plurality of sub-windows;
an arrangement module 1203, configured to rearrange data in remaining sub-windows except for the reference sub-window in the multiple sub-windows according to an arrangement rule, where the rearranged data in the remaining sub-windows is the same as the arrangement rule of the data in the reference sub-window;
and the processing module 1204 is used for processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm.
In one possible implementation, the dividing module 1201 is specifically configured to determine the number of sub-windows to be divided according to the size of the image window and/or the image processing algorithm; the image window is divided into a number of sub-windows.
In one possible implementation, the arrangement rules relate to the data operation requirements of the image processing algorithm.
In a possible implementation manner, the arrangement module 1203 is specifically configured to symmetrically rotate each data in the first sub-window with respect to the central column of the image window with reference to the reference sub-window according to the positional relationship between the first sub-window in the reference sub-window and the remaining sub-windows and the central pixel point of the image window.
In a possible implementation manner, the arrangement module 1203 is specifically configured to symmetrically rotate each data in the second sub-window with respect to the center line of the image window with reference to the reference sub-window according to the positional relationship between the reference sub-window and the second sub-window in the remaining sub-windows and the center pixel point of the image window.
In a possible implementation manner, the arrangement module 1203 is specifically configured to rotate each data in the third sub-window by a set angle with respect to the center pixel point with respect to the reference sub-window according to the positional relationship between the reference sub-window and the third sub-window in the remaining sub-windows and the center pixel point of the image window.
In one possible implementation, the image processing algorithm comprises a wavelet denoising algorithm.
Based on the same technical concept, the present application further provides an electronic device, and fig. 13 is a schematic structural diagram of the electronic device provided in the embodiment of the present application, as shown in fig. 13, including: processor 1301, communication interface 1302, memory 1303 and communication bus 1304, wherein processor 1301, communication interface 1302, memory 1303 complete the communication between each other through communication bus 1304;
the memory 1303 stores a computer program that, when executed by the processor 1301, causes the processor 1301 to perform the steps of:
if the obtained image window is determined to be divided, dividing the image window into a plurality of sub-windows;
determining a reference sub-window among the plurality of sub-windows;
rearranging data in other sub-windows except the reference sub-window in the plurality of sub-windows according to the arrangement rule, wherein the rearranged data in the other sub-windows are identical to the arrangement rule of the data in the reference sub-window;
and processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm.
In a possible implementation, the processor 1301 is specifically configured to determine the number of sub-windows to be divided according to the size of the image window and/or the image processing algorithm; the image window is divided into a number of sub-windows.
In one possible implementation, the arrangement rules relate to the data operation requirements of the image processing algorithm.
In a possible implementation manner, the processor 1301 is specifically configured to symmetrically rotate each data in the first sub-window with respect to the center column of the image window with respect to the reference sub-window according to the positional relationship between the reference sub-window and the first sub-window in the remaining sub-windows and the center pixel point of the image window.
In a possible implementation manner, the processor 1301 is specifically configured to symmetrically rotate each data in the second sub-window with respect to the center line of the image window with respect to the reference sub-window according to the positional relationship between the reference sub-window and the second sub-window in the remaining sub-windows and the center pixel point of the image window.
In a possible implementation manner, the processor 1301 is specifically configured to rotate each data in the third sub-window by a set angle with respect to the center pixel point with respect to the reference sub-window according to the positional relationship between the reference sub-window and the third sub-window in the remaining sub-windows and the center pixel point of the image window.
In one possible implementation, the image processing algorithm comprises a wavelet denoising algorithm.
The communication bus mentioned for the above-mentioned electronic devices may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture ) bus, or the like. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface 1302 is used for communication between the electronic device and other devices.
The Memory may include RAM (Random Access Memory ) or NVM (Non-Volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit, an NP (Network Processor ), etc.; but may also be a DSP (Digital Signal Processing, digital instruction processor), application specific integrated circuit, field programmable gate array or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.
Based on the same technical idea, the embodiments of the present application provide a computer readable storage medium, in which a computer program executable by an electronic device is stored, which when executed on the electronic device, causes the electronic device to implement any of the embodiments described above.
The computer readable storage medium may be any available medium or data storage device that can be accessed by a processor in an electronic device, including but not limited to magnetic memories such as floppy disks, hard disks, tapes, MO (magneto optical disks), etc., optical memories such as CD, DVD, BD, HVD, etc., and semiconductor memories such as ROM, EPROM, EEPROM, NAND FLASH (non-volatile memories), SSD (solid state disk), etc.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.

Claims (10)

1. A method of processing image window data, the method comprising:
if the obtained image window is determined to be divided, dividing the image window into a plurality of sub-windows;
determining a reference sub-window among the plurality of sub-windows;
rearranging data in other sub-windows except the reference sub-window in the plurality of sub-windows according to an arrangement rule, wherein the rearranged data in the other sub-windows are identical to the arrangement rule of the data in the reference sub-window;
and processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm.
2. The method of claim 1, wherein the dividing the image window into a plurality of sub-windows comprises:
determining the number of sub-windows to be divided according to the size of the image window and/or the image processing algorithm;
the image window is divided into the number of sub-windows.
3. The method of claim 1, wherein the arrangement rule relates to a data operation requirement of the image processing algorithm.
4. A method according to any one of claims 1-3, wherein rearranging the data in the remaining sub-windows of the plurality of sub-windows except the reference sub-window according to the arrangement rule comprises:
and according to the position relation between the reference sub-window and the central pixel point of the image window and the first sub-window in the rest sub-windows, taking the reference sub-window as a reference, and symmetrically rotating each data in the first sub-window relative to the central column of the image window.
5. A method according to any one of claims 1-3, wherein rearranging the data in the remaining sub-windows of the plurality of sub-windows except the reference sub-window according to the arrangement rule comprises:
and according to the position relation between the reference sub-window and the central pixel point of the image window and the second sub-window in the rest sub-windows, taking the reference sub-window as a reference, and symmetrically rotating each data in the second sub-window relative to the central line of the image window.
6. A method according to any one of claims 1-3, wherein rearranging the data in the remaining sub-windows of the plurality of sub-windows except the reference sub-window according to the arrangement rule comprises:
and rotating each data in the third sub-window relative to the central pixel point by a set angle by taking the reference sub-window as a reference according to the position relation between the reference sub-window and the third sub-window in the rest sub-windows and the central pixel point of the image window.
7. A method according to any of claims 1-3, wherein the image processing algorithm comprises a wavelet denoising algorithm.
8. An image window data processing apparatus, the apparatus comprising:
the dividing module is used for dividing the acquired image window into a plurality of sub-windows if the acquired image window is determined to be divided;
a determining module, configured to determine a reference sub-window from the plurality of sub-windows;
the arrangement module is used for rearranging the data in the rest of the sub-windows except the reference sub-window according to the arrangement rule, wherein the data in the rest of the rearranged sub-windows are identical to the arrangement rule of the data in the reference sub-window;
and the processing module is used for processing the data in the reference sub-window and the rearranged data in the rest sub-windows by adopting an image processing algorithm.
9. An electronic device comprising at least a processor and a memory, the processor being adapted to implement the steps of the image window data processing method according to any of claims 1-7 when executing a computer program stored in the memory.
10. A computer storage medium, characterized in that it stores a computer program executable by an electronic device, which program, when run on the electronic device, causes the electronic device to perform the steps of the image window data processing method of any of claims 1-7.
CN202310201079.3A 2023-03-06 2023-03-06 Image window data processing method, device, equipment and medium Active CN116071220B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310201079.3A CN116071220B (en) 2023-03-06 2023-03-06 Image window data processing method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310201079.3A CN116071220B (en) 2023-03-06 2023-03-06 Image window data processing method, device, equipment and medium

Publications (2)

Publication Number Publication Date
CN116071220A true CN116071220A (en) 2023-05-05
CN116071220B CN116071220B (en) 2023-06-20

Family

ID=86178577

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310201079.3A Active CN116071220B (en) 2023-03-06 2023-03-06 Image window data processing method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN116071220B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103167218A (en) * 2011-12-14 2013-06-19 北京大学 Super-resolution reconstruction method and equipment based on non-locality
US20150071538A1 (en) * 2013-09-11 2015-03-12 Digitalglobe, Inc. Automated and scalable object and feature extraction from imagery
CN106373098A (en) * 2016-08-30 2017-02-01 天津大学 Random pulse noise removal method based on non-similar pixel statistics
CN108596169A (en) * 2018-03-12 2018-09-28 北京建筑大学 Piecemeal signal conversion based on video streaming image and object detection method and device
CN112308806A (en) * 2020-11-11 2021-02-02 海宁奕斯伟集成电路设计有限公司 Image processing method, image processing device, electronic equipment and readable storage medium
CN113095316A (en) * 2021-04-15 2021-07-09 西安电子科技大学 Image rotation target detection method based on multilevel fusion and angular point offset
CN115035456A (en) * 2022-06-30 2022-09-09 维沃移动通信有限公司 Video denoising method and device, electronic equipment and readable storage medium
CN115170409A (en) * 2022-06-16 2022-10-11 北京奕斯伟计算技术股份有限公司 Image processing method, apparatus, device, readable storage medium and program product

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103167218A (en) * 2011-12-14 2013-06-19 北京大学 Super-resolution reconstruction method and equipment based on non-locality
US20150071538A1 (en) * 2013-09-11 2015-03-12 Digitalglobe, Inc. Automated and scalable object and feature extraction from imagery
CN106373098A (en) * 2016-08-30 2017-02-01 天津大学 Random pulse noise removal method based on non-similar pixel statistics
CN108596169A (en) * 2018-03-12 2018-09-28 北京建筑大学 Piecemeal signal conversion based on video streaming image and object detection method and device
CN112308806A (en) * 2020-11-11 2021-02-02 海宁奕斯伟集成电路设计有限公司 Image processing method, image processing device, electronic equipment and readable storage medium
WO2022100048A1 (en) * 2020-11-11 2022-05-19 海宁奕斯伟集成电路设计有限公司 Image processing method and apparatus, electronic device, and readable storage medium
CN113095316A (en) * 2021-04-15 2021-07-09 西安电子科技大学 Image rotation target detection method based on multilevel fusion and angular point offset
CN115170409A (en) * 2022-06-16 2022-10-11 北京奕斯伟计算技术股份有限公司 Image processing method, apparatus, device, readable storage medium and program product
CN115035456A (en) * 2022-06-30 2022-09-09 维沃移动通信有限公司 Video denoising method and device, electronic equipment and readable storage medium

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
HAOZHENG REN等: "Research of multi-focus image fusion based on M-band multi-wavelet transformation", IEEE *
沈德海;侯建;鄂旭;张龙昌;: "一种多子窗口中值加权的高斯噪声滤波算法", 现代电子技术, no. 09 *
管声启;石秀华;宋玉琴;: "基于小波分解的织物疵点检测", 昆明理工大学学报(理工版), no. 01 *
胡越黎;计慧杰;吴频;宣祥光;: "图像的中值滤波算法及其FPGA实现", 计算机测量与控制, no. 11 *

Also Published As

Publication number Publication date
CN116071220B (en) 2023-06-20

Similar Documents

Publication Publication Date Title
KR102631381B1 (en) Convolutional neural network processing method and apparatus
US11954583B2 (en) Transposed convolution using systolic array
CN108268931B (en) Data processing method, device and system
US11816559B2 (en) Dilated convolution using systolic array
KR20180060149A (en) Convolution processing apparatus and method
CN108073549B (en) Convolution operation device and method
CN111199273A (en) Convolution calculation method, device, equipment and storage medium
US20240062054A1 (en) Storage of input values across multiple cores of neural network inference circuit
KR102452951B1 (en) Method and apparatus for performing convolution operation in neural network
US20200218777A1 (en) Signal Processing Method and Apparatus
US11238560B2 (en) Image down-scaling with pixel sets selected via blue noise sampling
EP3766020A1 (en) Efficient convolutional engine
CN111709415B (en) Target detection method, device, computer equipment and storage medium
CN114443559A (en) Reconfigurable operator unit, processor, calculation method, device, equipment and medium
CN117437132A (en) Method and apparatus for image restoration
CN115883308A (en) Method and device for selecting main node in consensus algorithm, electronic equipment and storage medium
CN116071220B (en) Image window data processing method, device, equipment and medium
CN111448545A (en) Parallel multi-valued reduction
US20240046081A1 (en) Data transfer for non-dot product computations on neural network inference circuit
CN112766397A (en) Classification network and implementation method and device thereof
CN111860287A (en) Target detection method and device and storage medium
KR100561462B1 (en) Image processing method and apparatus in image forming system
CN110780849B (en) Matrix processing method, device, equipment and computer readable storage medium
CN110826694B (en) Image processing method and device based on convolutional neural network
CN111901500A (en) Image processing method and apparatus, storage medium, and electronic apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant