CN115760600B - Image processing method and device, electronic equipment and storage medium - Google Patents
Image processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115760600B CN115760600B CN202211352901.8A CN202211352901A CN115760600B CN 115760600 B CN115760600 B CN 115760600B CN 202211352901 A CN202211352901 A CN 202211352901A CN 115760600 B CN115760600 B CN 115760600B
- Authority
- CN
- China
- Prior art keywords
- image processing
- signal
- image
- field
- processing module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application discloses an image processing method, an image processing device, electronic equipment and a storage medium, and relates to the technical field of image processing, wherein the method comprises the following steps: acquiring a source image signal, and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series; the image processing operation of the target image processing module in the image processing system includes: performing image processing operation corresponding to the target image processing module on the input signal to obtain an output signal; judging whether matrix operation is included in image processing operation corresponding to the next image processing module of the target image processing module; if yes, the output signal is expanded, and the expanded output signal is used as an input signal of the next image processing module. The image edge data processing method and device improve image edge data processing quality and reduce processing complexity.
Description
Technical Field
The present invention relates to the field of image processing technology, and more particularly, to an image processing method and apparatus, an electronic device, and a computer readable storage medium.
Background
A schematic diagram of a general processing flow for video image processing is shown in FIG. 1, and the general processing flow comprises a source image output module, a common module n1 (without matrix operation), an algorithm module m1 with matrix operation, an algorithm module m2 with matrix operation, an algorithm module m3 with matrix operation, a common module n2 (without matrix operation), image display and other output applications. Typical synchronization signals for video image output are a line synchronization signal Hs and a field synchronization signal Vs, with Hs being high and Vs being high and with image sizes 768 by 576 being as an example, and the signal format is generally shown in fig. 2. Thus, in the case of performing the arithmetic sliding window processing including the matrix operation, taking a 7 by 7 square array as an example, when the matrix operation is performed on the top six rows of data, the bottom six rows of data, the left front six columns of data and the right rear six columns of data of the image, since the four edge data cannot fill the 7 by 7 array window, the normal operation cannot be performed on the upper, lower, left and right edge data of each frame of image in the matrix operation, and the processing method needs to be considered additionally.
As shown in fig. 3, fig. 3 is a schematic diagram of a sliding window processing of an image edge matrix algorithm. Conventional processing methods for image edges are generally divided into two types: the first method is that the upper and lower edges use the image data of the front and rear adjacent frames to carry out the array filling of the beginning line and the end line of the frame, the left and right edges use the image data of the front and rear adjacent lines to carry out the array filling of the beginning line and the end line of the line. The second method is to add an edge processing module in each algorithm module containing matrix operation or to process the image edge by adopting an edge processing method. By using the method, the control process is often complex, each algorithm module containing matrix operation needs to be processed by adopting a corresponding edge processing module or an edge processing method according to own algorithm characteristics, and along with the increase of the algorithm modules, logic resources are easy to consume, the system is complex, and the reliability of system processing is reduced.
When a video image is processed by a certain algorithm including matrix operation, effective data of the image edge cannot fill an array window, so that a certain problem exists that the effective data of the image edge cannot be filled, the additional processing is needed, the traditional processing method can cause that the image edge is easy to be severely distorted, line-field synchronous signals are consecutive and misplaced, or each algorithm module needs to be additionally added with a corresponding edge processing module or an edge processing method, the complexity of a system is increased, and the logic resource is easy to be consumed, and the reliability of the system is reduced.
Therefore, how to improve the image edge data processing quality and reduce the processing complexity is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
An object of the present application is to provide an image processing method and apparatus, an electronic device, and a computer readable storage medium, which solve the technical problems: how to improve the image edge data processing quality and reduce the processing complexity.
To achieve the above object, the present application provides an image processing method, including:
acquiring a source image signal, and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
The image processing operation of the target image processing module in the image processing system includes:
performing image processing operation corresponding to the target image processing module on the input signal to obtain an output signal;
judging whether matrix operation is included in image processing operation corresponding to the next image processing module of the target image processing module;
if yes, the output signal is expanded, and the expanded output signal is used as an input signal of the next image processing module.
Wherein expanding the output signal comprises:
expanding a line synchronization signal and a field synchronization signal of the output signal;
the image data is filled in an envelope common to the extended line sync signal and the field sync signal.
Wherein the expanded line synchronization signal is at least longer than the line synchronization signal before expansion
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
And a row period, wherein N is the side length of the array data window required in the image processing operation corresponding to the next image processing module.
Wherein the filling of the image data in the envelope common to the extended line sync signal and the field sync signal comprises:
And filling preset values in the envelope of the extended line synchronous signals and the field synchronous signals.
Wherein the filling of the image data in the envelope common to the extended line sync signal and the field sync signal comprises:
and filling the last pixel value of the corresponding row in the envelope which is common to the extended row synchronizing signal and the field synchronizing signal.
Wherein after the source image signal is acquired, the method further comprises:
expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid;
correspondingly, inputting the source image signal into an image processing system for image processing, comprising:
and inputting the expanded source image signal into an image processing system for image processing.
To achieve the above object, the present application provides an image processing method, including:
acquiring a source image signal, and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
the image processing operation of the target image processing module in the image processing system includes:
acquiring an output signal of a last image processing module of the target image processing module as an input signal;
Judging whether matrix operation is included in the image processing operation corresponding to the target image processing module;
if yes, the input signal is expanded, and image processing operation corresponding to the target image processing module is carried out on the expanded input signal to obtain an output signal of the target image processing module.
Wherein expanding the input signal comprises:
expanding a line synchronization signal and a field synchronization signal of the input signal;
the image data is filled in an envelope common to the extended line sync signal and the field sync signal.
Wherein the expanded line synchronization signal is at least longer than the line synchronization signal before expansion
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
And a row period, wherein N is the side length of the array data window required in the image processing operation corresponding to the next image processing module.
Wherein the filling of the image data in the envelope common to the extended line sync signal and the field sync signal comprises:
and filling preset values in the envelope of the extended line synchronous signals and the field synchronous signals.
Wherein the filling of the image data in the envelope common to the extended line sync signal and the field sync signal comprises:
And filling the last pixel value of the corresponding row in the envelope which is common to the extended row synchronizing signal and the field synchronizing signal.
Wherein after the source image signal is acquired, the method further comprises:
expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid;
correspondingly, inputting the source image signal into an image processing system for image processing, comprising:
and inputting the expanded source image signal into an image processing system for image processing.
In order to achieve the above object, the present application provides an image processing apparatus for acquiring a source image signal and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
the target image processing module in the image processing system includes:
the first processing unit is used for performing image processing operation corresponding to the target image processing module on the input signal to obtain an output signal;
a first judging unit, configured to judge whether a matrix operation is included in an image processing operation corresponding to a next image processing module of the target image processing module; if yes, starting the workflow of the first expansion unit;
The first expansion unit is configured to expand the output signal, and take the expanded output signal as an input signal of the next image processing module.
In order to achieve the above object, the present application provides an image processing apparatus for acquiring a source image signal and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
the target image processing module in the image processing system includes:
an acquisition unit configured to acquire an output signal of a previous image processing module of the target image processing module as an input signal;
a second judging unit, configured to judge whether matrix operation is included in the image processing operation corresponding to the target image processing module; if yes, starting the workflow of the second expansion unit;
the second expansion unit is used for expanding the input signal;
and the second processing unit is used for performing image processing operation corresponding to the target image processing module on the expanded input signal to obtain an output signal of the target image processing module.
To achieve the above object, the present application provides an electronic device, including:
a memory for storing a computer program;
and a processor for implementing the steps of the image processing method as described above when executing the computer program.
To achieve the above object, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the image processing method as described above.
According to the scheme, the image processing method provided by the application comprises the following steps: acquiring a source image signal, and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series; the image processing operation of the target image processing module in the image processing system includes: performing image processing operation corresponding to the target image processing module on the input signal to obtain an output signal; judging whether matrix operation is included in image processing operation corresponding to the next image processing module of the target image processing module; if yes, the output signal is expanded, and the expanded output signal is used as an input signal of the next image processing module.
According to the image processing method provided by the application, according to whether matrix operation is included in the image processing operation corresponding to the next image processing module, the output signal of the last image processing module is expanded and then input to the next image processing module, so that the algorithm module can process the image edge conveniently. The method simplifies the processing flow of the algorithm module containing matrix operation, ensures that the image edge data is the same as the image internal data, can be uniformly processed when the matrix operation is carried out, does not need to be additionally treated, can improve the image edge data processing quality to a certain extent, has simple expansion principle, is easy to realize, and requires very few design resources or logic resources. Therefore, the image processing method improves the image edge data processing quality and reduces the processing complexity. The application also discloses an image processing device, electronic equipment and a computer readable storage medium, and the technical effects can be achieved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:
FIG. 1 is a schematic diagram of a video image processing flow in the related art;
FIG. 2 is a schematic diagram of a typical signal format;
FIG. 3 is a schematic diagram of a sliding window processing of an image edge matrix algorithm;
FIG. 4 is a block diagram of a first image processing system;
FIG. 5 is a flowchart of a first image processing method;
FIG. 6 is a diagram of an extended forward synchronization signal;
FIG. 7 is a diagram of an extended backward synchronization signal;
FIG. 8 is a schematic diagram of an extended top field sync signal;
FIG. 9 is a diagram of an extended post-field sync signal;
FIG. 10 is a schematic diagram of an original source image signal;
FIG. 11 is a schematic illustration of an expanded source image signal;
FIG. 12 is a schematic diagram showing the expansion of a module in the subsequent stage when it needs to expand the field sync signal;
FIG. 13 is a block diagram of a second image processing system;
FIG. 14 is a flowchart of a second image processing method;
FIG. 15 is a schematic diagram of an image signal input to a Gaussian filter algorithm module by a pre-expansion stage module;
FIG. 16 is a schematic diagram of an image signal input to the Gaussian filter algorithm module by the post-expansion pre-stage module;
FIG. 17 is a schematic diagram of a sliding window processing of an edge matrix of an image before signal expansion;
FIG. 18 is a schematic diagram of a sliding window processing for edge matrix operation of an image after signal expansion;
FIG. 19 is a block diagram of a target image processing module in a first image processing system;
FIG. 20 is a block diagram of a target image processing module in a second image processing system;
fig. 21 is a structural diagram of an electronic device.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application. In addition, in the embodiments of the present application, "first," "second," and the like are used to distinguish similar objects, and are not necessarily used to describe a particular order or sequence.
The embodiment of the application discloses an image processing method, which improves the image edge data processing quality and reduces the processing complexity. Specifically, the method comprises the following steps:
acquiring a source image signal, and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
In this embodiment, a source image signal is acquired and input into an image processing system for image processing, the image processing system including a plurality of image processing modules connected in series, including an image processing module including a matrix operation and an image processing module not including a matrix operation.
Fig. 4 is a schematic diagram of a first image processing system, as shown in fig. 4, including a source image output module, a normal module n1 (without matrix operation), an algorithm module m1 with matrix operation, an algorithm module m2 with matrix operation, an algorithm module m3 with matrix operation, a normal module n2 (without matrix operation), and image display and other output applications.
Referring to fig. 5, a flowchart of an image processing operation of a target image processing module in the first image processing system, as shown in fig. 5, includes:
s101: performing image processing operation corresponding to the target image processing module on the input signal to obtain an output signal;
s102: judging whether matrix operation is included in image processing operation corresponding to the next image processing module of the target image processing module; if yes, go to S103;
s103: and expanding the output signal, and taking the expanded output signal as an input signal of the next image processing module.
In a specific implementation, each image module in the image processing system, that is, the target image processing module, executes the image processing operation of the module on the input signal to obtain an output signal, then determines whether matrix operation is included in the image processing operation corresponding to the next image processing module, if so, expands the output signal, then inputs the expanded output signal into the next image processing module for processing, and if not, directly inputs the output signal into the next image processing module for processing.
As a possible implementation, expanding the output signal includes: expanding a line synchronization signal and a field synchronization signal of the output signal; the image data is filled in an envelope common to the extended line sync signal and the field sync signal.
Wherein the expanded line synchronization signal is at least longer than the line synchronization signal before expansion
A pixel period, after expansionThe field sync signal is at least longer than the field sync signal before expansion>
And a row period, wherein N is the side length of the array data window required in the image processing operation corresponding to the next image processing module.
The length of the line synchronizing signal is increased on the basis of the actual line synchronizing signal, the minimum value of the increased length is half of the side length value of an array data window required in the image processing operation corresponding to the next image processing module, if the output of a certain stage of module is processed by a Gaussian filter module, the line synchronizing signal is expanded by adopting a 7-by-7 array matrix, and the envelope length of the expanded line synchronizing signal is at least 4 pixel periods longer than that of the actual line synchronizing signal. Taking the image size 768 by 576 as an example, the synchronization signal line expansion is performed for the 7 by 7 gaussian filter matrix operation, the expansion forward synchronization signal is shown in fig. 6, and the expansion backward synchronization signal is shown in fig. 7.
The length of the field synchronizing signal is increased on the basis of the actual field synchronizing signal, meanwhile, the line synchronizing signal is filled in the increased field synchronizing signal, the length of the increased field synchronizing signal is calculated by taking the image line period as a unit, the minimum value of the increased length is half of the side length value of an array data window required in the image processing operation corresponding to the next image processing module, if a certain stage of module outputs the field synchronizing signal and then is processed by a Gaussian filter module, the algorithm adopts a 7-by-7 array matrix, the field synchronizing signal is expanded, and the envelope length of the expanded field synchronizing signal is at least 4 line periods longer than that of the actual field synchronizing signal. Taking the image size 768 by 576 as an example, the field sync signal expansion is performed for the 7 by 7 gaussian filter matrix operation, the front field sync signal expansion is shown in fig. 8, and the back field sync signal expansion is shown in fig. 9.
Further, the image data is filled in an envelope common to the extended line sync signal and the field sync signal. The filled data may be different depending on the algorithm used by the later-stage algorithm module, and the embodiment is not particularly limited. As a possible embodiment, the envelope of the extended line sync signal together with the field sync signal may be filled with a preset value, for example 0. As another possible implementation, the last pixel value of the corresponding row may be filled in an envelope of the extended row synchronization signal and the field synchronization signal together. Of course, the first half of the data expanded by the gaussian filter may be filled with the last pixel value of each row, and the second half may be filled with 0.
According to the image processing method provided by the embodiment of the application, according to whether matrix operation is included in the image processing operation corresponding to the next image processing module, the output signal of the last image processing module is expanded and then input to the next image processing module, so that the algorithm module can process the image edge conveniently. The method simplifies the processing flow of the algorithm module containing matrix operation, ensures that the image edge data is the same as the image internal data, can be uniformly processed when the matrix operation is carried out, does not need to be additionally treated, can improve the image edge data processing quality to a certain extent, has simple expansion principle, is easy to realize, and requires very few design resources or logic resources. Therefore, the image processing method provided by the embodiment of the application improves the image edge data processing quality and reduces the processing complexity.
On the basis of the foregoing embodiment, as a preferred implementation manner, after the source image signal is acquired, the method further includes: expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid; correspondingly, inputting the source image signal into an image processing system for image processing, comprising: and inputting the expanded source image signal into an image processing system for image processing.
In order to facilitate the expansion of the field synchronizing signals by the modules at each level, the source image output signals can be processed, or the source image module is an autonomous controllable module, the source image module is processed, and the processing is that the line synchronizing signals are continuously output during the invalid period of the field synchronizing signals, so that the line synchronizing signals are not influenced by the field synchronizing signals, the line synchronizing signals are continuously output according to the line period from the source of the signal flow direction, and the line synchronizing signals are not required to be independently generated when the field synchronizing signals are required to be expanded by the subsequent modules, and the line synchronizing signals outside the envelope of the original actual field synchronizing signals are directly enveloped according to the requirement. The original source image signal is shown in fig. 10, and the extended source image signal is shown in fig. 11.
It can be understood that, although the line synchronization signal is outputted without interruption after the source image is outputted, it can still be determined by the envelope of the field synchronization signal, which line periods are valid lines and which line periods are invalid lines, so that the method has no influence on the common processing module without matrix operation.
After the output signal of the source image module is processed, the line synchronization signal is kept to be output uninterruptedly, and when a certain module at the later stage needs to expand the field synchronization signal, an expansion schematic diagram is shown in fig. 12.
Therefore, the embodiment expands the actual field synchronizing signal, the line synchronizing signal and the image data signal which are input originally, and increases the actual image edge data, so that when the algorithm module containing matrix operation is used for processing the image edge data, the expanded field synchronizing signal, line synchronizing signal and image data signal are used for filling the array window, thereby assisting the operation processing of the image edge, thus not only omitting the special edge processing module which is required by each algorithm module and is complex in control and easy to consume logic resources, but also avoiding the problems of serious distortion of the image edge and continuous dislocation of the line field synchronizing signal caused by the lack of the edge processing module.
The embodiment of the application discloses an image processing method, which improves the image edge data processing quality and reduces the processing complexity. Specifically, the method comprises the following steps:
acquiring a source image signal, and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
In this embodiment, a source image signal is acquired and input into an image processing system for image processing, the image processing system including a plurality of image processing modules connected in series, including an image processing module including a matrix operation and an image processing module not including a matrix operation.
Fig. 13 is a diagram of a second image processing system, as shown in fig. 13, including a source image output module, a normal module n1 (without matrix operation), an algorithm module m1 with matrix operation, an algorithm module m2 with matrix operation, an algorithm module m3 with matrix operation, a normal module n2 (without matrix operation), and image display and other output applications.
Referring to fig. 14, a flowchart of an image processing operation of a target image processing module in the second image processing system, as shown in fig. 14, includes:
s201: acquiring an output signal of a last image processing module of the target image processing module as an input signal;
s202: judging whether matrix operation is included in the image processing operation corresponding to the target image processing module; if yes, go to S203;
s203: and expanding the input signal, and performing image processing operation corresponding to the target image processing module on the expanded input signal to obtain an output signal of the target image processing module.
In a specific implementation, each image module in the image processing system, that is, the target image processing module, acquires an output signal of a previous image processing module as an input signal, determines whether matrix operation is included in image processing operations corresponding to the module, if so, expands the input signal, then executes the image processing operations of the module to obtain an output signal of the module, and if not, directly executes the image processing operations of the module on the input signal to obtain the output signal of the module.
As a possible implementation, expanding the input signal includes: expanding a line synchronization signal and a field synchronization signal of the input signal; the image data is filled in an envelope common to the extended line sync signal and the field sync signal.
Wherein the expanded line synchronization signal is at least longer than the line synchronization signal before expansion
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
And a row period, wherein N is the side length of the array data window required in the image processing operation corresponding to the next image processing module.
The length of the line synchronizing signal is increased on the basis of the actual line synchronizing signal, the minimum value of the increased length is half of the side length value of an array data window required in the image processing operation corresponding to the next image processing module, if the output of a certain stage of module is processed by a Gaussian filter module, the line synchronizing signal is expanded by adopting a 7-by-7 array matrix, and the envelope length of the expanded line synchronizing signal is at least 4 pixel periods longer than that of the actual line synchronizing signal.
The length of the field synchronizing signal is increased on the basis of the actual field synchronizing signal, meanwhile, the line synchronizing signal is filled in the increased field synchronizing signal, the length of the increased field synchronizing signal is calculated by taking the image line period as a unit, the minimum value of the increased length is half of the side length value of an array data window required in the image processing operation corresponding to the next image processing module, if a certain stage of module outputs the field synchronizing signal and then is processed by a Gaussian filter module, the algorithm adopts a 7-by-7 array matrix, the field synchronizing signal is expanded, and the envelope length of the expanded field synchronizing signal is at least 4 line periods longer than that of the actual field synchronizing signal.
Further, the image data is filled in an envelope common to the extended line sync signal and the field sync signal. The filled data may be different depending on the algorithm used by the later-stage algorithm module, and the embodiment is not particularly limited. As a possible embodiment, the envelope of the extended line sync signal together with the field sync signal may be filled with a preset value, for example 0. As another possible implementation, the last pixel value of the corresponding row may be filled in an envelope of the extended row synchronization signal and the field synchronization signal together. Of course, the first half of the data expanded by the gaussian filter may be filled with the last pixel value of each row, and the second half may be filled with 0.
According to the image processing method provided by the embodiment of the application, according to whether matrix operation is included in the image processing operation corresponding to the next image processing module, the output signal of the last image processing module is expanded and then input to the next image processing module, so that the algorithm module can process the image edge conveniently. The method simplifies the processing flow of the algorithm module containing matrix operation, ensures that the image edge data is the same as the image internal data, can be uniformly processed when the matrix operation is carried out, does not need to be additionally treated, can improve the image edge data processing quality to a certain extent, has simple expansion principle, is easy to realize, and requires very few design resources or logic resources. Therefore, the image processing method provided by the embodiment of the application improves the image edge data processing quality and reduces the processing complexity.
On the basis of the foregoing embodiment, as a preferred implementation manner, after the source image signal is acquired, the method further includes: expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid; correspondingly, inputting the source image signal into an image processing system for image processing, comprising: and inputting the expanded source image signal into an image processing system for image processing.
In order to facilitate the expansion of the field synchronizing signals by the modules at each level, the source image output signals can be processed, or the source image module is an autonomous controllable module, the source image module is processed, and the processing is that the line synchronizing signals are continuously output during the invalid period of the field synchronizing signals, so that the line synchronizing signals are not influenced by the field synchronizing signals, the line synchronizing signals are continuously output according to the line period from the source of the signal flow direction, and the line synchronizing signals are not required to be independently generated when the field synchronizing signals are required to be expanded by the subsequent modules, and the line synchronizing signals outside the envelope of the original actual field synchronizing signals are directly enveloped according to the requirement.
It can be understood that, although the line synchronization signal is outputted without interruption after the source image is outputted, it can still be determined by the envelope of the field synchronization signal, which line periods are valid lines and which line periods are invalid lines, so that the method has no influence on the common processing module without matrix operation.
Therefore, the embodiment expands the actual field synchronizing signal, the line synchronizing signal and the image data signal which are input originally, and increases the actual image edge data, so that when the algorithm module containing matrix operation is used for processing the image edge data, the expanded field synchronizing signal, line synchronizing signal and image data signal are used for filling the array window, thereby assisting the operation processing of the image edge, thus not only omitting the special edge processing module which is required by each algorithm module and is complex in control and easy to consume logic resources, but also avoiding the problems of serious distortion of the image edge and continuous dislocation of the line field synchronizing signal caused by the lack of the edge processing module.
In the following, an application embodiment provided in the present application is described, taking an image size 768 by 576 as an example, and referring to a 7 by 7 gaussian filter matrix operation, an image signal input to a gaussian filter algorithm module by an expansion front stage module is shown in fig. 15, and an image signal input to a gaussian filter algorithm module by an expansion rear stage module is shown in fig. 16. Fig. 17 shows a schematic diagram of the image edge matrix operation sliding window processing before signal expansion, fig. 17 shows an effective image, no image data exists in a portion between the dashed line frame and the solid line frame, fig. 18 shows a schematic diagram of the image edge matrix operation sliding window processing after signal expansion, fig. 18 shows an effective image in an inner solid line frame, and an outer solid line frame shows a portion where image data exists after filling the image data.
The method of pre-expanding input signals is adopted, the actual image edge part is expanded and amplified and then is input to the algorithm module containing matrix operation, so that when the algorithm module processes the image edge data by using the matrix operation, the flow is simple, convenient and reliable, the whole image operation processing flow is simplified and unified, the problem of coherent dislocation of line-field synchronous signals can be avoided when the image edge specific processing logic required by each algorithm module is omitted, the image quality can be improved to a certain extent, the design principle is simple, the realization is easy, the resources are saved, the reliability is improved, the defect of the traditional image edge processing method is overcome, the non-differential pipeline processing flow of the matrix operation module and the non-matrix operation module can be realized to a certain extent, the system unification is facilitated, and the efficiency is improved.
The method for expanding the actual signal is adopted to expand and amplify the image edge part for matrix operation required by the related algorithm module, so that the flow is simplified and unified when the matrix operation processing is carried out on the whole image, the image edge data is not required to be considered independently, the whole process is simple and reliable, and the image edge pixel quality can be improved to a certain extent. The present embodiment can be applied to a scene of processing an image using an ASIC (Application Specific Integrated Circuit ) or an FPGA (Field-Programmable Gate Array, field programmable logic device), is easy to implement, and consumes little logic resources or design resources.
An image processing apparatus according to an embodiment of the present application is described below, and an image processing apparatus described below and an image processing method described above may be referred to each other.
The image processing device is used for acquiring a source image signal and inputting the source image signal into the image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
referring to fig. 19, a block diagram of a target image processing module in a first image processing system, as shown in fig. 19, includes:
A first processing unit 101, configured to perform an image processing operation corresponding to the target image processing module on an input signal to obtain an output signal;
a first determining unit 102, configured to determine whether a matrix operation is included in an image processing operation corresponding to a next image processing module of the target image processing module; if yes, starting the workflow of the first expansion unit;
the first expansion unit 103 is configured to expand the output signal, and take the expanded output signal as an input signal of the next image processing module.
According to the image processing device provided by the embodiment of the application, according to whether matrix operation is included in the image processing operation corresponding to the next image processing module, the output signal of the last image processing module is expanded and then input to the next image processing module, so that the algorithm module can process the image edge conveniently. The method simplifies the processing flow of the algorithm module containing matrix operation, ensures that the image edge data is the same as the image internal data, can be uniformly processed when the matrix operation is carried out, does not need to be additionally treated, can improve the image edge data processing quality to a certain extent, has simple expansion principle, is easy to realize, and requires very few design resources or logic resources. Therefore, the image processing device provided by the embodiment of the application improves the image edge data processing quality and reduces the processing complexity.
On the basis of the above embodiment, as a preferred implementation manner, the first expansion unit 103 includes:
a first expansion subunit for expanding the line synchronization signal and the field synchronization signal of the output signal;
a first padding subunit for padding image data in an envelope common to the extended line sync signal and the field sync signal.
Based on the above embodiment, as a preferred embodiment, the extended line synchronization signal is at least longer than the line synchronization signal before extension
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
And a row period, wherein N is the side length of the array data window required in the image processing operation corresponding to the next image processing module.
On the basis of the above embodiment, as a preferred implementation manner, the first filling subunit is specifically configured to: and filling preset values in the envelope of the extended line synchronous signals and the field synchronous signals.
On the basis of the above embodiment, as a preferred implementation manner, the first filling subunit is specifically configured to: and filling the last pixel value of the corresponding row in the envelope which is common to the extended row synchronizing signal and the field synchronizing signal.
On the basis of the above embodiment, as a preferred implementation manner, the image processing apparatus is specifically configured to: acquiring a source image signal; expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid; and inputting the expanded source image signal into an image processing system for image processing.
An image processing apparatus according to an embodiment of the present application is described below, and an image processing apparatus described below and an image processing method described above may be referred to each other.
The image processing device is used for acquiring a source image signal and inputting the source image signal into the image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
referring to fig. 20, a block diagram of a target image processing module in a second image processing system, as shown in fig. 20, includes:
an acquisition unit 201 for acquiring an output signal of a previous image processing module of the target image processing module as an input signal;
a second judging unit 202, configured to judge whether a matrix operation is included in the image processing operation corresponding to the target image processing module; if yes, starting the workflow of the second expansion unit;
The second spreading unit 203 is configured to spread the input signal;
and the second processing unit 204 is configured to perform an image processing operation corresponding to the target image processing module on the expanded input signal to obtain an output signal of the target image processing module.
According to the image processing device provided by the embodiment of the application, according to whether matrix operation is included in the image processing operation corresponding to the next image processing module, the output signal of the last image processing module is expanded and then input to the next image processing module, so that the algorithm module can process the image edge conveniently. The method simplifies the processing flow of the algorithm module containing matrix operation, ensures that the image edge data is the same as the image internal data, can be uniformly processed when the matrix operation is carried out, does not need to be additionally treated, can improve the image edge data processing quality to a certain extent, has simple expansion principle, is easy to realize, and requires very few design resources or logic resources. Therefore, the image processing device provided by the embodiment of the application improves the image edge data processing quality and reduces the processing complexity.
On the basis of the above embodiment, as a preferred implementation manner, the second expansion unit 203 includes:
A second expansion subunit for expanding the line synchronization signal and the field synchronization signal of the input signal;
and a second padding subunit for padding image data in an envelope common to the extended line sync signal and the field sync signal.
Based on the above embodiment, as a preferred embodiment, the extended line synchronization signal is at least longer than the line synchronization signal before extension
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
And a row period, wherein N is the side length of the array data window required in the image processing operation corresponding to the next image processing module.
On the basis of the above embodiment, as a preferred implementation manner, the second filling subunit is specifically configured to: and filling preset values in the envelope of the extended line synchronous signals and the field synchronous signals.
On the basis of the above embodiment, as a preferred implementation manner, the second filling subunit is specifically configured to: and filling the last pixel value of the corresponding row in the envelope which is common to the extended row synchronizing signal and the field synchronizing signal.
On the basis of the above embodiment, as a preferred implementation manner, the image processing apparatus is specifically configured to: acquiring a source image signal; expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid; and inputting the expanded source image signal into an image processing system for image processing.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Based on the hardware implementation of the program modules, and in order to implement the method of the embodiments of the present application, the embodiments of the present application further provide an electronic device, fig. 21 is a structural diagram of an electronic device, and as shown in fig. 21, the electronic device includes:
a communication interface 1 capable of information interaction with other devices such as network devices and the like;
and the processor 2 is connected with the communication interface 1 to realize information interaction with other devices and is used for executing the image processing method provided by one or more technical schemes when running the computer program. And the computer program is stored on the memory 3.
Of course, in practice, the various components in the electronic device are coupled together by a bus system 4. It will be appreciated that the bus system 4 is used to enable connected communications between these components. The bus system 4 comprises, in addition to a data bus, a power bus, a control bus and a status signal bus. But for clarity of illustration the various buses are labeled as bus system 4 in fig. 21.
The memory 3 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program for operating on an electronic device.
The present embodiment also provides a storage medium, i.e. a computer storage medium, in particular a computer readable storage medium, for example comprising a memory 3 storing a computer program executable by the processor 2 for performing the steps of the method described above. The computer readable storage medium may be a Memory such as FRAM (magnetic random Access Memory), ROM (Read Only Memory), PROM (programmable Read Only Memory, programmable Read-Only Memory), EPROM (erasable programmable Read Only Memory, erasable Programmable Read-Only Memory), EEPROM (electrically erasable programmable Read Only Memory, electrically Erasable Programmable Read-Only Memory), flash Memory, magnetic surface Memory, optical disk, or CD-ROM (compact disk Read Only Memory).
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (18)
1. An image processing method, comprising:
acquiring a source image signal, and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
the image processing operation of the target image processing module in the image processing system includes:
performing image processing operation corresponding to the target image processing module on the input signal to obtain an output signal;
judging whether matrix operation is included in image processing operation corresponding to the next image processing module of the target image processing module;
if yes, expanding the output signal, and taking the expanded output signal as an input signal of the next image processing module;
wherein expanding the output signal comprises:
expanding a line synchronization signal and a field synchronization signal of the output signal; wherein the expanded line synchronization signal is at least longer than the line synchronization signal before expansion
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
A row period, wherein N is the side length of an array data window required in the image processing operation corresponding to the next image processing module;
The image data is filled in an envelope common to the extended line sync signal and the field sync signal.
2. The image processing method according to claim 1, wherein the filling the image data in an envelope common to the extended line synchronization signal and the field synchronization signal includes:
and filling preset values in the envelope of the extended line synchronous signals and the field synchronous signals.
3. The image processing method according to claim 1, wherein the filling the image data in an envelope common to the extended line synchronization signal and the field synchronization signal includes:
and filling the last pixel value of the corresponding row in the envelope which is common to the extended row synchronizing signal and the field synchronizing signal.
4. The image processing method according to claim 1, wherein after the acquisition of the source image signal, further comprising:
expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid;
correspondingly, inputting the source image signal into an image processing system for image processing, comprising:
and inputting the expanded source image signal into an image processing system for image processing.
5. An image processing method, comprising:
acquiring a source image signal, and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
the image processing operation of the target image processing module in the image processing system includes:
acquiring an output signal of a last image processing module of the target image processing module as an input signal;
judging whether matrix operation is included in the image processing operation corresponding to the target image processing module;
if yes, expanding the input signal, and performing image processing operation corresponding to the target image processing module on the expanded input signal to obtain an output signal of the target image processing module;
wherein expanding the input signal comprises:
expanding a line synchronization signal and a field synchronization signal of the input signal; wherein the expanded line synchronization signal is at least longer than the line synchronization signal before expansion
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
N is the side length of the array data window required in the image processing operation corresponding to the target image processing module;
The image data is filled in an envelope common to the extended line sync signal and the field sync signal.
6. The image processing method according to claim 5, wherein the filling the image data in an envelope common to the extended line synchronization signal and the field synchronization signal includes:
and filling preset values in the envelope of the extended line synchronous signals and the field synchronous signals.
7. The image processing method according to claim 5, wherein the filling the image data in an envelope common to the extended line synchronization signal and the field synchronization signal includes:
and filling the last pixel value of the corresponding row in the envelope which is common to the extended row synchronizing signal and the field synchronizing signal.
8. The image processing method according to claim 5, further comprising, after the acquisition of the source image signal:
expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid;
correspondingly, inputting the source image signal into an image processing system for image processing, comprising:
and inputting the expanded source image signal into an image processing system for image processing.
9. An image processing device, wherein the image processing device is used for acquiring a source image signal and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
the target image processing module in the image processing system includes:
the first processing unit is used for performing image processing operation corresponding to the target image processing module on the input signal to obtain an output signal;
a first judging unit, configured to judge whether a matrix operation is included in an image processing operation corresponding to a next image processing module of the target image processing module; if yes, starting the workflow of the first expansion unit;
the first expansion unit is used for expanding the output signal and taking the expanded output signal as an input signal of the next image processing module;
wherein the first expansion unit includes:
a first expansion subunit for expanding the line synchronization signal and the field synchronization signal of the output signal;
a first padding subunit for padding image data in an envelope common to the extended line sync signal and the field sync signal;
Wherein the expanded line synchronization signal is at least longer than the line synchronization signal before expansion
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
And a row period, wherein N is the side length of the array data window required in the image processing operation corresponding to the next image processing module.
10. The image processing apparatus according to claim 9, wherein the first filling subunit is specifically configured to: and filling preset values in the envelope of the extended line synchronous signals and the field synchronous signals.
11. The image processing apparatus according to claim 9, wherein the first filling subunit is specifically configured to: and filling the last pixel value of the corresponding row in the envelope which is common to the extended row synchronizing signal and the field synchronizing signal.
12. The image processing apparatus according to claim 9, wherein the image processing apparatus is specifically configured to: acquiring a source image signal; expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid; and inputting the expanded source image signal into an image processing system for image processing.
13. An image processing device, wherein the image processing device is used for acquiring a source image signal and inputting the source image signal into an image processing system for image processing; the image processing system comprises a plurality of image processing modules which are connected in series;
the target image processing module in the image processing system includes:
an acquisition unit configured to acquire an output signal of a previous image processing module of the target image processing module as an input signal;
a second judging unit, configured to judge whether matrix operation is included in the image processing operation corresponding to the target image processing module; if yes, starting the workflow of the second expansion unit;
the second expansion unit is used for expanding the input signal;
the second processing unit is used for performing image processing operation corresponding to the target image processing module on the expanded input signal to obtain an output signal of the target image processing module;
wherein the second expansion unit includes:
a second expansion subunit for expanding the line synchronization signal and the field synchronization signal of the input signal;
a second padding subunit for padding image data in an envelope common to the extended line sync signal and the field sync signal;
Wherein the expanded line synchronization signal is at least longer than the line synchronization signal before expansion
The field sync signal after expansion is at least longer than the field sync signal before expansion by one pixel period>
And the line period N is the side length of the array data window required in the image processing operation corresponding to the target image processing module.
14. The image processing apparatus according to claim 13, wherein the second filling subunit is specifically configured to: and filling preset values in the envelope of the extended line synchronous signals and the field synchronous signals.
15. The image processing apparatus according to claim 13, wherein the second filling subunit is specifically configured to: and filling the last pixel value of the corresponding row in the envelope which is common to the extended row synchronizing signal and the field synchronizing signal.
16. The image processing apparatus according to claim 13, wherein the image processing apparatus is specifically configured to: acquiring a source image signal; expanding the source image signal to enable the expanded source image signal to continuously output a line synchronization signal in a period when the field synchronization signal is invalid; and inputting the expanded source image signal into an image processing system for image processing.
17. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the image processing method according to any one of claims 1 to 8 when executing said computer program.
18. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the image processing method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211352901.8A CN115760600B (en) | 2022-11-01 | 2022-11-01 | Image processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211352901.8A CN115760600B (en) | 2022-11-01 | 2022-11-01 | Image processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115760600A CN115760600A (en) | 2023-03-07 |
| CN115760600B true CN115760600B (en) | 2023-04-25 |
Family
ID=85354928
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211352901.8A Active CN115760600B (en) | 2022-11-01 | 2022-11-01 | Image processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115760600B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110771159A (en) * | 2018-09-26 | 2020-02-07 | 深圳市大疆创新科技有限公司 | Image processing system and image processing method |
| CN115103097A (en) * | 2022-06-17 | 2022-09-23 | 深圳市雨滴科技有限公司 | Image processing device, method, computer equipment and readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR0169541B1 (en) * | 1993-06-30 | 1999-03-20 | 이리마지리 쇼우이찌로 | Image processing device and method therefor and electronic device having image processing device |
| US6750835B2 (en) * | 1999-12-27 | 2004-06-15 | Semiconductor Energy Laboratory Co., Ltd. | Image display device and driving method thereof |
| CN114519690A (en) * | 2020-11-19 | 2022-05-20 | 京东方科技集团股份有限公司 | Image processing method and device, image detection method and system, and storage medium |
-
2022
- 2022-11-01 CN CN202211352901.8A patent/CN115760600B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110771159A (en) * | 2018-09-26 | 2020-02-07 | 深圳市大疆创新科技有限公司 | Image processing system and image processing method |
| CN115103097A (en) * | 2022-06-17 | 2022-09-23 | 深圳市雨滴科技有限公司 | Image processing device, method, computer equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115760600A (en) | 2023-03-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11222240B2 (en) | Data processing method and apparatus for convolutional neural network | |
| CN109064399B (en) | Image super-resolution reconstruction method and system, computer device and storage medium thereof | |
| US8724923B2 (en) | Image processing apparatus and image processing method for correcting distorted image data | |
| CN105306866A (en) | Frame rate conversion method and device | |
| CN109325901B (en) | Method and device for realizing image processing | |
| US10158839B2 (en) | Non-transitory storage medium and apparatus for processing source image to generate target image | |
| CN115760600B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN103578077A (en) | Image zooming method and related device | |
| US20110102681A1 (en) | Image converting method and apparatus therefor based on motion vector-sharing | |
| JP2002077612A (en) | Image processing method and image processing apparatus | |
| CN113658049A (en) | Image transposition method, equipment and computer readable storage medium | |
| US8947436B2 (en) | Method, apparatus and system for dense graph simplification, and recording medium for the same | |
| US20070040842A1 (en) | Buffer memory system and method | |
| CN105491363A (en) | LED panel pixel correction method and apparatus | |
| CN116862756B (en) | Line data processing method, line buffer, electronic device and storage medium | |
| WO2005065027A2 (en) | Method and system of updating a memory of a color display | |
| CN114727051B (en) | Media resource transmission device, system and method | |
| US11972504B2 (en) | Method and system for overlapping sliding window segmentation of image based on FPGA | |
| CN108632499B (en) | Image signal processor and time sequence generating device thereof | |
| CN117011146B (en) | Image scaling method and device, electronic equipment and storage medium | |
| CN115829838B (en) | Image edge-expanding circuit, chip and method | |
| US20150229876A1 (en) | Image processor, image-processing method and program | |
| US10672107B2 (en) | Image processing apparatus and method of controlling the same | |
| JP2007058513A (en) | Controller and method, and program | |
| CN117479029A (en) | Image sensor and output control method and device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |