CN115619630A - Image processing method and apparatus - Google Patents

Abstract

The embodiment of the invention discloses an image processing method, which is executed in an image processor, wherein the image processor comprises a plurality of layer processing channels, each layer processing channel is provided with a scaling module, and the image processing method comprises the following steps: receiving an input image to be processed; obtaining a scaling mode corresponding to each image to be processed; determining a processing sequence corresponding to the target image to be processed according to the corresponding scaling mode; at least performing caching processing and zooming processing on the target image to be processed according to the processing sequence to obtain a zoomed image; and the scaling processing is carried out by the same scaling module. The invention also discloses an image processing device. The invention can reduce the resource consumption in the image processing process.

Description

Image processing method and apparatus

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an image processing method and an image processing apparatus.

Background

In a video processing process, a video image needs to be scaled, a Scaler (scaling module) is a scaling engine widely used in an image processing device, and in an image processor based on an FPGA (field programmable gate array), scaling by the Scaler often consumes a large amount of resources, for example, a large amount of Lut (lookup table) resources and REG (register) resources, and the more the number of scalers performing scaling, the more the consumed resources.

Therefore, how to reduce the resource consumption in the image scaling process becomes a problem to be solved in the present invention.

Disclosure of Invention

Therefore, to overcome the prior art, the present invention proposes an image processing method and an image processing apparatus to reduce resource consumption during image scaling processing.

Specifically, in a first aspect, an embodiment of the present invention discloses an image processing method, which is executed in an image processor, where the image processor includes a plurality of layer processing channels, each layer processing channel is provided with a scaling module, and the image processing method includes:

receiving an input image to be processed;

acquiring a scaling mode corresponding to each image to be processed;

determining a processing sequence corresponding to the target image to be processed according to the corresponding scaling mode;

at least performing caching processing and zooming processing on the target image to be processed according to the processing sequence to obtain a zoomed image; and the scaling processing is carried out by the same scaling module.

In the existing scaling processing of a single layer, generally, a scaling module is respectively and fixedly arranged in front of and behind a memory, the scaling module in front of the memory only performs scaling processing, and the scaling module behind the memory only performs scaling processing, and scaling of an image in any proportion can be realized through a fixed processing sequence. The image processing method disclosed by the embodiment of the invention only sets one zooming module in each layer processing channel, and performs zooming processing on a target image to be processed through the same zooming module, thereby reducing the number of zooming modules, ensuring the zooming function of the image in any proportion and reducing the consumption of resources.

In one embodiment of the invention, each of the scaling modules comprises: a first direction scaling unit and a second direction scaling unit;

the scaling by the same scaling module includes:

performing first-direction scaling processing by the first-direction scaling unit to output an image after the first-direction scaling processing, and/or

Performing second direction scaling processing through the corresponding second direction scaling unit to output an image subjected to the second direction scaling processing;

the caching process comprises the following steps: and caching one of the target image to be processed, the image subjected to the first-direction scaling processing and the image subjected to the second-direction scaling processing to a memory, and obtaining the image subjected to the caching processing.

By separating the first direction scaling unit and the second direction scaling unit in each scaling module, the scaling module in at least one direction can be used for scaling the target image to be processed in at least one direction, so that the image processing method is flexibly applicable to scaling the image in any proportion and has the characteristics of flexibility and comprehensiveness.

In an embodiment of the present invention, each layer processing channel further includes: the image processing device comprises an input module, a first selector, a second selector and a third selector, wherein the input module is used for receiving the target image to be processed;

the determining a processing sequence corresponding to the target image to be processed according to the corresponding scaling mode comprises:

receiving, by the first selector, a first control signal corresponding to the scaling manner, and selecting one of the target image to be processed, the image subjected to the second-direction scaling processing, or the image subjected to the cache processing based on the first control signal, and transmitting the selected one to the first-direction scaling unit; and

receiving, by the second selector, a second control signal corresponding to the scaling manner, and selecting, based on the second control signal, one of the target to-be-processed image, the image scaled in the first direction, or the image cached in the first direction, and transmitting the selected image to a second direction scaling unit; and

and receiving a third control signal corresponding to the scaling mode by the third selector, and selecting one of the target image to be processed, the first-direction scaled image or the second-direction scaled image to be transmitted to the memory based on the third control signal, so as to determine the processing order.

In the image processing method disclosed in this embodiment, a first selector, a second selector, and a third selector are arranged in each layer processing channel, and according to different scaling modes, the first selector, the second selector, and the third selector are configured to receive a corresponding first control signal, a corresponding second control signal, and a corresponding third control signal, respectively, and select a path of processed image transmission based on the corresponding control signal, and determine a corresponding processing order from respective input and output data.

In an embodiment of the present invention, the determining a processing order corresponding to the target to-be-processed image according to the corresponding scaling manner includes:

the zooming mode is a first-direction zooming-in and second-direction zooming-out mode, a first-direction zooming-in and second-direction zooming-in mode or a first-direction zooming-out and second-direction zooming-out mode, the first selector selects to transmit the cached images to the first-direction zooming unit based on the first control signal; or

The zooming mode is a first-direction zooming and second-direction zooming, or a first-direction zooming and second-direction zooming, and the second direction is unchanged, and the first selector selects to transmit the target image to be processed to the first-direction zooming unit based on the first control signal; or

The scaling mode is a first direction unchanged and a second direction enlarged, and the first selector selects to transmit one of the buffered image or the target image to be processed to the first direction scaling unit based on the first control signal.

No matter what scaling mode, the first selector only selects to transmit one of the cached image or the target image to be processed to the first direction scaling unit, and does not receive the image scaled in the second direction, so that the scheduling is more convenient, the output transmission link is simpler, and the resource occupation is less.

In an embodiment of the present invention, the determining a processing order corresponding to the target image to be processed according to the corresponding scaling manner further includes:

and generating the first control signal, the second control signal and the third control signal according to the corresponding scaling modes and outputting the first control signal, the second control signal and the third control signal to the first selector, the second selector and the third selector respectively.

In an embodiment of the present invention, the determining a processing order corresponding to the target to-be-processed image according to the corresponding scaling manner includes:

the zooming mode is a first-direction zooming-in mode and a second-direction zooming-out mode, a first-direction zooming-in mode and a second-direction zooming-in mode are unchanged, or a first-direction zooming-in mode and a second-direction zooming-out mode are unchanged, and the first-direction zooming processing is performed after the caching processing; or

The zooming mode is zooming in the first direction and zooming out in the second direction, zooming out in the first direction and zooming in the second direction or zooming out in the first direction and zooming out in the second direction and not changing, and the zooming processing in the first direction is performed before the caching processing; or

The scaling mode is that the first direction is unchanged and the second direction is enlarged, and the first direction scaling processing is performed before the caching processing or the caching processing is performed before the first direction scaling processing.

Different processing sequences are set through different scaling modes, the method is applicable to various scaling modes, scaling processing in the first direction is performed after caching processing or before caching processing and before scaling processing in the second direction, scheduling is more convenient, an output transmission link is simpler, and less resources are occupied.

In an embodiment of the present invention, the determining a processing order corresponding to the target to-be-processed image according to the corresponding scaling manner includes:

the zooming mode is a first-direction zooming-in mode and a second-direction zooming-out mode, the first-direction zooming-in mode and the second-direction zooming-out mode are unchanged, or the first-direction zooming-out mode and the second-direction zooming-out mode are unchanged, and the processing sequence is that the second-direction zooming processing, the cache processing and the first-direction zooming processing are carried out in sequence; or alternatively

The zooming mode is a first-direction zooming and a second-direction zooming, and the processing sequence is that the cache processing, the first-direction zooming and the second-direction zooming are carried out in sequence; or alternatively

The scaling mode is a first-direction scaling mode and a second-direction scaling mode, and the processing sequence is that the first-direction scaling processing, the second-direction scaling processing and the cache processing are performed in sequence;

the zooming mode is zooming in a first direction and zooming in a second direction, and the processing sequence is zooming in the first direction, caching and zooming in the second direction in sequence; or

The zooming mode is a first-direction constant zooming mode and a second-direction zooming mode, and the processing sequence is that the first-direction zooming processing, the caching processing and the second-direction zooming processing are carried out in sequence or the caching processing, the first-direction processing and the second-direction zooming processing are carried out in sequence; or

The scaling mode is a first-direction scaling mode and a second-direction scaling mode, and the processing sequence is that the first-direction scaling processing, the caching processing and the second-direction scaling processing are performed in sequence or the first-direction scaling processing, the second-direction processing and the caching processing are performed in sequence.

Different processing sequences are set through different zooming modes, the method is suitable for various zooming modes, and the cached images are not amplified, so that the bandwidth consumption of caching processing is reduced.

In an embodiment of the present invention, each layer processing channel further includes: the system comprises an input module, a first target selector and a second target selector, wherein the input module is used for receiving the target image to be processed;

the determining a processing sequence corresponding to the target image to be processed according to the corresponding scaling mode comprises:

receiving, by the first target selector, a first selection signal corresponding to the scaling manner, and selecting, based on the first selection signal, one of the target to-be-processed image and the cached image to transmit to one of the first direction scaling unit and the second direction scaling unit; and

receiving a second selection signal corresponding to the scaling mode by the second target selector, and selecting one of the image subjected to scaling processing in the first direction and the image subjected to scaling processing in the second direction and the target image to be processed based on the second selection signal and transmitting the selected one to the memory, thereby determining the processing order;

wherein the scaling mode is as follows: the first direction is reduced and the second direction is unchanged, the first direction is enlarged and the second direction is unchanged, the first direction is unchanged and the second direction is reduced, or the first direction is unchanged and the second direction is enlarged.

In the image processing method disclosed in this embodiment, when the scaling manner of the image is only one direction and the other direction is unchanged, the processing of the scaling unit corresponding to the unchanged direction is skipped by controlling different selections of the first target selector and the second target selector, so that the processing time can be saved, and the effect of reducing resource consumption can be achieved.

In an embodiment of the present invention, the image processing method further includes:

and superposing the zoomed images output by the layer processing channels to obtain a pre-monitoring picture, and outputting the pre-monitoring picture to an external display for display.

In a second aspect, an embodiment of the present invention discloses an image processing apparatus, including: an image processor and a memory connected to the image processor; wherein the image processor is configured to perform the image processing method according to any one of the above embodiments.

In a third aspect, an embodiment of the present invention discloses another image processing apparatus, including: the device comprises an image processor, a memory connected with the image processor and a controller connected with the image processor; the image processor is configured to execute the image processing method of the foregoing embodiment, and the controller is configured to generate the first control signal, the second control signal, and the third control signal according to the scaling manner, output the first control signal to the first selector, output the second control signal to the second selector, and output the third control signal to the third selector.

As can be seen from the above, the embodiments of the present invention may achieve one or more of the following advantages: only one zooming module is arranged in a single layer processing channel, zooming processing of any proportion can be realized through the same zooming module for a target image to be processed, the number of the zooming modules is reduced, and the zooming function of any proportion of the image is guaranteed, so that the occupancy rate of the zooming module on the image processing equipment resource is reduced, the image processing efficiency is improved, and the resource consumption is reduced.

Other aspects and features of the present invention will become apparent from the following detailed description, which proceeds with reference to the accompanying drawings. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the invention. It should be further understood that the drawings are not necessarily drawn to scale and that, unless otherwise indicated, they are merely intended to conceptually illustrate the structures and procedures described herein.

Drawings

The following detailed description of embodiments of the invention will be made with reference to the accompanying drawings.

Fig. 1 is a flowchart illustrating a step of an image processing method according to an embodiment of the present invention.

Fig. 2 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.

Fig. 3 is a diagram of a connection relationship between a controller and a first selector, a second selector, and a third selector in another image processing apparatus according to an embodiment of the present invention.

[ description of reference ]

10: an image processor; 11: a layer processing channel; 111: scaling module, 1111: a first direction scaling unit; 1112: a second direction scaling unit; 112: an input module; 1131: a first selector; 1132: a second selector; 1133: a third selector; 20: a memory; 30: a controller; 200: an image processing apparatus; 300: an image processing apparatus; m100: provided is an image processing method.

S1-S4: the image processing method comprises the following steps.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, embodiments accompanying figures are described in detail below.

In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not a whole embodiment. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be noted that the division of the embodiments of the present invention is only for convenience of description and should not be construed as a limitation, and features of various embodiments may be combined and referred to each other without contradiction.

In an existing image processor, when scaling processing is performed on a single layer, it is generally required to fixedly set a scaling module respectively in front of and behind a memory, where each scaling module includes a vertical scaling unit V Scaler and a horizontal scaling unit H Scaler for scaling processing in vertical and horizontal directions, respectively, a scaling module located in front of the memory, such as Scaler 0, is only used for reduction processing, and a scaling module located behind the memory, such as Scaler 1, is only used for enlargement processing, so as to ensure minimum bandwidth consumption on the memory and smooth transition when a video is dynamically adjusted at any scaling ratio. In the process of scaling, regardless of the scaling mode, the processing path of the image is always: input image → Scaler 0 → memory → Scaler 1 → output image, so that the image entering the memory does not exceed the size of the original input image, and the scaling of the input image in any proportion can be realized.

However, the scaling module itself occupies larger consumed resources as a core module of the scaling process, and the more the scaling module is, the more consumed resources are. If the number of scaling modules is directly reduced, for example, if the image is scaled only by Scaler 0, only the image can be scaled down vertically and horizontally, and if the image is scaled only by Scaler 1, only the image can be scaled up vertically and horizontally, which results in a limited scaling ratio of the image.

To this end, an embodiment of the present invention provides an image processing apparatus 200, as shown in fig. 2, the image processing apparatus 200 includes an image processor 10 and a memory 20, the image processor 10 is, for example, a Programmable logic device, specifically, an FPGA (Field-Programmable Gate Array) device or other similar logic devices. The memory 20 is used for caching images, the memory 20 is, for example, DDR, and the memory 20 may also be a memory inside FPGA when the resource capacity allows, which is not limited in this embodiment. When the image to be processed is large, a plurality of memories 20 may be provided according to actual requirements to meet the processing requirements.

Referring to fig. 2, the image processor 10 includes at least one layer processing channel 11 (only one layer processing channel is shown in fig. 2, but this embodiment is not limited to this, and for example, there may also be multiple layer processing channels 11), and each layer processing channel 11 is provided with a scaling module 111.

The image processor 10 is configured to execute an image processing method, as shown in fig. 1, an embodiment of the present invention discloses an image processing method M100, where the image processing method M100 includes, for example:

s1: receiving an input image to be processed;

s2: acquiring a scaling mode corresponding to each image to be processed;

s3: determining a processing sequence corresponding to the target image to be processed according to the corresponding scaling mode;

s4: at least performing caching processing and zooming processing on the target image to be processed according to the processing sequence to obtain a zoomed image; wherein, the scaling process is performed by the same scaling module 111.

The number of the to-be-processed images received in step S1 may be multiple, each to-be-processed image corresponds to one layer processing channel 11, and the target to-be-processed image mentioned in step S3 may refer to any one of the multiple to-be-processed images. Specifically, the scaling manner mentioned in step S2 may be obtained by directly inputting control information (for example, directly inputting a scaling ratio of an image), or may be obtained by obtaining a current resolution and a target resolution of an image to be processed, and then comparing the current resolution and the target resolution. For example, the initial resolution is 128 × 512 pixels, the target resolution is 1024 × 256 pixels, and comparison is performed, because 128< -1024, the zoom manner in the horizontal direction is zoom-in, because 512>256, the zoom manner in the vertical direction is zoom-out, and the zoom manner of the image is zoom-out vertically and zoom-in horizontally. Thus, the scaling method can be classified into the following cases: both directions zoom out (vertical and horizontal zoom out), both directions zoom in (vertical and horizontal zoom in and horizontal zoom out), one direction zoom out and the other direction zoom in (vertical and horizontal zoom in/vertical and horizontal zoom out), only one direction zoom the other direction invariant (vertical and horizontal/vertical zoom in and horizontal/vertical and horizontal zoom out/vertical and horizontal zoom in).

In this embodiment, only one scaling module 111 is needed to be arranged in each layer processing channel 11 of the image processing apparatus 200, and for a target image to be processed, scaling processing of any proportion on the target image to be processed can be implemented only by one scaling module 111 in the layer processing channel 11 corresponding to the target image to be processed according to the above different scaling modes.

In an embodiment of the present invention, referring to fig. 2, each scaling module 111 includes a first direction scaling unit 1111 and a second direction scaling unit 1112, and the scaling in the aforementioned step S4 by the same scaling module 111 includes: the first direction scaling process is performed by the first direction scaling unit 1111 to output the first direction scaled image, and/or the second direction scaling process is performed by the corresponding second direction scaling unit 1112 to output the second direction scaled image. The caching process includes, for example, caching one of the target to-be-processed image, the first-direction processed image, and the second-direction processed image in the memory 20, and obtaining the cached image, where specific caching contents are selected differently according to different scaling modes.

In the above embodiment, at least one of the first direction scaling unit 1111 and the second direction scaling unit 1112 in the scaling module 111 performs scaling processing in at least one direction on the target image to be processed. Specifically, the first direction scaling unit 1111 is configured to perform scaling processing in a first direction on the input image pair, and the second direction scaling unit 1112 is configured to perform scaling processing in a second direction on the input image pair. The first direction is, for example, a vertical direction, the second direction is, for example, a horizontal direction, the first direction scaling unit 1111 is a vertical scaling unit, and the second direction scaling unit 1112 is a horizontal scaling unit; of course, the opposite is also possible, i.e. the first direction is the horizontal direction, the second direction is the vertical direction, the first direction scaling unit 1111 is the horizontal scaling unit, and the second direction 1112 is the vertical scaling unit. For convenience of understanding, the examples in the embodiments of the present invention are all described with the first direction being a vertical direction and the second direction being a horizontal direction.

As in the foregoing prior art, the scaling modules and the memories have fixed processing sequences, the positions of the scaling units in two directions in each scaling module relative to the memory are also fixed, and when only one scaling module is provided, only processing of enlarging an image in two directions or reducing the image in two directions can be performed, resulting in limited image scaling. On the basis that only one scaling module 111 is arranged in one layer processing channel 11, the image processing apparatus 200 disclosed in this embodiment can complete processing of all scaling modes through the first direction scaling unit 1111 and the second direction scaling unit 1112 in the same scaling module 111, and the image processing apparatus 200 disclosed in this embodiment can be flexibly applied to scaling of an image in any proportion, and has the characteristics of flexibility and comprehensiveness.

In an embodiment of the present invention, referring to fig. 2, each layer processing channel 11 further includes, for example, an input module 112, a first selector 1131, a second selector 1132, and a third selector 1133.

The aforementioned step S1 is executed in the input module 112, for example, and the input module 112 further executes processing such as cropping of the image after executing the step S1, so as to facilitate scaling of the subsequent steps.

For example, the step S3 includes:

s311: the first selector 1131 receives the first control signal corresponding to the scaling manner, and selects one of the target to-be-processed image, the post-image subjected to the second-direction scaling processing, and the image subjected to the buffering processing based on the first control signal corresponding to the scaling manner, and transmits the selected one to the first-direction scaling unit 1111. The second selector 1132 receives the second control signal corresponding to the scaling manner, and selects one of the target to-be-processed image, the post-image subjected to the scaling processing in the first direction, and the image subjected to the buffering processing based on the second control signal corresponding to the scaling manner, and transmits the selected one to the second direction scaling unit 1112. And the third selector 1133 receives the third control signal corresponding to the scaling mode, and selects one of the target to-be-processed image, the post-image subjected to the second-direction scaling processing, and the image subjected to the first-direction scaling processing based on the third control signal corresponding to the scaling mode, and transmits the selected one to the memory 20.

It should be noted that in step S311, the first control signals corresponding to different scaling modes are different, the second control signals corresponding to different scaling modes are different, and the third control signals corresponding to different scaling modes are different. Specifically, for example, the scaling mode is a first-direction scaling and a second-direction scaling, the first selector 1131 receives a first control signal, selects to transmit the target image to be processed to the first-direction scaling unit 1111 based on the first control signal, the second selector 1132 receives a second control signal, selects to transmit the image after the buffer processing to the second-direction scaling unit 1112 based on the second control signal, and the third selector 1133 receives a third control signal, selects to transmit the image after the first-direction scaling to the memory 20 based on the third control signal, and thus determines the processing sequence as follows: the first direction scaling processing, the caching processing and the second direction scaling processing are sequentially performed. For another example, the scaling mode is a first-direction scaling mode and a second-direction scaling mode, the first selector 1131 receives a first control signal, and selects to transmit the buffered image to the first-direction scaling unit 1111 based on the first control signal, the second selector 1132 receives a second control signal, and selects to output the target image to be processed to the second-direction scaling unit 1112 based on the second control signal, the third selector 1133 receives a third control signal, and selects to transmit the second-direction scaled image to the memory 20 based on the third control signal, and thus the processing sequence is determined to be the second-direction scaling processing, the buffering processing, and the first-direction scaling processing. Obviously, the two first control signals in the two aforementioned scaling manners are different, the two second control signals are also different, and the two third control signals are different, and of course, in some scaling manners, the two third control signals may also be the same, and the present embodiment is not limited thereto. Of course, the process of determining the processing sequence by the first selector 1131, the second selector 1132 and the third selector 1133 corresponding to the two scaling manners illustrated above based on the received selection control signal is also only for illustration and should not be taken as a limitation to the process of determining the processing sequence in this embodiment.

For example, each layer processing channel 11 further includes, for example, a fourth selector (not shown in fig. 2), and the foregoing step S3 further includes, for example, S312: and receiving a fourth control signal corresponding to the scaling mode by a fourth selector, and selecting one of the image subjected to the scaling processing in the first direction, the image subjected to the scaling processing in the second direction and the image subjected to the buffering processing based on the fourth control signal corresponding to the scaling mode to be output as the image subjected to the scaling processing. The fourth control signals corresponding to different scaling modes are also different. For example, the foregoing exemplary scaling manner is a first-direction scaling and a second-direction scaling, and the fourth selector receives a fourth control signal and selects, based on the fourth control signal, to output the image subjected to the second-direction scaling as the scaled image. The zoom mode is a first-direction zoom-in mode and a second-direction zoom-out mode, and the fourth selector receives a fourth control signal and selects and outputs the image subjected to the zoom processing in the first direction as the image subjected to the zoom processing based on the fourth control signal. Obviously, these two fourth control signals are also different.

Preferably, the scaling mode is a first-direction scaling mode and a second-direction scaling mode, the first-direction scaling mode and the second-direction scaling mode are respectively selected, when the first direction is enlarged and the second direction is reduced, the first direction is enlarged and the second direction is unchanged, or the first direction is unchanged and the second direction is reduced, the first selector selects to transmit the image after the buffer processing to the first-direction scaling unit based on the first control signal. When the zooming mode is a first-direction zooming and second-direction zooming, or a first-direction zooming and second-direction zooming, the first selector selects to transmit the target image to be processed to the first-direction zooming unit based on the first control signal. The scaling mode is that the first direction is unchanged and the second direction is enlarged, and the first selector selects to transmit one of the buffered image or the target image to be processed to the first direction scaling unit based on the first control signal. As mentioned above, the first direction is a vertical direction, the second direction is a horizontal direction, and in the image scaling process, for example, the scaling mode is horizontal scaling and vertical scaling needs to be performed first for vertical scaling and then horizontal scaling, so that the buffering of line data is less, and the resource consumption is less, at this time, the processing sequence is that the buffering processing, the vertical scaling processing and the horizontal scaling processing are performed in sequence, and the first selector 1131 corresponding to the vertical scaling unit transmits the buffered data to the vertical scaling unit. If the scaling mode is vertical scaling and horizontal scaling, in order to minimize the bandwidth consumption of the memory 20, the processing sequence should be vertical scaling, buffering and horizontal scaling, and the first selector 1131 corresponding to the vertical scaling unit inputs the target input image to the vertical scaling unit. Therefore, the vertical scaling unit always selects one of the buffered image and the target image to be processed for processing, without directly processing the horizontally scaled image. Therefore, in order to facilitate the scheduling of the scaling module 111 and simplify the data transmission path, no matter what the scaling manner is, the first selector 1131 is configured to select one of the target to-be-processed image and the buffered image to be transmitted to the first direction scaling unit 1111, and not to receive the second direction scaled image. Of course, the first selector 1131 may not be limited thereto in other embodiments of the present invention.

For example, the processing sequence determined according to the step S3 specifically includes:

when the zooming mode is zooming in the first direction and zooming in the second direction, the processing sequence is zooming in the first direction, zooming in the second direction and caching; or, the second direction scaling processing, the first direction scaling processing and the buffering processing are performed in sequence.

When the zooming mode is the first-direction zooming and the second-direction zooming, the processing sequence is that the cache processing, the first-direction zooming processing and the second-direction zooming processing are carried out in sequence; or, the caching process, the second direction scaling process and the first direction scaling process are performed in sequence. Preferably, the vertical direction is first enlarged, and then the horizontal direction is enlarged, for example, the first direction is the vertical direction, and the second direction is the horizontal direction, when both the directions are enlarged, the processing sequence is the buffer processing, the first direction scaling processing, and the second direction scaling processing. The data cache of the first vertical amplification processing is less, and the resource consumption is less.

When the zooming mode is zooming-out in the first direction and zooming-in the second direction, the processing sequence is zooming processing in the first direction, caching processing and zooming processing in the second direction.

When the zooming mode is the zooming in the first direction and the zooming in the second direction, the processing sequence is the zooming in the second direction, the caching and the zooming in the first direction.

When the scaling manner is the first-direction scaling and the second direction is not changed, the processing sequence is that the first-direction scaling processing and the buffer processing are performed in sequence, the second-direction scaling processing may be before, after, or between the first-direction scaling processing and the buffer processing, and preferably, the second-direction scaling processing is performed after the first-direction scaling processing, so that the data processed by the second-direction scaling unit 1112 is the data after being scaled once, and the occupied resources are less.

When the zooming mode is zooming in the first direction and the second direction is not changed, the processing sequence is that the caching processing and the zooming processing in the first direction are performed in sequence, the zooming processing in the second direction can be before, after or between the caching processing and the zooming processing in the first direction, and the preferable zooming processing in the second direction is after the zooming processing in the first direction.

When the scaling manner is that the first direction is unchanged and the second direction is reduced, the processing sequence is that the second-direction scaling processing and the caching processing are sequentially performed, the first-direction scaling processing may be before, after, or between the second-direction scaling processing and the caching processing, and preferably, the first-direction scaling processing is after the caching processing.

When the zooming mode is the first-direction unchanged and the second-direction zooming is performed, the processing sequence is the buffering processing and the second-direction zooming sequentially, the first-direction zooming may be before, after, or between the buffering processing and the second-direction zooming, and preferably, the first-direction zooming is before the second-direction zooming.

The method sets different processing sequences through different zooming modes, is suitable for various zooming modes, is more flexible in image zooming, and reduces the instantaneous bandwidth of the memory to the maximum extent. The selector 1131 in the foregoing embodiment selects only one of the target to-be-processed image and the buffered image to be transmitted to the first direction scaling unit 1111, and the corresponding processing sequence is also preferably the first direction scaling processing, either after the buffering processing or before the buffering processing and before the second direction scaling processing, and the preferred processing sequence corresponding to each scaling manner can refer to the foregoing contents, and is not specifically illustrated for each scaling manner here.

In another embodiment of the present invention, when the scaling manner is that only one direction is scaled and the other direction is not changed, another specific implementation manner of the foregoing step S3 includes, for example:

s321: receiving a first selection signal corresponding to a scaling manner by a first target selector, and selecting one of a target image to be processed and a buffered image based on the first selection signal to be transmitted to one of a first direction scaling unit 1111 and a second direction scaling unit 1112; and receiving a second selection signal corresponding to the scaling mode by the second target selector, and selecting one of the image subjected to the scaling processing in the first direction and the image subjected to the scaling processing in the second direction and the target image to be processed based on the second selection signal and transmitting the selected one to the memory 20, thereby determining the processing order.

The second target selector is the aforementioned third selector 1133, the second selection signal is, for example, the aforementioned third control signal, and the first target selector is selected from one of the first selector 1131 and the second selector 1132 according to different scaling manners, specifically, when the scaling manner is scaling in the first direction and the second direction is not changed, the first target selector is the first selector 1131, the corresponding first selection signal is the aforementioned first control signal, when the scaling manner is not changed in the first direction and scaling in the second direction, the first target selector is the second selector 1132, the corresponding first selection signal is the aforementioned second control signal, and the data transmission relationships between the first target selector and the second target selector and the target image to be processed, the image scaled in the first direction, the image scaled in the second direction, and the image subjected to cache processing are all referred to the aforementioned first selector 1131, the second selector 1132, and the specific description of the selector 1133. As described above, each layer processing channel 11 further includes, for example, a fourth selector, and step S3 further includes, for example, S322: the zooming mode is zooming in the first direction and the second direction is not changed, the fourth selector receives a fourth control signal and selects one of the zoomed image in the first direction and the cached image to be output as the zoomed image based on the fourth control signal; the scaling mode is that the first direction is unchanged and the second direction is scaled, and the fourth selector receives a fourth control signal and selects one of the image after the scaling processing in the second direction and the image after the buffering processing based on the fourth control signal to be output as the image after the scaling processing.

For example, when the scaling manner is the first direction is reduced and the second direction is not changed, the first target selector is the first selector 1131, and the second target selector is the third selector 1133. Receiving a first control signal by the first selector 1131, and selecting a target to-be-processed image from the target to-be-processed image and the buffer-processed image based on the first control signal to transmit to the first direction scaling unit 1111; and the third selector receives a third control signal, selects an image subjected to the first direction processing from the target image to be processed and the image subjected to the first direction scaling processing based on the third control signal, and transmits the image subjected to the first direction processing to the memory 20, so that the processing sequence is determined to be that the first direction scaling processing and the cache processing are sequentially performed, and the second direction scaling processing is skipped. The fourth control signal is also received, for example, by a fourth selector, and the buffered image is selected as the scaled image output based on the fourth control signal. The other zooming manners can be derived by referring to the present example, and are not repeated herein.

For example, the processing sequence determined according to step S321 includes, for example:

the scaling mode is that the first direction is reduced and the second direction is not changed, the processing sequence is that the scaling processing and the cache processing in the first direction are carried out in sequence, and the scaling processing in the second direction is skipped.

The zooming mode is that the first direction is enlarged and the second direction is not changed, the processing sequence is that the cache processing and the zooming processing of the first direction are carried out in sequence, and the zooming processing of the second direction is skipped.

The scaling mode is that the first direction is unchanged and the second direction is reduced, the processing sequence is that the scaling processing and the cache processing in the second direction are sequentially carried out, and the scaling processing in the first direction is skipped.

The zooming mode is that the first direction is unchanged and the second direction is enlarged, the processing sequence is that the caching processing and the second direction zooming processing are carried out in sequence, and the first direction zooming processing is skipped.

In the two specific embodiments of step S3, step S311 and step S321, step S311 is applicable to any scaling manner, where the first selector 1131, the second selector 1132 and the third selector 1133 all receive corresponding control signals, and perform corresponding selection based on the received selection control signals, so as to determine a processing sequence corresponding to the scaling manner, and the control is simpler. Step S321 is applicable to a situation where the scaling manner is that only one direction is scaled and another direction is unchanged, where the selector connected to the scaling unit corresponding to the unchanged direction does not receive the corresponding control signal, that is, the output interface of the selector does not output data, so that the scaling unit corresponding to the unchanged direction is skipped, and the processing time and the resources occupied in the processing process of the scaling unit corresponding to the unchanged direction can be saved.

For example, taking the scaling manner as vertical invariance and horizontal enlargement (i.e. first direction invariance and second direction enlargement) as an example, if step S311 is selected, the specific processing content of step S4 is that the first direction scaling unit 1111 (vertical scaling unit) performs the vertical invariance processing (first direction scaling processing) on the target image to be processed, then outputs the image after the first direction scaling processing to the memory 20 for buffering, reads out the image after the buffering processing and transmits the image to the second direction scaling unit 1112 (horizontal scaling unit) for performing the horizontal direction enlargement processing (second direction scaling processing), and finally outputs the image after the second direction scaling processing as the image after the scaling processing. Or, the target image to be processed is input into the memory 20 for buffering, the buffered image is transmitted to the first direction scaling unit 1111 (vertical scaling unit) for vertical invariant processing (first direction scaling processing), and finally the image after the first direction scaling processing output by the first direction scaling unit 1111 (vertical scaling unit) is transmitted to the second direction scaling unit 1112 (horizontal scaling unit) for horizontal enlargement processing (second direction scaling processing), and finally the image after the second direction scaling processing is output as the scaled image.

After step S321, the specific scaling processing content is to skip the processing in the vertical direction (the first-direction scaling processing), directly input the target image to be processed into the memory 20 for buffering, transmit the image after the buffering processing to the second-direction scaling unit 1112 (the horizontal scaling unit) for performing the amplification processing in the horizontal direction (the second-direction scaling processing), and finally output the image after the second-direction scaling processing as the scaled image. Here, it should be noted that the process of keeping the vertical unchanged may be specifically understood that the first direction scaling unit 1111 scales the input image by a scaling ratio of 1: the scaling processing of 1, although the output image and the input image are not changed, occupies resources and processing time, therefore, skipping the processing in the vertical direction reduces the occupation of resources and saves the processing time.

In summary, in the above embodiment, by providing the first selector 1131, the second selector 1132, and the third selector 1133 in each layer processing channel 11, and determining the data transmission sequence among the first direction scaling unit 1111, the second direction scaling unit 1112, and the memory 20 through a specific working manner, only a specific example of the implementation of step S3 in the image processor 10 in the image processing apparatus provided by the embodiment of the present invention is provided, and it is able to conveniently and flexibly implement scaling processing of an image at any ratio by adding fewer modules in each layer processing channel 11, and has the characteristics of simplicity and high efficiency. However, the above-mentioned specific embodiment is not intended to limit the image processor 10, and any other structure capable of implementing the step S3 to determine the foregoing processing sequence according to the scaling manner is within the scope of the present invention.

Further, the image processor 10 is for example also configured to perform step S5 (performed between step S2 and step S3): and generating a first control signal, a second control signal and a third control signal according to the scaling manner, and outputting the first control signal to the first selector 1131, the second control signal to the second selector 1132, and the third control signal to the third selector 1133. As in the foregoing embodiment, for example, each layer processing channel 11 further includes a fourth selector, and then step S5 further includes, for example, generating a fourth control signal according to a scaling manner, and outputting the fourth control signal to the fourth selector.

According to the different embodiments described above, the image processor 10 may generate only the first control signal and the third control signal, or only the second control signal and the third control signal, or generate the first control signal, the second control signal, and the third control signal, according to the scaling mode.

In another embodiment of the present invention, there is also provided another image processing apparatus 300 for performing the foregoing image processing method, based on the image processing apparatus 200, the image processing apparatus 300 further comprising a controller 30 connected to the image processor 10, specifically, referring to fig. 3, the controller 30 is connected to a first selector 1131, a second selector 1132 and a third selector 1133 inside the image processor, respectively, the controller 30 is configured to generate the first control signal and the third control signal, and output the first control signal to the first selector 1131, the second control signal to the second selector 1132 and the third control signal to the third selector 1133. As in the foregoing embodiment, each layer processing channel 11 further includes, for example, a fourth selector, and then the controller 30 is further connected to, for example, the fourth selector (not shown in fig. 3), and the controller 30 generates a fourth control signal according to the scaling manner and outputs the fourth control signal to the fourth selector.

According to the different embodiments described above, the controller may generate only the first control signal and the third control signal, or only the second control signal and the third control signal, or the first control signal, the second control signal, and the third control signal, according to the scaling mode. The controller 30 is specifically, for example, an MCU or an ARM processor, or a controller with similar control functions, and the embodiment is not limited.

In an embodiment of the present invention, the image processor 10 related to the foregoing embodiment further includes, for example, a superposition module, configured to execute step S6: and superposing the zoomed images output by the layer processing channels 11 to obtain a pre-monitoring picture, and outputting the pre-monitoring picture to an external display for display.

After step S6, the pre-monitoring screen may be displayed on screen, for example. The images to be processed input into the layer processing channels 11 may be a plurality of sub-images obtained by splitting a source image, and the plurality of sub-images are respectively scaled and then superimposed to obtain a scaled image of the source image. Of course, the plurality of images to be processed may be a plurality of different images, and the scaling manners thereof may be the same or different. The image processing apparatus disclosed in this embodiment can be applied to zoom pre-monitoring of a video processor, and in practical applications, a plurality of zoomed images are output and then, for example, buffered or otherwise processed.

In the above embodiment, the number of scaling modules in a single layer in the image processing apparatus is reduced to one, the processing order is determined according to the scaling mode of the image to be processed, and the image to be processed is scaled based on the processing order, so that scaling of the image in any proportion is ensured, and the total number of scaling modules in the whole image processing apparatus is reduced by half, so that the number of resources occupied by the scaling modules in scaling of the image with the same number of layers is reduced, and resource consumption is reduced. Or, with the same total number of scaling modules, the image processing apparatus can process scaling of more image layers and achieve more functions with the same resource consumption compared with the prior art. The image processing method M100 is executed in the image processing apparatus, and has the same advantageous effects as the image processing apparatus.

Although the present invention has been described with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the present invention.

Claims (10)

1. An image processing method, implemented in an image processor, where the image processor includes a plurality of layer processing channels, and each layer processing channel is provided with a scaling module, and the image processing method includes:

receiving an input image to be processed;

obtaining a scaling mode corresponding to each image to be processed;

determining a processing sequence corresponding to the target image to be processed according to the corresponding scaling mode;

at least performing caching processing and zooming processing on the target image to be processed according to the processing sequence to obtain a zoomed image; and the scaling processing is carried out by the same scaling module.

2. The image processing method according to claim 1, wherein each of the scaling modules comprises: a first direction scaling unit and a second direction scaling unit;

the scaling by the same scaling module includes:

performing first-direction scaling processing by the first-direction scaling unit to output an image after the first-direction scaling processing, and/or

Performing second direction scaling processing through the corresponding second direction scaling unit to output an image subjected to the second direction scaling processing;

the caching process comprises the following steps: and caching one of the target image to be processed, the image subjected to the first-direction scaling processing and the image subjected to the second-direction scaling processing to a memory, and obtaining the image subjected to the caching processing.

3. The image processing method according to claim 2, wherein each of the layer processing channels further comprises: the image processing device comprises an input module, a first selector, a second selector and a third selector, wherein the input module is used for receiving the target image to be processed;

the determining a processing sequence corresponding to the target image to be processed according to the corresponding scaling mode comprises:

receiving, by the first selector, a first control signal corresponding to the scaling manner, and selecting, based on the first control signal, one of the target to-be-processed image, the image subjected to the second-direction scaling processing, or the image subjected to the cache processing, and transmitting the selected one to the first-direction scaling unit; and

receiving, by the second selector, a second control signal corresponding to the scaling manner, and selecting, based on the second control signal, one of the target to-be-processed image, the image scaled in the first direction, or the image cached in the first direction, and transmitting the selected image to a second direction scaling unit; and

and receiving a third control signal corresponding to the scaling mode by the third selector, and selecting one of the target image to be processed, the first-direction scaled image or the second-direction scaled image to be transmitted to the memory based on the third control signal, so as to determine the processing order.

4. The image processing method according to claim 3, wherein the determining a processing order corresponding to the target image to be processed according to the corresponding scaling manner comprises:

the zooming mode is a first-direction zooming-in mode and a second-direction zooming-out mode, a first-direction zooming-in mode and a second-direction zooming-in mode are unchanged, or a first-direction zooming-out mode and a second-direction zooming-out mode are unchanged, and the first selector selects to transmit the cached images to the first-direction zooming unit based on the first control signal; or

The zooming mode is a first-direction zooming and second-direction zooming, or a first-direction zooming and second-direction zooming, and the first selector selects to transmit the target image to be processed to the first-direction zooming unit based on the first control signal; or alternatively

The scaling mode is a first direction unchanged and a second direction enlarged, and the first selector selects to transmit one of the buffered image or the target image to be processed to the first direction scaling unit based on the first control signal.

5. The image processing method according to claim 3, wherein the determining a processing order corresponding to the target image to be processed according to the corresponding scaling manner further comprises:

and generating the first control signal, the second control signal and the third control signal according to the corresponding scaling modes and outputting the first control signal, the second control signal and the third control signal to the first selector, the second selector and the third selector respectively.

6. The image processing method according to claim 3, wherein the determining a processing order corresponding to the target image to be processed according to the corresponding scaling manner comprises:

the zooming mode is a first-direction zooming-in and second-direction zooming-out mode, a first-direction zooming-in and second-direction zooming-in mode or a first-direction zooming-in and second-direction zooming-out mode, wherein the first-direction zooming processing is performed after the caching processing; or

The zooming mode is zooming in the first direction and zooming out in the second direction, zooming out in the first direction and zooming in the second direction or zooming out in the first direction and zooming out in the second direction and not changing, and the zooming processing in the first direction is performed before the caching processing; or

The scaling mode is that the first direction is unchanged and the second direction is enlarged, and the first direction scaling processing is before the caching processing or the caching processing is before the first direction scaling processing.

7. The image processing method according to claim 6, wherein the determining a processing order corresponding to the target image to be processed according to the corresponding scaling manner comprises:

the zooming mode is a first-direction zooming-in mode and a second-direction zooming-out mode, the first-direction zooming-in mode and the second-direction zooming-out mode are unchanged, or the first-direction zooming-out mode and the second-direction zooming-out mode are unchanged, and the processing sequence is that the second-direction zooming processing, the cache processing and the first-direction zooming processing are carried out in sequence; or

The zooming mode is a first-direction zooming and a second-direction zooming, and the processing sequence is that the cache processing, the first-direction zooming and the second-direction zooming are carried out in sequence; or

The scaling mode is a first-direction scaling mode and a second-direction scaling mode, and the processing sequence is that the first-direction scaling processing, the second-direction scaling processing and the cache processing are performed in sequence;

the zooming mode is zooming in a first direction and zooming in a second direction, and the processing sequence is that the zooming processing in the first direction, the caching processing and the zooming processing in the second direction are carried out in sequence; or

The zooming mode is a first-direction constant zooming mode and a second-direction zooming mode, and the processing sequence is that the first-direction zooming processing, the caching processing and the second-direction zooming processing are carried out in sequence or the caching processing, the first-direction processing and the second-direction zooming processing are carried out in sequence; or alternatively

The scaling mode is a first direction scaling mode and a second direction scaling mode, and the processing sequence is that the first direction scaling processing, the cache processing and the second direction scaling processing are performed in sequence or the first direction scaling processing, the second direction processing and the cache processing are performed in sequence.

8. The image processing method according to claim 2, wherein each of the layer processing channels further comprises: the system comprises an input module, a first target selector and a second target selector, wherein the input module is used for receiving the target image to be processed;

the determining a processing sequence corresponding to the target image to be processed according to the corresponding scaling mode comprises:

receiving, by the first target selector, a first selection signal corresponding to the scaling manner, and selecting, based on the first selection signal, one of the target to-be-processed image and the cached image to transmit to one of the first direction scaling unit and the second direction scaling unit; and

receiving, by the second target selector, a second selection signal corresponding to the scaling manner, and selecting one of the image subjected to the scaling processing in the first direction and the image subjected to the scaling processing in the second direction and the target image to be processed based on the second selection signal, and transmitting the selected one to the memory, thereby determining the processing order;

wherein the scaling mode is as follows: the first direction is reduced and the second direction is unchanged, the first direction is enlarged and the second direction is unchanged, the first direction is unchanged and the second direction is reduced, or the first direction is unchanged and the second direction is enlarged.

9. The image processing method according to any one of claims 1 to 8, characterized by further comprising:

and superposing the zoomed images output by the layer processing channels to obtain a pre-monitoring picture, and outputting the pre-monitoring picture to an external display for display.

10. An image processing apparatus characterized by comprising: an image processor and a memory connected to said image processor, wherein said image processor is adapted to perform the image processing method of any one of claims 1-8.

Images