TWI811043B - Image processing system and image object superimposition apparatus and method thereof - Google Patents

Abstract

An image object superimposition apparatus that includes an object identification circuit, a graphic drawing circuit, a graphic processing circuit and a superimposition circuit is provided. The object identification circuit identifies objects included in an original image to generate object identification information. The graphic drawing circuit draws object frames, to be stored in a memory, according to the object identification information. The object frames correspond to the objects and have a total memory usage smaller than a data size of the original image. The graphic processing circuit includes graphic processing unit circuits that process the object frames to generate processed object frames. The superimposition circuit superimposes the processed object frames on the original image to output a synthesized image.

Description

Image processing system and its image object overlay device and method

The present invention relates to image object overlay technology, in particular to an image processing system and its image object overlay device and method.

In modern human life, image capture devices are widely used. For example, mobile phones that can shoot videos or surveillance systems that record images for a long time will use image capture devices. Among them, object recognition is an important technology derived from image capture devices, and can be used to recognize specific objects such as human faces and license plates.

In the past, after object recognition, image processing devices usually set a large frame to cover all objects, so as to mark and store the range with objects. However, when objects are dispersed in such a processing method, too many image ranges without objects will be stored, resulting in a waste of memory. Moreover, the form of a single frame cannot be set differently for individual objects, and the flexibility of display is insufficient.

In view of the problems in the prior art, an object of the present invention is to provide an image processing system and an image object overlay device and method therefor, so as to improve the prior art.

The invention includes an image object overlay device, which includes: an object identification circuit, a graphics drawing circuit, a graphics processing circuit and a superposition circuit. The object identification circuit is configured to identify multiple objects included in the original image to generate object identification information. The graphics drawing circuit is configured to draw a plurality of object frames according to the object identification information and store them in the memory, wherein the object frames respectively correspond to the objects, and the total memory space used by the object frames is smaller than the data size of the original image. The graphics processing circuit includes a plurality of graphics processing unit circuits, and the graphics processing unit circuits process object frames to generate a plurality of processed object frames. The overlay circuit is configured to overlay the processed object frame on the original image, and then output a composite image.

The present invention also includes a method for overlaying image objects, which is applied to an image object overlay device, comprising: an object identification circuit identifies a plurality of objects included in the original image to generate object identification information; a graphics drawing circuit draws the plurality of objects according to the object identification information The frame is stored in the memory, wherein the object frame corresponds to the object respectively, and the total space of the memory used by the object frame is smaller than the data size of the original image; the plurality of graphics processing unit circuits included in the graphics processing circuit process the object frame to generate a plurality of processed object frames; and the superimposed circuit superimposes the processed object frames on the original image, and then outputs a synthetic image.

The present invention further includes an image processing system, including: an image capture device, a memory, an image object overlay device and a display device. The image capture device is configured to capture images to generate original images. The image object overlay device includes: an object identification circuit, a graphics drawing circuit, a graphics processing circuit and a superposition circuit. The object identification circuit is configured to receive the original image, and identify multiple objects included in the original image to generate object identification information. The graphics drawing circuit is configured to draw a plurality of object frames according to the object identification information and store them in the memory, wherein the object frames respectively correspond to the objects, and the total memory space used by the object frames is smaller than the data size of the original image. The graphics processing circuit includes a plurality of graphics processing unit circuits, and the graphics processing unit circuits process object frames to generate a plurality of processed object frames. The overlay circuit is configured to overlay the processed object frame on the original image, and then output a composite image. The display device is configured to receive and display the composite image.

About the feature, implementation and effect of this case, hereby cooperate with drawing as preferred embodiment and describe in detail as follows.

An object of the present invention is to provide an image processing system and its image object overlay device and method, which can identify individual objects and draw object frames, which not only greatly reduces the amount of memory used, but also allows object frames to be processed according to requirements. Different processing to achieve flexible display effect.

Please refer to Figure 1. FIG. 1 shows a block diagram of an image processing system 100 in an embodiment of the present invention. The image processing system 100 includes: an image capture device 110 , a memory 120 , an image object overlay device 130 and a display device 140 .

The image capture device 110 may include one or more photosensitive elements, and is configured to perform image capture to generate an original image OI.

The image object superimposing device 130 is configured to receive the original image OI for object recognition, draw a corresponding object frame and superimpose it on the original image OI to generate a synthetic image SI. In one embodiment, in addition to object superimposition, the image object superimposition device 130 can also selectively perform, for example, but not limited to, auto-focus, auto-exposure, auto-white balance, high dynamic range imaging (high dynamic range imaging; HDRI), image reduction noise or a combination thereof to generate a synthetic image SI.

The display device 140 is configured to receive and display the synthesized image SI.

In different applications, the image processing system 100 can be an image monitoring system, an image capture system or other electronic systems capable of capturing images for processing and displaying. Wherein, the original image OI can be an image in an original image stream, and the synthetic image SI can also be an image in a synthetic image stream. Through the processing of the image object superimposing device 130, the synthetic image SI can include object frames corresponding to different objects without affecting the original image OI, so as to facilitate subsequent data processing.

The structure and operation of the image object superimposing device 130 will be described in more detail below.

Please refer to Figure 2 and Figure 3 at the same time. FIG. 2 shows a block diagram of the memory 120 and the image object overlay device 130 in an embodiment of the present invention. FIG. 3 shows a schematic diagram of an original image OI in an embodiment of the present invention.

As shown in FIG. 2 , the image object superimposing device 130 includes: a processor 200 , an object identification circuit 210 , a graphic drawing circuit 220 , a graphic processing circuit 230 , a superimposing circuit 240 and a memory interface circuit 250 .

The processor 200 is configured to run the resource allocation software RGN to set and control each circuit.

The object identification circuit 210 is configured to receive the original image OI, and identify a plurality of objects included in the original image OI to generate object identification information ODI.

As shown in FIG. 3 , the original image OI includes six objects OB1 - OB6 . In FIG. 3 , the objects OB1 - OB6 are human faces as an example, but in other embodiments, they can also be, for example but not limited to, license plates or other objects that can be processed and recognized according to actual needs.

In one embodiment, the object identification information ODI may include but not limited to distance information of each object OB1 - OB6 in the original image OI. In one embodiment, when the objects are all of the same attribute (for example, both are human faces), the distance information can be determined by the size of the objects. For example, when the size of the object is larger, it can be determined that it has a relatively closer position. And when the size of the object is smaller, it can be judged to have a relatively far position.

It should be noted that the definition and acquisition method of the above information is just an example. The present invention is not limited thereto.

The memory 120 is configured to store data, and can be accessed by the processor 200 , the object recognition circuit 210 , the graphics rendering circuit 220 , the graphics processing circuit 230 and the superimposition circuit 240 . In one embodiment, the memory 120 is, for example, but not limited to dynamic random access memory (DRAM), and these circuits can access the memory 120 through the memory interface circuit 250 .

The memory 120 includes an original image storage block IB and a plurality of object storage blocks BB1-BB6. The original image storage block IB is configured to store the original image OI. In one embodiment, after receiving the original image OI, the object recognition circuit 210 may store the original image OI in the original image storage block IB.

The graphics rendering circuit 220 receives the object identification information ODI and operates accordingly. In one embodiment, the object identification information ODI can be selectively transmitted to the graphics rendering circuit 220 by the resource allocation software RGN operated by the processor 200 , or directly transmitted from the object identification circuit 210 to the graphics rendering circuit 220 .

The graphic drawing circuit 220 is configured to draw a plurality of object frames OF1-OF6 according to the object identification information ODI and store them in the object storage blocks BB1-BB6 of the memory 120.

Please also refer to Figure 4. FIG. 4 shows a schematic diagram of object frames OF1-OF6 in an embodiment of the present invention.

As shown in FIG. 4 , the object frames OF1-OF6 respectively correspond to the objects OB1-OB6 in FIG. 3 , and the borders drawn by solid lines in the object frames OF1-OF6 respectively correspond to the ranges of objects indicated by dotted lines in FIG. 3 . In one embodiment, the graphic drawing circuit 220 is a circuit with a drawing engine, and is configured to draw object frames OF1˜OF6 according to the object identification information ODI.

In one embodiment, the corresponding relationship between the object frames OF1-OF6 and the object storage blocks BB1-BB6 can be set by the resource configuration software RGN according to the object identification information ODI. More specifically, after receiving the object identification information ODI, the resource allocation software RGN can know the number, coordinates and sizes of the objects OB1-OB6.

The resource allocation software RGN will then request the corresponding object storage blocks BB1-BB6 from the memory 120, wherein these object storage blocks BB1-BB6 have the number and size that can store the object frames OF1-OF6. The resource configuration software RGN can record the size and address of the object storage blocks BB1~BB6. Wherein, the total space of the memory 120 used by the object frames OF1-OF6 is smaller than the data size of the original image OI.

The graphics processing circuit 230 includes a plurality of graphics processing unit circuits PD1-PD6, respectively processing the object frames OF1-OF6 to generate a plurality of processed object frames OA1-OA6.

Please also refer to Figure 5. FIG. 5 shows a block diagram of the graphics processing circuit 230 in an embodiment of the present invention. The graphics processing circuit 230 includes a plurality of graphics processing unit circuits PD1 - PD6 .

After receiving the object identification information ODI, the resource allocation software RGN can judge the processing required for each object frame OF1~OF6, and store the size and size of the object storage blocks BB1~BB6 corresponding to the object frame OF1~OF6. The address is notified to the graphics processing unit circuits PD1-PD6, so that the graphics processing unit circuits PD1-PD6 read the object frames OF1-OF6 for processing.

The processing performed by the graphics processing circuit 230 on the object frames OF1-OF6 includes, for example but not limited to, enlargement and reduction of the frames, conversion of graphic formats, or a combination thereof.

In one embodiment, the graphics processing circuit 230 can process the object frames OF1 - OF6 according to the graphics processing parameters GS1 - GS6 including scaling parameters. The resource configuration software RGN can write the graphics processing parameters GS1~GS6 and the addresses of the object storage blocks BB1~BB6 into the registers (not shown) corresponding to the graphics processing unit circuits PD1~PD6, so that the graphics The processing unit circuits PD1-PD6 read and operate accordingly.

In one embodiment, the graphics processing unit circuits PD1-PD6 perform format conversion on the object frames OF1-OF6 to have the same image format as the original image OI. For example, the image format of the original image OI may be, for example, but not limited to ARGB8888 or AYUV8888. The graphics processing unit circuits PD1~PD6 can convert the format of the object frames OF1~OF6 to be set to one of the above two formats, corresponding to the image format of the original image OI.

In practice, the graphics processing unit circuits PD1 - PD6 mentioned above can be implemented by, for example, but not limited to, application specific integrated circuits (ASIC).

The superimposing circuit 240 is configured to superimpose the processed object frames OA1 - OA6 on the original image OI, and then output the synthesized image SI. In one embodiment, the resource configuration software RGN can configure a set of superposition parameters SP1-SP6 according to the object identification information ODI, and then make the superposition circuit 240 perform superposition according to the superposition parameters SP1-SP6. The resource allocation software RGN can write the information of the superimposition parameters SP1-SP6 into the corresponding registers (not shown) of the superposition circuit 240, so that the superposition circuit 240 can operate accordingly after being read.

In one embodiment, the overlay parameters SP1 - SP6 include graphic order levels. For example, the resource configuration software RGN can set the graphics sequence level according to the distance information of the object identification information ODI. When one of the objects (such as the object OB1 ) is at a relatively close position, the corresponding one of the processed object frames (such as the processed object frame OA1 ) has a higher graphic sequence level. When one of the objects (such as the object OB3 ) is at a relatively far position, the corresponding one of the processed object frames (such as the processed object frame OA3 ) has a lower graphic sequence level. In one embodiment, the original image OI is set to have the lowest graphic order level.

In another embodiment, the superposition parameters SP1-SP6 include transparency, and the transparency can be set corresponding to the sequence level of the graphics. When one of the objects is at a relatively close position, the corresponding frame of one of the processed objects not only has a higher graphic sequence level, but also has a lower transparency. When one of the objects is at a relatively far position, the corresponding frame of one of the processed objects not only has a lower graphic sequence level, but also has a higher transparency.

In one embodiment, the overlay circuit 240 can perform overlay on a pixel-by-pixel basis. With the superposition position set by the resource configuration software RGN, the superimposition circuit 240 can determine whether each target pixel in the original image OI corresponds to any position of the processed object frame OA1-OA6. If the target pixel corresponds to any range of the processed object frame OA1-OA6, the overlay circuit 240 will receive the pixel data of the corresponding processed object frame OA1-OA6, and superimpose it with the target pixel of the original image OI. If the target pixel does not correspond to any range of the processed object frame OA1 - OA6 , the overlay circuit 240 will directly output the target pixel of the original image OI.

In one embodiment, when the target pixel corresponds to the range of multiple processed object frames OA1-OA6, the overlay circuit 240 can output the pixel data of one of the processed object frames OA1-OA6 according to the sequence level of the graphics (for example The one with the highest graphics order level), and superimposed with the target pixel of the original image OI.

Please refer to Figure 6. FIG. 6 shows a block diagram of a graphic processing circuit 230 and an overlay circuit 240 in another embodiment of the present invention. Similar to the graphics processing circuit 230 in FIG. 5 , the graphics processing circuit 230 in FIG. 6 includes a plurality of graphics processing unit circuits PD1 - PD6 . However, in this embodiment, the superposition circuit 240 further includes complex superposition sub-circuits SC1 - SC6 .

The superposition sub-circuits SC1~SC6 respectively correspond to one of the graphics processing unit circuits PD1~PD6, and sequentially convert the processed object frame OA1~OA6 according to the superposition parameters SP1~SP6 from the first superimposition subcircuit SC1~SC6. One of them is superimposed on the original image OI, and the final superimposition sub-circuit SC6 outputs the synthesized image SI.

In an embodiment, the superimposing sub-circuits SC1 - SC6 can be configured to superimpose the respective object frames OA1 - OA6 according to the order of the graphic sequence level, instead of the order shown in FIG. 6 .

In some techniques, although the objects in the original image can be identified separately, it is necessary to draw a single large frame covering all objects. When objects are scattered in the original image, the frame will contain too much non-object image content, resulting in waste of memory usage. Moreover, the form of a single frame cannot be set differently for individual objects, and the flexibility of display is insufficient.

The image object superimposing device of the present invention can identify individual objects and draw object frames, which not only greatly reduces the usage of memory, but also allows different settings for object frames according to requirements to achieve flexible display effects.

It should be noted that, in the above embodiments, the numbers of objects, object frames and processed object frames are just an example. In practice, the numbers of objects, object frames and processed object frames vary with the received original images. The present invention is not limited thereto.

Moreover, in the above-mentioned embodiments, the content contained in the object identification information, graphic setting parameters, and superposition parameters is just an example. In practice, the content contained in the object identification information, graphics setting parameters and overlay parameters may vary according to actual needs. The present invention is not limited thereto.

Please refer to Figure 7. FIG. 7 shows a flow chart of a method 700 for overlaying image objects in an embodiment of the present invention.

In addition to the aforementioned devices, the present invention further discloses a method 700 for superimposing image objects, which is applied to, for example, but not limited to, the device 130 for superimposing image objects in FIG. 2 . An embodiment of the method 700 for overlaying image objects is shown in FIG. 7 , which includes the following steps.

In step S710 : the object identification circuit 210 identifies the plurality of objects OB1 - OB6 included in the original image OI to generate object identification information ODI.

In step S720: the graphics drawing circuit 220 draws a plurality of object frames OF1-OF6 according to the object identification information ODI to store in the memory 120, wherein the object frames OF1-OF6 correspond to the objects OB1-OB6 respectively, and the object frames OF1-OF6 The total space of the memory 120 is smaller than the data size of the original image OI.

In step S730: the plurality of graphics processing unit circuits PD1-PD6 included in the graphics processing circuit 230 process the object frames OF1-OF6 to generate a plurality of processed object frames OA1-OA6.

In step S740 : the superimposition circuit 240 superimposes the processed object frames OA1 - OA6 on the original image OI, and then outputs the synthesized image SI.

In practice, the numbers of objects, object frames and processed object frames vary with the received original images. Moreover, the content contained in the object identification information, graphic setting parameters and overlay parameters may vary according to actual needs. The present invention is not limited thereto.

It should be noted that the above-mentioned implementation is just an example. In other embodiments, those skilled in the art can make modifications without departing from the spirit of the present invention. It should be understood that, unless the order of the steps mentioned in the above embodiments is specifically stated, the order of the steps may be adjusted according to actual needs, and may even be executed simultaneously or partly simultaneously.

The image processing system and its image object overlay device and method in the present invention can identify individual objects and draw object frames, which not only greatly reduces the amount of memory used, but also allows different settings for object frames according to requirements. A flexible display effect is achieved.

Although the embodiments of the present invention are as described above, these embodiments are not intended to limit the present invention, and those skilled in the art can make changes to the technical characteristics of the present invention according to the explicit or implicit contents of the present invention. All these changes may belong to the scope of patent protection sought by the present invention. In other words, the scope of patent protection of the present invention must be defined by the scope of patent application in this specification.

100: Image processing system 110: image capture device 120: memory 130:Image object overlay device 140: display device 200: Processor 210: Object identification circuit 230: Graphic drawing circuit 230: graphics processing circuit 240:Superposition circuit 250: memory interface circuit 700: Image object overlay method S710~S740: Steps BB1~BB6: object storage block GS1~GS6: Graphics processing parameters IB: Original image storage block OA1~OA6: object frame after processing OB1~OB6: Object ODI: Object Identification Information OF1~OF6: object frame OI: original image PD1~PD6: graphics processing unit circuit RGN: Resource Allocation Software SC1~SC6: superposition sub-circuit SI: Synthetic Image SP1~SP6: Overlay parameters

[Fig. 1] shows a block diagram of an image processing system in one embodiment of the present invention; [Fig. 2] shows a block diagram of the memory and image object superimposition device in one embodiment of the present invention; [Fig. 3] shows a schematic diagram of an original image in an embodiment of the present invention; [Fig. 4] shows a schematic diagram of an object frame in an embodiment of the present invention; [Fig. 5] shows a block diagram of a graphics processing circuit in an embodiment of the present invention; [Fig. 6] shows a block diagram of a graphics processing circuit and an overlay circuit in another embodiment of the present invention; and [ FIG. 7 ] shows a flow chart of a method for overlaying image objects in an embodiment of the present invention.

120: memory

130:Image object overlay device

200: Processor

210: Object identification circuit

220: Graphic drawing circuit

230: graphics processing circuit

240:Superposition circuit

250: memory interface circuit

BB1~BB6: object storage block

IB: Original image storage block

GS1~GS6: Graphics processing parameters

OA1~OA6: object frame after processing

ODI: Object Identification Information

OF1~OF6: object frame

OI: original image

PD1~PD6: graphics processing unit circuit

RGN: Resource Allocation Software

SI: Synthetic Image

SP1~SP6: Overlay parameters

Claims (15)

An image object superposition device, comprising: an object identification circuit configured to identify a plurality of objects included in an original image to generate object identification information, wherein the object identification information includes a distance of each of the objects in the original image information; a graphics drawing circuit configured to draw a plurality of object frames according to the object identification information to be stored in a memory, wherein the object frames respectively correspond to the objects, and the object frames use the space of the memory The sum is smaller than the data size of the original image; a graphics processing circuit, including a plurality of graphics processing unit circuits, and the graphics processing unit circuits process the object frames to generate a plurality of processed object frames; and an overlay circuit configured The processed object frames are superimposed on the original image, and then a synthesized image is output. The image object overlay device as described in Claim 1 further includes a processor configured to run a resource allocation software, and the resource allocation software configures a set of overlay parameters according to the object identification information. The image object overlay device as described in claim 2, wherein the set of overlay parameters includes a graphic sequence level. The image object overlay device as described in claim 3, wherein the resource allocation software sets the graphic sequence level, and when one of the objects is in a relatively close position, one of the corresponding processed objects The drawing frame has a higher hierarchy of the graphic order, and when one of these objects When an object is at a relatively far position, one of the processed object frames corresponding to it has a lower graphic order level, wherein the original image has the lowest graphic order level. The image object overlay device as described in claim 1, wherein the graphics processing unit circuits process the object frames according to a scaling parameter. The image object overlay device as described in Claim 1, wherein the graphics processing unit circuits perform format conversion processing on the object frames according to an image format of the original image. The image object overlay device as described in claim 3, wherein the overlay circuit includes a plurality of overlay sub-circuits, and the overlay sub-circuits correspond to the graphics processing unit circuits respectively, so as to sequentially process the overlays according to the set of overlay parameters The object frame is superimposed on the original image. A method for overlaying image objects, applied in an image object overlay device, comprising: an object identification circuit identifies multiple objects included in an original image to generate object identification information; a processor operates a resource allocation software to Configure a set of overlay parameters according to the object identification information; use a graphics drawing circuit to draw a plurality of object frames according to the object identification information and store them in a memory, wherein the object frames respectively correspond to the objects, and the objects The sum of the memory space used by the frames is smaller than the data size of the original image; the object frames are processed by a plurality of graphics processing unit circuits included in a graphics processing circuit to generate a plurality of processed object frames; and The processed object frames are superimposed on the original image by an overlay circuit, and then a composite image is output. The image object overlay method as described in Claim 8, wherein the object identification information includes a distance information of each of the objects in the original image. The image object overlay method as described in claim 8, wherein the set of overlay parameters includes a graphic sequence level. The image object overlay method as described in claim 10, wherein, when one of the objects is at a relatively close position, the corresponding one of the processed object frames has a higher graphics order level, and when one of the objects is at a relatively far position, its corresponding one of the processed object frames has a lower level of the graphic order, wherein the original image has the lowest level of the graphic sequential hierarchy. The image object overlay method as described in claim 8, wherein the graphics processing unit circuits process the object frames according to a scaling parameter. The image object overlay method as described in Claim 8, wherein the graphics processing unit circuits perform format conversion processing on the processed object frames according to an image format of the original image. The image object overlay method as described in Claim 10, wherein the overlay circuit includes a plurality of overlay sub-circuits, and the overlay sub-circuits respectively correspond to the graphics processing unit circuits, so as to sequentially process them respectively according to the set of overlay parameters The after object frame is superimposed on the original image. An image processing system, comprising: an image capture device configured to perform image capture to generate an original image; a memory; an image object superposition device comprising: an object recognition circuit configured to receive the original image, and Identifying multiple objects contained in the original image to generate object identification information, wherein the object identification information includes a distance information of each of the objects in the original image; a graphics drawing circuit configured to draw according to the object identification information A plurality of object frames are stored in the memory, wherein the object frames respectively correspond to the objects, and the total space of the memory used by the object frames is smaller than the data size of the original image; a graphics processing circuit, comprising a plurality of graphics processing unit circuits that process the object frames to generate a plurality of processed object frames; and an overlay circuit configured to overlay the processed object frames on the original image , and then output a synthesized image; and a display device configured to receive and display the synthesized image.

Images