TWI826336B - Frame image acquisition method - Google Patents

Abstract

Disclosed is a frame image acquisition method, comprising: an image block range identification step of identifying frame image block range information and object image block range information of a comic page image; a image block relationship determination step of determining an additive relationship image block corresponding to a frame image block based on the frame image block range information and the object image block range information; and an image block synthesis step of merging the frame image block and the additive relationship image block to obtain a comic frame image with better integrity, solving the problem of fragmented pictures and text.

Description

How to capture frame images

The present invention relates to an image capturing technology for digital comics, and in particular, to a frame image capturing method.

Digital comics generally refer to paper comics that are stored as digital data, and also include comic data that are directly drawn digitally. This type of digital comics needs to be displayed by a computer. With the popularization of mobile devices, the way people use computers to read digital comics has gradually shifted from desktop computers to mobile phones.

Comics usually contain multiple frames on a single page, thereby illustrating the story with images displayed in sequence, functioning as a storyboard. However, when a single page of a digital comic is displayed entirely on the screen of a mobile phone, it is often too crowded and difficult to read. Therefore, there is currently a technology that extracts each frame from a single page of a comic and displays it individually on the screen of a mobile phone. However, this often leads to the problem of fragmented images and text as the frames are extracted. .

Therefore, the purpose of the present invention is to provide a method for capturing a frame image, which can capture a comic frame while maintaining its integrity, so as to solve the problems of the conventional technology.

The technical means adopted by the present invention to solve the problems of the conventional technology is to provide a frame image capturing method for obtaining each comic frame image in a comic page image. The capture of the frame image The obtaining method includes the following steps: a block range identification step, using an artificial intelligence image recognition unit to perform an image recognition process on the comic page image to obtain the picture of each frame block in the comic page image; frame block range information and object block range information of each object block; a block relationship determination step, using a judgment unit to determine from the frame block based on the frame block range information and the object block range information and the relative positional relationship between each of the object blocks, determine each addition relationship block corresponding to the picture frame block, and obtain the block addition relationship information, wherein the addition relationship block is related to the picture frame block The object block has a position overlap and belongs to the same comic frame as the frame block; and a block synthesis step is to use an image processing unit according to the additive relationship information of the block, for the Each picture frame block in the comic page image corresponds to each additive relationship block corresponding to the picture frame block while maintaining the relative positional relationship between each other unchanged. Merge to obtain the comic frame image.

In one embodiment of the present invention, a frame image capturing method is provided, wherein in the block range recognition step, the image recognition processing performed by the artificial intelligence image recognition unit is to convert the comic page A character pattern in the image is partially identified as the object tile.

In one embodiment of the present invention, a frame image capturing method is provided, wherein in the block range recognition step, the image recognition processing performed by the artificial intelligence image recognition unit is to convert the comic page The text dialog portion of the image located within a dialog box pattern is identified as the object tile.

In one embodiment of the present invention, a method for capturing a frame image is provided, wherein in the block range identification step, the frame block range information is two diagonal ends of the frame block. The coordinate information of the point. The object block range information is the coordinate information of the two diagonal endpoints of the object block.

In one embodiment of the present invention, a method for capturing a frame image is provided, wherein in the block relationship determination step, the additive relationship block is further at least partially protruding outside the frame block. of this object's tiles.

In one embodiment of the present invention, a method for capturing a frame image is provided, wherein the block relationship determination step further includes: a subtraction relationship determination sub-step, based on the frame block range information and the object block Range information, from the relative positional relationship between the picture frame block and each object block, determines each subtraction relationship block corresponding to the picture frame block, and obtains block subtraction relationship information, wherein the subtraction relationship The block is the object block that overlaps with the frame block but belongs to a different comic frame in position than the frame block, and the block synthesis step further includes: a subtractive synthesizer Step: According to the block subtraction relationship information, for each of the picture frame blocks in the comic page image, remove each of the subtraction relationship blocks corresponding to the picture frame block from the picture frame block.

In an embodiment of the present invention, a method for capturing a frame image is provided, wherein in the subtractive synthesis sub-step of the block synthesis step, each corresponding frame image block is left blank. The subtraction relation tile is removed from the frame tile.

In an embodiment of the present invention, a frame image capturing method is provided, wherein the block synthesis step further includes: an image repair sub-step, and after the subtraction synthesis sub-step, an image repair algorithm is used Method to fill in the removed portion of the frame tile in the subtractive synthesis sub-step.

Through the technical means adopted in the present invention, the frame image capturing method of the present invention not only captures the frame tiles in the comic page image, but also acquires the object tiles, and converts the frame tiles into And all the object tiles belonging to the same comic frame are correspondingly merged to obtain the comic frame image, thereby maintaining the integrity of all the object tiles in the comic frame, thereby solving the problem of fragmentation of images and text. problem. Furthermore, in a preferred embodiment, the frame image capturing method of the present invention will remove the object tiles belonging to different comic frames from the frame tiles to reduce the number of comic frames. Frame foreign objects in the image to improve readability. In addition, the frame image capturing method of the present invention can also further fill in the removed parts of the image to eliminate the abruptness caused by the lack or blankness of part of the image, so as not to reduce the attractiveness of the comic. .

The embodiments of the present invention will be described below based on FIGS. 1 to 7 . This description is not intended to limit the implementation of the present invention, but is one example of the present invention.

As shown in Figures 1 to 7, a method for capturing a frame image according to an embodiment of the present invention is used to obtain each comic frame image in a comic page image MP. The frame image The image capturing method includes: a block range identification step S10, a block relationship determination step S20 and a block synthesis step.

As shown in Figures 1 to 3, in the block range identification step S10, an artificial intelligence image recognition unit 1 is used to perform an image recognition process on the comic page image MP to obtain the comic page The frame block range information (indicated by a single-point chain line) of each frame block F1, F2, F3, F4, F5, F6 in the image MP and each object block C1, C2, C3, C4, C5 , object block range information of T1, T2, and T3 (indicated by a two-point chain line).

Specifically, in the block range identification step S10, the artificial intelligence image recognition unit 1 is used to find each of the picture frame blocks, each of the object blocks, and the other from the comic page image MP. For the corresponding corresponding frame block range information and object block range information, it is better to use deep learning to perform image recognition processing, for example: using the YOLO (you only look once) series of object recognition Neural network model. Of course, the present invention is not limited to this, as long as the image recognition processing can be effectively performed, and the frame block range information of each frame block and the information of each object block are obtained from the comic page image MP. For object block range information, other artificial intelligence image recognition units can also be used.

Furthermore, in this embodiment, the image recognition processing performed by the artificial intelligence image recognition unit 1 recognizes a character pattern part in the comic page image MP as the object tiles C1, C2, and C3. , C4, C5, and the text dialogue part located in a dialog box pattern in the comic page image MP is identified as the object tiles T1, T2, and T3. Specifically, various objects (such as: characters, buildings, backgrounds, text dialogues, effect patterns) in the comic page image MP can be the objects of the image recognition processing of the artificial intelligence image recognition unit 1 Object tiles. However, considering the execution efficiency of the image recognition process and the effect reflected in comic reading, only the object tiles C1, C2, C3, C4, and C5 of the character pattern part and the object tiles of the text dialogue part are T1, T2, and T3 are a better choice for image recognition processing.

In addition, in this embodiment, the picture frame block range information is the coordinate information of the two diagonal endpoints of the picture frame blocks F1, F2, F3, F4, F5, and F6, and the object block range information is It is the coordinate information of the two diagonal endpoints of the object's blocks C1, C2, C3, C4, C5, T1, T2, and T3. In other words, the frame block range information and the object block range information use a rectangular frame to define the range of the frame block and the object block, thereby greatly reducing the burden of subsequent calculations of positional relationships. It should be noted that the frame block range information and the object block range information are only used for subsequent calculations of the positional relationship between the frame block and the object block, and are not used to represent the picture. The actual outline shape of the frame block and the object block. For example, as shown in Figure 3, the object blocks C1, C2, C3, C4, and C5 are all blocks with human outline shapes, and the object blocks C1, C2, C3, C4, and C5 are The object block range information represented by the two-point chain line is all rectangular.

As shown in Figures 1 and 3, in the block relationship judgment step S20, a judgment unit 2 (Figures 4 and 5) is used based on the frame block range information and the object block range. Information, from the relative positional relationship between the picture frame block and each of the object blocks, determine each addition relationship block corresponding to the picture frame block, and obtain the block addition relationship information, wherein the addition relationship diagram The block is the object block that overlaps the frame block and belongs to the same comic frame as the frame block.

Specifically, in the block relationship determination step S20, the judgment unit 2 is used to determine which object blocks belong to the same comic frame as a specific frame block and belong to the frame. These object blocks of the comic frame will be determined as additive relationship blocks corresponding to the frame block (additional relationship determination sub-step S21). For example, in the comic page image MP in Figure 3, because the object block C1 and the object block T1 both overlap with the frame block F1, and both overlap with the frame block F1 The position belongs to the comic frame in the upper right corner. Therefore, in the addition relationship determination sub-step S21, both the object block C1 and the object block T1 will be determined by the judgment unit 2 as corresponding to the frame picture. Additive relationship diagram of block F1. Similarly, in the comic page image MP in Figure 3, since the object block C5 and the object block T3 both overlap with the frame block F6, although the object block C5 is also overlapped with the frame block F6, Both the frame block F5 and the picture frame block F6 have overlapping positions, but from the overlapping position, the overlapping proportion, ... and other positional relationships, it can be judged that the object block C5 and the object block T3 are both related to the picture. The frame block F6 belongs to the lowermost comic frame in terms of position, so in the addition relationship judgment sub-step S21, the object block C5 and the object block T3 will both be judged as corresponding by the judgment unit 2 The additive relation block to the picture frame block F6.

As shown in Figure 1 and Figures 3 to 5, in the block synthesis step S30, an image processing unit 3 is used to calculate each element in the comic page image MP based on the block addition relationship information. The picture frame blocks F1, F2, F3, F4, F5, F6, the picture frame blocks F1, F2, F3, F4, F5, F6 and the picture frame blocks F1, F2, F3, F4, F5, Each of the additive relationship tiles corresponding to F6 is merged while maintaining the relative positional relationship between each other to obtain the comic frame image.

Specifically, as shown in Figures 3 to 5, in the block synthesis step S30, the image processing unit 3 combines the frame blocks belonging to the same comic frame with the corresponding Each of the additive relationship blocks is merged through image processing (additive synthesis sub-step S31). For example, as shown in Figure 4, the frame block F6 belongs to the same comic frame as the object block C5 and the object block T3. Therefore, in the additive synthesis sub-step S31, the The image processing unit 3 will correspondingly merge the frame block F6, the object block C5 and the object block T3 while maintaining the relative positional relationship between each other to obtain a character pattern (the object Block C5) Complete and unfragmented comic frame image MF6. As shown in Figure 5, the frame block F1 belongs to the same comic frame as the object block C1 and the object block T1. Therefore, in the additive synthesis sub-step S31, the image processing unit 3. The frame block F1, the object block C1 and the object block T1 will be merged accordingly while maintaining the relative positional relationship between each other to obtain a text dialogue (the object block T1). Complete and unfragmented comic frame image MF1.

It should be noted that the objects merged by the image processing unit 3 are "object tiles", not necessarily the range specified by the object tile range information. Therefore, as shown in Figure 4, when the object block C5 and the frame block F6 are merged, the object blocks C4 and C3 located within the range specified by the object block range information of the object block C5 The part will not be merged with the frame block F6. As shown in Figure 5, when the object block T1 is merged with the frame block F1, the dialog box pattern located outside the range specified by the object block range information of the object block T1 is not the object in the first place. It is part of the block T1, so of course it will not be merged with the frame block F1.

Through the above method, the frame image capturing method of the present invention acquires the object tile while capturing the frame tile in the comic page image, and combines the frame tile and All the object tiles belonging to the same comic frame are correspondingly merged to obtain the comic frame image, thereby maintaining the integrity of all the object tiles in the comic frame, thereby solving the problem of images and text The problem of fragmentation.

Preferably, in this embodiment, in the block relationship determination step S20, the additive relationship block is further at least partially protruding from the object block outside the picture frame block. Specifically, most of the object tiles are located entirely within the frame tiles of the same comic frame. No matter whether they are merged into the frame tiles or not, the resulting comic will not be modified. Frame images are affected. For example, whether the object block T3 in Figure 4 is merged into the frame block F6 or not, it will not affect the obtained comic frame image MF6. Regardless of whether the object block C1 in Figure 5 is merged into the frame block F1, it will not affect the obtained comic frame image MF1. Therefore, by preliminarily excluding the object blocks that are entirely within the frame block of the same comic frame (that is, not protruding outside the frame block), it is possible to significantly reduce the number of subsequent errors in the frame block. The number of additive relationship blocks to be processed in the block synthesis step S30 is thereby improved to improve processing efficiency. In addition, in the additive synthesis sub-step S31, it is also possible to combine the additive relationship block with the frame diagram only by combining the portion of the additive relationship diagram that protrudes outside the frame diagram instead of the entire additive relationship block. Block merging further improves processing efficiency.

Preferably, as shown in Figures 1, 3 and 6, in the frame image capturing method according to the embodiment of the present invention, the block relationship determination step S20 further includes: a subtraction relationship determination. Sub-step S22: Based on the frame block range information and the object block range information, from the relative position relationship between the picture frame block and each of the object blocks, determine each corresponding to the picture frame block. Subtract the relationship block to obtain block subtraction relationship information, where the subtraction relationship block is the comic frame that has a position overlap with the frame block but belongs to a different position from the frame block. The object tiles and the tile synthesis step S30 further include: a subtraction synthesis sub-step S32. According to the subtraction relationship information of the tiles, for each frame tile in the comic page image, the frame tile is Each corresponding subtraction relation block is removed from the picture frame block.

Specifically, in the subtraction relationship determination sub-step S22, the determination unit 2 is used to determine which object tiles overlap with a specific frame tile but belong to different comic frames, and Such object blocks will be judged as subtraction relationship blocks corresponding to the frame block. For example, in the comic page image MP in Figure 3, the object block C5 overlaps with the frame block F5, but it can be judged from the positional relationship such as the overlapping position, the overlapping proportion, etc. It is found that the object block C5 belongs to the lowermost comic frame rather than the comic frame of the frame block F5, so the object block C5 will be judged by the judgment unit 2 as corresponding to The subtractive relationship block of this picture frame block F5. Next, in the subtraction synthesis sub-step S32, the image processing unit 3 will remove the object block C5 determined to be the subtraction relationship block from the frame block F5. The image processing unit 3 mainly has two methods for removing the subtraction related tiles: the interception method and the blank method. The cutting method is to cut the part to be removed from the frame block F5, and the blanking method is to paint the part to be removed white. Since the cutting method will easily make the shape of the frame block F5 complicated, in the subtractive synthesis sub-step S32 of the block synthesis step S30, the corresponding frame block F5 is left blank. Preferably, each subtractive relationship tile is removed from the frame tile. It should be noted that the object removed by the image processing unit 3 is the "object tile", not the range specified by the object tile range information. Therefore, as shown in Figure 6, only the portion of the frame block F5 that overlaps with the character outline shape of the object block C5 is removed, rather than the part specified by the object block range information of the object block C5. ranges are removed. Through the above method, the object tiles belonging to different comic frames can be removed from the frame tiles to reduce foreign object interference in the comic frame image and improve readability.

In addition, although it is shown in Figure 1 that the subtraction relationship determination sub-step S22 and the addition relationship determination sub-step S21 are performed simultaneously, and the subtraction synthesis sub-step S32 and the addition synthesis sub-step S31 are performed simultaneously, the present invention does not use the method. To this extent, the subtraction relationship determination sub-step S22 can be performed before, after, or at the same time as the addition relationship determination sub-step S21. The subtraction synthesis sub-step S32 can also be performed before, after, or at the same time as the addition synthesis sub-step S31.

Preferably, as shown in Figures 1 and 7, in the frame image capturing method according to the embodiment of the present invention, the block synthesis step S30 further includes: an image repair sub-step S33. After the subtractive synthesis sub-step S32, an image restoration algorithm is used to fill in the removed portion R of the frame block in the subtractive synthesis sub-step S32.

Specifically, in the image repair sub-step S33, the image processing unit 3 cuts off, leaves blank or otherwise removes the portion of the frame block in the subtractive synthesis sub-step S32 ( That is, the removed part) is filled into a complete block using image inpainting (Inpainting). For example, as shown in Figure 7, although the removed portion R of the frame block F5 was originally part of the character pattern of the object block C5, the image processing unit 3 will The removed part R is regarded as a damaged or missing part of the frame block F5. Therefore, after image repair, there are no elements of the object block C5 in the frame block F5. Through the above method, the removed parts of the image can be filled in to eliminate the abrupt feeling caused by the lack or blank space of some images, so as not to reduce the attractiveness of the comic.

The above descriptions and explanations are only descriptions of the preferred embodiments of the present invention. Those with ordinary knowledge of this technology may make other modifications based on the patent scope defined below and the above explanations, but these modifications should still be made. It is for the spirit of the present invention and within the scope of rights of the present invention.

1: Artificial intelligence image recognition unit 2: Judgment unit 3:Image processing unit C1: Object tile C2: Object tile C3: Object tile C4: Object tile C5: Object tile F1: Frame tile F2: Frame tile F3: Frame tile F4: Frame tile F5: Frame tile F6: Frame tile MF1: Comic frame image MF6: Comic frame image MP: Comic page image R: removed part S10: Block range identification step S20: Steps to determine block relationship S21: Addition relationship judgment sub-step S22: substep of subtraction relationship judgment S30: Tile synthesis steps S31: Additive synthesis sub-step S32: Subtractive synthesis sub-step S33: Image restoration sub-step T1: Object tile T2: Object tile T3: Object tile

[Figure 1] is a schematic flowchart showing a method for capturing a frame image according to an embodiment of the present invention; [Picture 2] is a schematic diagram showing an image of a comic page; [Figure 3] is a schematic diagram showing the block range identification step of the frame image capturing method according to an embodiment of the present invention; [Figure 4] is a schematic diagram showing the block synthesis steps of the frame image capturing method according to an embodiment of the present invention; [Figure 5] is a schematic diagram showing the block synthesis steps of the frame image capturing method according to an embodiment of the present invention; [Figure 6] is a schematic diagram showing the subtractive synthesis sub-step of the block synthesis step of the frame image capturing method according to an embodiment of the present invention; [Fig. 7] is a schematic diagram showing the image repair sub-step of the block synthesis step of the frame image capturing method according to an embodiment of the present invention.

S10: Block range identification step

S20: Steps to determine block relationship

S21: Addition relationship judgment sub-step

S22: substep of subtraction relationship judgment

S30: Tile synthesis steps

S31: Additive synthesis sub-step

S32: Subtractive synthesis sub-step

S33: Image restoration sub-step

Claims (7)

A method for capturing frame images is used to obtain each comic frame image in a comic page image. The method for capturing frame images includes the following steps: a block range identification step, using a manual The smart image recognition unit performs an image recognition process on the comic page image, and obtains the frame block range information of each frame block and the object block range information of each object block in the comic page image. ; A block relationship judgment step, using a judgment unit to determine the correspondence from the relative positional relationship between the frame block and each object block based on the frame block range information and the object block range information. For each addition relationship block of the picture frame block, the block addition relationship information is obtained, wherein the addition relationship block has a position overlap with the picture frame block and belongs to the position of the picture frame block The object tiles of the same comic frame; and a tile synthesis step, using an image processing unit according to the additive relationship information of the tiles, for each frame tile in the comic page image, the drawing is The frame block and each additive relationship block corresponding to the picture frame block are correspondingly merged while maintaining the relative position relationship between each other, and the comic frame image is obtained, wherein the block relationship The judgment step further includes: a subtraction relationship judgment sub-step, based on the frame block range information and the object block range information, from the relative position relationship between the picture frame block and each of the object blocks, determine the correspondence For each subtraction relationship block of the picture frame block, the subtraction relationship information of the block is obtained, wherein the subtraction relationship block has a position overlap with the picture frame block but belongs to the position of the picture frame block The different object blocks of the comic frame and the block synthesis steps further include: A subtraction synthesis sub-step, according to the block subtraction relationship information, for each of the picture frame blocks in the comic page image, each of the subtraction relationship blocks corresponding to the picture frame block is derived from the picture frame block Remove. The method for capturing a frame image as described in claim 1, wherein in the block range recognition step, the image recognition processing performed by the artificial intelligence image recognition unit is to convert the image in the comic page image into A character pattern is partially recognized as an object tile. The method for capturing a frame image as described in claim 1, wherein in the block range recognition step, the image recognition processing performed by the artificial intelligence image recognition unit is to convert the image in the comic page image into Text dialog portions within a dialog box pattern are recognized as object tiles. The method for capturing a frame image as described in claim 1, wherein in the block range identification step, the frame block range information is the coordinate information of the two diagonal endpoints of the frame block. , the object block range information is the coordinate information of the two diagonal endpoints of the object block. The method for capturing a frame image as described in claim 1, wherein in the block relationship determination step, the additive relationship block is further at least partially protruding from the object image outside the frame block. block. The method for capturing a frame image as described in claim 1, wherein in the subtraction synthesis sub-step of the block synthesis step, each subtraction relationship diagram corresponding to the frame block is left blank. Block is removed from this frame block. The frame image capturing method as described in claim 1 or 6, wherein the block synthesis step further includes: an image repair sub-step, and after the subtractive synthesis sub-step, an image repair algorithm is used to Fill in the removed portions of the frame tile in the subtractive synthesis sub-step.

Images