JP6910605B2 - Image processing device and image processing method - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method for generating composite image data in which texture image data is superimposed on subject image data.

In recent years, research and development in a technical field called Augmented Reality (AR) has been promoted, and by superimposing various images on the image of the actual subject and displaying it on the display, more images can be obtained together with the image of the subject. Information can be visually recognized.

By using AR technology, it becomes possible to easily superimpose a texture image on an image of a real subject and virtually change the pattern and color tone of the surface of the subject, and it is applied to various fields. It is becoming more like. For example, Patent Document 1 describes an image processing device that photographs a user and uses a motion capture technique to replace the clothes actually worn by the user with virtual clothes and display an image. Further, in Patent Document 2, a marker is recognized from an image captured by a camera, the image is acquired as a user image, the position of the marker in the user image is associated with the position of the user's body, and the clothing acquired from the database. An image processing device that generates and displays a fitting image obtained by synthesizing an image with a user image is described.

In such a method of identifying the markers displayed on the subject image and superimposing the texture images, by associating the markers with the texture coordinates of the texture image in advance, it is easy to identify the markers in the subject image. The texture coordinates can be determined. As a method for identifying markers in a subject image, for example, in Non-Patent Document 1, a plurality of single-color circular markers selected from five types of colors are arranged in an area where texture images are superimposed, and three vertical columns × A method of identifying the marker at the center of the range by referring to the color array of the markers in the range of three horizontal columns is described. Further, in Non-Patent Document 2, four types of markers having different numbers of black dots are arranged, and a plurality of markers arranged in a local range are referred to in the same manner as the identification method described in Non-Patent Document 1. Describes how to identify the marker in.

Japanese Unexamined Patent Publication No. 2012-252437 Japanese Unexamined Patent Publication No. 2012-144827

Scholz, V. and Magnor, M., “Texture replacement of garments in monocular video sequences”, In: Heidrich, W., Akenine- Moeller, T. (eds.) Rendering Techniques 2006, Eurographics Symposium on Rendering, pp. 305312 . Eu-rographics (EG), ACM SIGGRAPH, Eurographics, Nicosia, Cyprus (2006) 27 Gaku Narita, Yoshihiro Watanabe, Masatoshi Ishikawa, “Dynamic Projection Mapping onto Deforming Non-rigid Surface using Deformable Dot Cluster Marker”, IEEE Transactions on Visualization and Computer Graphics., 2016

In Patent Documents 1 and 2, it is possible to display a composite image in which a texture image such as a clothing image is superimposed on a user image, but it is difficult to display it realistically according to the movement of the user. In particular, for an easily deformable subject such as clothing, image processing for superimposing texture images following the deformation is required for realistic image display, and Patent Documents 1 and 2 sufficiently support such image processing. Can not.

In Non-Patent Documents 1 and 2, the marker displayed on the deformed subject is detected, and the texture image is corrected based on the texture coordinates associated with the marker, so that the texture image is superimposed according to the deformation of the subject. It is possible to match. However, Non-Patent Document 1 does not assume online processing because the fabric on which the marker is printed is colored in black and it takes time to interpolate the shadows. Further, in Non-Patent Document 2, equipment such as a high-speed camera and IR ink is required, and there is an upper limit to the number of markers that can be arranged, and it is difficult to extend the whole body of the subject.

In addition, since a plurality of markers in a local range are detected to identify the markers, for example, a wrinkle on a subject such as a skirt may partially block the local range and make it impossible to identify. In that case, there is a problem that the marker cannot be accurately identified.

Therefore, an object of the present invention is to provide an image processing apparatus and an image processing method capable of accurately superimposing a texture image on a deformed subject image.

The image processing apparatus according to the present invention has an image data storage unit that stores texture image data and a pattern that is displayed in an array on the surface of the subject and can be individually identified and can identify each block unit for each predetermined number. An identification data storage unit that stores a plurality of generated markers in association with the texture coordinate position of the texture image data and stores identification information for identifying each block unit in association with other block units arranged around the block unit. It is divided into a detection processing unit that detects the marker and its display position based on the image data of the subject, and a classification processing unit that divides the marker into the block units based on the detected pattern of the marker. The block unit is associated with the identification processing unit that identifies the block unit based on the block identification information and identifies the marker in the identified block unit, the display position of the identified marker, and the marker. It is provided with a compositing processing unit that corrects the texture image data based on the texture coordinate position and generates a composite image data in which the corrected texture image data is superimposed on the image data of the subject. Further, the marker is generated by combining a common first pattern for identifying the block unit and an individual second pattern for identifying the marker included in the block unit, and the block identification information is generated by combining the block identification information. It is generated based on the first pattern included in each of the units.

The image processing method according to the present invention generates a plurality of markers in a pattern that can be individually identified and can identify each block unit for each predetermined number, and associates each block unit with other block units arranged around the block unit. The block identification information to be identified is generated, the texture coordinate position of the stored texture image data is stored in association with the marker, and the markers are arranged in block units based on the image data of the subject displayed on the surface. The marker and its display position are detected, the marker is divided into the block units based on the pattern of the detected markers, and the block units are identified based on the block identification information for the divided block units. The texture image data is corrected and the corrected texture is corrected based on the display position of the identified marker and the texture coordinate position associated with the marker by identifying the marker in the identified block unit. A composite image data is generated by superimposing the image data on the image data of the subject. Further, the marker is generated by combining a common first pattern for identifying the block unit and an individual second pattern for identifying the marker included in the block unit, and the block identification information is for the block unit. It is generated based on the first pattern included in each.

The present invention uses a plurality of markers generated in a pattern that is arranged and displayed on the surface of the subject, is individually identifiable, and can identify each block unit for each predetermined number, by having the above configuration. Therefore, even if a part of the marker is shielded due to the deformation of the subject, it is possible to identify the block unit and individually identify the marker. Therefore, the texture image can be accurately superimposed on the deformed subject image.

It is a functional block block diagram which concerns on the image processing apparatus which concerns on this invention. It is explanatory drawing which shows an example of the marker which has distinctiveness. It is a schematic diagram which shows an example of an array pattern in which a plurality of markers are displayed in an array. It is explanatory drawing about the case where the array pattern is divided into block units. It is explanatory drawing which illustrates the case where the marker and the texture coordinates of a texture image are associated. It is explanatory drawing about the case where a marker displayed on a subject is partially shielded. This is a processing flow related to image processing. This is an image example related to a subject image. It is explanatory drawing about the case where the mesh model is generated in the whole area which superimposes the texture image based on the display position and the texture coordinate position of the identified marker. This is an image example showing the identification result in a state where the T-shirt is wrinkled vertically and the marker is hidden. This is an example of an image in which the corrected texture image is combined with the subject image. It is explanatory drawing which shows the processing process which adds a shadow. It is explanatory drawing which shows the processing process of another image processing.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Since the embodiments described below are preferable specific examples for carrying out the present invention, various technical restrictions are made, but the present invention particularly limits the present invention in the following description. Unless otherwise stated, it is not limited to these forms.

FIG. 1 is a functional block configuration diagram relating to an image processing device according to the present invention. The image processing device 1 processes the subject image input from the image input unit 2, corrects the texture image according to the subject image, and generates a composite image in which the corrected texture image is superimposed on the subject image to generate an image. It has a function to display on the output unit 3.

The image input unit 2 is designed to input a live-action image of the subject, a computer graphics image that virtually represents the subject, and the like. For example, an image or communication of an object deformed by a photographing device such as a camera. The image received via the network or the like is input. The subject includes not only a real object that is deformed but also a virtual object that is image-represented based on such a deformed object.

The image output unit 3 can be equipped with various display devices such as a television, a personal computer, a mobile terminal such as a smartphone, and a large display according to the purpose, and can be used with the image processing device 1 via a communication network or the like. By connecting, the image output unit 3 can be installed at various places to display an image.

The image processing device 1 includes an image processing unit 10, an image data storage unit 11 that stores image data such as texture image data, and an identification data storage unit 12 that stores data related to identification such as markers. The image processing unit 10 stores image data such as subject image data and texture image data input from the image input unit 2 in the image data storage unit 11. The texture image data is image data to be superimposed on the subject image data. For example, when the subject is clothing, image data related to clothing design such as a pattern such as a color or a pattern to be displayed on the clothing can be mentioned.

The identification data storage unit 12 stores identification data relating to a plurality of markers arranged and displayed on the surface of the subject, and these markers can be individually identified and a predetermined number of block units can be identified. It is generated by a pattern. Here, the block unit is set in a predetermined range, so that the block unit includes a predetermined number of markers arranged in the range.

FIG. 2 is an explanatory diagram showing an example of a marker having such distinctiveness. In this example, the marker M is composed of concentric double circles, one color selected from a plurality of kinds of colors is displayed in the circular inner region RA, and the outer ring-shaped region is the upper and lower halves. It is divided into outer regions RB and RC, and one selected color is displayed in each of the outer regions RB and RC.

The color displayed in the inner region RA is set so that all the markers in the block unit display the same color. Therefore, the color and range of each block can be identified by dividing the range in which the inner region RA of the marker M is a common color.

The combination of the display colors of the outer regions RB and RC is set so that the markers M in the block unit can be individually identified. Specifically, when N types of colors are used for the outer regions RB and RC, N ² combinations patterns are set, so the number of color types to be used is set according to the number of markers in the block unit. By deciding, the marker M in the block unit can be individually identified by the combination pattern of the display colors of the outer regions RB and RC.

Then, by setting the arrangement of the color combination patterns of the outer regions RB and RC in the block unit to be common, the marker M at the same position in each block unit has the color combination patterns of the outer regions RB and RC. It becomes the same, and the position of the marker M in the block unit can be identified by the color combination pattern. By numbering such color combination patterns in advance corresponding to each combination pattern, it is possible to efficiently process the identification processing.

Each block unit is identified by block identification information associated with other block units arranged around the block unit. The range of each block unit can be specified by the distribution range of the display color of the inner region RA and the range of the combination pattern of the outer region RB and RC, and from the entire display color of the inner region RA of each identified block unit. Based on the arrangement relationship seen, it is possible to generate block identification information associated with the display colors of other block units arranged around each block unit. For example, by using the method of arranging the display colors of the markers described in Non-Patent Document 1, it is possible to arrange the display colors of the block units so that each block unit can be individually identified, and each block unit can be arranged. It is possible to create block identification information that can identify.

When identifying using the marker display colors described above, it is preferable to select a color in which the display colors of the markers M displayed on the live-action image of the subject can be clearly distinguished from each other. For example, in a color space such as the HSV color system, it is preferable to use colors that are as far apart from each other as possible. As the marker, an identifiable display means other than the display color may be used, and for example, the marker can be configured by using a display means that can be detected by an infrared sensor or the like.

In the above example, the marker pattern based on the color combination has been described, but each marker is a combination of hierarchical patterns by combining a common pattern for identifying a block unit and an individual pattern for identifying a marker within a block unit. It may be configured. Therefore, the design of the marker can be freely changed, and for example, it can be configured by a combination of points and lines, and is not limited to a combination of colors. Further, the color originally possessed by the subject can be used for one pattern of the marker, and such a pattern makes it possible to simplify the marker detection process.

FIG. 3 is a schematic diagram showing an example of an array pattern H in which a plurality of markers M are arranged and displayed. Further, FIG. 4 is an explanatory diagram (FIG. 4A) illustrating the case where the array pattern H shown in FIG. 3 is divided into block units K, and the display color of the divided block unit K (display color of the inner region RA). ) Is replaced with a circular mark (FIG. 4 (b)) and an explanatory diagram (FIG. 4 (c)) illustrating the arrangement of markers in the block unit.

As shown in FIG. 3, the difference in the display color of each region of the marker M is schematically displayed by the difference in shade. In this example, 81 markers M are arranged at equal intervals of 9 in each of the vertical and horizontal directions, and the block unit K is set by dividing them into 3 in each of the vertical and horizontal directions. The method of arranging the markers M may be appropriately set according to the texture coordinates of the texture image, and the range of each block may be set assuming the shielding range due to the deformation of the subject.

The display color of the inner region RA of the marker M, which is the display color of each block unit K, is selected from, for example, four colors of red, green, blue, and black, and is the same as the known selection method described in Non-Patent Document 1. You can select and place it in. Since the number of types used for the display colors of the outer regions RB and RC is 9 for each block unit K, it may be 3 or more colors, and a color that can be clearly distinguished from each other and detected should be selected. Just do it. As the number of colors increases, the possibility of erroneously detecting the marker increases, so in this example, three colors are selected. For example, two colors are selected from the three colors of yellow, cyan, and magenta, and the colors are arranged according to the combination pattern of the outer regions RB and RC in the block unit K.

As shown in FIG. 4B, in this example, the block identification information for identifying the block unit corresponds to a total of nine display colors for one block unit and eight block units arranged around the block unit. The identification information (ID) can be sequentially arranged clockwise and created. For example, numbers 1 to 4 may be applied to each of the four display colors, and a number string corresponding to the array of display colors may be created and used as block identification information. In addition, when it is assumed that the arrangement pattern of the subject image is rotated and displayed, a number string is created in the arrangement order according to the rotation, and a plurality of block identification information is created for one block unit. It may be. In addition, in the corner block unit, the number of block units arranged around is three, and in the side block unit, the number of block units arranged around is five. These are also arranged clockwise in the display color. Block identification information can be created in order.

By identifying the block unit by the block identification information and individually identifying the markers in the block unit in this way, each marker can be hierarchically identified. In this example, in order to identify each marker individually, as shown in FIG. 4C, numbers 1 to 9 are applied and identified according to the combination pattern of the outer region of each marker.

Then, by associating each marker with the texture coordinates of the texture image, it is possible to superimpose the texture image in association with the display position of the marker displayed on the subject image. FIG. 5 is an explanatory diagram illustrating a case where the marker and the texture coordinates of the texture image are associated with each other. In this example, the texture coordinates of the texture image G composed of the pattern pattern are associated with a plurality of markers constituting the array pattern H.

As described above, the identification data storage unit 12 contains data for associating each marker with the texture coordinate position together with data for identifying the generated marker pattern and block units set for each predetermined number of markers. It is registered.

The image processing unit 10 includes a detection processing unit 101, a classification processing unit 102, an identification processing unit 103, and a synthesis processing unit 104, and the detection processing unit 101 detects a marker and its display position based on the image data of the subject. Perform the processing to be performed. In the case of a photographed image in which the subject is actually photographed, it is preferable to perform mask processing for separating the background image for the subject and perform preprocessing for reducing false detection of markers. When the marker is composed of a color pattern as in the above-mentioned example, the marker pattern and its display position can be detected by performing the process of detecting the constituent colors from the image.

Specifically, a binary image of each color is created, labeling processing is performed, and those having an area within a certain range are classified as a marker area. For binary images, noise can be reduced and the risk of false positives can be reduced by performing median filters and morphology calculations.

Based on the obtained binary image, the color combination of the inner region and the outer region of the pattern is determined, and the distance from the center of gravity position of the image region corresponding to the inner region to the center of gravity position of the image region corresponding to the outer region is calculated. However, if the value is equal to or less than a certain value, it can be determined that the markers are the same, and the marker can be detected. At that time, each marker can be specified by the combination pattern of the outer region, and for example, it may be specified by the numbered data. As for the display position of the marker, for example, the coordinates of the center of gravity of the inner region may be used as the display position. In this way, the marker displayed on the subject image and its display position can be detected.

The classification processing unit 102 performs a process of classifying the markers into block units based on the marker pattern detected by the detection processing unit 101. In the above example, the color of the inner region of the marker is the same for each block, and the color combination pattern of the outer region is common for each block. Therefore, the combination of the outer regions of each marker is used. By checking, it is possible to identify the block unit and to specify and classify the range. In a subject that is easily deformed, such as clothing, some markers are shielded in the arrangement pattern (see FIG. 6A) displayed on the subject, as illustrated in FIG. 6 (see FIG. 6B). In some cases, if the markers constituting the block unit are partially displayed in the subject image as shown in FIG. 6 (b), the combination of the color of the inner region and the outer region of the displayed markers should be confirmed. Can be divided into blocks.

The identification processing unit 103 identifies the divided block units based on the block identification information, and also identifies the markers in the identified block units. In the above example, since the block identification information is created based on the display color of the inner area of the marker of each block unit, the display color is clockwise for one block unit and the block units arranged around it. The block identification information is sequentially checked to determine whether or not they match, and if they match, the block identification information is identified as a registered block unit.

Also in the example shown in FIG. 6, since some markers of the block unit are displayed, the display color of the block unit can be identified, and the block unit can be identified based on the block identification information.

By identifying the block unit, the markers in the block unit can also be easily identified based on the color combination of the outer region. Then, the display positions detected for the individually identified markers are used as they are in the synthesis processing unit 104.

The compositing processing unit 104 corrects the texture image data based on the display position of the identified marker and the texture coordinate position associated with the marker, and superimposes the corrected texture image data on the image data of the subject. Perform the process of generating. In the above-described example, the display position of the marker registered in the storage unit 12 and the texture coordinate position corresponding to the marker registered in advance are read out, and are described in Non-Patent Document 1 based on the positional relationship of both. The known interpolation processing is performed to correct the texture image stored in the storage unit 11 so as to match the subject image, and generate composite image data superimposed on the subject image.

When the subject image is a moving image, the markers are identified for each frame constituting the moving image as described above, and the texture image is corrected and linked based on the variation between the frames of the identified markers. By doing so, it is possible to generate a composite image by following the moving image.

Further, when the subject image is composed of a three-dimensional image having a shadow, the texture image does not include the shadow, so it is necessary to perform a process of adding a shadow to the superimposed composite image. Become. Specifically, the shadow processing can be performed by reflecting the image on the texture image interpolated based on the brightness of the pixels constituting the subject image. However, since the reflectance differs depending on the colors constituting the marker, it is necessary to interpolate the brightness of the mark area by referring to the brightness of the pixels in the area other than the marker.

FIG. 7 is a processing flow related to image processing. First, an input image in which the subject is displayed is acquired (S100). For example, as shown in FIG. 8, an image taken with the worn T-shirt as a subject is acquired as an input image. The marker arrangement pattern shown in FIG. 3 is color-printed on the front side of the plain T-shirt, and the image shown in FIG. 8 is an image taken by the photographing device from the front side. Further, as described with reference to FIG. 5, the texture coordinate positions of the texture image are associated and stored in advance corresponding to the arrangement pattern of the markers.

Next, a marker detection process is performed on the acquired subject image (S101). As described above, a binary image is generated for a plurality of types of colors used for the marker, and the marker and the display position in the subject image are detected. Then, as described above, the block unit division processing is performed based on the detected marker pattern (S102), the divided block units are identified based on the block identification information, and the markers in the block units are identified. (S103).

Next, the texture image is corrected based on the display position and the texture coordinate position of the identified marker (S104), and a composite image is generated by superimposing it on the subject image (S105), and the generated composite image is used as an image output unit. Output processing to 3 (S106).

FIG. 9 is an explanatory diagram regarding a case where a mesh model is generated over the entire region where the texture images are superimposed based on the display position and the texture coordinate position of the identified markers. Based on the display position of the marker that has been successfully identified, the mesh model is deformed according to the area where the subject image is superimposed by a known method such as interpolation processing using the Radical Basis Function.

The black dots in the squares shown in FIG. 9 indicate the display positions of the identified markers, and the identification accuracy of the markers can be improved by performing the identification in block units and the identification of the markers hierarchically. FIG. 10 is an image example showing a marker identification result in a state where vertical wrinkles are formed on the T-shirt and one or more rows of markers are hidden. FIG. 10B shows the result of performing marker identification processing by the conventional method shown in Non-Patent Document 1 on the input image shown in FIG. 10A, and FIG. 10D shows the result. The result of hierarchical identification processing for identifying the block unit and identifying the marker is shown for the input image shown in FIG. 10 (c).

In the input images shown in FIGS. 10 (a) and 10 (c), the markers are partially hidden by vertical wrinkles. In FIGS. 10 (b) and 10 (d), the identified markers are indicated by square black dots. Looking at the identification results, in FIG. 10B, the markers could not be identified along the portion where the vertical wrinkles were formed, and the identification accuracy was lowered. Since one row of markers is shielded at the wrinkled part, the method of identifying the marker based on other markers arranged around the marker is a marker arranged around the shielded marker. Can no longer be identified. On the other hand, in FIG. 10D, since the block unit including the shielded marker can be identified, it is possible to identify a marker other than the shielded marker. Therefore, the identification accuracy of the marker can be improved, and the subsequent correction of the texture image can be performed with high accuracy.

FIG. 11 shows an example of an image in which the corrected texture image is combined with the subject image. In the composite image example shown in FIG. 11, since the shadow due to the wrinkles of the T-shirt displayed in the subject image is not displayed, the texture image is superimposed on the subject image without discomfort by performing the process of adding the original shadow. A composite image can be generated.

FIG. 12 is an explanatory diagram showing a processing process for adding a shadow. FIG. 12A shows a brightness image of the subject image shown in FIG. 8, and the brightness due to the color of the marker is expressed in addition to the shadow due to the wrinkles. Therefore, the interpolation process of the brightness image is performed with reference to the brightness of the region other than the marker, and the interpolated image as shown in FIG. 12B is generated.

FIG. 13 is an explanatory diagram showing a processing process of another image processing. The upper image string and the lower image string are the input image from the left side, the mesh model image based on the marker (white square) identified in the input image, the composite image by the mesh model, the correction image, and the error map image. Shown. The modified image is an image modified by performing supplementary processing for associating the display position of the marker that could not be identified in the mesh model image with the texture coordinate position. The error map image is an image in which the magnitude of the color shift between the composite image and the corrected image is displayed for each pixel in different colors. In this example, the resolution of the input image is 1080 pixel × 1080 pixel, and the maximum value of color shift is 30 pixels, which is displayed in five stages.

The upper image sequence shows the result of image processing using the conventional method described in Non-Patent Document 1. In the input image, the marker is partially shielded by vertical wrinkles, and the area where the marker cannot be identified extends from the shielded portion. Therefore, an error occurs in the area where the marker cannot be identified, and the color shift becomes larger in the area closer to the shielded portion. On the other hand, the lower image sequence shows the result of processing using the image processing according to the present invention. In the input image, the markers are almost shielded by vertical wrinkles, but since the displayed markers can be almost identified, the error area is small even in the shielded portion. Therefore, the composite image almost matches the corrected image, and it can be confirmed that the composite image accurately follows the deformation of the subject.

As described above, since the block unit is set for each predetermined number of markers to identify the block unit and individually identify the markers in the block unit, some markers in the block unit are shielded. Even if it is done, it is possible to identify other unobstructed markers. Therefore, in the case of a subject that is easily deformed such as a cloth, even if the marker displayed on the subject is partially shielded by the deformation, the marker near the shielded portion can be identified by referring to the peripheral markers globally. It becomes possible to improve the identification accuracy of the marker. By associating the marker with the texture coordinate position of the texture image, it is possible to correct the texture image based on the display position of the marker identified in the subject image and superimpose it on the subject image to generate a composite image. It is possible to follow the deformed subject and perform image processing without discomfort.

1 ... image processing device, 2 ... image input unit, 3 ... image output unit, 10 ... image processing unit, 11 ... image data storage unit, 12 ... identification data storage unit, 101 ... Detection processing unit, 102 ... Classification processing unit, 103 ... Identification processing unit, 104 ... Synthesis processing unit, G ... Texture image, H ... Arrangement pattern, K ... Block unit, M ... marker

Claims (4)

A texture image of an image data storage unit that stores texture image data, and a plurality of markers that are displayed in an array on the surface of the subject and are generated in a pattern that can be individually identified and can identify each block unit for each predetermined number. The identification data storage unit that stores the block identification information that stores the data in association with the texture coordinate position of the data and identifies the block unit in association with other block units arranged around each block unit, and the identification data storage unit that stores the image data of the subject. The detection processing unit that detects the marker and its display position, the division processing unit that divides the marker into the block units based on the detected pattern of the marker, and the block identification information for the divided block units. The texture is based on the identification processing unit that identifies the block unit and identifies the marker in the identified block unit, the display position of the identified marker, and the texture coordinate position associated with the marker. An image processing apparatus including a composition processing unit that corrects image data and generates composite image data in which the corrected texture image data is superimposed on the image data of the subject. The marker is generated by combining a common first pattern for identifying the block unit and an individual second pattern for identifying the marker included in the block unit, and the block identification information is for the block unit. The image processing apparatus according to claim 1, which is generated based on the first pattern included in each. Multiple markers are generated in a pattern that can be individually identified and each block unit can be identified, and block identification information that identifies each block unit in association with other block units arranged around it is generated. The texture coordinate position of the stored texture image data is stored in association with the marker, and the marker and its display position are detected based on the image data of the subject displayed on the surface by arranging the markers in block units. , The marker is divided into the block units based on the pattern of the detected markers, and the block units are identified and identified within the block units based on the block identification information for the divided block units. The marker is identified, the texture image data is corrected based on the display position of the identified marker and the texture coordinate position associated with the marker, and the corrected texture image data is used as the image data of the subject. An image processing method that generates superimposed composite image data. The marker is generated by combining a common first pattern for identifying the block unit and an individual second pattern for identifying the marker included in the block unit, and the block identification information is provided for each of the block units. The image processing method according to claim 3, which is generated based on the first pattern included.

Images