JP5003790B2 - Image processing apparatus, image processing method, and computer program - Google Patents

Description

  The present invention relates to an image processing technique for performing deformation processing on an image.

  In the image transformation process for transforming an image, a pixel (reference source pixel) constituting the transformed image is associated with a pixel (reference destination pixel) of the transformation source image (referred to as “mapping”), and a reference source pixel Is set to the pixel value of the reference pixel.

  As described above, in the image transformation process using mapping, the positional relationship between the reference source pixel and the reference destination pixel varies depending on the position of the reference source pixel in the transformed image depending on the content of the transformation process. Therefore, in the image deformation process using mapping, conventionally, a memory area that can store the entire image before and after the deformation is secured, and the deformation process is performed in the secured memory area.

JP 2004-318204 A

  However, depending on the size of the image to be deformed and the memory capacity of the image processing apparatus that performs image processing, it may not be possible to secure a memory area that can store the entire image before and after deformation, and deformation processing may not be performed. .

  The present invention has been made to solve the above-described conventional problems, and an object thereof is to reduce the amount of memory required for image deformation processing for deforming an image.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

[Application Example 1]
An image processing apparatus, comprising: an image deformation processing unit that performs a deformation process on a pre-deformation image and generates a post-deformation image in the post-deformation image region, wherein the image deformation processing unit And a target block setting unit that sets a target block that is a destination of the transformation process, and is included in a reference block in the pre-transformation image that is associated with the target block and serves as a source of the transformation process A block deformation processing unit that generates an image in the target block by referring to the image, and sequentially moves the target block to a different position in the post-deformation image area, and the image in the target block after the movement Is generated by the block deformation processing unit, and a block scanning unit that generates the entire image after deformation, and the target block by the block deformation processing unit. Prior to the generation of the image, a maximum block size calculation unit that calculates a maximum block size of the reference block at all positions where the target block is arranged, and a maximum block size calculated by the maximum block size calculation unit A source memory area securing unit that secures a source memory area so that an image of a size can be stored, and the block transformation processing unit receives an image included in the reference block associated with the target block. An image processing apparatus that generates an image of the target block with reference to an image included in the reference block stored in the source memory area and stored in the source memory area.

  In this application example, prior to the generation of the image of the target block by the block deformation processing unit, the maximum block size of the reference block at all positions where the target block is arranged is calculated. Then, a source memory area is secured so that an image having the maximum block size can be stored. Therefore, since the pre-deformation image required for generating the image in the target block can be stored in the source memory area, the deformation process can be performed in units of blocks. By performing the deformation process in units of blocks, the memory area used for the deformation process can be reduced, and the amount of memory required for the image deformation process can be reduced.

[Application Example 2]
In the image processing device according to application example 1, the image deformation processing unit may generate an image after multiple deformation by performing the deformation processing on the original image N times (N is an integer of 2 or more). The maximum block size calculation unit can calculate a logical sum of N reference blocks in the N deformation processes referenced from individual target blocks set in an image area of the multiple-deformed image. An upper limit block size calculating unit that calculates the upper limit block size prior to the N deformation processes, and the memory area securing unit generates an image of the upper limit block size corresponding to each deformation process. N source memory areas that can be stored are secured, and among the N source memory areas, the i-th source memory area (i is an integer of 1 or more and N or less) is the source of the i-th deformation process. To be used, it is used as the destination of the i-1 th transformation processing, the image processing apparatus.

  In this application example, N source memory areas capable of storing an image having an upper limit block size corresponding to each deformation process are secured. Therefore, in any of the N deformation processes, the source image required for generating the destination image of the deformation process can be stored in the source memory area. Therefore, even when the deformation process is performed a plurality of times, the deformation process can be performed in units of blocks, and the amount of memory required for the image deformation process can be reduced.

  Note that the present invention can be realized in various modes, for example, an image processing method and apparatus, an image deformation method and apparatus, an image correction method and apparatus, and a function of these methods or apparatuses. The present invention can be realized in the form of a computer program, a recording medium recording the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

1 is an explanatory diagram schematically showing a configuration of a printer to which an image processing apparatus as an embodiment of the present invention is applied. It is explanatory drawing which shows an example of the user interface containing the list display of an image. 7 is a flowchart illustrating a face shape correction printing routine executed when face shape correction printing is performed in a printer. Explanatory drawing which shows an example of the user interface for setting the type and degree of image deformation. Explanatory drawing which shows an example of the setting result of a deformation | transformation area | region. Explanatory drawing which shows a mode that each of two deformation | transformation area | regions is divided | segmented into a small area | region. Explanatory drawing which shows a mode that a deformation | transformation parameter is set. Explanatory drawing which shows a mode that the deformation | transformation parameter was set about each of the two deformation | transformation area | regions 2 in step S150. The flowchart which shows the flow of the band buffer securing process performed in step S200. Explanatory drawing which shows the bandwidth required when transforming an image. 10 is a flowchart showing the flow of a maximum bandwidth calculation process executed in step S210 of FIG. Explanatory drawing which shows a bandwidth required when a some deformation | transformation process is performed. The flowchart which shows the flow of the line deformation | transformation process performed in step S300. Explanatory drawing which shows a mode that the image already deform | transformed by the line deformation | transformation process is produced | generated. The flowchart which shows the flow of the transformed image generation process of the reference range performed in step S330. Explanatory drawing which shows the state of the band buffer at the time of the deformation | transformation process of an image.

Next, embodiments of the present invention will be described in the following order based on examples.
A. Example:
B. Variations:

A. Example:
FIG. 1 is an explanatory diagram schematically showing the configuration of a printer 100 to which an image processing apparatus as an embodiment of the present invention is applied. The printer 100 is a color inkjet printer compatible with so-called direct printing, in which an image is printed based on image data acquired from a memory card MC or the like. The printer 100 includes a CPU 110 that controls each unit of the printer 100, an internal memory 120 configured by, for example, a ROM and a RAM, an operation unit 140 configured by buttons and a touch panel, a display unit 150 configured by a liquid crystal display, A printer engine 160 and a card interface (card I / F) 170 are provided. The printer 100 may further include an interface for performing data communication with other devices (for example, a digital still camera or a personal computer). Each component of the printer 100 is connected to each other via a bus.

  The printer engine 160 is a printing mechanism that performs printing based on print data. The card interface 170 is an interface for exchanging data with the memory card MC inserted in the card slot 172. In the present embodiment, image data as RGB data is stored in the memory card MC, and the printer 100 acquires the image data stored in the memory card MC via the card interface 170.

  The internal memory 120 stores a face shape correction unit 200, a display processing unit 310, and a print processing unit 320. The face shape correction unit 200 is a computer program for executing a face shape correction process described later under a predetermined operating system. The display processing unit 310 is a display driver that controls the display unit 150 to display processing menus and messages on the display unit 150. The print processing unit 320 is a computer program for generating print data from image data, controlling the printer engine 160, and printing an image based on the print data. The CPU 110 implements the functions of these units by reading and executing these programs from the internal memory 120.

  The face shape correction unit 200 includes a deformation mode setting unit 210, a deformation region setting unit 220, a deformation region dividing unit 230, a deformation parameter setting unit 240, and a divided region deformation unit 250 as program modules. . The divided region transformation unit 250 includes a maximum bandwidth calculation unit 252, a band buffer allocation unit 254, and a transformation processing unit 256. The functions of these units will be described later.

  The internal memory 120 also stores a dividing point arrangement pattern table 410 and a dividing point movement table 420. The band buffer 430 is a storage area secured in the internal memory 120 by the band buffer allocation unit 254 as will be described later. The contents of the dividing point arrangement pattern table 410 and the dividing point movement table 420 and the function of the band buffer 430 will be described later.

  The printer 100 prints an image based on the image data stored in the memory card MC. When the memory card MC is inserted into the card slot 172, the display processing unit 310 displays a user interface including a list display of images stored in the memory card MC on the display unit 150. FIG. 2 is an explanatory diagram illustrating an example of a user interface including a list display of images. In this embodiment, the list display of images is realized using thumbnail images included in image data (image files) stored in the memory card MC. In the user interface shown in FIG. 2, eight thumbnail images TN1 to TN8 and five buttons BN1 to BN5 are displayed.

  When the user selects an image on the user interface shown in FIG. 2 and operates the normal print button BN3, the printer 100 executes normal print processing for printing the selected image normally. On the other hand, when the user selects an image on the user interface and operates the face shape correction print button BN4, the printer 100 corrects the shape of the face in the image and prints the corrected image for the selected image. The face shape correction printing process is executed. In the example of FIG. 2, the thumbnail image TN1 is selected, and the face shape correction print button BN4 is operated. Therefore, the printer 100 corrects the face shape of the image corresponding to the thumbnail image TN1, and prints the corrected image.

  FIG. 3 is a flowchart illustrating a flow of face shape correction printing processing executed when the printer 100 performs face shape correction printing. As described above, the face shape correction printing process is executed by the printer 100 in accordance with the user's operation of the face shape correction print button BN4 on the user interface shown in FIG.

  In step S110, the face shape correction unit 200 (FIG. 1) sets a target image to be subjected to face shape correction processing. The face shape correction unit 200 sets an image corresponding to the thumbnail image TN1 selected by the user in the user interface shown in FIG. 2 as a target image. In the following, the original image that is the target of face shape correction is also referred to as “original image”, and the image data representing the original image is also referred to as “original image data”.

  In step S120, the deformation mode setting unit 210 (FIG. 1) sets the image deformation type and the degree of image deformation for face shape correction. The deformation mode setting unit 210 instructs the display processing unit 310 to display a user interface for setting the type and degree of image deformation on the display unit 150, and the image deformation type designated by the user through the user interface. And the degree are selected and set as the image deformation type and degree used for processing.

  FIG. 4 is an explanatory diagram showing an example of a user interface for setting the type and degree of image deformation. As shown in FIG. 4, the user interface includes an interface for setting the image deformation type and a preview image of the correction result. As the image deformation type, a plurality of options such as a deformation type “type A” that sharpens the shape of the face and a deformation type “type B” that increases the shape of the eyes are set in advance. The user operates the interface to select the image deformation type from a plurality of options. The deformation mode setting unit 210 sets the image deformation type selected by the user as an image deformation type used for actual processing.

  The user interface shown in FIG. 4 includes an interface for setting the degree (degree) of image deformation. As the degree of image deformation, for example, three levels of options of strong (S), medium (M), and weak (W) are set in advance. The user operates this interface to specify the degree of image deformation. The deformation mode setting unit 210 sets the degree of image deformation designated by the user as the degree of image deformation used for actual processing. A check box provided in the user interface is checked when the user desires detailed designation of the deformation mode.

  When the user operates various interfaces provided in the user interface shown in FIG. 4, the face shape correction unit performs a deformation process on the reduced image obtained by reducing the original image, and the deformed image is displayed as a thumbnail image. Is done. However, it is also possible to omit the display of thumbnail images.

  In the following, the deformation type “type A” for sharpening the shape of the face is set as the image deformation type, the degree of “medium” is set as the degree of image deformation, and the user desires detailed designation The explanation will be given assuming that there was no.

  In step S130 (FIG. 3), the deformation area setting unit 220 sets an area (deformation area) on which image deformation processing for face shape correction is performed. In setting the deformation area, first, the face area is detected. The position and direction of the detected face area are adjusted. And a deformation | transformation area | region is set based on the result in which the position and direction were adjusted. The face area is detected by a known face detection method such as a pattern matching method using a template (see Japanese Patent Application Laid-Open No. 2004-318204).

  FIG. 5 is an explanatory diagram showing an example of the result of setting the deformation area. 5A shows the detection result of the face area, and FIG. 5B shows the deformation area set based on the detected face area. In the example of FIG. 5, the target image TI includes two persons. Therefore, as indicated by the thick line in FIG. 5A, two face areas FA1 and FA2 corresponding to two persons are detected from the target image TI. These face areas FA1 and FA2 are rectangular areas each including an image of each eye, nose and mouth. The broken line in FIG. 5B indicates two face areas FA1 and FA2, and the thick line indicates two deformation areas TA1 and TA2 set for each of the two face areas FA1 and FA2.

  In step S140 of FIG. 3, the deformation area dividing unit 230 divides the deformation area set in step S130 into a plurality of small areas. Specifically, the deformation area dividing unit 230 arranges dividing points with respect to the deformation area set in step S130. And a deformation | transformation area | region is divided | segmented into a several small area | region using the straight line which ties a dividing point. Note that the arrangement mode (number and position of division points) of the division points in the deformation area is defined in association with the deformation type set in step S120 (FIG. 3) by the division point arrangement pattern table 410 (FIG. 1). Has been. As shown in FIG. 4, when the deformation “type A” for sharpening the face is selected as the deformation type, the deformation area is divided into 15 rectangular small areas.

  FIG. 6 is an explanatory diagram showing how each of the two deformation areas TA1, TA2 set in step S130 of FIG. 3 is divided into small areas. FIG. 6A shows the arrangement of the two deformation areas TA1 and TA2 set in step S130 of FIG. FIG. 6B shows how each of the two deformation areas TA1 and TA2 is divided into small areas in step S140. In this embodiment, the deformation “type A” is selected as the deformation type. Therefore, as shown in FIG. 6B, the two deformation areas TA1 and TA2 are each divided into 15 rectangular small areas.

  In step S150 of FIG. 3, the deformation parameter setting unit 240 sets various parameters when performing deformation processing on the deformation area. Specifically, the movement mode (movement direction and movement distance) of the dividing points used for dividing the deformation area into small areas is set. The movement mode of the dividing points is determined in advance by the dividing point movement table 420 (FIG. 1) in association with the combination of the deformation type and the degree of deformation set in step S120 (FIG. 3). As described above, when the deformation “type A” (see FIG. 4) for sharpening the face is set as the deformation type and the degree of “medium” is set as the deformation degree, The movement mode is set to a movement direction and a movement distance associated with a combination of these deformation types and deformation degrees.

  FIG. 7 is an explanatory diagram showing how the deformation parameters are set. FIG. 7A shows an example of the contents of the dividing point movement table 420 referred to by the deformation parameter setting unit 240. FIG. 7B shows how the division points move according to the division point movement table 420 shown in FIG.

  The dividing point movement table 420 shows the amount of movement along the direction (H direction) orthogonal to the reference line RL and the direction parallel to the reference line RL (V direction) for each of the dividing points D11 to D42. Here, the reference line RL is a direction that bisects the deformation area TA, and is a direction that corresponds to the vertical direction of the face. The unit of the movement amount shown in the dividing point movement table 420 is the pixel pitch PP on the image. For the H direction, the amount of movement to the right side is represented as a positive value, the amount of movement to the left side is represented as a negative value, and for the V direction, the amount of movement upward is positive. It is expressed as a value, and the downward movement amount is expressed as a negative value. For example, the dividing point D11 is moved to the right along the H direction by a distance that is seven times the pixel pitch PP, and is moved upward along the V direction by a distance that is 14 times the pixel pitch PP. For example, the division point D22 is not moved because the movement amount is zero in both the H direction and the V direction.

  Thus, by setting the movement mode of the dividing points, the dividing points in the deformation area TA move from the positions indicated by white circles to the positions indicated by black circles as shown in FIG. 7B. In FIG. 7B, the positions of the division points that are not moved are also indicated by black circles. As described above, by setting the deformation parameter, in the deformation process, the image in the small area indicated by the broken line in the deformation area TA is deformed to the image in the small area indicated by the solid line. For example, an image in a small area (small area shown with hatching) having the divide points D11, D21, D22, and D12 as vertices has the divide points D′ 11, D′ 21, D22, and D′ 12 as vertices. It is transformed into an image in a small area.

  The deformation of the image in the small area is performed by mapping (mapping) a point in the small area after the deformation to a point in the small area before the deformation. In the deformation by mapping, the position (reference position) of the corresponding pixel (reference pixel) in the small area before deformation is calculated from the position (target pixel position) of the pixel (target pixel) in the small area after deformation. . Then, by setting the pixel value of the reference pixel in the small area before the deformation to the pixel value of the target pixel, the image in the small area is deformed. When the reference position is shifted from the position of the pixel in the small area before the deformation, the pixel value of the target pixel is calculated from the pixels near the reference position by interpolation processing (for example, bilinear method or bicubic method). Set pixel values. In the following, a virtual pixel whose pixel value is calculated by interpolation processing in this way is also referred to as a reference pixel.

  As shown in FIG. 6A, the two deformation areas TA1 and TA2 set in step S130 are in an overlapping state. When the deformation areas overlap in this way, the deformation parameter setting unit 240 determines the priority of deformation of the overlapping deformation areas in step S140. The priority of deformation is set by the position of the deformation area and the like. In this case, for example, the deformation area TA2 in which the upper end position of the deformation area is higher is given priority. However, the priority order of deformation may be set based on other parameters such as the size of the deformation area. In the following description, a deformation area having a higher priority is also referred to as an “upper” deformation area, and a process for deforming an upper deformation area is also referred to as an “upper” deformation process. In addition, a deformation area having a lower priority is also referred to as a “lower” deformation area, and a process for deforming a lower deformation area is also referred to as a “lower” deformation process.

  FIG. 8 is an explanatory diagram showing how the deformation parameters are set for each of the two deformation areas TA1 and TA2 in step S150. As shown in FIG. 8, the dividing points indicated by the white circles of the two deformation areas TA1 and TA2 are moved to the positions indicated by the black circles. For this reason, the small areas in the two deformation areas TA1 and TA2 indicated by the dotted lines are respectively transformed into small areas indicated by the solid lines.

  In step S200 of FIG. 3, the band buffer allocating unit 254 (FIG. 1) secures a band buffer in the internal memory 120 (FIG. 1). FIG. 9 is a flowchart showing the flow of the band buffer securing process executed in step S200.

  In step S210, the band buffer allocation unit 254 calculates the maximum bandwidth. Here, the band is a band-like region extending in the horizontal direction of an image used in a deformation process described later, and the band width refers to the number of pixels in the vertical direction included in the band.

  FIG. 10 is an explanatory diagram showing a bandwidth necessary for deforming an image. In the example of FIG. 10, the image is deformed so that the two vertices below the figure (arrow) shown in FIG. 10A are moved as shown in FIG. The squares in FIG. 10B indicate vertices V1 to V3, V6, and V7 that are not moved. The white circles in FIG. 10B indicate the positions of the vertices V4 and V5 before the deformation, and the black circles indicate the positions of the vertices V4 'and V5' after the deformation.

  When the transformed vertex V4 'is the target pixel, the reference pixel is the vertex V4. Therefore, in the case of performing the transformation as shown in FIG. 10A to FIG. 10B, the pixel value of the reference pixel V4 is referred to in order to determine the pixel value of the target pixel V4 '. Therefore, when the image is deformed as shown in FIG. 10, the target pixel V4 ′ and the reference pixel V4 that is a distance dd (hereinafter also referred to as “lower displacement”) below the target pixel V4 ′ are banded. The bandwidth is set to be included in. Similarly, the pixel value of the reference pixel V5 is referred to in order to determine the pixel value of the target pixel V5 '. Therefore, when the image is deformed as shown in FIG. 10, the target pixel V5 ′ and the reference pixel V5 that is away from the target pixel V5 ′ by a distance du (hereinafter also referred to as “upper displacement”) are included in the band. The bandwidth is set so that

  FIG. 11 is a flowchart showing the flow of the maximum bandwidth calculation process executed in step S210 of FIG. In step S211, the maximum bandwidth calculation unit 252 (FIG. 1) sets both the maximum values of the upper side displacement and the lower side displacement to zero. Next, the maximum bandwidth calculation unit 252 sets the upper left pixel of the target pixel as the target pixel (step S212).

  In step S213, the maximum bandwidth calculation unit 252 acquires the reference position of the target pixel. In step S214, an upper displacement or a lower displacement (hereinafter, the upper displacement and the lower displacement are simply referred to as “displacement”) between the reference position and the target pixel position is acquired. The acquired displacement is compared with the maximum displacement value by the maximum bandwidth calculation unit 252 and the maximum displacement value is updated (step S215).

  In step S216, the maximum bandwidth calculation unit 252 determines whether there is a higher-order deformation process. If it is determined that there is a higher-order transformation process, control is transferred to step S217. On the other hand, if it is determined that there is no higher-order deformation process, control is transferred to step S218.

  In step S217, the maximum bandwidth calculation unit 252 acquires the reference position of the reference pixel by the higher-order deformation process. After obtaining the reference position of the reference pixel, the control is returned to step S214, and steps S214 to S217 are repeatedly executed until there is no higher-order deformation process for which the reference position is to be obtained.

  FIG. 12 is an explanatory diagram showing a bandwidth required when a plurality of deformation processes are performed. In the example of FIG. 12, two deformation processes, a first deformation process and a second deformation process, are performed. FIG. 12A shows a state in which the first deformation process, which is a higher-order deformation process, is performed, and FIG. 12B shows a state in which a second deformation process, which is a lower-order deformation process, is performed. Yes. FIG. 12C shows a state in which the second deformation process is performed after the first deformation process. In FIGS. 12A to 12C, a broken line represents the shape of the small area before the deformation process, and a solid line represents the shape of the small area after the deformation process.

  As shown in FIG. 12A, in the first deformation process, a small rectangular area having a white square as a vertex is transformed into a rectangular small area having a black square as a vertex. Therefore, the pixel P0 in the small region before deformation indicated by a white circle is moved to the pixel P1 above the distance d1 by the first deformation processing. Similarly, as shown in FIG. 12B, in the second deformation process, a small rectangular area having a white triangle as a vertex is transformed into a rectangular small area having a black triangle as a vertex, and is represented by a white circle. The pixel P1 in the previous small area is moved to the pixel P2 above the distance d2.

  Thus, by sequentially performing the first deformation process and the second deformation process, the position of the pixel P0 is moved to the position of the pixel P2, as shown in FIG. If the target pixel is set to the pixel P2 in step S213 in FIG. 11, first, the lower displacement d2 is acquired in step S214. Since there is a higher-order deformation process (first deformation process), in step S217, the reference position of the reference pixel (pixel P1), that is, the position of the pixel P0 is acquired. Therefore, in step S214, the sum of the distance d1 and the distance d2 is acquired as the lower displacement of the target pixel P2, and the maximum value of the displacement is updated in step S215. As described above, by repeatedly executing Steps S214 to S217, even when a plurality of deformation processes are performed sequentially, the upper displacement, the lower displacement, and the width of the target pixel (that is, 1) are added. By using the band, the deformation process of the target pixel P2 can be performed.

  In step S218 of FIG. 11, the maximum bandwidth calculation unit 252 determines whether the bandwidth calculation has been performed for all the pixels of the target image. When the bandwidth is calculated for all the pixels of the target image, the control is returned to the band buffer securing process in FIG. On the other hand, if there is a pixel whose bandwidth has not been calculated, control is transferred to step S219.

  In step S219, the maximum bandwidth calculation unit 252 sets a pixel next to the target pixel as a new target pixel. Specifically, when the target pixel is not the rightmost pixel of the image, the pixel on the right side of the target pixel is set as a new target pixel. On the other hand, when the target pixel is the rightmost pixel of the target image, the leftmost pixel in the column immediately below the target pixel is set as a new target pixel. In step S218, steps S213 to S219 are repeatedly executed until it is determined that the bandwidth has been calculated for all the pixels of the target image.

  In this way, the maximum bandwidth calculation unit 252 calculates the maximum bandwidth value (maximum bandwidth) required for all pixels. After the calculation of the maximum bandwidth, in step S220 of FIG. 9, the band buffer allocating unit 254 (FIG. 1) uses the maximum bandwidth band buffer (original image buffer) for storing the original image that is the image before the transformation. Secure. Next, in step S230, the band buffer allocating unit 254 secures a band buffer (deformed image buffer) having the maximum bandwidth for storing the image after the deformation process for each deformation process. After securing the deformed image buffer, control is returned to the face shape correction printing process of FIG. Note that the deformed image buffer is an area where an image subjected to the deformation process is generated, and thus can also be referred to as a “deformed image area”.

  In step S300 of FIG. 3, the deformation processing unit 256 (FIG. 1) performs line deformation processing for deforming an image by setting pixel values of individual pixels in one line. Here, the line refers to a pixel column of one vertical pixel extending in the entire horizontal direction of the image.

  FIG. 13 is a flowchart showing the flow of the line deformation process executed in step S300. FIG. 14 is an explanatory diagram showing a state where a deformed image is generated by the line deformation process. Note that the left column in FIG. 14 represents an image before deformation. Further, the middle column in FIG. 14 represents the image after the first deformation process, and the right column represents the image after the second deformation process.

  FIG. 14A shows a state before the first line deformation process is performed. The first line transformation process is performed on the top line of the image. Therefore, as shown in FIG. 14A, the generation line (hatched portion) is set to the uppermost line of the image. In the state shown in FIG. 14A, since the line deformation process for setting pixel values is not performed, both the image after the first deformation process and the image after the second deformation process are generated. It has not been done.

  In step S <b> 310, the deformation processing unit 256 identifies a reference range (“source” of deformation processing) of a line that is a target of deformation processing (“destination” of deformation processing). Note that the line that is the target of the deformation process is a line where a deformed image is generated, and is also referred to as a “generation line” below. Specifically, the deformation processing unit 256 calculates the position of the reference pixel for each pixel in the generation line. A range in which the vertical direction extends over the upper and lower ends of the reference pixel and the horizontal direction extends to the entire image is specified as the reference range. FIG. 14B shows a state where the reference range of the generation line is specified, and the reference range of the generation line in the image after the first deformation process is specified as indicated by an arrow.

  In step S320, the deformation processing unit 256 determines whether or not the reference range has been deformed in the upper deformation process, that is, whether or not an image has been generated by the upper deformation process. If it is determined that the reference range has been transformed in the upper transformation process, control is transferred to step S340. On the other hand, if it is determined that the reference range has not been transformed in the higher-order transformation process, control is transferred to step S330. Note that whether or not the reference range has been modified can be determined by providing a flag indicating whether or not an image on the line has been generated for each of a plurality of lines constituting the upper transformed image. . In the example of FIG. 14, as shown in FIG. 14B, a deformed image by the first deformation process is not generated in the reference range. Therefore, it is determined that the reference range indicated by hatching has not been transformed, and the control is transferred to step S330.

  In step S330, the deformation processing unit 256 generates a deformed image within the reference range by the upper deformation process. In the example of FIG. 14, the reference destination in the original image is specified as indicated by an arrow from the reference range indicated by hatching as shown in FIG. Then, as indicated by an arrow in FIG. 14D, an image in which the first deformation process is performed on the reference destination image in the original image is generated within the reference range. The specific content of the process of generating a deformed image by the upper deformation process within the reference range (reference range image generation process) will be described later. As can be seen from the above description, the source in the lower deformation process is the destination in the upper deformation process.

  In step S340, the deformation processing unit 256 generates a deformed image on the generation line. Specifically, as shown in FIG. 14E, the second deformation process is performed on the image generated in the reference range, thereby generating the image subjected to the second deformation process on the generation line. . As a result, an image (multi-deformed image) in which the first deformation process and the second deformation process are sequentially performed on the generation line is generated.

  FIG. 15 is a flowchart showing the flow of the reference range image generation process executed in step S330. FIG. 16 is an explanatory diagram showing the state of the band buffer when the image deformation process is performed. Each column on the left side of FIG. 16 indicates the state of the original image buffer. The center column in FIG. 16 shows the state of the first modified image buffer for the first transformation process, and the right column in FIG. 16 shows the second transformed image buffer for the second transformation process. Indicates the state.

  FIG. 16A shows a state where a deformed image is generated on the upper generation line. As described above, in order to generate a deformed image on the generation line, an image after the first deformation process is generated in the reference range of the generation line in the first deformed image buffer. When the generation of the deformed image on the generation line is completed, the line is moved downward by one pixel as shown in FIG. At this time, an image for one line is taken into the original image buffer from the memory card MC (FIG. 1). Then, the upper end line of the band buffer indicated by a broken line is discarded.

  The hatched portion of the second modified image buffer in FIG. 16B shows the generated line after the line movement. Then, the reference range of this generation line is specified as the range within the thick frame of the first modified image buffer in step S310 of FIG. As shown in FIG. 16B, a deformed image has not been generated for one line at the lower end of the reference range of the generation line. Therefore, in step S320 in FIG. 13, it is determined that the reference range has not been transformed, and in step S330, the reference range image generation process shown in FIG. 15 is executed.

  In step S331, the transformation processing unit 256 sets the uppermost line of the reference range as the target line. In step S332, it is determined whether the target line has been deformed, that is, whether a deformed image has been generated. If it is determined that the target line has been transformed, control is transferred to step S338. On the other hand, if it is determined that the target line has not been deformed, control is transferred to step S333.

  In the example of FIG. 16, as shown in FIG. 16A, when a deformed image is generated on the upper generation line, an image after the first deformation process is generated in the first deformed image buffer. Yes. Therefore, in step S332, it is determined that the target line has been deformed, and control is transferred to step S337.

  In step S337, the deformation processing unit 256 determines whether the target line is at the lower end of the reference range. When it is determined that the target line is the lower end of the reference range, the processing in FIG. 16 is finished and the control is returned. On the other hand, when it is determined that the target line is not the lower end of the reference range, the control is moved to step S338. In step S338, the deformation processing unit 256 sets the next (lower) line of the target line as a new target line.

  In this way, the target line is sequentially moved downward from the upper end of the reference range, and the two steps S332 and S338 are performed while the target line is within the upper part of the reference range indicated by the thick frame in FIG. Is repeatedly executed. When the target line is sequentially moved downward and one line at the lower end of the reference range is set as the target line. In step S332, it is determined that the target line has not been deformed. Then, control proceeds from step S332 to step S333.

  In step S333, the deformation processing unit 256 determines whether or not the deformation process for generating an image on the target line is the highest-level deformation process. If it is determined that the deformation process is the highest order, control is transferred to step S336, and a deformed image is generated on the target line. On the other hand, if it is determined that the deformation process is not the highest order, the control is moved to step S334.

  In step S334, the transformation processing unit 256 identifies the reference range of the target line, similarly to step S310 in FIG. Next, in step S335, the deformation processing unit 256 determines whether or not the reference range of the target line has been deformed in the higher-level deformation processing, similar to step S310 in FIG. If it is determined that the reference range has been transformed in the higher-order transformation process, control is transferred to step S336. On the other hand, if it is determined that the reference range has not been transformed in the higher-order transformation process, control is transferred to step S330, and the reference range image generation process is recursively executed. As described above, by executing the reference range image generation process recursively, even when a plurality of deformation processes are performed, a deformed image is generated on the target line by a higher-order deformation process.

  In the example of FIG. 16, the first deformation process is the highest deformation process. Therefore, in step S333, it is determined that the deformation process is the highest-order deformation process. Next, in step S336, as shown in FIG. 16D, an image obtained by performing the first deformation process on the original image is generated on one line at the lower end of the reference range. As described above, an image subjected to the first deformation process is generated in the reference range in the first deformed image buffer by the reference range image generating process. Then, on the generation line in the second deformed image buffer, as shown in FIG. 16 (e), the second deforming process is performed on the image in the reference range in the first deformed image buffer. A completed image is generated.

  When the deformed image is generated on the generation line in this way, the line deformation process shown in FIG. 13 is finished, and the control is returned to the face shape correction printing process shown in FIG.

  In step S410 in FIG. 3, a print process of a generation line in which a deformed image is generated is performed. Specifically, data representing the image of the generated line (line data) is supplied to the print processing unit 320. The print processing unit 320 accumulates the supplied line data as necessary, and performs processing such as resolution conversion and halftone processing to generate print data. The generated print data is supplied from the print processing unit 320 to the printer engine 160.

  In step S420, it is determined whether the line transformation process (step S300) and the line printing process (step S410) have been performed for all lines of the target image. If it is determined that these processes have been completed for all lines, the face shape correction printing process of FIG. 3 ends. On the other hand, if it is determined that there is an unprocessed line, control is transferred to step S430.

  In step S430, the generation line to be deformed is moved to the next line as described above. In step S420, steps S300 to S430 are repeatedly executed until it is determined that the line transformation process (step S300) and the line printing process (step S410) have been completed for all lines. As a result, the printer engine 160 prints an image obtained by subjecting the target image to deformation processing.

  As described above, in this embodiment, when the target image is deformed, the maximum bandwidth necessary for determining the pixel values of all the pixels constituting the deformed image is calculated in advance. Then, by securing a band buffer with the calculated maximum bandwidth, the target image can be transformed using the image stored in the band buffer. Therefore, the amount of memory required for the deformation process of the target image can be reduced.

  In this embodiment, when one target image is deformed, the bandwidth of the band buffer is fixedly set to the maximum bandwidth. Therefore, it is possible to reduce the number of times of performing band buffer securing, compared to the case where the necessary bandwidth is determined for each line and the band buffer having the determined bandwidth is secured. Therefore, the processing time for deforming the target image can be shortened.

  Furthermore, in this embodiment, by securing the band buffer with the maximum bandwidth calculated in advance, even when the line for generating the deformed image moves, the image or band stored in the band buffer before the line movement is moved. A part of the image generated in the buffer can be used as it is. Therefore, storing an image in the band buffer and regenerating the image in the band buffer can be omitted, so that the processing time for deforming the target image can be shortened.

B. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

B1. Modification 1:
In the above embodiment, the deformed image is generated for each line. However, the deformed image may be generated for each block in any form as long as it is for each block that is a part of the deformed image. As the block, for example, a rectangular area that is a part of the deformed image or a pixel block that is a part of a line can be used.

B2. Modification 2:
In the above embodiment, the present invention is applied to the face shape deformation process, but the present invention can be applied to a deformation process different from the face shape deformation process. The present invention can be generally applied to deformation processing of an object (object) included in an image.

B3. Modification 3:
In the above embodiment, the deformation area is divided into small areas, and the entire deformation area is deformed by deforming the divided small areas. However, the division of the deformation area may be omitted. In general, the present invention can be applied to any form of deformation processing as long as it is a deformation processing by mapping.

B4. Modification 4:
In the above embodiment, the second deformed image buffer is secured and the band of the deformed image is stored. However, if the area that can be stored by the generation line is secured, the band of the deformed image is stored. It is also possible to omit the band buffer. Even in this way, it is possible to generate a deformed image on the generation line.

B5. Modification 5:
In the above-described embodiment, the deformation processes of the plurality of deformation areas are performed separately, but the deformation of the plurality of deformation areas can also be performed by a single deformation process. In this case, the position on the original image of the pixel associated with the target pixel of the finally generated deformed image is obtained. Then, the pixel value of the target pixel is set from the pixel value of the pixel associated with the target pixel.

B6. Modification 6:
In the above embodiment, the printing process is performed for each generated line, but the printing process is not necessarily performed for each line. For example, an area for storing the entire deformed image finally generated may be secured, and an image may be stored for each line generated in the area.

B7. Modification 7:
In the above embodiment, the present invention is applied to the printer 100. However, the present invention is an apparatus that performs a deformation process for generating a deformed image from an original image. It can be applied to the device. The present invention can be applied to, for example, a personal computer or a digital camera as long as it has a function of performing image deformation processing.

B8. Modification 8:
In each of the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware. .

100 ... Printer 110 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Internal memory 140 ... Operation part 150 ... Display part 160 ... Printer engine 170 ... Card interface 172 ... Card slot 200 ... Face shape correction part 210 ... Deformation mode setting part 220 ... Deformation area setting part 230 ... Deformation area division part 240 ... Deformation parameter setting unit 250 ... Divided area transforming unit 252 ... Maximum bandwidth calculating unit 254 ... Band buffer allocating unit 256 ... Deformation processing unit 310 ... Display processing unit 320 ... Print processing unit 410 ... Dividing point arrangement pattern table 420 ... Dividing point movement Table 430 ... Band buffer

Claims (3)

An image processing apparatus that executes a deformation process for changing the shape of an object included in an image,
Storage means;
A band buffer allocating unit for securing a buffer which is a storage area for storing pixel values in the storage unit before the transformation process;
By using the reserved buffer as a storage area for pixel values associated with the execution of the deformation process , each pixel value of the reference source pixel constituting the object in the image after the deformation process is converted to a value before the deformation process. A deformation processing unit that executes the deformation processing by setting each pixel value of a reference pixel that is a pixel corresponding to each reference source pixel in the image;
An output processing unit that outputs an image including the object subjected to the deformation process,
The image processing apparatus, wherein the band buffer allocating unit secures the buffer so that pixel values of the reference source pixel and the reference destination pixel included in a part of the object , not the entire object , can be stored . An image processing method for executing deformation processing for changing the shape of an object included in an image,
(A) securing a buffer which is a storage area for storing pixel values in the storage means of the computer before the deformation process;
(B) By using the reserved buffer as a storage area for pixel values associated with the execution of the deformation process, the respective pixel values of the reference source pixels constituting the object in the image after the deformation process are deformed. Executing the deformation process by setting each pixel value of a reference destination pixel that is a pixel corresponding to each reference source pixel in the image before processing;
(C) outputting an image including the object subjected to the deformation process,
The step (a) of securing the buffer is a step of securing the buffer so that pixel values of the reference source pixel and the reference destination pixel included in a part of the object , not the entire object , can be stored. There is an image processing method. A computer program for executing a deformation process for changing the shape of an object included in an image,
A band buffer allocation function for securing a buffer that is a storage area for storing pixel values in the storage means of the computer before the transformation process;
By using the reserved buffer as a storage area for pixel values associated with the execution of the deformation process , each pixel value of the reference source pixel constituting the object in the image after the deformation process is converted to a value before the deformation process. A deformation processing function for executing the deformation processing by setting each pixel value of a reference pixel that is a pixel corresponding to each reference source pixel in the image;
And an output processing function of outputting an image including an object that is executed in the modification process is implemented in the computer,
The band buffer allocation function is a function of securing the buffer so that pixel values of the reference source pixel and the reference destination pixel included in a part of the object , not the entire object , can be stored. .

Images