WO2016158310A1 - 画像処理システムおよび画像処理方法 - Google Patents

画像処理システムおよび画像処理方法 Download PDF

Info

Publication number
WO2016158310A1
WO2016158310A1 PCT/JP2016/057721 JP2016057721W WO2016158310A1 WO 2016158310 A1 WO2016158310 A1 WO 2016158310A1 JP 2016057721 W JP2016057721 W JP 2016057721W WO 2016158310 A1 WO2016158310 A1 WO 2016158310A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
image
mask
basic
range
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2016/057721
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
水野　雄介
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MegaChips Corp
Original Assignee
MegaChips Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by MegaChips Corp filed Critical MegaChips Corp
Publication of WO2016158310A1 publication Critical patent/WO2016158310A1/ja
Priority to US15/712,881 priority Critical patent/US10410079B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/60—Creating or editing images; Combining images with text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/18—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10112—Digital tomosynthesis [DTS]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20064—Wavelet transform [DWT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]

Definitions

  • the present invention relates to an image processing system and an image processing method.
  • Patent Documents 1 to 3 disclose techniques for detecting a moving object in a moving image.
  • Patent Documents 4 and 5 disclose techniques for tracking moving objects using a particle filter. By using such a moving object detection technique, a moving object in an image can be cut out as an ROI (region of interest).
  • a technique for separating a foreground image and a background image from an entire image by graph cut for a still image taken by a digital camera is known.
  • a foreground image can be cut out as an ROI.
  • JP 2013-254291 A Japanese Patent Laid-Open No. 2007-088897 JP 2006-093784 A JP 2009-199363 A JP 2005-165688 A JP 2006-203409 A JP-T-2001-520466 JP 2003-324613 A
  • the ROI may be set with a portion that locally protrudes from the intended region, or conversely, a portion that locally erodes the intended region.
  • the ROI may be set with a defect in the intended region. That is, where the entire region of the intended region is supposed to be set as the ROI, it is determined that a non-ROI exists in the region, and the non-ROI forms a ROI defect. Combining an insufficient ROI with such defects in the outline and / or interior with another image will feel unnatural.
  • an image processing system including an image composition system.
  • the image synthesis system includes a synthesis unit that synthesizes a ROI (region of interest) in the first target image and the second target image.
  • the second target image has a similar shape to the first target image, and a similarity ratio of the second target image to the first target image is 1 or less.
  • the synthesizing unit obtains first basic image data, basic mask data, second basic image data, and synthesis control data.
  • the first basic image data is data of a first basic image that is a source of the first target image.
  • the basic mask data is basic mask data that is a source of a target mask for discriminating between the ROI and the non-ROI in the range of the first target image.
  • the second basic image data is data of a second basic image that is a source of the second target image.
  • the synthesis control data is data for controlling the degree of synthesis.
  • the synthesis unit includes a wavelet conversion unit.
  • the wavelet transform unit performs a wavelet transform on the first target image data, which is the data of the first target image, to a designated decomposition level designated by the synthesis control data, whereby a first wavelet transform is performed.
  • Generate coefficient data The wavelet transform unit performs the wavelet transform on the second target image data, which is the data of the second target image, to the decomposition level determined according to the designated decomposition level and the similarity ratio, Second wavelet coefficient data is generated.
  • the synthesis unit further includes a synthesis execution unit and an inverse wavelet conversion unit.
  • the synthesis execution unit is a coefficient that discriminates the ROI coefficient related to the ROI and the non-ROI coefficient related to the non-ROI with respect to the first wavelet coefficient data based on target mask data that is data of the target mask. Perform discrimination processing.
  • the synthesis executing unit performs a coefficient synthesis process for synthesizing the ROI coefficient in the first wavelet coefficient data and the wavelet coefficient in the second wavelet coefficient data.
  • the synthesis execution unit generates synthesized coefficient data having the same image size and decomposition level as the second wavelet coefficient data.
  • the inverse wavelet transform unit generates composite image data by performing inverse wavelet transform on the combined coefficient data until a decomposition level reaches a predetermined end level.
  • the image processing system according to the second aspect is the image processing system according to the first aspect, and further includes a data supply system that outputs the first basic image data and the basic mask data.
  • An image processing system is the image processing system according to the first aspect, wherein the image composition system includes the medium for supplying the composition control data, the first basic image data, and the basic mask data. Obtained by the same medium.
  • An image processing system is the image processing system according to the third aspect, and outputs the first basic image data, the basic mask data, and the synthesis control data by a wireless or wired communication medium.
  • a data supply system is further included.
  • An image processing system is the image processing system according to the first aspect, wherein the image composition system includes the medium for supplying the composition control data, the first basic image data, and the basic mask data. Are acquired by different media.
  • An image processing system is the image processing system according to the fifth aspect, wherein the image composition system stores the first basic image data and the basic mask data in a wireless or wired communication medium or an external storage. Obtained by a medium, and the composition control data is obtained from a user by an operation medium provided on the image composition system side.
  • An image processing system is the image processing system according to the fifth or sixth aspect, wherein the first basic image data and the basic mask data are output by a wireless or wired communication medium. Further includes a system.
  • An image processing system is the image processing system according to any one of the first to seventh aspects, wherein the image composition system selectively selects one of a plurality of composition control data. Includes a selector to be supplied to the combining unit.
  • An image processing system is the image processing system according to the eighth aspect, wherein the plurality of composition control data is supplied by a user inputting to an operation medium provided on the image composition system side. First synthesized control data, and second synthesized control data supplied by a medium different from the operation medium. The selector preferentially selects the first synthesis control data.
  • the image processing system is any one of the first to ninth aspects, and the image composition system further includes an image decoder and a mask data restoration unit.
  • the image decoder acquires an image bit stream that is an encoded bit stream of the first basic image data, and acquires the first basic image data by decoding the image bit stream.
  • the mask data restoration unit obtains a mask bit stream which is a bit stream of the basic mask data, and restores the basic mask data from the mask bit stream by an algorithm independent of decoding of the image bit stream.
  • the image composition system acquires the image bitstream and the mask bitstream by separate bitstreams.
  • the image processing system is the image processing system according to the tenth aspect, and further includes a data supply system that outputs the first basic image data and the basic mask data.
  • the data supply system includes an image data output unit and a mask data output unit.
  • the image data output unit includes an image encoder that encodes the first basic image data to generate the image bitstream.
  • the mask data output unit includes a mask generation unit and a mask bitstream generation unit.
  • the mask generation unit generates the basic mask data using an algorithm independent of the encoding of the first basic image data.
  • the mask bit stream generation unit generates the mask bit stream from the basic mask data.
  • the first basic image data is output from the data supply system by the image bitstream.
  • the basic mask data is output from the data supply system according to the mask bitstream.
  • the data supply system outputs the image bit stream and the mask bit stream as separate bit streams.
  • An image processing system is the image processing system according to the tenth aspect, wherein the synthesis control data is embedded in an area that does not affect the basic mask data in the mask bitstream, Supplied to the image composition system.
  • the mask data restoration unit extracts the synthesis control data from the mask bitstream.
  • the image processing system is the image processing system according to the twelfth aspect, and further includes a data supply system that outputs the first basic image data and the basic mask data.
  • the data supply system includes an image data output unit and a mask data output unit.
  • the image data output unit includes an image encoder that encodes the first basic image data to generate the image bitstream.
  • the mask data output unit includes a mask generation unit and a mask bitstream generation unit.
  • the partial mask generation unit generates the basic mask data using an algorithm independent of the encoding of the first basic image data.
  • the mask bit stream generation unit generates the mask bit stream from the basic mask data.
  • the mask bit stream generation unit embeds the synthesis control data in the area of the mask bit stream.
  • the first basic image data is output from the data supply system by the image bitstream.
  • the basic mask data and the synthesis control data are output from the data supply system by the mask bitstream.
  • the data supply system outputs the image bit stream and the mask bit stream as separate bit streams.
  • the image processing system of the fourteenth aspect is any one of the first to ninth aspects, and the image composition system further includes an image decoder and a mask data restoration unit.
  • the image decoder acquires an image bit stream that is an encoded bit stream of the first basic image data, and acquires the first basic image data by decoding the image bit stream.
  • the mask data restoration unit obtains a mask bit stream which is a bit stream of the basic mask data, and restores the basic mask data from the mask bit stream by an algorithm independent of decoding of the image bit stream.
  • the mask bit stream is supplied to the image composition system in a state where the mask bit stream is embedded in an area of the image bit stream that does not affect the first basic image data.
  • the image decoder extracts the mask bitstream from the image bitstream and supplies it to the mask data restoration unit.
  • the image processing system is the image processing system according to the fourteenth aspect, and further includes a data supply system that outputs the first basic image data and the basic mask data.
  • the data supply system includes an image data output unit and a mask data output unit.
  • the image data output unit includes an image encoder that encodes the first basic image data to generate the image bitstream.
  • the mask data output unit includes a mask generation unit and a mask bitstream generation unit.
  • the mask generation unit generates the basic mask data using an algorithm independent of the encoding of the first basic image data.
  • the mask bit stream generation unit generates the mask bit stream from the basic mask data.
  • the first basic image data is output from the data supply system by the image bitstream.
  • the basic mask data is output from the data supply system according to the mask bitstream.
  • the image encoder embeds the mask bitstream in the region of the image bitstream. Accordingly, the data supply system outputs the image bit stream and the mask bit stream as a single bit stream.
  • An image processing system is the image processing system according to the fourteenth aspect, wherein the composition control data is supplied to the image composition system in a state embedded in the area of the image bitstream. .
  • the image decoder extracts the synthesis control data from the image bitstream.
  • the image processing system is the image processing system according to the sixteenth aspect, and further includes a data supply system that outputs the first basic image data and the basic mask data.
  • the data supply system includes an image data output unit and a mask data output unit.
  • the image data output unit includes an image encoder that encodes the first basic image data to generate the image bitstream.
  • the mask data output unit includes a mask generation unit and a mask bitstream generation unit.
  • the mask generation unit generates the basic mask data using an algorithm independent of the encoding of the first basic image data.
  • the mask bit stream generation unit generates the mask bit stream from the basic mask data.
  • the first basic image data is output from the data supply system by the image bitstream.
  • the basic mask data is output from the data supply system according to the mask bitstream.
  • the image encoder embeds the mask bit stream and the synthesis control data in the area of the image bit stream. Accordingly, the data supply system outputs the image bit stream, the mask bit stream, and the synthesis control data as a single bit stream.
  • An image processing system is the image processing system according to any one of the first to seventeenth aspects, wherein the coefficient synthesizing process includes the non-ROI coefficient in the first wavelet coefficient data. Is replaced with the corresponding wavelet coefficient in the second wavelet coefficient data, and the ROI coefficient in the first wavelet coefficient data is replaced with the ROI coefficient in the second wavelet coefficient data.
  • An image processing system is the image processing system according to any one of the first to eighteenth aspects, wherein the basic mask is an image in a state where the wavelet transformation is not performed.
  • This is an original mask for the original image.
  • the original mask indicates whether each pixel in the original image belongs to the ROI or the non-ROI, and the ROI corresponding portion and the non-ROI corresponding portion corresponding to the ROI and the non-ROI in the original image.
  • the synthesis execution unit performs a mask expansion process for expanding the ROI corresponding part and the non-ROI corresponding part of the target mask included in the basic mask for each band component included in the first wavelet coefficient data.
  • a mask developing unit for generating a development mask for the first wavelet coefficient data is provided.
  • the synthesis execution unit performs the coefficient determination processing based on the distinction between the ROI corresponding part and the non-ROI corresponding part in the development mask.
  • the image processing system is the image processing system according to the nineteenth aspect, further including a data supply system that outputs the first basic image data and the basic mask data.
  • the data supply system generates the original mask for the range of the basic image and outputs the original mask data as the basic mask data.
  • the image processing system of the twenty-first aspect is any one of the first to eighteenth aspects, and the basic mask is a development mask for the first wavelet coefficient data.
  • the expansion mask is a mask expansion process for expanding an original mask for an original image, which is an image that has not been subjected to the wavelet transform, for each band component included in the first wavelet coefficient data. It is generated by doing.
  • the original mask indicates whether each pixel in the original image belongs to the ROI or the non-ROI, and the ROI corresponding portion and the non-ROI corresponding portion corresponding to the ROI and the non-ROI in the original image.
  • the synthesis execution unit performs the coefficient discrimination process based on the distinction between the ROI corresponding part developed on the development mask and the non-ROI corresponding part.
  • the image processing system is the image processing system according to the twenty-first aspect, and further includes a data supply system that outputs the first basic image data and the basic mask data.
  • the data supply system generates an original mask for the range of the basic image, and an original mask generation unit that generates the development mask by performing the mask development process on the original mask. Including.
  • the data supply system outputs the development mask data as the basic mask data.
  • An image processing system is the image processing system according to any one of the first to twenty-second aspects, wherein the synthesizing unit applies the entire range of the first basic image to the first target image. And the entire range of the basic mask is used as the target mask.
  • the image processing system is the image processing system according to the twenty-third aspect, and further includes a data supply system.
  • the data supply system outputs the first target image data as the first basic image data.
  • the data supply system generates the target mask for the range of the first target image and outputs the target mask data as the basic mask data.
  • the image processing system according to the 25th aspect is the image processing system according to the 23rd aspect, and further includes a data supply system.
  • the data supply system cuts the first target image from the entire image including the ROI and outputs the first target image data as the first basic image data.
  • the data supply system generates the target mask for the range of the first target image and outputs the target mask data as the basic mask data.
  • An image processing system is the image processing system according to the twenty-fifth aspect, wherein the data supply system includes an overall mask generation unit, a cutout range determination unit, a mask cutout unit, and a first image cutout unit.
  • the whole mask generation unit generates whole mask data that is data of a whole mask for discriminating between the ROI and the non-ROI in a range of the whole image based on the whole image data that is data of the whole image. To do.
  • the cutout range determination unit performs a cutout range determination process for determining a cutout range that is a range of the first target image in the whole image based on the whole mask data and the synthesis control data, and specifies the cutout range The cut range specifying data for generating is generated.
  • the mask cutting unit cuts data in the cut range among the whole mask data for the basic mask data based on the cut range specifying data.
  • the first image cutout unit cuts out data in the cutout range as the first basic image data out of the whole image data based on the cutout range specifying data.
  • An image processing system is the image processing system according to any one of the first to twenty-second aspects, wherein the synthesizing unit sets a part of the first basic image as the first target image. And use the entire range of the basic mask as the target mask.
  • the synthesizing unit further acquires cutout range specifying data for specifying a cutout range that is a range of the first target image in the first basic image.
  • the synthesizing unit further includes a first image cutout unit that cuts out data in the cutout range as the first target image data from the first basic image data based on the cutout range specifying data.
  • the image processing system of the twenty-eighth aspect is the image processing system of the twenty-seventh aspect and further includes a data supply system.
  • the data supply system outputs, as the first basic image data, whole image data that is data of a whole image including the first target image as a part thereof.
  • the data supply system generates the target mask for the range of the first target image and outputs the target mask data as the basic mask data.
  • the data supply system includes an overall mask generation unit, a cut range determination unit, and a mask cut unit.
  • the overall mask generation unit generates overall mask data that is data of an overall mask for discriminating between the ROI and the non-ROI in the range of the overall image based on the overall image data.
  • the cutout range determination unit performs a cutout range determination process for determining the cutout range that is the range of the first target image in the whole image based on the whole mask data and the synthesis control data, and specifies the cutout range Generate data.
  • the mask cutting unit cuts data in the cut range among the whole mask data for the basic mask data based on the cut range specifying data.
  • the data supply system further outputs the cut range specifying data.
  • An image processing system is the image processing system according to any one of the first to twenty-second aspects, wherein the synthesizing unit sets a part of the first basic image as the first target image. And a portion of the base mask is used as the target mask.
  • the synthesizing unit further acquires cutout range specifying data for specifying a cutout range that is a range of the first target image in the first basic image.
  • the combining unit further includes a mask cutout unit and a first image cutout unit.
  • the mask cutting unit cuts data within the cut range among the basic mask data for the target mask data based on the cut range specifying data.
  • the first image cutout unit cuts out data in the cutout range as the first target image data from the first basic image data based on the cutout range specifying data.
  • the image processing system is the image processing system according to the 29th aspect, and further includes a data supply system.
  • the data supply system outputs, as the first basic image data, whole image data that is data of a whole image including the first target image as a part thereof.
  • the data supply system generates and outputs the basic mask data for the entire image range.
  • the data supply system includes an overall mask generation unit and a cutout range determination unit.
  • the overall mask generation unit generates overall mask data that is data of an overall mask for discriminating between the ROI and the non-ROI in the range of the overall image based on the overall image data.
  • the cutout range determination unit performs a cutout range determination process for determining the cutout range that is the range of the first target image in the whole image based on the whole mask data and the synthesis control data, and specifies the cutout range Generate data.
  • the data supply system further outputs the cut range specifying data.
  • the image processing system is the image processing system according to any one of the first to twenty-second aspects, wherein the synthesizing unit applies the entire range of the first basic image to the first target image. And a part of the basic mask is used as the target mask.
  • the synthesizing unit further acquires cut range specifying data for specifying a cut range that is a range of the target mask in the basic mask.
  • the synthesizing unit further includes a mask cutout unit that cuts out data within the cutout range for the target mask data from the basic mask data based on the cutout range specifying data.
  • the image processing system is the image processing system according to the thirty-first aspect and further includes a data supply system.
  • the data supply system cuts the first target image from the entire image including the ROI and outputs the first target image data as the first basic image data.
  • the data supply system generates and outputs the basic mask data for the entire image range.
  • the data supply system includes an overall mask generation unit, a cutout range determination unit, and a first image cutout unit.
  • the whole mask generation unit generates whole mask data that is data of a whole mask for discriminating between the ROI and the non-ROI in a range of the whole image based on the whole image data that is data of the whole image. To do.
  • the cutout range determination unit performs a cutout range determination process for determining the cutout range that is the range of the first target image in the whole image based on the whole mask data and the synthesis control data, and specifies the cutout range Generate data.
  • the first image cutout unit cuts out data in the cutout range as the first basic image data out of the whole image data based on the cutout range specifying data.
  • the data supply system further outputs the cut range specifying data.
  • An image processing system is the image processing system according to any one of the first to twenty-second aspects, wherein the synthesizing unit uses a part of the first basic image as the first target image. And a portion of the base mask is used as the target mask.
  • the synthesizing unit further includes a cutting range determining unit, a mask cutting unit, and a first image cutting unit.
  • the cutout range determination unit performs a cutout range determination process for determining a cutout range that is a range of the first target image in the first basic image based on the basic mask data and the synthesis control data, and the cutout range is determined.
  • the cut range specifying data for specifying is generated.
  • the mask cutting unit cuts data within the cut range among the basic mask data for the target mask data based on the cut range specifying data.
  • the first image cutout unit cuts out data in the cutout range as the first target image data from the basic image data based on the cutout range specifying data.
  • An image processing system is the image processing system according to the first aspect, and further includes a data supply system that outputs the first basic image data and the basic mask data.
  • the data supply system includes: a small image generation unit that generates a small image by reducing an entire image including an ROI; a small mask generation unit that generates a small mask for the small image; and A mask enlargement unit that enlarges to the same image size as the image and generates an enlarged mask.
  • the data supply system outputs the enlarged mask data as the basic mask data.
  • An image processing system is the image processing system according to the first aspect, wherein the basic mask is the entire small mask for the reduced image of the first basic image or the small mask. It is a part including the ROI corresponding part.
  • the synthesizing unit further includes a mask enlarging unit that enlarges the basic mask to an image size of the first target image to generate an enlarged mask.
  • the synthesizing unit uses the enlarged mask as the target mask.
  • the image processing system of the thirty-sixth aspect is the image processing system of the thirty-fifth aspect and further includes a data supply system.
  • the data supply system generates a small image by reducing the entire image including the ROI, generates the small mask for the small image, and outputs the whole or a part of the small mask as the basic mask. To do.
  • the image processing system of the 37th aspect is the one of the image processing system of the first 34, second 36 embodiment, the reduction ratio of the small mask with respect to the first basic image is a 1/2 R (R is a natural number).
  • the mask enlargement unit generates the enlargement mask by performing enlargement processing with an enlargement ratio of R times for the small mask.
  • the i-th data i is an integer
  • An image processing system is the image processing system according to the thirty-seventh aspect, wherein the mask enlarging unit performs the enlargement of the mask at the end of at least one of the R times of enlargement processing. Is smoothed.
  • the image processing system is the image processing system according to the thirty-sixth aspect, wherein the data supply system further includes a cut-out range determining unit and a mask cut-out unit.
  • the cut range determination unit performs a cut range determination process based on the enlarged mask data and the synthesis control data. In the cut-out range determination process, a cut-out range that is the range of the first target image in the entire image and a small mask cut-out range corresponding to the cut-out range in the small mask are determined.
  • the cut range determining unit generates cut range specifying data for specifying the cut range and small mask cut range specifying data for specifying the small mask cut range.
  • the mask cutout unit cuts out data in the small mask cutout range for the basic mask data from the small mask data based on the small mask cutout range specifying data.
  • the data supply system further outputs the cut range specifying data.
  • An image processing system is the image processing system according to the thirty-sixth aspect, wherein the data supply system receives an encoded bit stream of the basic mask data which is data of the whole or a part of the small mask. And output as a mask bitstream.
  • the image synthesis system obtains the basic mask by decoding the mask bitstream.
  • An image processing system is the image processing system according to the thirty-sixth aspect, wherein the data supply system encodes the basic mask data, which is data of the whole or a part of the small mask.
  • the basic mask data bit stream is generated as a mask bit stream.
  • the image composition system restores the basic mask from the mask bitstream.
  • the image processing system is the image processing system according to any one of the first to thirty-third aspects, and the combining unit further includes a second image cutout unit.
  • the second image cutout unit sets a similarity range that is similar to the first target image with the similarity ratio in the second basic image, and the similarity is included in the second basic image data. Data within the shape range is cut out as the second target image data.
  • An image processing system is the image processing system according to the forty-second aspect, in which the composition unit obtains the composite image data obtained by performing the inverse wavelet transform until the decomposition level becomes zero. Is embedded in the range of the similar shape in the second basic image data.
  • An image processing system is the image processing system according to any one of the first to thirty-third aspects, wherein the synthesizing unit reduces at least a part of the second basic image to reduce the first basic image.
  • An image reduction unit that generates two target images is further provided.
  • An image processing system is the image processing system according to the first aspect, and the image composition system further includes a memory and a mask generation unit connected to the composition unit via a bus.
  • the memory is provided for storing whole image data, which is whole image data serving as a source of the first basic image, and the second basic image data.
  • the mask generation unit generates the basic mask data based on the entire image data.
  • an image processing system including a data supply system.
  • the data supply system outputs data used for an image composition process for compositing an ROI (region of interest) in the first target image and the second target image.
  • the second target image has a similar shape to the first target image, and a similarity ratio of the second target image to the first target image is 1 or less.
  • the image composition process includes a wavelet conversion process. In the wavelet conversion process, first wavelet coefficient data is generated by performing wavelet conversion on the first target image data, which is data of the first target image, to a specified decomposition level.
  • Second wavelet coefficient data is generated.
  • the specified decomposition level of the first wavelet coefficient data is P1
  • the decomposition level of the second wavelet coefficient data is P2
  • the similarity ratio is 1/2 P3
  • P2 P1 ⁇ P3 is there.
  • the image composition process further includes a composition execution process and an inverse wavelet transform process. In the synthesis execution process, the ROI coefficient related to the ROI and the non-ROI coefficient related to the non-ROI are determined for the first wavelet coefficient data, and the ROI and the non-ROI are determined in the range of the first target image.
  • a coefficient determination process is performed.
  • a coefficient synthesis process for synthesizing the ROI coefficient in the first wavelet coefficient data and the wavelet coefficient in the second wavelet coefficient data is performed.
  • synthesized coefficient data having the same image size and decomposition level as the second wavelet coefficient data is generated.
  • composite image data is generated by performing inverse wavelet transform on the combined coefficient data until the decomposition level reaches a predetermined end level.
  • the data supply system includes an image data output unit and a mask data output unit.
  • the image data output unit has an image encoder.
  • the image encoder encodes first basic image data that is data of a first basic image that is a source of the first target image, and generates an image bit stream that is an encoded bit stream of the first basic image data.
  • the mask data output unit includes a mask generation unit and a mask bitstream generation unit.
  • the mask generation unit generates basic mask data, which is basic mask data serving as a source of the target mask, by an algorithm independent of the encoding of the first basic image data.
  • the mask bitstream generation unit generates a mask bitstream that is a bitstream of the basic mask data.
  • the data supply system outputs the first basic image data by the image bit stream.
  • the data supply system outputs the basic mask data by the mask bitstream.
  • the data supply system outputs synthesis control data for designating the designated decomposition level in the wavelet transform.
  • An image processing system is the image processing system according to the 46th aspect, wherein the data supply system outputs the image bitstream and the mask bitstream as separate bitstreams.
  • An image processing system is the image processing system according to the 47th aspect, wherein the mask bit stream generation unit applies the synthesis control to a region of the mask bit stream that does not affect the basic mask data. Embed the data.
  • An image processing system is the image processing system according to the 46th aspect, wherein the mask bit stream is provided in an area where the image encoder does not affect the first basic image data in the image bit stream. Embed. Accordingly, the data supply system outputs the image bit stream and the mask bit stream as a single bit stream.
  • the image processing system according to the 50th aspect is the image processing system according to the 49th aspect, wherein the image encoder embeds the mask bitstream and the synthesis control data in the area of the image bitstream. Accordingly, the data supply system outputs the image bit stream, the mask bit stream, and the synthesis control data as a single bit stream.
  • An image processing system is the image processing system according to any one of 46th to 50th aspects, wherein the basic mask is an image in a state where the wavelet transform is not performed.
  • This is an original mask for the original image.
  • the original mask indicates whether each pixel in the original image belongs to the ROI or the non-ROI, and the ROI corresponding portion and the non-ROI corresponding portion corresponding to the ROI and the non-ROI in the original image.
  • the mask generation unit generates the original mask for the range of the basic image, and outputs the original mask data as the basic mask data.
  • the image processing system is any one of the 46th to 50th aspects, wherein the basic mask is a development mask for the first wavelet coefficient data.
  • the expansion mask is a mask expansion process for expanding an original mask for an original image, which is an image that has not been subjected to the wavelet transform, for each band component included in the first wavelet coefficient data. It is generated by doing.
  • the mask generation unit generates the original mask for the range of the basic image, and a mask expansion unit that generates the expansion mask by performing the mask expansion process on the original mask. And have.
  • the data supply system outputs the development mask data as the basic mask data.
  • An image processing system is the image processing system according to any one of the 46th to 52nd aspects, wherein the image data output unit converts the first target image data into the first basic image. Output as data.
  • the mask data output unit generates the target mask for the range of the first target image, and outputs the target mask data as the basic mask data.
  • An image processing system is the image processing system according to any one of the 46th to 52nd aspects, in which the image data output unit starts from the entire image including the ROI to the first target image. And the first target image data is output as the first basic image data.
  • the mask data output unit generates the target mask for the range of the first target image, and outputs the target mask data as the basic mask data.
  • An image processing system is the image processing system according to the 54th aspect, wherein the mask generation unit includes an overall mask generation unit, a cutout range determination unit, and a mask cutout unit.
  • the whole mask generation unit generates whole mask data that is data of a whole mask for discriminating between the ROI and the non-ROI in a range of the whole image based on the whole image data that is data of the whole image.
  • the cutout range determination unit performs a cutout range determination process for determining a cutout range that is a range of the first target image in the whole image based on the whole mask data and the synthesis control data, and specifies the cutout range The cut range specifying data for generating is generated.
  • the mask cutting unit cuts data in the cut range among the whole mask data for the basic mask data based on the cut range specifying data.
  • the image data output unit further includes a first image cutout unit that cuts out data in the cutout area as the first basic image data from the entire image data based on the cutout area specifying data.
  • An image processing system is the image processing system according to any one of the forty-sixth to fifty-second aspects, wherein the image data output unit is an entire image including the first target image as a part thereof. Whole image data as data is output as the first basic image data.
  • the mask data output unit generates the target mask for the range of the first target image, and outputs the target mask data as the basic mask data.
  • the mask generation unit includes an overall mask generation unit, a cutout range determination unit, and a mask cutout unit.
  • the overall mask generation unit generates overall mask data that is data of an overall mask for discriminating between the ROI and the non-ROI in the range of the overall image based on the overall image data.
  • the cutout range determination unit performs a cutout range determination process for determining a cutout range that is a range of the first target image in the whole image based on the whole mask data and the synthesis control data, and specifies the cutout range The cut range specifying data for generating is generated.
  • the mask cutting unit cuts data in the cut range among the whole mask data for the basic mask data based on the cut range specifying data.
  • the data supply system further outputs the cut range specifying data.
  • An image processing system is the image processing system according to any one of the 46th to 52nd aspects, wherein the image data output unit is configured to output an entire image including the first target image as a part thereof. Whole image data as data is output as the first basic image data.
  • the mask data output unit generates and outputs the basic mask data for the entire image range.
  • the mask generation unit includes an overall mask generation unit and a cutout range determination unit.
  • the overall mask generation unit generates overall mask data that is data of an overall mask for discriminating between the ROI and the non-ROI in the range of the overall image based on the overall image data.
  • the cutout range determination unit performs a cutout range determination process for determining a cutout range that is a range of the first target image in the whole image based on the whole mask data and the synthesis control data, and specifies the cutout range The cut range specifying data for generating is generated.
  • the data supply system further outputs the cut range specifying data.
  • An image processing system is the image processing system according to any one of the 46th to 52nd aspects, wherein the image data output unit starts from the entire image including the ROI with the first target image. And the first target image data is output as the first basic image data.
  • the mask data output unit generates and outputs the basic mask data for the entire image range.
  • the mask generation unit includes an overall mask generation unit and a cutout range determination unit.
  • the whole mask generation unit generates whole mask data that is data of a whole mask for discriminating between the ROI and the non-ROI in a range of the whole image based on the whole image data that is data of the whole image. To do.
  • the cutout range determination unit performs a cutout range determination process for determining a cutout range that is a range of the first target image in the whole image based on the whole mask data and the synthesis control data, and specifies the cutout range
  • the cut range specifying data for generating is generated.
  • the image data output unit further includes a first image cutout unit that cuts out data in the cutout area as the first basic image data from the entire image data based on the cutout area specifying data.
  • the data supply system further outputs the cut range specifying data.
  • An image processing system is the image processing system according to the 46th aspect, wherein the mask generation unit generates a small image by reducing the entire image including the ROI, A small mask generation unit that generates a small mask for a small image.
  • An image processing system is the image processing system according to the 59th aspect, wherein the mask generation unit expands the small mask to the same image size as the entire image and generates an expansion mask. It further has a part.
  • the image processing system is the image processing system according to the 60th aspect, wherein the small image generation unit generates the small image at a reduction ratio of 1/2 R of the entire image (R is Natural number).
  • the mask enlargement unit generates the enlargement mask by performing enlargement processing with an enlargement ratio of R times for the small mask.
  • the i-th data i is an integer
  • An image processing system is the image processing system according to the 61st aspect, wherein the mask enlarging unit performs the enlargement of the mask at the end of at least one of the R enlargement processes. Is smoothed.
  • the image processing system is the image processing system according to any one of the 60th to 62nd aspects, wherein the mask generation unit further includes a cutout range determination unit and a mask cutout unit.
  • the cut range determination unit performs a cut range determination process based on the enlarged mask data and the synthesis control data. In the cut-out range determination process, a cut-out range that is the range of the first target image in the entire image and a small mask cut-out range corresponding to the cut-out range in the small mask are determined.
  • the cut range determining unit generates cut range specifying data for specifying the cut range and small mask cut range specifying data for specifying the small mask cut range.
  • the mask cutout unit cuts out data in the small mask cutout range for the basic mask data from the small mask data based on the small mask cutout range specifying data.
  • the data supply system further outputs the cut range specifying data.
  • An image processing system is the image processing system according to the 59th or 63rd aspect, wherein the mask bitstream generation unit is configured to detect the entirety of the small mask or a portion cut out of the small mask.
  • the basic mask data which is data is acquired, and an encoded bit stream of the basic mask data is generated as the mask bit stream.
  • An image processing system is the image processing system according to the 59th or 63rd aspect, wherein the mask bitstream generation unit is configured to detect the whole of the small mask or a portion cut out of the small mask.
  • a bit stream of the basic mask data is generated as the mask bit stream without encoding the basic mask data which is data.
  • An image processing system is the image processing system according to any one of the 46th to 65th aspects, wherein the data supply system includes the wavelet conversion process, the synthesis execution process, The image processing apparatus further includes a synthesis unit that performs the inverse wavelet transform process.
  • An image processing system is the image processing system according to the 66th aspect, wherein the synthesizing unit outputs second basic image data which is data of a second basic image serving as a source of the second target image. get.
  • the synthesis unit further includes a second image cutout unit.
  • the second image cutout unit sets a similarity range that is similar to the first target image with the similarity ratio in the second basic image, and the similarity is included in the second basic image data. Data within the shape range is cut out as the second target image data.
  • An image processing system is the image processing system according to the 67th aspect, wherein the composition unit obtains the composite image data obtained by performing the inverse wavelet transform until the decomposition level becomes zero. Is embedded in the range of the similar shape in the second basic image data.
  • An image processing system is the image processing system according to the 66th aspect, wherein the synthesizing unit outputs second basic image data that is data of a second basic image that is a source of the second target image. get.
  • the synthesizing unit further includes an image reducing unit that reduces the at least part of the second basic image to generate the second target image.
  • the image processing system according to the 70th aspect is the image processing system according to any one of the 19th, 21st, 22nd and 52nd aspects, wherein the mask development process is for the first wavelet plane.
  • a unit process is performed for converting the first mask into a second mask for the second wavelet plane whose decomposition level is one step higher than that of the first wavelet plane, based on the mask development conditions.
  • the unit processing is recursively performed according to the wavelet transform method until the decomposition level of the second wavelet plane becomes the specified decomposition level. Do it.
  • the mask expansion condition when a 5 ⁇ 3 filter is used for the wavelet transform includes a first condition and a second condition, where n is an integer.
  • the first condition is that when 2n-th data on the first wavelet plane is associated with the ROI by the first mask, the n-th and high-frequency components of the low frequency component in the second wavelet plane.
  • the condition is that the second mask is formed so that the ⁇ n ⁇ 1 ⁇ th and n th data of the component are associated with the ROI.
  • the second condition is that when the ⁇ 2n + 1 ⁇ th data on the first wavelet plane is associated with the ROI by the first mask, the nth of the low-frequency component in the second wavelet plane.
  • ⁇ n + 1 ⁇ th and ⁇ n ⁇ 1 ⁇ th to ⁇ n + 1 ⁇ th data of the high frequency component are formed such that the second mask is formed so as to be associated with the ROI.
  • An image processing system is the image processing system according to any one of the 19th, 21st, 22nd and 52nd aspects, wherein the mask development process is performed for the first wavelet plane.
  • a unit process is performed for converting the first mask into a second mask for the second wavelet plane whose decomposition level is one step higher than that of the first wavelet plane, based on the mask development conditions.
  • the unit processing is recursively performed according to the wavelet transform method until the decomposition level of the second wavelet plane becomes the specified decomposition level. Do it.
  • the mask expansion condition when a Daubechies 9 ⁇ 7 filter is used for the wavelet transform includes a third condition and a fourth condition, where n is an integer.
  • the third condition is that when the 2n-th data on the first wavelet plane is associated with the ROI by the first mask, ⁇ n ⁇ 1 ⁇ of low-frequency components in the second wavelet plane
  • the second mask is formed so that ⁇ n + 1 ⁇ th to ⁇ n + 1 ⁇ th to ⁇ n-2 ⁇ th to ⁇ n + 1 ⁇ th data of the high frequency component are associated with the ROI.
  • the fourth condition is that when the ⁇ 2n + 1 ⁇ -th data on the first wavelet plane is associated with the ROI by the first mask, ⁇ n
  • the condition is that the second mask is formed so that the (n-1) th to ⁇ n + 2 ⁇ th and the ⁇ n-2 ⁇ th to ⁇ n + 2 ⁇ th data of the high frequency component are associated with the ROI.
  • An image processing system is the image processing system according to any one of the 26th, 28th, 30th, 32nd, 33rd and 55th to 58th aspects, wherein the cutout range determination process Includes a minimum range specifying process, a tracking process, and a necessary range specifying process.
  • the minimum range specifying process the minimum range of the rectangle including the ROI is specified based on the original mask for the original image before cutting.
  • a range corresponding to the minimum range is specified as a minimum tracking range in the most resolved highest band component of the final wavelet plane of the specified decomposition level specified by the synthesis control data.
  • the necessary range specifying process it is specified in which range of the original image before cropping the data necessary to calculate the wavelet coefficient within the minimum tracking range.
  • the range specified by the necessary range specifying process is set as the cutout range.
  • An image processing system is the image processing system according to the 39th or 63rd aspect, wherein the cutout range determining process includes a minimum range specifying process, a tracking process, a necessary range specifying process, and a small mask. Including necessary range identification processing.
  • the minimum range specifying process a minimum range of a rectangle including the ROI in the entire image is specified based on the enlarged mask.
  • the tracking process a range corresponding to the minimum range is specified as a minimum tracking range in the most resolved highest band component of the final wavelet plane of the specified decomposition level specified by the synthesis control data. .
  • the necessary range specifying process it is specified in which range of the entire image the data necessary for calculating the wavelet coefficient within the minimum tracking range exists.
  • the small mask necessary range specifying process it is specified to which range of the small mask the range specified by the necessary range specifying process corresponds.
  • the range specified by the necessary range specifying process is set as the cutout range.
  • the range specified by the small mask necessary range specifying process is set as the small mask cut-out range.
  • An image processing system is the image processing system according to the 73rd aspect, wherein the upper left corner coordinates of the cutout range are (AX3, AY3), and the lower right corner coordinates of the cutout range are (BX3, BY3).
  • the upper left coordinates of the small mask cutout range are (AX4, AY4), the lower right coordinates of the small mask cutout range are (BX4, BY4), and the reduction ratio of the small mask to the entire image is 1/2.
  • R is a natural number
  • each value of the AX3, AY3, BX3, BY3 is right-shifted by R bits in the small mask necessary range specifying process, and the obtained values are converted to the AX4, AY4, BX4. , BY4.
  • An image processing system is the image processing system according to any one of the 72nd to 74th aspects, wherein the wavelet transform includes a lowest range included in a wavelet plane of each decomposition level. This is a method for recursively decomposing band components. The highest band component is the lowest band component in the final wavelet plane.
  • An image processing system is the image processing system according to any one of the 72nd to 75th aspects, wherein the upper left coordinates of the minimum range are (AX1, AY1), and the minimum range
  • the bottom right coordinates are (BX1, BY1)
  • the top left coordinates of the minimum tracking range are (AX2, AY2)
  • the bottom right coordinates of the minimum tracking range are (BX2, BY2)
  • the top left coordinates of the cut range Is AX3, AY3
  • the lower right coordinate of the cutout range is (BX3, BY3)
  • a 5 ⁇ 3 filter is used for the wavelet transform
  • the minimum range specifying process uses the AX1 and the AY1.
  • BX1 and BY1 are obtained.
  • the second recursive process is set such that if AY1 is an even number, AY1 / 2 is set to a new AY1, and if AY1 is an odd number, ⁇ AY1-1 ⁇ / 2 is set to a new AY1.
  • the AY1 obtained finally is set to the AY2.
  • a third recursive process is performed for the specified number of times, wherein BX1 / 2 is set to a new BX1 if BX1 is an even number, and ⁇ BX1 + 1 ⁇ / 2 is set to a new BX1 if BX1 is an odd number. BX1 finally obtained is set as the BX2.
  • a fourth recursion process is performed for the designated number of times, in which BY1 / 2 is set to a new BY1 if BY1 is an even number, and ⁇ BY1 + 1 ⁇ / 2 is set to a new BY1 if BY1 is an odd number.
  • the BY1 finally obtained is set to the BY2.
  • the fifth recursive process of setting ⁇ AX2 ⁇ 2-2 ⁇ to a new AX2 is performed the specified number of times, and the finally obtained AX2 is set to the AX3.
  • a sixth recursive process of setting ⁇ AY2 ⁇ 2-2 ⁇ to a new AY2 is performed the specified number of times, and finally obtained AY2 is set to the AY3.
  • the seventh recursive process of setting ⁇ BX2 ⁇ 2 + 2 ⁇ to a new BX2 is performed the specified number of times, and the finally obtained BX2 is set to the BX3.
  • the eighth recursion process of setting ⁇ BY2 ⁇ 2 + 2 ⁇ to a new BY2 is performed the specified number of times, and finally obtained BY2 is set to BY3.
  • An image processing system is the image processing system according to any one of the 72nd to 75th aspects, wherein the upper left corner coordinates of the minimum range are (AX1, AY1), and the minimum range
  • the bottom right coordinates are (BX1, BY1)
  • the top left coordinates of the minimum tracking range are (AX2, AY2)
  • the bottom right coordinates of the minimum tracking range are (BX2, BY2)
  • the top left coordinates of the cut range Is AX3, AY3
  • the lower right coordinate of the cutout range is (BX3, BY3)
  • a Daubechies 9 ⁇ 7 filter is used for the wavelet transform
  • the AX1 and the AY1 are used in the minimum range specifying process.
  • BX1 and BY1 are obtained.
  • the AX1 is an even number
  • ⁇ AX1 / 2-1 ⁇ is set to a new AX1
  • ⁇ AX1-3 ⁇ / 2 is set to a new AX1.
  • the process is performed the designated number of times designated by the synthesis control data, and the finally obtained AX1 is set to the AX2.
  • AY1 is an even number
  • ⁇ AY1 / 2-1 ⁇ is set to a new AY1
  • if AY1 is an odd number ⁇ AY1-3 ⁇ / 2 is set to a new AY1.
  • the process is performed the specified number of times, and finally obtained AY1 is set to AY2.
  • the eleventh recursive process is set such that ⁇ BX1 + 2 ⁇ / 2 is set to a new BX1 if the BX1 is an even number, and ⁇ BX1 + 3 ⁇ / 2 is set to a new BX1 if the BX1 is an odd number.
  • the BX1 finally obtained is set to the BX2.
  • the 12th recursive process of setting ⁇ BY1 + 2 ⁇ / 2 to a new BY1 if the BY1 is an even number and setting ⁇ BY1 + 3 ⁇ / 2 to a new BY1 if the BY1 is an odd number is the designation.
  • the BY1 obtained finally is set to the BY2.
  • the thirteenth recursive process of setting ⁇ AX2 ⁇ 2-4 ⁇ to a new AX2 is performed the specified number of times, and the finally obtained AX2 is set to the AX3.
  • the 14th recursive process of setting ⁇ AY2 ⁇ 2-4 ⁇ to a new AY2 is performed the specified number of times, and finally obtained AY2 is set to the AY3.
  • the fifteenth recursive process of setting ⁇ BX2 ⁇ 2 + 4 ⁇ to a new BX2 is performed the specified number of times, and the finally obtained BX2 is set to the BX3.
  • the 16th recursive process of setting ⁇ BY2 ⁇ 2 + 4 ⁇ to a new BY2 is performed the specified number of times, and finally obtained BY2 is set to BY3.
  • an image processing method including steps (a) to (d) is provided.
  • first target image data, target mask data, second target image data, and synthesis control data are acquired.
  • the first target image data is data of a first target image having an ROI (region of interest).
  • the target mask data is target mask data for discriminating between the ROI and the non-ROI in the range of the first target image.
  • the second target image data is data of a second target image having a similar shape to the first target image and having a similarity ratio of 1 or less with respect to the first target image.
  • the synthesis control data is data for controlling the degree of synthesis of the ROI in the first target image and the second target image.
  • first wavelet coefficient data is generated by performing wavelet transformation on the first target image data to a designated decomposition level designated by the synthesis control data.
  • second wavelet coefficient data is generated by performing the wavelet transform on the second target image data to a decomposition level determined according to the designated decomposition level and the similarity ratio. To do.
  • the specified decomposition level of the first wavelet coefficient data is P1
  • the decomposition level of the second wavelet coefficient data is P2
  • the similarity ratio is 1/2 P3
  • P2 P1 ⁇ P3 is there.
  • step (c) coefficient discrimination processing is performed for discriminating, based on the target mask data, ROI coefficients related to the ROI and non-ROI coefficients related to the non-ROI with respect to the first wavelet coefficient data.
  • step (c) a coefficient synthesis process for synthesizing the ROI coefficient in the first wavelet coefficient data and the wavelet coefficient in the second wavelet coefficient data is performed.
  • synthesized coefficient data having the same image size and decomposition level as the second wavelet coefficient data is generated.
  • step (d) composite image data is generated by performing inverse wavelet transform on the combined coefficient data until the decomposition level reaches a predetermined end level.
  • the image processing method is the image processing method according to the 78th aspect, and further includes step (e).
  • the step (e) the first target image data and the target mask data are supplied.
  • the step (e) includes steps (e-1), (e-2) and (e-3).
  • first basic image data that is data of a first basic image that is a source of the first target image is encoded, and an image that is an encoded bit stream of the first basic image data Generate a bitstream.
  • basic mask data which is basic mask data serving as a source of the target mask, is generated by an algorithm independent of the encoding of the first basic image data.
  • a mask bit stream that is a bit stream of the basic mask data is generated.
  • the first target image data is supplied by the image bit stream, and the target mask data is supplied by the mask bit stream.
  • the image processing method according to the 80th aspect is the image processing method according to the 79th aspect, wherein the step (a) includes steps (a-1) and (a-2).
  • the step (a-1) the first basic image data is obtained by decoding the image bitstream, and the first target image data is obtained by the first basic image data.
  • the step (a-2) the basic mask data is acquired from the mask bitstream by an algorithm independent of decoding of the image bitstream, and the target mask data is acquired by the basic mask data.
  • the ROI in the first target image and the second target image are synthesized by combining the first wavelet coefficient data obtained by converting the first target image data and the second target image data, and the second target image data. This is done using wavelet coefficient data.
  • the ROI in the first target image is determined by determining the ROI coefficient for the first wavelet coefficient data. For this reason, even if insufficient ROI is used, a repaired ROI can be provided on the composite image. That is, a better composite image can be obtained as compared with the case where the first target image data and the second target image data are combined as they are.
  • the degree of synthesis ROI restoration, ROI expansion, etc.
  • the image size of the composite image can be adjusted.
  • FIG. 1 is a conceptual diagram of an image processing system according to Embodiment 1.
  • FIG. 1 is a block diagram illustrating an image processing system according to Embodiment 1.
  • FIG. 1 is a block diagram illustrating an image processing system according to Embodiment 1.
  • FIG. 3 is a block diagram illustrating an image encoder (for moving images) in the first embodiment.
  • FIG. 3 is a block diagram illustrating an image encoder (for still images) in the first embodiment.
  • 6 is a diagram illustrating an example of an entire image according to Embodiment 1.
  • FIG. 5 is a diagram illustrating an example of an entire mask in the first embodiment.
  • FIG. FIG. 3 is a block diagram illustrating a mask encoder according to the first embodiment.
  • FIG. 4 is a flowchart for explaining the operation of the data supply system according to the first embodiment.
  • FIG. 3 is a block diagram illustrating an image decoder (for moving images) in the first embodiment.
  • FIG. 3 is a block diagram illustrating an image decoder (for still images) in the first embodiment.
  • 5 is a block diagram illustrating a mask decoder in the first embodiment.
  • FIG. FIG. 3 is a block diagram illustrating a combining unit in the first embodiment. It is a figure explaining a Mallat type wavelet plane about Embodiment 1 (decomposition level 1). It is a figure explaining a Mallat type wavelet plane about Embodiment 1 (decomposition level 2).
  • FIG. 10 is a diagram for explaining mask development processing when a 5 ⁇ 3 filter is used for wavelet transformation in the first embodiment.
  • FIG. 10 is a diagram for describing mask development processing when a Daubechies 9 ⁇ 7 filter is used for wavelet transformation in the first embodiment.
  • FIG. 6 is a flowchart for explaining a coefficient synthesis process (first coefficient synthesis process) in the first embodiment.
  • 5 is a flowchart for explaining a coefficient synthesis process (second coefficient synthesis process) in the first embodiment.
  • 4 is a flowchart for explaining the operation of the image composition system in the first embodiment.
  • FIG. 3 is a conceptual diagram of image composition in the first embodiment.
  • 6 is a diagram illustrating an example of a first target image in the first embodiment.
  • FIG. 6 is a diagram illustrating an example of a target mask in the first embodiment.
  • FIG. It is a figure which shows the example of an expansion
  • FIG. 6 is a diagram illustrating an example of a second target image in the first embodiment.
  • FIG. 6 is a diagram illustrating an example of a composite image for comparison that does not depend on Embodiment 1.
  • FIG. 6 is a diagram illustrating an example of a composite image according to Embodiment 1 (composite at a decomposition level 1).
  • FIG. 5 is a diagram illustrating an example of a composite image according to Embodiment 1 (composite at a decomposition level 2).
  • FIG. 5 is a diagram illustrating an example of a composite image according to Embodiment 1 (composite at a decomposition level 3).
  • FIG. 5 is a diagram illustrating an example of a composite image according to Embodiment 1 (composite at a decomposition level 5).
  • 6 is a block diagram illustrating an image processing system according to Embodiment 2.
  • FIG. 10 is a block diagram illustrating an image encoder according to Embodiment 2 (for moving images).
  • FIG. 10 is a block diagram illustrating an image encoder according to Embodiment 2 (for still images). It is a block diagram explaining an image decoder about Embodiment 2 (for moving pictures). It is a block diagram explaining an image decoder about Embodiment 2 (for still pictures).
  • FIG. 10 is a flowchart for explaining the operation of the data supply system according to the second embodiment. 10 is a flowchart for explaining the operation of the image composition system in the second embodiment. 10 is a block diagram illustrating an image processing system according to Embodiment 3. FIG. 14 is a flowchart for explaining the operation of the data supply system according to the third embodiment. 10 is a flowchart for explaining the operation of the image composition system in the third embodiment.
  • FIG. 10 is a block diagram illustrating an image processing system according to a fourth embodiment.
  • FIG. 10 is a block diagram illustrating an image processing system according to a fifth embodiment.
  • FIG. 20 is a block diagram illustrating a mask data output unit according to the sixth embodiment.
  • FIG. 20 is a block diagram illustrating a combining unit in the sixth embodiment.
  • FIG. 16 is a conceptual diagram of image composition in the sixth embodiment.
  • 24 is a flowchart for explaining the operation of the data supply system in the sixth embodiment.
  • 20 is a flowchart for explaining the operation of the image composition system in the sixth embodiment.
  • FIG. 20 is a conceptual diagram of image composition in the seventh embodiment.
  • FIG. 20 is a diagram for explaining a cut-out range in the seventh embodiment.
  • FIG. 20 is a diagram for explaining a cut-out range in the seventh embodiment.
  • FIG. 20 is a diagram for explaining a cut-out range in the seventh embodiment.
  • FIG. 20 is a diagram for explaining a cut-out range in the seventh embodiment.
  • 24 is a flowchart for explaining a cut-out range determination process in the seventh embodiment.
  • FIG. 20 is a diagram for explaining a pixel range necessary for obtaining an output for one pixel by wavelet conversion in the seventh embodiment (5 ⁇ 3 filter).
  • FIG. 20 is a diagram for explaining a pixel range necessary for obtaining an output for one pixel by wavelet conversion in the seventh embodiment (5 ⁇ 3 filter). It is a figure explaining the range of the pixel required in order to obtain the output for 1 pixel by wavelet transformation about Embodiment 7 (Daubechies 9x7 filter). It is a figure explaining the range of the pixel required in order to obtain the output for 1 pixel by wavelet transformation about Embodiment 7 (Daubechies 9x7 filter).
  • FIG. 20 is a diagram for explaining a minimum tracking range in the seventh embodiment.
  • FIG. 20 is a block diagram illustrating a mask data output unit according to the seventh embodiment.
  • FIG. 20 is a block diagram illustrating an image composition system according to a seventh embodiment.
  • FIG. 20 is a block diagram illustrating a combining unit in the seventh embodiment.
  • 42 is a flowchart for explaining the operation of the data supply system in the seventh embodiment.
  • 24 is a flowchart for explaining the operation of the image composition system in the seventh embodiment.
  • FIG. 20 is a conceptual diagram of image composition in the eighth embodiment.
  • FIG. 20 is a block diagram illustrating a data supply system according to an eighth embodiment.
  • FIG. 20 is a block diagram illustrating a combining unit in the eighth embodiment.
  • 42 is a flowchart for explaining the operation of the data supply system in the eighth embodiment.
  • 42 is a flowchart for explaining the operation of the image composition system in the eighth embodiment.
  • FIG. 19 is a conceptual diagram of image composition in the ninth embodiment.
  • FIG. 20 is a block diagram illustrating a mask data output unit according to the ninth embodiment.
  • FIG. 25 is a block diagram illustrating a combining unit in the ninth embodiment.
  • 42 is a flowchart for explaining the operation of the data supply system in the ninth embodiment.
  • 38 is a flowchart for explaining the operation of the image composition system in the ninth embodiment.
  • FIG. 20 is a conceptual diagram of image composition in the tenth embodiment.
  • FIG. 10 is a block diagram illustrating a data supply system according to a tenth embodiment.
  • FIG. 20 is a block diagram illustrating a combining unit in the tenth embodiment.
  • 42 is a flowchart for explaining the operation of the data supply system in the tenth embodiment.
  • 42 is a flowchart for explaining the operation of the image composition system in the tenth embodiment.
  • FIG. 20 is a conceptual diagram of image composition in the eleventh embodiment.
  • FIG. 38 is a block diagram illustrating a combining unit in the eleventh embodiment. 42 is a flowchart for explaining the operation of the image composition system in the eleventh embodiment.
  • FIG. 20 is a conceptual diagram of image composition in the twelfth embodiment.
  • FIG. 38 is a block diagram illustrating a mask data output unit in the twelfth embodiment.
  • FIG. 20 is a block diagram illustrating a combining unit in the twelfth embodiment. 42 is a flowchart for explaining the operation of the data supply system in the twelfth embodiment. 38 is a flowchart for explaining the operation of the image composition system in the twelfth embodiment.
  • FIG. 20 is a conceptual diagram of image composition in the thirteenth embodiment.
  • FIG. 38 is a block diagram illustrating an inverse wavelet transform unit in the thirteenth embodiment.
  • FIG. 24 is a conceptual diagram of image composition in the fourteenth embodiment.
  • FIG. 24 is a conceptual diagram of image composition in the fourteenth embodiment.
  • FIG. 24 is a conceptual diagram of image composition in the fourteenth embodiment.
  • FIG. 24 is a conceptual diagram of image composition in the fourteenth embodiment.
  • FIG. 24 is a conceptual diagram of image composition in the fourteenth embodiment.
  • FIG. 24 is a conceptual diagram of image composition in
  • FIG. 40 is a block diagram illustrating a combining unit in the fourteenth embodiment. 40 is a flowchart for explaining the operation of the image composition system in the fourteenth embodiment.
  • FIG. 20 is a conceptual diagram of image composition in the fifteenth embodiment.
  • FIG. 25 is a block diagram illustrating a combining unit in the fifteenth embodiment.
  • 38 is a flowchart for explaining the operation of the image composition system in the fifteenth embodiment.
  • FIG. 20 is a conceptual diagram of image composition in the sixteenth embodiment.
  • FIG. 38 is a block diagram illustrating a mask data output unit in the sixteenth embodiment. 38 is a flowchart for describing mask enlargement processing according to the sixteenth embodiment. It is a figure explaining mask expansion processing about Embodiment 16 (one dimension).
  • Embodiment 16 It is a figure explaining mask expansion processing about Embodiment 16 (two dimensions). It is a figure explaining mask expansion processing about Embodiment 16 (two dimensions). 38 is a flowchart in the case where a smoothing process is performed after the last enlargement process in the sixteenth embodiment. 38 is a flowchart in the case where a smoothing process is performed after each enlargement process in the sixteenth embodiment.
  • it is a figure which shows the example of an expansion mask (when smoothing processing is not performed). It is a figure which shows the example of an expansion mask about Embodiment 16 (when an expansion process is performed twice). It is a figure which shows the example of an expansion mask about Embodiment 16 (when a contraction process is performed twice after performing an expansion process twice).
  • FIG. 42 is a flowchart for explaining the operation of the data supply system in the sixteenth embodiment.
  • FIG. 19 is a conceptual diagram of image composition in the seventeenth embodiment.
  • FIG. 38 is a block diagram illustrating a mask data output unit in the seventeenth embodiment.
  • FIG. 38 is a block diagram illustrating a combining unit in the seventeenth embodiment.
  • 42 is a flowchart for explaining the operation of the data supply system in the seventeenth embodiment.
  • 42 is a flowchart for explaining the operation of the image composition system in the seventeenth embodiment.
  • FIG. 38 is a block diagram for explaining an image processing system when basic mask data is not encoded and decoded according to the seventeenth embodiment.
  • FIG. 38 is a conceptual diagram of image composition in the eighteenth embodiment.
  • FIG. 38 is a block diagram illustrating a mask data output unit in the eighteenth embodiment.
  • FIG. 38 is a block diagram illustrating a combining unit in the eighteenth embodiment.
  • FIG. 38 is a flowchart for explaining cutout range determination processing in the eighteenth embodiment.
  • FIG. 38 is a conceptual diagram illustrating small mask cut-out range determination processing in the eighteenth embodiment.
  • FIG. 38 is a flowchart for describing a small mask cutout range determination process in the eighteenth embodiment.
  • 42 is a flowchart for explaining the operation of the data supply system in the eighteenth embodiment.
  • 42 is a flowchart for explaining the operation of the image composition system in the eighteenth embodiment.
  • FIG. 42 is a flowchart for explaining the operation of the image composition system in the eighteenth embodiment.
  • FIG. 38 is a conceptual diagram of image composition in the eighteenth embodiment.
  • FIG. 38 is a block diagram illustrating an image processing system according to the nineteenth embodiment.
  • FIG. 38 is a hardware configuration diagram for describing an image processing apparatus according to the nineteenth embodiment;
  • FIG. 1 shows a conceptual diagram of an image processing system 1 according to the first embodiment.
  • the image processing system 1 includes two image processing systems 10 and 20.
  • One image processing system 10 includes a data supply system 11
  • the other image processing system 20 includes an image composition system 21.
  • the image composition system 21 executes image composition processing.
  • the data supply system 11 outputs data used for image composition processing.
  • the image processing system 1 may be referred to as an overall system 1
  • the data supply system 11 may be referred to as a supply system 11
  • the image composition system 21 may be referred to as a composition system 21.
  • the image processing system 10 may be configured only by the supply system 11 or may further include another processing system.
  • the image processing system 20 may be configured by only the synthesis system 21 or may further include another processing system.
  • the supply system 11 is included in the image processing system 10 and also in the overall system 1.
  • the composition system 21 is included in the image processing system 20 and also in the overall system 1.
  • the image processing systems 10 and 20 are provided by a semiconductor integrated circuit. That is, the various functions and processes of the image processing systems 10 and 20 are realized in a circuit, in other words, in hardware. However, part or all of the functions and processes can be realized by a program that causes the microprocessor to function, in other words, by software.
  • FIG. 2 shows an application example of the entire system 1.
  • the image processing system 10 is provided in the data supply side device 30, and the image processing system 20 is provided in the image composition side device 40.
  • the data supply device 30 may be referred to as a supply device 30 and the image composition device 40 may be referred to as a composition device 40.
  • the user of the supply side device 30 is different from the user of the composition side device 40, but is not limited to this example.
  • the supply side device 30 includes a display unit 31, an operation unit 32, an external interface 33, and an image input unit 34.
  • the composition side device 40 includes a display unit 41, an operation unit 42, an external interface 43, and an image input unit 44.
  • the external interfaces 33 and 43 may be referred to as I / Fs 33 and 43.
  • the display units 31 and 41 are configured by, for example, a liquid crystal display device, but the display units 31 and 41 may be configured by different types of display devices.
  • the operation units 32 and 42 are operation media for the user to input instructions, data, and the like to the devices 30 and 40, in other words, the image processing systems 10 and 20.
  • the operation units 32 and 42 are configured by one or a plurality of devices such as a keyboard, a mouse, a button, and a switch.
  • the I / F 33 is a part where the supply side device 30 inputs and outputs signals with the outside of the device.
  • the I / F 43 is a part where the synthesizing apparatus 40 inputs and outputs signals with the outside of the apparatus.
  • the I / Fs 33 and 43 include a communication interface, so that the supply side device 30 and the composition side device 40 can communicate with each other through the I / Fs 33 and 43.
  • the communication method between the devices 30 and 40 may be any of wired, wireless, and combinations thereof.
  • a medium 50 is interposed for information transmission between the devices 30 and 40. In the case of communication as described above, the medium 50 is a wireless or wired communication medium (in other words, a communication path).
  • the I / Fs 33 and 34 may include an interface for an external storage medium in addition to or instead of the communication interface.
  • information transmission between the supply-side device 30 and the composition-side device 40 can be performed via an external storage medium, and the external storage medium corresponds to the medium 50 interposed between the devices 30 and 40. .
  • the image input unit 34 is configured by a digital camera. Alternatively, the image input unit 34 may be a storage device that supplies image data. The image input unit 44 is similarly configured. The image input units 34 and 44 may be configured by different types of devices.
  • combination side apparatus 40 is not limited to the example of FIG. That is, some of the above components may be omitted, or other components may be added.
  • FIG. 3 shows a more specific configuration example of the supply system 11 and the synthesis system 21.
  • the display units 31 and 41 are omitted in order to avoid complication of the drawing.
  • the supply system 11 includes an image data output unit 100 and a mask data output unit 200.
  • the image data output unit 100 includes an image encoder 150
  • the mask data output unit 200 includes a mask generation unit 210 and a mask encoder 250.
  • the composition system 21 includes an image decoder 300, a mask decoder 400, and a composition unit 500.
  • the image data output unit 100 acquires the entire image data A10 that is the data of the entire image, and outputs the first basic image data A50 that is the data of the first basic image.
  • the entire image data A10 is supplied from, for example, the image input unit 34 or another system in the image processing system 10.
  • the first basic image data A50 is output from the supply system 11.
  • the whole image provides a main image including an ROI (region of interest) for image synthesis in the synthesis system 21. That is, the entire image includes the ROI.
  • the first basic image is an image output from the image data output unit 100, and the first basic image is constituted by a part of the entire image including the ROI or the entire range of the entire image. In other words, the entire image is an image that is a source of the first basic image.
  • the image data output unit 100 outputs the entire range of the entire image as the first basic image (see FIG. 26 described later).
  • a range of the first basic image that is the target of the synthesis process in the synthesis unit 500 is referred to as a first target image.
  • the first target image is configured by a part including the ROI in the first basic image supplied to the synthesis system 21 or the entire range of the first basic image.
  • the first target image is configured by a part of the entire image including the ROI or the entire range of the entire image. That is, the whole image and the first basic image are images that are sources of the first target image.
  • the whole image, the first basic image, and the first target image may be images taken by a digital camera or the like, or may be computer graphics.
  • the image data output unit 100 includes the image encoder 150.
  • the image encoder 150 generates the encoded bit stream Abs of the first basic image data A50 by encoding the first basic image data A50 that is output data.
  • This encoded bit stream Abs may be referred to as an image bit stream Abs.
  • the image bit stream Abs is output from the I / F 33 to the outside of the supply side device 30.
  • the image encoder 150 can be configured by various image encoders, for example, encoders compliant with known codec standards (H.264, Moving Pictures Experts Group (MPEG), Joint Photographic Experts Group (JPEG), JPEG 2000, etc.). That is, the specification of the image encoder 150 is not particularly limited.
  • codec standards H.264, Moving Pictures Experts Group (MPEG), Joint Photographic Experts Group (JPEG), JPEG 2000, etc.
  • FIG. 4 shows a block diagram when a general image encoder 150s for moving images is used as the image encoder 150.
  • the motion compensation unit performs motion compensation on the input image (specifically, the image of each frame constituting the moving image).
  • the image after motion compensation is frequency-converted by the frequency converter, and then quantized by the quantizer.
  • the quantized data is encoded by the entropy encoding unit.
  • the encoded data is converted into a bit stream compliant with the codec specification of the image encoder 150s by the bit stream generation unit.
  • the quantized data generated by the quantization unit is inversely quantized by the inverse quantization unit and inversely frequency transformed by the inverse frequency transform unit in order to feed back to motion compensation.
  • the inverse frequency converted data is input to the motion compensation unit.
  • the motion compensation unit calculates the motion difference between the previous and next frames, and the calculation result is input to the frequency conversion unit.
  • FIG. 5 shows a block diagram when a general image encoder 150 t for still images is used as the image encoder 150.
  • the image encoder 150t has a configuration in which the motion compensation unit, the inverse quantization unit, and the inverse frequency conversion unit are removed from the image encoder 150s of FIG. That is, the input still image is converted into a bit stream that conforms to the codec specification of the image encoder 150t by the frequency conversion unit, the quantization unit, the entropy encoding, and the bit stream generation unit.
  • the mask data output unit 200 acquires the entire image data A10, generates basic mask data B50 that is basic mask data based on the entire image data A10, and outputs the basic mask data B50.
  • the basic mask data B50 is output from the supply system 11.
  • the mask data output unit 200 includes the mask generation unit 210 and the mask encoder 250. As shown in FIG. 3, the mask generation unit 210 includes an overall mask generation unit 211. The overall mask generation unit 211 generates overall mask data B10, which is overall mask data, based on the overall image data A10.
  • the whole mask is a mask for discriminating between ROI and non-ROI in the whole range of the whole image.
  • the basic mask is a mask output from the mask data output unit 200, and is a mask for the range of the basic image.
  • the basic mask is a mask for the range of the entire image (see FIG. 26 described later).
  • a mask for the range of the first target image is referred to as a target mask. That is, the whole mask is a mask that is a source of the basic mask and the target mask, and the basic mask is a mask that is a source of the target mask.
  • the mask encoder 250 acquires the entire mask data B10 as basic mask data B50, which is output data, and encodes the basic mask data B50 to generate an encoded bit stream Bbs of the basic mask data B50.
  • This encoded bit stream Bbs may be referred to as a mask bit stream Bbs.
  • the mask bit stream Bbs is output from the I / F 33 to the outside of the supply side device 30.
  • the mask generation unit 210 can be configured by various mask generation techniques. For example, a technique for detecting a moving object in a moving image is known. By using this moving object detection technique, a mask in which a moving object in an image is set to ROI can be generated. See, for example, Patent Documents 1 to 3 regarding the moving object detection technology. For example, Patent Documents 4 and 5 describe a technique for tracking a moving object using a particle filter. According to such a moving object tracking technique, the accuracy of detecting a moving object can be improved and the amount of calculation can be reduced.
  • a technique for separating a foreground image and a background image from an entire image by graph cut for a still image taken by a digital camera is known. If the mask generation unit 210 is configured using this image separation technique, a mask in which the foreground image is set to ROI can be generated.
  • the mask generation unit 210 performs preprocessing as appropriate when generating a mask. For example, when the entire image data A10 is image data (Bayer data) taken by a digital camera, the Bayer data is converted into RGB color data. When the size of the entire image is large, reduction processing is performed to reduce the amount of calculation. In addition, color space conversion to black and white, YUV, HSV, or the like is performed in order to extract feature amounts.
  • image data A10 image data (Bayer data) taken by a digital camera
  • the Bayer data is converted into RGB color data.
  • reduction processing is performed to reduce the amount of calculation.
  • color space conversion to black and white, YUV, HSV, or the like is performed in order to extract feature amounts.
  • an example of the entire image 60 is shown in FIG.
  • the part of the person who is moving is set to the ROI 60a, and the other part is set to the non-ROI 60b.
  • FIG. 7 shows the entire mask 70 corresponding to the entire range of the entire image 60.
  • the whole mask 70 can be understood as an image indicating whether each pixel in the whole image 60 belongs to the ROI 60a or the non-ROI 60b.
  • the entire mask 70 has an ROI corresponding portion 70a and a non-ROI corresponding portion 70b corresponding to the ROI 60a and the non-ROI 60b in the entire image 60.
  • the white portion is the ROI corresponding portion 70a
  • the black portion is the non-ROI corresponding portion 70b.
  • the entire mask 70 may be generated for all frame images, or the entire mask 70 may be generated, for example, every fixed frame or every fixed time. The same applies to the case where still images are sequentially input.
  • the mask encoder 250 can be configured by various image encoders.
  • the entire mask 70 (inputted to the mask encoder 250 as a basic mask) only needs to distinguish between the ROI corresponding portion 70a and the non-ROI corresponding portion 70b. For this reason, the entire mask 70 is essentially a binary image.
  • the mask encoder 250 can be configured by an encoder conforming to a codec standard used in a facsimile. Codec standards used in facsimile include JBIG (Joint Bi-level Image Experts Group), JBIG2, MH (Modified Huffman), MR (Modified READ (Relative Element Address Designate)), MMR (Modified Modified READ), and the like.
  • FIG. 8 shows a block diagram when a general image encoder 250u for facsimile is used as the mask encoder 250.
  • the preprocessing unit performs predetermined preprocessing (for example, run length processing) on the input mask image.
  • the preprocessed data is encoded by the entropy encoding unit, and the entropy encoding unit outputs an encoded bit stream that conforms to the codec specification of the image encoder 250u.
  • Entropy coding includes arithmetic coding, variable length coding, and the like. Since the mask is a binary image, a certain degree of compression can be obtained by, for example, run-length processing (in other words, run-length encoding) without using advanced encoding such as entropy encoding.
  • the mask encoder 250 acquires the synthesis control data C50 for controlling the degree of synthesis, and embeds the synthesis control data C50 as additional information in the mask bitstream Bbs.
  • the mask bit stream Bbs has a region that does not affect the basic mask data B50, and the entropy encoding unit embeds the synthesis control data C50 in that region.
  • An example of such an area of the mask bitstream Bbs is a header area.
  • the composition control data C50 is supplied to the supply system 11 when the user inputs the operation unit 32 provided on the supply system 11 side.
  • the mask encoder 250 generates a mask bit stream Bbs, which is a bit stream of the basic mask data B50 (whole mask data B10 in the example of FIG. 3), in accordance with a predetermined specification. It is an example.
  • the encoded bit stream Bbs is an example of the mask bit stream Bbs.
  • the mask bitstream Bbs may be a bitstream of unencoded data. In that case, a bitstream generation specification without encoding is adopted. Since the mask is a binary image as described above, the data size of the basic mask is often smaller than that of the first basic image. For this reason, a certain level of practicality can be obtained without encoding the basic mask data B50. In addition, the amount of calculation related to encoding can be reduced.
  • the first basic image data A50 is output to the outside of the supply-side device 30 (and thus to the outside of the supply system 11) by the image bitstream Abs.
  • the basic mask data B50 is output to the outside of the supply-side device 30 by the mask bit stream Bbs.
  • the synthesis control data C50 is output to the outside of the supply side device 30 by the mask bit stream Bbs together with the basic mask data B50.
  • the image bit stream Abs and the mask bit stream Bbs are output to the outside of the supply side device 30 as separate bit streams.
  • the mask generation unit 210 generates the basic mask data B50 by an algorithm independent from the encoding (encoding algorithm) of the first basic image data A50.
  • the Max-shift method which is an optional function of JPEG2000
  • a generation algorithm for information specifying ROI is incorporated in an image data encoding algorithm.
  • the basic mask data B50 is generated by an algorithm different from the Max-shift method or the like.
  • FIG. 9 shows a flowchart for explaining the operation of the supply system 11.
  • the image encoder 150 acquires the whole image data A10, and the whole image data
  • the image bit stream Abs is generated by encoding the data A10 as the first basic image data A50 (step S112).
  • the mask generation unit 210 generates the entire mask data B10 as the basic mask data B50 based on the entire image data A10 (step S122). Then, the mask encoder 250 acquires the entire mask data B10 as the basic mask data B50, encodes the basic mask data B50, and generates a mask bit stream Bbs (step S123). Thereafter, the mask encoder 250 embeds the synthesis control data C50 in the mask bit stream Bbs (step S124).
  • the image bit stream Abs is output from the image data output unit 100, and the mask bit stream Bbs is output from the mask data output unit 200 (step S105).
  • step S112 and steps S122 to S124 are executed in parallel, but step S112 and steps S122 to S124 may be executed sequentially.
  • the synthesis control data C50 may be input in step S101, or the synthesis control data C50 that has been input and held in advance may be used in step S124.
  • the first basic image data A50, the basic mask data B50, and the composition control data C50 are output from the supply system 11 via the I / F 33 and input to the composition system 21 via the I / F 43.
  • the medium 50 (see FIG. 2) is used.
  • the first basic image data A50, the basic mask data B50, and the composition control data C50 are supplied to the composition system 21 by the same medium 50 (for example, a communication medium or an external storage medium).
  • a different medium 50 may be used.
  • a communication medium may be used for supplying the first basic image data A50
  • an external storage medium may be used for supplying the basic mask data B50 and the composition control data C50.
  • the image decoder 300 obtains the first basic image data A50 by obtaining the image bitstream Abs and decoding the image bitstream Abs.
  • the image decoder 300 only needs to correspond to the image encoder 150, and the image decoder 300 can be configured by various image decoders, for example, decoders based on known codec standards. That is, the specification of the image decoder 300 is not particularly limited.
  • FIG. 10 shows a block diagram when a general image decoder 300 s for moving images is used as the image decoder 300.
  • the image bitstream Abs is analyzed by the bitstream analysis unit, decoded by the entropy decoding unit, dequantized by the inverse quantization unit, and inverse frequency converted by the inverse frequency conversion unit.
  • the motion compensation unit performs motion compensation from the correlation with the previous and subsequent frames, thereby generating decoded image data, that is, first basic image data A50.
  • FIG. 11 shows a block diagram when a general image decoder 300t for still images is used as the image decoder 300.
  • the image decoder 300t has a configuration in which the motion compensation unit is removed from the image decoder 300s of FIG. That is, the first basic image data A50 is acquired from the image bitstream Abs by the bitstream analysis unit, the entropy decoding unit, the inverse quantization unit, and the inverse frequency conversion unit.
  • the mask decoder 400 acquires the mask bit stream Bbs, and acquires the basic mask data B50 by decoding the mask bit stream Bbs.
  • the supply system 11 outputs the mask bit stream Bbs as a bit stream separate from the image bit stream Abs. Therefore, the synthesis system 21 acquires the mask bit stream Bbs and the image bit stream Abs by separate bit streams.
  • the mask decoder 400 corresponds to the mask encoder 250, it can be constituted by various image decoders. That is, the specification of the mask decoder 400 is not particularly limited.
  • FIG. 12 shows a block diagram of a general image decoder 400u for facsimile. According to the image decoder 400u, the mask bitstream Bbs is decoded by the entropy decoding unit, and the post-processing unit performs predetermined post-processing on the decoded data.
  • the entropy decoding unit analyzes the mask bitstream Bbs before decoding. In the analysis, it is checked whether data is embedded in a predetermined area of the mask bit stream Bbs. If data is embedded, the entropy decoding unit extracts the data. In this case, since the synthesis control data C50 is acquired by the synthesis system 21 in a state embedded in the mask bitstream Bbs, the entropy decoding unit extracts the synthesis control data C50.
  • the mask bit stream Bbs is decoded by an algorithm independent from the decoding of the image bit stream Abs in the image decoder 300.
  • the mask decoder 400 is an example of the mask data restoration unit 400 that restores the basic mask data B50 from the mask bitstream Bbs by an algorithm independent from the decoding of the image bitstream Abs.
  • the mask bitstream Bbs may not be an encoded bitstream.
  • a data restoration specification that does not perform decryption is adopted, and the amount of computation related to decryption can be reduced.
  • the synthesis unit 500 acquires the first basic image data A50, the basic mask data B50, the synthesis control data C50, and the second basic image data D50, and uses these data to perform image synthesis. And composite image data E100, which is composite image data, is output.
  • the second basic image provides a background image for image composition.
  • the second basic image data D50 which is the data of the second basic image, is supplied from, for example, the image input unit 44 provided on the synthesis system 21 side or another system of the image processing system 20 (see FIG. 1).
  • the Note that the second basic image and the second target image described later may be images captured by a digital camera or the like, or may be computer graphics.
  • FIG. 13 shows a configuration example of the synthesis unit 500.
  • the synthesis unit 500 includes a wavelet conversion unit 510, a synthesis execution unit 520, and an inverse wavelet conversion unit 530.
  • the wavelet transform unit 510 performs discrete processing of integer type or real number type for each of the first target image data A60 that is data of the first target image and the second target image data D60 that is data of the second target image.
  • Wavelet transform DWT
  • the transform coefficient may be referred to as, for example, a wavelet transform coefficient or a wavelet coefficient.
  • the first target image is an image in a range where wavelet transformation is performed in the first basic image.
  • the synthesis unit 500 treats the entire range of the first basic image as the first target image, and thus the first basic image data A50 is processed as the first target image data A60 (see FIG. 26 described later).
  • the second target image is an image in a range where wavelet transformation is performed in the second basic image. That is, the second basic image is an image that is a source of the second target image.
  • the synthesis unit 500 treats the entire range of the second basic image as the second target image, so that the second basic image data D50 is processed as the second target image data D60 (see FIG. 26 described later). ).
  • the second target image has the same shape and size as the first target image
  • the expression that the images are congruent basically means that the outer shape of the entire image is congruent. That is, whether or not to include the specific display content of the image as a comparison target is determined according to the purpose of the comparison or the like.
  • first wavelet coefficient data A61 Data (wavelet coefficient group) generated by performing wavelet transform on the first target image data A60 will be referred to as first wavelet coefficient data A61.
  • second wavelet coefficient data D61 data (wavelet coefficient group) generated by performing wavelet transform on the second target image data D60 will be referred to as second wavelet coefficient data D61.
  • a high frequency component in other words, a high frequency component
  • a low frequency component in other words, a low frequency component
  • Such frequency decomposition is also called, for example, band division.
  • Each band component obtained by frequency decomposition (that is, each of a low-frequency component and a high-frequency component) is also called a subband.
  • an octave division method is adopted in which, following the basic method of JPEG 2000, only the band component divided in the low frequency side in both the vertical direction and the horizontal direction is recursively divided into bands.
  • the number of recursive band divisions is called a decomposition level.
  • the decomposition level of wavelet transform is specified by the synthesis control data C50.
  • the wavelet conversion unit 510 performs decomposition level 3 wavelet conversion.
  • 14 to 16 show Mallat-type wavelet planes 61 to 63 for two-dimensional wavelet transformation.
  • the input image two-dimensional image
  • the input image is subjected to frequency decomposition in the vertical direction and the horizontal direction at decomposition level 1 (see FIG. 14).
  • decomposition level 1 see FIG. 14
  • the band component LL1 obtained at the decomposition level 1 is further decomposed into four band components HH2, HL2, LH2, and LL2 at the decomposition level 2 (see the wavelet plane 62 in FIG. 15).
  • the band component LL2 obtained at the decomposition level 2 is further decomposed into four band components HH3, HL3, LH3, and LL3 at the decomposition level 3 (see the wavelet plane 63 in FIG. 16). Note that the designated value of the decomposition level by the synthesis control data C50 is not limited to 3.
  • HL1 is a band component composed of a horizontal high-frequency component H and a vertical low-frequency component L at decomposition level 1.
  • the notation is generalized as “XYm” (X and Y are either H or L. m is an integer of 1 or more). That is, the band component composed of the horizontal band component X and the vertical band component Y at the decomposition level m is denoted as “XYm”.
  • the wavelet plane (see FIGS. 14 to 16) is a two-dimensional map in which the wavelet transform calculation result data is associated with a sequence of pixels in the original image (an image in which the wavelet transform is not performed).
  • a group of data arranged.
  • the calculation result data (LL component data) obtained by using a certain pixel in the original image as the target pixel is the value of the target pixel in the original image. It is arranged according to the position.
  • the wavelet plane is sometimes called a wavelet space, a wavelet region, or a wavelet image.
  • the band component LL1 corresponds to the essential information of the image. Note that according to the band component LL1, it is possible to provide an image having a quarter size of the image before decomposition (in other words, an image having a reduction ratio of 1/2 with respect to the image before decomposition).
  • the band component HL1 corresponds to edge information extending in the vertical direction
  • the band component LH1 corresponds to edge information extending in the horizontal direction
  • the band component HH corresponds to information on an edge extending in an oblique direction.
  • an original image that has not been subjected to wavelet conversion may be associated with decomposition level 0, and the original image may be expressed as a decomposition level 0 wavelet plane.
  • the band component most decomposed in the wavelet plane will be referred to as the highest band component.
  • the highest band components are LL3, HL3, LH3, and HH3.
  • the highest band components are LLk, HLk, LHk, and HHk.
  • the band component LL is referred to as the lowest band component and the band component HH is referred to as the highest band component.
  • the LL component is recursively decomposed by the same number of times in each of the horizontal direction and the vertical direction.
  • band components are synthesized in the reverse order of the decomposition.
  • the upper left corner is taken as the origin of the coordinate system, the origin is treated as 0, the wavelet transform L component output is treated as an even number, and the H component output is treated as an odd number. However, it is also possible to treat the L component output as an odd number and the H component output as an even number.
  • the wavelet plane (see FIGS. 14 to 16) is a conceptual plane in which the even-numbered and odd-numbered outputs of the wavelet transform are rearranged for each band component.
  • the synthesis execution unit 520 determines, for the first wavelet coefficient data A61, a coefficient related to ROI (hereinafter also referred to as ROI coefficient) and a coefficient related to non-ROI (hereinafter also referred to as non-ROI coefficient). Perform discrimination processing. Then, the synthesis execution unit 520 performs a coefficient synthesis process for synthesizing the ROI coefficient in the first wavelet coefficient data A61 and the wavelet coefficient in the second wavelet coefficient data D61, whereby synthesized coefficient data E61. Is generated.
  • ROI coefficient a coefficient related to ROI
  • non-ROI coefficient hereinafter also referred to as non-ROI coefficient
  • the coefficient determination process is performed based on target mask data B60 which is target mask data for determining ROI and non-ROI in the range of the first target image.
  • target mask data B60 which is target mask data for determining ROI and non-ROI in the range of the first target image.
  • the basic mask for the range of the first basic image becomes the target mask for the range of the first target image (FIG. 26 described later). reference).
  • the target mask targets the first target image that is an original image (an image in a state where wavelet transformation is not performed).
  • the composition execution unit 520 includes a mask development unit 521.
  • the mask for the original image is called an original mask.
  • the original mask can be understood as an image indicating whether each pixel in the original image belongs to ROI or non-ROI.
  • the original mask has an ROI corresponding portion and a non-ROI corresponding portion corresponding to the ROI and the non-ROI in the original image.
  • the whole mask 70 (see FIG. 7) is an original mask.
  • the basic mask and the target mask are also original masks.
  • the mask development unit 521 uses the ROI corresponding portion and the non-ROI corresponding portion of the original mask for each band component included in the first wavelet coefficient data A61 (in other words, the wave corresponding to the first wavelet coefficient data A61). Expand for each band component contained in the let plane. By such mask development processing, a development mask which is a mask for the first wavelet coefficient data A61 is generated. Regarding the development of the mask, see, for example, Patent Documents 6 and 7 and Non-Patent Document 1.
  • FIG. 17 to 19 show unfolded masks 71, 72, and 73, respectively, in which the entire mask 70 of FIG.
  • ROI corresponding portions 71a, 72a, 73a are illustrated in white, and non-ROI corresponding portions 71b, 72b, 73b are illustrated in black.
  • FIG. 20 shows a flowchart of the mask development process.
  • a process hereinafter also referred to as a unit process
  • S202 for increasing the mask decomposition level by one step is performed. If the decomposition level specified by the synthesis control data C50 is 2 or more, the unit process S202 is repeated until a mask of the specified decomposition level is obtained (see step S201).
  • the first mask for the first wavelet plane is converted into a second mask for the second wavelet plane whose decomposition level is one step higher than that of the first wavelet plane.
  • the first mask to be developed is an original mask
  • the original image before wavelet conversion corresponds to the first wavelet plane.
  • the unit process S202 is repeated recursively. That is, the unit process S202 is performed again by setting the second mask as a new first mask.
  • the unit process S202 is repeated according to the wavelet conversion method. For example, when the above Mallat type method is adopted (see FIGS. 14 to 16), only the lowest band component LL is recursively decomposed on the wavelet plane. For this reason, mask development is also performed recursively only for the portion corresponding to the band component LL.
  • the unit processing S202 is performed based on a predetermined mask development condition, and the mask development condition depends on the number of taps of the wavelet transform filter.
  • the mask development condition includes two conditions (referred to as a first condition and a second condition) based on FIG.
  • the decomposition-side low-pass filter has 5 taps
  • the decomposition-side high-pass filter has 3 taps.
  • n can be expressed as 2nth, where n is an integer
  • the second mask is formed so that the n-th data (corresponding to the output data on the low-pass filter side) is associated with the ROI.
  • a second mask is formed so that the ⁇ n ⁇ 1 ⁇ th and nth data of the high-frequency component (corresponding to the output data on the high-pass filter side) are associated with the ROI in the second wavelet plane.
  • Second condition When the ⁇ 2n + 1 ⁇ th data on the first wavelet plane is associated with the ROI by the first mask, the nth and ⁇ n + 1 ⁇ th and high frequencies of the low-frequency component in the second wavelet plane The second mask is formed so that the ⁇ n ⁇ 1 ⁇ th to ⁇ n + 1 ⁇ th data of the band components are associated with the ROI.
  • the mask expansion condition includes two conditions (referred to as a third condition and a fourth condition) based on FIG.
  • the decomposition-side low-pass filter has 9 taps
  • the decomposition-side high-pass filter has 7 taps.
  • the synthesis execution unit 520 When the decomposition level specified by the synthesis control data C50 is 3, the synthesis execution unit 520 performs coefficient discrimination processing based on the distinction between the ROI corresponding portion 73a and the non-ROI corresponding portion 73b in the development mask 73 (see FIG. 19). I do.
  • the synthesis execution unit 520 performs synthesis by performing a coefficient synthesis process for synthesizing the ROI coefficient in the first wavelet coefficient data A61 and the wavelet coefficient in the second wavelet coefficient data D61.
  • Coefficient data E61 is generated.
  • the non-ROI coefficient is set as the second wavelet.
  • the coefficient data D61 is replaced with data existing at the same position on the wavelet plane. According to this, the first wavelet coefficient data A61 is converted into the combined coefficient data E61.
  • FIG. 23 shows a flowchart of the coefficient synthesis process. 23, first, one wavelet coefficient to be inspected is selected from the first wavelet coefficient data A61 (step S301). Then, it is determined whether the inspection object coefficient is an ROI coefficient or a non-ROI coefficient (step S302). That is, in step S302, coefficient determination processing is performed.
  • step S303 When the inspection target coefficient is a non-ROI coefficient, the inspection target coefficient is replaced with the corresponding wavelet coefficient in the second wavelet coefficient data D61 (step S303). On the other hand, when the inspection target coefficient is an ROI coefficient, step S303 is not performed. Steps S301 to S303 are repeated until all the wavelet coefficients in the first wavelet coefficient data A61 are inspected (step S304).
  • FIG. 24 shows another flowchart of the coefficient synthesis process.
  • step S303B is provided instead of step S303. That is, when it is determined in the coefficient determination processing step S302 that the inspection target coefficient is an ROI coefficient, the ROI coefficient in the first wavelet coefficient data A61 is associated with the corresponding one in the second wavelet coefficient data D61 in step S303B. Embed in position. According to this, the second wavelet coefficient data D61 is converted into the combined coefficient data E61.
  • composition of the ROI coefficient in the first wavelet coefficient data A61 and the wavelet coefficient in the second wavelet coefficient data D61 includes two specific methods (first coefficient synthesis process and second coefficient). (Referred to as synthesis processing).
  • First coefficient synthesis process (see S300 in FIG. 23): The non-ROI coefficient in the first wavelet coefficient data A61 is replaced with the corresponding wavelet coefficient in the second wavelet coefficient data D61.
  • Second coefficient synthesis process (see S300B in FIG. 24): The ROI coefficient in the first wavelet coefficient data A61 is embedded in the corresponding position in the second wavelet coefficient data D61.
  • the inverse wavelet transform unit 530 performs inverse wavelet transform (IDWT) on the combined coefficient data E61 generated by the composition execution unit 520 until the decomposition level becomes zero. Thereby, synthesized image data E80 is generated from the synthesized coefficient data E61.
  • the composite image data E80 is output from the composite unit 500 and becomes composite image data E100 that is output data.
  • the inverse wavelet conversion is a process opposite to the wavelet conversion performed by the wavelet conversion unit 510.
  • band components are recursively synthesized.
  • the number of synthesis in the inverse wavelet transform is called a synthesis level. Note that the synthesis level in the state before the inverse wavelet transform is expressed as 0. For example, when the decomposition level specified by the synthesis control data C50 is 3, the decomposition level 3 corresponds to the synthesis level 0.
  • FIG. 25 shows a flowchart for explaining the operation of the synthesis system 21.
  • FIG. 26 shows a conceptual diagram of image composition.
  • the operation flow S400 of FIG. 25 is roughly divided into step S410 of data acquisition processing, step S440 of wavelet conversion processing, step S450 of synthesis execution processing, and step S460 of inverse wavelet conversion processing.
  • the image decoder 300 decodes the image bitstream Abs to One basic image data A50 is acquired (step S422).
  • the mask decoder 400 extracts the synthesis control data C50 from the mask bitstream Bbs (step S432). Thereafter, the mask decoder 400 decodes the mask bitstream Bbs to obtain basic mask data B50 (step S433).
  • the entire range of the first basic image is treated as the first target image
  • the entire range of the basic mask is treated as the target mask
  • the entire range of the second basic image is processed.
  • the first basic image data A50, the basic mask data B50, and the second basic image data D50 are treated as the first target image data A60, the target mask data B60, and the second target image data D60 in the synthesis unit 500 ( (See FIG. 13).
  • the wavelet conversion unit 510 performs wavelet conversion on each of the first target image data A60 and the second target image data D60 to the decomposition level designated by the synthesis control data C50. Thereby, the first wavelet coefficient data A61 and the second wavelet coefficient data D61 are generated from the first target image data A60 and the second target image data D60.
  • the mask development unit 521 In the synthesis execution step S450, the mask development unit 521 generates decomposition mask data at a decomposition level designated by the synthesis control data C50 based on the target mask data B60 (step S451). For mask development, refer to mask development processing S200 in FIG. Then, the synthesis execution unit 520 performs coefficient discrimination processing and coefficient synthesis processing to generate synthesized coefficient data E61 (step S452).
  • the inverse wavelet transform unit 530 performs inverse wavelet on the combined coefficient data E61 to generate composite image data E80 (which is output data E100 in the first embodiment). .
  • step S422 and steps S432 to S433 are executed in parallel, but step S422 and steps S432 to S433 may be executed sequentially. Further, step S440 and step S451 may be executed in parallel.
  • the synthesis of the ROI in the first target image and the second target image is performed using the first wavelet coefficient data A61 and the second wavelet coefficient data D61. Also, the ROI in the first target image is determined by determining the ROI coefficient for the first wavelet coefficient data A61. Such coefficient determination is performed based on the target mask data B60, more specifically, based on the development mask data for the first wavelet coefficient data A61 (generated based on the target mask data B60).
  • the synthesis condition (ROI repair, ROI expansion, etc.) can be adjusted.
  • FIG. 27 shows an example of the first target image.
  • the child part is designated as ROI.
  • FIG. 28 shows an original mask corresponding to FIG.
  • FIG. 29 to FIG. 31 show development masks of decomposition levels 1, 2 and 3 generated from the original mask of FIG. 28 (which is also the target mask in the first embodiment).
  • the white portion is the ROI corresponding portion
  • the black portion is the non-ROI corresponding portion.
  • FIG. 32 is an example of the second target image.
  • FIG. 33 is a composite image that does not depend on the first embodiment. Specifically, the ROI is cut out from the image of FIG. 27 based on the mask of FIG. 28, and the cut ROI and the image of FIG. 32 are combined to obtain the composite image of FIG.
  • FIGS. 34 to 36 show images synthesized by the first embodiment. 34 to 36 correspond to images synthesized at the decomposition levels 1, 2, and 3, respectively. For reference, an image synthesized at the decomposition level 5 is shown in FIG.
  • the ROI contour here, the contour of the child
  • the contour of the right arm of a child lacks accuracy.
  • non-ROIs that form defects are sometimes found in ROIs. For this reason, as shown in FIG. 33, the unnaturalness is conspicuous in the composite image not based on the first embodiment.
  • the range cut out from the first target image in FIG. 27 becomes wider as the decomposition level for synthesis increases.
  • the original ROI children's part
  • the expanded part becomes wider as the decomposition level for the synthesis increases. It should be noted that the higher the decomposition level, the more the boundary between the extended portion and the second target image can be blurred.
  • the mask is generated by an algorithm independent of the encoding of the first basic image. Further, the decoding of the basic mask data in the synthesis system 21 is performed by an algorithm independent from the decoding of the first basic image. For this reason, the freedom degree of selection of the image encoder 150 and the image decoder 300 becomes high.
  • FIG. 38 shows a configuration example of the supply system 11B and the synthesis system 21B according to the second embodiment.
  • Supply system 11B and composition system 21B can be applied to image processing systems 1, 10, 20 and the like instead of supply system 11 and composition system 21 according to the first embodiment.
  • the supply system 11B and the synthesis system 21B are basically the same as the supply system 11 and the synthesis system 21 (see FIG. 3) according to the first embodiment, except for the following points.
  • an image data output unit 100B instead of the image data output unit 100, an image data output unit 100B having an image encoder 150B is provided.
  • an image decoder 300B is provided instead of the image decoder 300.
  • the image encoder 150B acquires the mask bit stream Bbs from the mask encoder 250, and embeds the mask bit stream Bbs as additional information in the image bit stream Abs. Specifically, the image bit stream Abs has an area that does not affect the first basic image data A50, and the mask bit stream Bbs is embedded in the area. Other operations of the image encoder 150B are the same as those of the image encoder 150.
  • a comment area in the format of the encoded stream can be used.
  • an application marker (APP marker) area may be used instead of or in addition to the comment area.
  • FIG. 39 shows a block diagram when a general image encoder 150u for moving images is used as the image encoder 150B.
  • the bit stream generation unit embeds the mask bit stream Bbs in the image bit stream Abs.
  • the other configuration of the image encoder 150u is the same as that of the image encoder 150s in FIG.
  • FIG. 40 shows a block diagram when a general image encoder 150v for still images is used as the image encoder 150B.
  • the bit stream generation unit embeds the mask bit stream Bbs in the image bit stream Abs.
  • the other configuration of the image encoder 150v is the same as that of the image encoder 150t in FIG.
  • the image decoder 300B acquires the image bit stream Abs, extracts the mask bit stream Bbs from the image bit stream Abs, and supplies the mask bit stream Bbs to the mask decoder 400. Other operations of the image decoder 300B are the same as those of the image decoder 300.
  • FIG. 41 shows a block diagram when a general image decoder 300u for moving images is used as the image decoder 300B.
  • the bit stream analysis unit extracts the mask bit stream Bbs from the image bit stream Abs.
  • the other configuration of the image decoder 300u is the same as that of the image decoder 300s in FIG.
  • FIG. 42 shows a block diagram when a general image decoder 300v for still images is used as the image decoder 300B.
  • the bit stream analysis unit extracts the mask bit stream Bbs from the image bit stream Abs.
  • the other configuration of the image decoder 300v is the same as that of the image decoder 300t in FIG.
  • the image bitstream Abs and the mask bitstream Bbs are output from the supply system 11B as a single bitstream and acquired by the synthesis system 21B.
  • the first basic image data A50 and the basic mask data B50 exist as separate data. This is because the generation algorithm of the basic mask data B50 is independent of the encoding algorithm of the first basic image data A50, as in the first embodiment.
  • the synthesis control data C50 is embedded in the mask bitstream Bbs. Therefore, the first basic image data A50, the basic mask data B50, and the synthesis control data C50 are output from the supply system 11B as a single bit stream and acquired by the synthesis system 21B.
  • the image encoder 150B embeds both the mask bit stream Bbs (in other words, the basic mask data B50) and the synthesis control data C50 in the image bit stream Abs.
  • the image decoder 300B extracts both the mask bit stream Bbs and the synthesis control data C50 from the image bit stream Abs.
  • the image encoder 150B acquires the synthesis control data C50 in a state where it is not embedded in the mask bit stream Bbs (thus acquiring the synthesis control data C50 and the mask bit stream Bbs separately), and the synthesis control data C50. Both the mask bit stream Bbs and the mask bit stream Bbs may be embedded in the area of the image bit stream Abs.
  • FIG. 43 shows a flowchart for explaining the operation of the supply system 11B.
  • step S106 is added to the operation flow S100 (see FIG. 9) according to the first embodiment.
  • the image encoder 150B embeds the mask bit stream Bbs in the image bit stream Abs.
  • FIG. 44 shows a flowchart for explaining the operation of the synthesis system 21B.
  • the operation flow S400B of FIG. 44 is basically the same as the operation flow S400 (see FIG. 25) according to the first embodiment, except for the following points.
  • a data acquisition step S410B to which step S408 is added is provided.
  • the image decoder 300B extracts the mask bit stream Bbs from the image bit stream Abs.
  • the same effect as in the first embodiment can be obtained.
  • the mask bit stream Bbs is embedded in an area of the image bit stream Abs that does not affect the first basic image data A50. For this reason, the same effect as Embodiment 1 can be acquired, ensuring the backward compatibility with respect to the existing encoding stream.
  • FIG. 45 shows a configuration example of the supply system 11C and the synthesis system 21C according to the third embodiment.
  • the supply system 11C and the composition system 21C can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • the supply system 11C and the synthesis system 21C are basically the same as the supply system 11 and the synthesis system 21 (see FIG. 3) according to the first embodiment, except for the following points.
  • a mask data output unit 200 ⁇ / b> C is provided instead of the mask data output unit 200
  • the mask data output unit 200 ⁇ / b> C includes a mask encoder 250 ⁇ / b> C instead of the mask encoder 250.
  • a mask decoder 400 ⁇ / b> C is provided instead of the mask decoder 400.
  • the synthesis control data C50 input by the user is output without being embedded in the mask bitstream Bbs. Therefore, the mask encoder 250C does not embed the synthesis control data C50, and the mask decoder 400C does not extract the synthesis control data C50.
  • FIG. 46 shows a flowchart for explaining the operation of the supply system 11C.
  • step S124 is deleted from the operation flow S100 (see FIG. 9) according to the first embodiment.
  • step S105 the image bit stream Abs, the mask bit stream Bbs, and the synthesis control data C50 are output.
  • FIG. 47 shows a flowchart for explaining the operation of the synthesis system 21C.
  • step S432 is deleted from the operation flow S400 (see FIG. 25) according to the first embodiment.
  • step S411 the image bit stream Abs, the mask bit stream Bbs, and the synthesis control data C50 are acquired.
  • the first basic image data A50, the basic mask data B50, and the composition control data C50 are supplied to the composition system 21 by the same medium 50 (for example, a communication medium or an external storage medium).
  • a different medium 50 may be used.
  • a communication medium may be used for supplying the first basic image data A50 and the basic mask data B50
  • an external storage medium may be used for supplying the synthesis control data C50.
  • FIG. 48 shows a configuration example of the supply system 11D and the synthesis system 21D according to the fourth embodiment.
  • Supply system 11D and composition system 21D can be applied to image processing systems 1, 10, 20 and the like instead of supply system 11 and composition system 21 according to the first embodiment.
  • the supply system 11D and the synthesis system 21D are basically the same as the supply system 11C and the synthesis system 21C (see FIG. 45) according to the third embodiment, except for the following points.
  • the synthesis control data C50 is supplied to the synthesis system 21D when the user inputs it to the operation unit 42 (see FIG. 2) provided on the synthesis system 21D side.
  • the first basic image data A50 and the basic mask data B50 are supplied to the synthesis system 21D by the medium 50 (for example, a communication medium or an external storage medium), as in the third embodiment. That is, the composition system 21D acquires the composition control data C50 using a medium different from the medium that supplies the first basic image data A50 and the basic mask data B50.
  • the same effect as in the first embodiment can be obtained.
  • the synthesis control data C50 can be input on the synthesis system 21D side, the synthesis condition can be adjusted on the synthesis system 21D side.
  • FIG. 49 shows a configuration example of the supply system 11E and the synthesis system 21E according to the fifth embodiment.
  • the supply system 11E and the composition system 21E can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • the supply system 11E is the same as the supply system 11 according to the first embodiment.
  • a selector 600 is added in the synthesis system 21E.
  • the other configuration of the synthesis system 21E is the same as that of the synthesis system 21 according to the first embodiment.
  • the selector 600 selectively supplies one of a plurality of composition control data to the composition unit 500.
  • two synthesis control data C41 and C42 are input to the selector 600, and one of the synthesis control data is output as the synthesis control data C50 and supplied to the synthesis unit 500.
  • the second synthesis control data C42 is supplied to the selector 600 by the medium 50 from the supply system 11E. More specifically, the second synthesis control data C42 is embedded in the mask bit stream Bbs in the supply system 11E, and the mask bit stream Bbs is supplied to the mask decoder 400 by the medium 50. The second synthesis control data C42 is extracted by the mask decoder 400 and input to the selector 600. That is, the second synthesis control data C42 is supplied to the synthesis unit 500 by a medium different from the first synthesis control data C41.
  • the selector 600 is set to preferentially select and output the first synthesis control data C41. According to this, as in the fourth embodiment, the composition level can be adjusted on the synthesis system 21E side. On the other hand, when the first synthesis control data C41 is not input, the second synthesis control data C42 is output, so that the labor on the synthesis system 21E side can be saved.
  • the selection priority of a plurality of composition control data may be fixed or may be changeable. For example, it is assumed that the selection priority is changed by one or both of the operation unit 42 provided on the synthesis system 21E side and the operation unit 32 provided on the supply system 11E side.
  • Embodiment 5 can be combined with Embodiment 2 or the like.
  • FIG. 52 is a conceptual diagram of image composition for the supply system 11F and the composition system 21F according to the sixth embodiment.
  • the mask data output unit 200F constitutes the supply system 11F in combination with the image data output unit 100 described above.
  • the combining unit 500F is combined with the image decoder 300 and the mask decoder 400 described above to configure the combining system 21F.
  • the supply system 11F and the composition system 21F can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • the synthesis execution unit 520F of the synthesis unit 500F does not have the mask development unit 521.
  • a mask development unit 521 is provided in the mask generation unit 210F of the mask data output unit 200F. That is, in the sixth embodiment, the mask development unit 521 is provided in the supply system 11F.
  • the mask generation unit 210F is provided with an original mask generation unit 220.
  • the original mask generation unit 220 generates an original mask for the range of the basic image.
  • the original mask generation unit 220 includes the aforementioned entire mask generation unit 211, and the entire mask data B10 generated by the entire mask generation unit 211 is output as the original mask data B20.
  • the mask development unit 521 generates development mask data B21 by performing the above-described mask development processing on the original mask data B20. Note that the mask development unit 521 acquires the synthesis control data C50 during the mask development process.
  • the original mask and the unfolded mask are generated for the entire range of the entire image, and the unfolded mask data B21 is output from the mask generating unit 210F as the basic mask data B50.
  • the mask encoder 250 encodes the expanded mask data B21 as the basic mask data B50 to generate a mask bit stream Bbs.
  • FIG. 53 shows a flowchart for explaining the operation of the supply system 11F.
  • a basic mask generation step S122F is provided instead of the basic mask generation step S122 (see FIG. 9).
  • the other steps of the operation flow S100F are the same as the operation flow S100 of FIG.
  • the original mask generation unit 220 generates the original mask data B20 (original mask generation processing) in step S131, and the mask expansion unit 521 generates the expansion mask data B21 from the original mask data B20 in step S132. Generate (mask expansion processing).
  • the mask development step S132 is the same as the mask development step S451 (see FIG. 25) performed on the synthesis system 21 side in the first embodiment.
  • the expanded mask data B21 is then processed as basic mask data B50. That is, in the next step S123, the expanded mask data B21 is encoded to generate a mask bit stream Bbs.
  • FIG. 54 shows a flowchart for explaining the operation of the synthesis system 21F.
  • a synthesis execution step S450F is provided instead of the synthesis execution step S450 (see FIG. 25).
  • the mask development step S451 is deleted from the synthesis execution step S450.
  • the other steps of the operation flow S400F are the same as those of the operation flow S400 of FIG.
  • a development mask is generated in the supply system 11F, and the development mask is supplied to the synthesis system 21F as a basic mask.
  • Embodiment 6 can be combined with Embodiment 2 or the like.
  • the same effect as in the first embodiment can be obtained.
  • Embodiment 7 In the first embodiment, the example in which the entire mask for the entire range of the entire image is supplied to the synthesis system 21 has been described. The same applies to the second to sixth embodiments. Here, considering that the mask only needs to be able to identify the ROI, an example in which only a part of the entire mask including the ROI-corresponding portion is supplied to the synthesis system 21 can be considered. Embodiment 7 describes such an example.
  • the supply system 11G cuts out a part of the entire mask including the ROI-corresponding part as a partial mask.
  • the partial mask is output from the supply system 11G as a basic mask.
  • the synthesis system 21G the acquired basic mask is treated as a target mask, and a development mask is generated based on the target mask. That is, the supply system 11G cuts out the target mask that is a mask for the range of the first target image that is the target of wavelet transformation from the entire mask. Then, the supply system 11G outputs the target mask data as basic mask data.
  • composition system 21G acquires the entire range of the entire image as the first basic image. Then, a range corresponding to the target mask in the first basic image is cut out as the first target image, and wavelet transformation is performed on the first target image.
  • the second target image is cut out from the second basic image, and wavelet transformation is performed on the second target image.
  • the second target image is cut out with the same shape and size as the first target image, but the position of the second target image in the second basic image can be arbitrarily set. That is, the position of the second target image in the second basic image does not have to be the same as the position of the first target image in the entire image.
  • the first wavelet coefficient data and the second wavelet coefficient data generated from the first target image and the second target image are combined in the same manner as in the first embodiment, and inverse wavelet transform is performed on the combined coefficient data. Done. Thereby, the ROI in the first target image and the second target image are combined. Then, the composite image is embedded at the position of the second target image in the second basic image. As a result, a composite image having the same size as that of the second basic image is obtained as in the first embodiment.
  • the size of the basic mask data can be reduced, and therefore the size of the mask bitstream can be reduced. For this reason, when the mask bit stream is transmitted by wired or wireless communication, the reduction of the basic mask data can reduce the amount of communication, which is useful for immediate transmission. Further, as the basic mask data is reduced, the data sizes of the first target image and the second target image are also reduced, so that the calculation load on the synthesis system 21G can be reduced. This is useful for speeding up image synthesis. In view of these, the immediacy of image synthesis is improved.
  • the data size of the basic mask output from the supply system 11G can be reduced. For this reason, basically, an arbitrary range including the ROI-corresponding portion in the entire mask can be set as the basic mask.
  • the basic mask is preferably as small as possible.
  • the range cut out from the entire mask as the basic mask is also the range where the first target image is cut out from the first basic image (equal to the whole image in the seventh embodiment).
  • the first target image and the second target image are synthesized after wavelet transform, cropping is performed so that the ROI range is not distorted also in the wavelet coefficient data (in other words, on the wavelet plane). The range needs to be determined.
  • FIG. 56 shows a rectangular minimum range 81 including the ROI 60a by taking the entire image 60 of FIG. 6 as an example. Since the minimum range 81 is set to a rectangle, the position and range of the minimum range 81 can be specified by the upper left coordinates (AX1, AY1) and the lower right coordinates (BX1, BY1) of the minimum range 81.
  • the upper left end of the entire image 60 is taken as the origin O (0, 0) of the coordinate system, and two axes orthogonal to the horizontal direction and the vertical direction of the whole image 60 are taken.
  • FIG. 57 shows a range 83 to be cut out as the first target image.
  • the cut-out range 83 includes the minimum range 81 and is larger than the minimum range 81.
  • the upper left corner coordinates are (AX3, AY3), and the lower right corner coordinates are (BX3, BY3).
  • FIG. 58 and 59 are diagrams showing the entire mask 70 with the minimum range 81 and the cut-out range 83.
  • FIG. 58 and 59 are diagrams showing the entire mask 70 with the minimum range 81 and the cut-out range 83.
  • FIG. 60 shows a flowchart of the process for determining the cut-out range 83.
  • the minimum range specifying process is performed in step S501
  • the tracking process is performed in step S502
  • the necessary range specifying process is performed in step S503.
  • a rectangular minimum range 81 including the ROI 60a is specified in the entire image 60 that is the original image before cutting.
  • the minimum range 81 is specified based on the original mask for the entire image 60, that is, the entire mask 70.
  • each row of the whole mask 70 is selected in order from the top, and it is determined whether or not the selected row has pixels belonging to the ROI corresponding portion 70a.
  • the position of the row that is first determined to have pixels belonging to the ROI corresponding portion 70a corresponds to AY1.
  • BY1 can be obtained by selecting each row of the entire mask 70 in order from the bottom.
  • AX1 can be obtained by selecting each column of the overall mask 70 in order from the left, and BX1 can be obtained by selecting each column of the overall mask 70 in order from the right.
  • ⁇ Tracking process> In order to prevent the ROI range from being distorted on the wavelet plane, it is necessary to be able to perform wavelet transformation on the entire area of the minimum range 81. In the wavelet transform, not only the data of the pixel of interest but also the data of the pixels on both sides thereof are used. For this reason, when performing wavelet transform on pixels near the outer edge of the minimum range 81, data of pixels outside the minimum range 81 is required. For this reason, the cutout range 83 is larger than the minimum range 81.
  • Patent Document 8 can be referred to for obtaining a pixel range necessary outside the minimum range 81.
  • the range of pixels required outside the minimum range 81 depends on the number of taps of the wavelet transform division filter.
  • the range of pixels required outside the minimum range 81 also depends on the decomposition level of wavelet transform, that is, the specified value of the decomposition level by the combination control data. This is because, for the highest band component (that is, the most decomposed band component) in the wavelet plane, the processing by the division filter is repeated for the number of decomposition levels.
  • step S502 of the tracking process in the most significant band component of the final wavelet plane (that is, the wavelet plane of the designated decomposition level specified by the synthesis control data C50), the minimum range 81 is set. Identify the corresponding range as the minimum tracking range.
  • FIG. 65 shows a diagram for explaining the minimum tracking range.
  • the uppermost band components LL3, HL3, LH3, and HH3 in the wavelet plane 63 at the decomposition level 3 are shown enlarged.
  • FIG. 65 shows a case where the minimum tracking range 82 corresponding to the minimum range 81 is specified in the lowest band component LL3 among the highest band components LL3, HL3, LH3, and HH3.
  • the origin O (0, 0) of the wavelet plane corresponds to the origin O of the original image (that is, the entire image 60).
  • the upper left corner coordinates are (AX2, AY2)
  • the lower right corner coordinates are (BX2, BY2).
  • 66 and 67 show flowcharts for obtaining the minimum tracking range 82 when a 5 ⁇ 3 filter is used for wavelet transformation.
  • 66 shows how to obtain the upper left corner coordinates (AX2, AY2)
  • FIG. 67 shows how to obtain the lower right corner coordinates (BX2, BY2).
  • the upper left coordinates (AX1, AY1) of the minimum range 81 are set in the parameter q in step S511.
  • q AX1 is set first.
  • step S515 if it is determined in step S515 that the current decomposition level has reached the specified decomposition level, the value of p at that time is set to AX2 of the minimum tracking range 82 in step S518.
  • the tracking process when a 5 ⁇ 3 filter is used for wavelet conversion can be expressed as follows.
  • AX1 is an even number
  • AX1 / 2 is set to a new AX1
  • ⁇ AX1-1 ⁇ / 2 is set to a new AX1 (referred to as a first recursive process).
  • the number of times designated by the synthesis control data C50 is performed, and the finally obtained AX1 is set to AX2.
  • AY1 is an even number
  • AY1 / 2 is set to a new AY1
  • ⁇ AY1-1 ⁇ / 2 is set to a new AY1 (hereinafter referred to as a second recursion process). The specified number of times is performed, and finally obtained AY1 is set to AY2.
  • BX1 is an even number
  • BX1 / 2 is set to a new BX1
  • a process of setting ⁇ BX1 + 1 ⁇ / 2 to a new BX1 (referred to as a third recursion process) is performed for the specified number of times.
  • BX1 finally obtained is set to BX2.
  • BY1 is an even number
  • BY1 / 2 is set to a new BY1.
  • ⁇ BY1 + 1 ⁇ / 2 is set to a new BY1 (referred to as a fourth recursion process).
  • the BY1 finally obtained is set to BY2.
  • 68 and 69 are flowcharts for obtaining the minimum tracking range 82 when the Daubechies 9 ⁇ 7 filter is used for wavelet transformation.
  • 68 shows how to obtain the upper left corner coordinates (AX2, AY2)
  • FIG. 69 shows how to obtain the lower right corner coordinates (BX2, BY2).
  • AX2 of the minimum tracking range 82 is obtained from AX1 of the minimum range 81
  • AY2 of the minimum tracking range 82 is obtained from AY1 of the minimum range 81.
  • the tracking process when the Daubechies 9 ⁇ 7 filter is used for wavelet conversion can also be expressed as follows.
  • AX1 is an even number
  • ⁇ AX1 / 2-1 ⁇ is set to a new AX1
  • ⁇ AX1-3 ⁇ / 2 is set to a new AX1 (referred to as a ninth recursion process). Is performed the designated number of times designated by the synthesis control data C50, and finally obtained AX1 is set to AX2.
  • AY1 is an even number
  • ⁇ AY1 / 2-1 ⁇ is set to a new AY1
  • ⁇ AY1-3 ⁇ / 2 is set to a new AY1 (referred to as a tenth recursive process). ) Is performed the number of times specified above, and finally obtained AY1 is set to AY2.
  • BX1 is an even number
  • ⁇ BX1 + 2 ⁇ / 2 is set to a new BX1
  • ⁇ BX1 + 3 ⁇ / 2 is set to a new BX1 (referred to as the eleventh recursive process). The designated number of times is performed, and finally obtained BX1 is set to BX2.
  • BY1 is an even number
  • ⁇ BY1 + 2 ⁇ / 2 is set to a new BY1
  • ⁇ BY1 + 3 ⁇ / 2 is set to a new BY1 (referred to as the 12th recursion process). This is performed the specified number of times, and finally obtained BY1 is set to BY2.
  • steps S513, S514, S523, S524, S533, S534, S543, and S544 are defined according to the number of taps of the filter. Also, in consideration of ease of calculation, carry to even number units is performed.
  • step S503 of the necessary range specifying process it is specified in which range of the entire image 60 before cutting the data necessary for calculating the wavelet coefficient within the minimum tracking range 82 is specified.
  • the specified necessary range becomes the cutout range 83.
  • FIG. 70 shows a flowchart for obtaining the cut-out range 83 from the minimum tracking range 82 when a 5 ⁇ 3 filter is used for wavelet conversion.
  • AX2, AY2, BX2, BY2 of the minimum tracking range 82 is set to the parameter r in step S551.
  • r AX2 is set first.
  • step S555 determines whether the current decomposition level has reached 0 or not. If it is determined in step S555 that the current decomposition level has reached 0, the value of s at that time is set to AX3 of the cut-out range 83 in step S558.
  • the necessary range specifying process when a 5 ⁇ 3 filter is used for wavelet conversion can be expressed as follows.
  • the process of setting ⁇ AX2 ⁇ 2-2 ⁇ to a new AX2 (referred to as a fifth recursion process) is performed the number of times specified by the synthesis control data C50, and the finally obtained AX2 is set to AX3. Set to.
  • the process of setting ⁇ BX2 ⁇ 2 + 2 ⁇ to a new BX2 (referred to as the seventh recursion process) is performed for the specified number of times, and the finally obtained BX2 is set to BX3.
  • the process of setting ⁇ BY2 ⁇ 2 + 2 ⁇ to a new BY2 (referred to as the 8th recursion process) is performed the specified number of times, and finally obtained BY2 is set to BY3.
  • FIG. 71 shows a flowchart for obtaining the cut-out range 83 from the minimum tracking range 82 when a Daubechies 9 ⁇ 7 filter is used for wavelet conversion.
  • the necessary range specifying process when the Daubechies 9 ⁇ 7 filter is used for wavelet conversion can be expressed as follows.
  • the process of setting ⁇ AX2 ⁇ 2-4 ⁇ to a new AX2 (referred to as the thirteenth recursion process) is performed the number of times specified by the synthesis control data C50, and the finally obtained AX2 is set to AX3. Set to.
  • the process of setting ⁇ BX2 ⁇ 2 + 4 ⁇ to a new BX2 (referred to as the fifteenth recursion process) is performed for the specified number of times, and the finally obtained BX2 is set to BX3.
  • the process of setting ⁇ BY2 ⁇ 2 + 4 ⁇ to a new BY2 (referred to as the 16th recursion process) is performed the specified number of times, and finally obtained BY2 is set to BY3.
  • FIG. 72 shows a configuration example of a mask data output unit 200G according to the seventh embodiment.
  • the mask data output unit 200G constitutes the supply system 11G in combination with the image data output unit 100 described above.
  • the supply system 11G can be applied to the image processing systems 1 and 10 in place of the supply system 11 according to the first embodiment.
  • the mask data output unit 200G includes a mask generation unit 210G and the mask encoder 250 described above.
  • the mask generation unit 210G includes the above-described entire mask generation unit 211, a cut range determination unit 232, and a mask cut unit 233.
  • the cut range determination unit 232 determines the cut range 83 (the range of the first target image in the whole image and the range of the target mask in the whole mask). Specifically, the cut range determination unit 232 performs the cut range determination process S500 (see FIG. 60) based on the entire mask data B10 and the synthesis control data C50. Then, the cut range determination unit 232 generates cut range specifying data F50, which is data for specifying the cut range 83.
  • the cut area specifying data F50 is specifically data of the upper left corner coordinates (AX3, AY3) and the lower right corner coordinates (BX3, BY3) of the cut area 83.
  • the mask cutout unit 233 cuts the data in the cutout area 83 out of the entire mask data B10 for the basic mask data based on the cutout area specifying data F50.
  • the data within the cutout range 83 among the whole mask data B10 is cut out as the basic mask data B50.
  • the cut data constitutes target mask data B60.
  • the mask encoder 250 acquires the target mask data B60 as the basic mask data B50, encodes the target mask data B60, and generates a mask bit stream Bbs. Further, the mask encoder 250 embeds the synthesis control data C50 and the cut range specifying data F50 in the mask bit stream Bbs.
  • FIG. 73 shows a configuration example of the synthesis system 21G according to the seventh embodiment.
  • the synthesizing system 21G can be applied to the image processing systems 1 and 20 in place of the synthesizing system 21 according to the first embodiment.
  • a mask decoder 400G and a synthesis unit 500G are provided instead of the mask decoder 400 and the synthesis unit 500 (see FIG. 3).
  • Other configurations of the synthesis system 21G are the same as those of the synthesis system 21 according to the first embodiment.
  • the mask decoder 400G extracts the synthesis control data C50 and the cut range specifying data F50 from the mask bit stream Bbs. Further, as described above, the mask decoder 400G decodes the mask bit stream Bbs to obtain the basic mask data B50. The basic mask data B50, the composition control data C50, and the cut-out range specifying data F50 are supplied to the composition unit 500G.
  • FIG. 74 shows a configuration example of the synthesis unit 500G.
  • the synthesizing unit 500G includes a first image cutting unit 540, a second image cutting unit 550, and an embedding unit 560 in the synthesizing unit 500 according to the first embodiment. Have been added.
  • the other configuration of the combining unit 500G is basically the same as that of the combining unit 500 according to the first embodiment.
  • the first image cutout unit 540 acquires the first basic image data A50 and the cutout range specifying data F50. Then, the first image cutout unit 540 cuts out the data in the cutout range 83 from the first basic image data A50 (here, the entire image data A10) based on the cutout range specifying data F50 (see FIG. 55). ). The cut data is supplied to the wavelet conversion unit 510 as the first target image data A60.
  • the second image cutout unit 550 acquires the second basic image data D50, the cutout range specifying data F50, and the composite position designation data G50.
  • the synthesis position designation data G50 is data for designating a position where the ROI in the first target image is synthesized in the second basic image.
  • the composite position designation data G50 can be understood as, for example, embedded position data of the composite image data E80 as described later. More specifically, the embedding position of the composite image data E80 can be specified by the position of the upper left end of the embedding range of the composite image data E80, the center position of the embedding range, or the like.
  • the composition position designation data G50 is supplied to the composition unit 500G when the user of the composition system 21G inputs the operation unit 42 (see FIG. 2) provided on the composition system 21G side.
  • the second image cutout unit 550 sets a synthesis destination range in the second basic image based on the synthesis position designation data G50.
  • the synthesis destination range is a range having the same shape and size as the first target image, and is therefore congruent with the first target image.
  • the composition destination range may be expressed as a similar shape range having a similarity ratio of 1 to the first target image.
  • the synthesis destination range is specified by the cut range specifying data F50.
  • the same shape and size of the first target image may be specified directly from the first target image data A60 instead of the cut-out range specifying data F50.
  • the second image cutout unit 550 cuts out data within the composition destination range from the second basic image data D50. The cut data is supplied to the wavelet conversion unit 510 as the second target image data D60.
  • the wavelet conversion unit 510, the composition execution unit 520, and the inverse wavelet conversion unit 530 operate in the same manner as in the first embodiment. Thereby, the composite image data E80 is generated from the first target image data A60 and the second target image data D60 cut out as described above.
  • the embedding unit 560 embeds the synthesized image data E80 generated by the inverse wavelet transform unit 530 in the synthesis destination range in the second basic image data D50.
  • the second basic image data D50 in which the composite image data E80 is inserted is output as composite image data E100 that is output data of the composite unit 500G.
  • FIG. 75 shows a flowchart for explaining the operation of the supply system 11G.
  • the operation flow S100G in FIG. 75 is basically the same as the operation flow S100 (see FIG. 9) according to the first embodiment, but differs in the following points.
  • a basic mask generation step S122G is provided instead of the basic mask generation step S122.
  • a data embedding step S124G is provided instead of the data embedding step S124.
  • the overall mask generation unit 221 generates the overall mask data B10 in the overall mask generation step S141.
  • the cutting step S142 the cutting range determining unit 232 and the mask cutting unit 233 generate target mask data B60 from the entire mask data B10.
  • the target mask data B60 is then treated as basic mask data B50. That is, in the next step S123, the target mask data B60 is encoded, and a mask bit stream Bbs is generated.
  • the mask encoder 250 embeds the synthesis control data C50 and the cut range specifying data F50 in the mask bitstream Bbs.
  • FIG. 76 shows a flowchart for explaining the operation of the synthesis system 21G.
  • the operation flow S400G of FIG. 76 is basically the same as the operation flow S400 (see FIG. 25) according to the first embodiment, except for the following points.
  • a data acquisition step S410G is provided instead of the data acquisition step S410 (see FIG. 25).
  • an image embedding step S480 is added.
  • a data extraction step S432G is provided instead of step S432, and a cutting step S470 is added.
  • the mask decoder 400G extracts the synthesis control data C50 and the cut range specifying data F50 from the mask bit stream Bbs.
  • the cutting step S470 the first image cutting unit 540 cuts the first target image data A60 from the first basic image data A50, and the second image cutting unit 550 cuts the second target image data D60 from the second basic image data D50. .
  • the embedding unit 560 embeds the composite image data E80 generated by the inverse wavelet transform unit 530 in the composite destination range in the second basic image data D50.
  • the supply system 11G cuts the basic mask from the entire mask and outputs the cut basic mask by the mask bitstream Bbs. For this reason, the size of the basic mask data B50 can be reduced, and therefore the size of the mask bitstream Bbs can be reduced. For this reason, when the mask bit stream is transmitted by wired or wireless communication, the amount of communication can be reduced, which helps the immediacy of transmission. Further, as the size of the basic mask is reduced, the sizes of the first target image and the second target image are also reduced, so that the calculation load on the synthesis system 21G can be reduced. This is useful for speeding up image synthesis. In view of these, the immediacy of image synthesis is improved.
  • Embodiment 7 can be combined with Embodiment 2 or the like.
  • the highest band component to be tracked Is defined as the lowest band component LL3 in the final wavelet plane.
  • the wavelet transform employs a method of recursively decomposing the highest band component of the wavelet plane
  • the highest band component in the final wavelet plane is the target of the tracking process.
  • the high-frequency component that is, the high-pass filter side
  • the output on the left side (equivalent to the upper side) of the high frequency component is p
  • q 2n + 1
  • p n ⁇ 1
  • Solving this results in p (q ⁇ 3) / 2 (Formula 6).
  • the high-frequency component that is, the high-pass filter side
  • the output on the left side (equivalent to the upper side) of the high frequency component is p
  • q 2n
  • p n ⁇ 1
  • n p.
  • r p (Formula 13)
  • s 2p-2 (Formula 14)
  • Solving this results in s 2r-2 (Equation 15). This is an equation according to step S553.
  • Equation 19 the left end coordinate AX1 or the upper end coordinate AY1 of the minimum range 81 is an odd number in step S512 of FIG. 66
  • Equations 20 and 22 the same result is obtained for both the low-frequency component and the high-frequency component.
  • the high-frequency component that is, the high-pass filter side
  • the output on the left side (equivalent to the upper side) of the high frequency component is p
  • q 2n + 1
  • p n ⁇ 2
  • Solving this results in p (q-5) / 2 (Equation 32).
  • the high-frequency component that is, the high-pass filter side
  • the output on the left side (equivalent to the upper side) of the high frequency component is p
  • q 2n
  • Solving this results in p q / 2-2 (Equation 38).
  • n p.
  • r p (Formula 39)
  • s 2p-4 (Formula 40)
  • Solving this results in s 2r-4 (Equation 41). This is an equation related to step S563.
  • n p.
  • FIG. 77 is a conceptual diagram of image composition for supply system 11H and composition system 21H according to the eighth embodiment.
  • the supply system 11H and the composition system 21H can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • the supply system 11H cuts not only the entire mask but also the entire image.
  • the supply system 11H cuts out the target mask that is a mask for the range of the first target image that is the target of wavelet transformation from the entire mask, as in the seventh embodiment, and uses the target mask data B60 as the basic mask data. Output as B50. Further, the supply system 11H cuts the first target image from the entire image and outputs the first target image data A60 as the first basic image data A50.
  • FIG. 78 shows a configuration example of the supply system 11H.
  • Supply system 11H includes an image data output unit 100H according to the eighth embodiment and a mask data output unit 200G according to the seventh embodiment.
  • the image data output unit 100H includes the first image cutout unit 540 provided in the synthesis system 21G in the seventh embodiment and the image encoder 150 described above.
  • the first image cutout unit 540 acquires the entire image data A10 in the supply system 11H, and acquires the cutout range specifying data F50 from the cutout range determination unit 232. Then, the first image cutout unit 540 cuts out the data in the cutout area 83 from the entire image data A10 based on the cutout area specifying data F50.
  • the cut partial image data is first target image data A60 that is a target of wavelet conversion in the synthesis system 21H.
  • the first target image data A60 is supplied to the image encoder 150 as first basic image data A50. Accordingly, the first target image data A60 is output from the supply system 11H as the first basic image data A50 by the image bitstream Abs.
  • the composition unit 500H (see FIG. 79) of the composition system 21H has a configuration in which the first image cutout unit 540 is deleted from the composition unit 500G (see FIG. 74) according to the seventh embodiment.
  • FIG. 80 shows a flowchart for explaining the operation of the supply system 11H.
  • a cutting step S151 is added to the operation flow S100G in FIG.
  • the first image cutting unit 540 cuts the first target image data A60 from the entire image data A10 based on the cutting range specifying data F50 generated in the cutting step S142.
  • the first target image data A60 cut out in step S151 is processed as first basic image data A50 in step S112. That is, the image bit stream Abs of the first basic image data A50 is generated from the first target image data A60 by the image encoder 150.
  • FIG. 81 shows a flowchart for explaining the operation of the synthesis system 21H.
  • the operation flow S400H of FIG. 81 is the same as the operation flow S400G of FIG. 76 except that step S470H is provided instead of step S470.
  • step S470 (see FIG. 76) according to Embodiment 7, the first target image and the second target image are cut out from the first basic image and the second basic image.
  • Embodiment 8 since the first basic image is supplied as the first target image, only the second target image is cut out in step S470H.
  • the cut area specifying data F50 is supplied from the supply system 11H to the combining system 21H, and the cut area specifying data F50 is used by the second image cutout unit 550.
  • the eighth embodiment it is possible not to supply the cut range specifying data F50 from the supply system 11H to the synthesis system 21H.
  • the cutting range specifying data F50 is not used for the first target image.
  • the second target image is referred to as the first target image. This is because it can be cut out with the same shape and size.
  • the supply system 11H cuts the first basic image from the whole image and outputs the cut first basic image by the image bitstream Abs. For this reason, not only the basic mask data B50 but also the first basic image data A50 can reduce the data size. For this reason, when the image bit stream Abs and the mask bit stream Bbs are transmitted by wired or wireless communication, the communication amount can be reduced, which is useful for immediate transmission.
  • FIG. 82 shows a conceptual diagram of image composition for supply system 11I and composition system 21I according to the ninth embodiment.
  • the supply system 11I and the composition system 21I can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • the synthesis system 21I cuts out the entire mask and the entire image.
  • the supply system 11I outputs, as the first basic image data A50, the entire image data A10, which is the entire image data including the first target image as a part, in the same manner as in the first embodiment.
  • Mask data B50 is generated and output. Further, the supply system 11I generates and outputs the cut-out range specifying data F50 as in the seventh embodiment.
  • composition system 21I cuts out the range specified by the cut range specifying data F50 from the first basic image and the basic mask, and uses the cut portion as the first target image and the target mask.
  • synthesis system 21I cuts out the second target image from the second basic image.
  • FIG. 83 shows a configuration example of the mask data output unit 200I of the supply system 11I.
  • the mask data output unit 200I is combined with the image data output unit 100 (see FIG. 3) according to the first embodiment to constitute the supply system 11I.
  • the mask data output unit 200I includes a mask generation unit 210I according to the ninth embodiment and the mask encoder 250 described above.
  • the mask generation unit 210I has a configuration in which the mask cutout unit 233 is deleted from the mask generation unit 210G (see FIG. 72) according to the seventh embodiment. Therefore, the mask generation unit 210I supplies the entire mask data B10 as the basic mask data B50 to the mask encoder 250.
  • the cut range determination unit 232 performs cut range determination processing in the same manner as in the seventh embodiment, and supplies the cut range specifying data F50 to the mask encoder 250.
  • the synthesis unit 500I (see FIG. 84) of the synthesis system 21I has a configuration in which a mask cutout unit 233 is added to the synthesis unit 500G (see FIG. 74) according to the seventh embodiment.
  • the mask cutout unit 233 acquires the entire mask data B10 as the basic mask data B50, and acquires cutout range specifying data F50. Then, the mask cutout unit 233 cuts the target mask data B60 from the entire mask data B10 based on the cutout range specifying data F50, as in the seventh embodiment.
  • the target mask data B60 is supplied to the synthesis execution unit 520.
  • FIG. 85 shows a flowchart for explaining the operation of the supply system 11I.
  • a cut range determination step S143 is provided instead of the cut step S142 in the operation flow S100G in FIG.
  • the cut range determining unit 232 generates cut range specifying data F50.
  • FIG. 86 shows a flowchart for explaining the operation of the synthesis system 21I.
  • the data acquisition step S410I of the operation flow S400I of FIG. 86 is provided with a cutting step S470I instead of the cutting step S470 in the data acquisition step S410G of the operation flow S400G (see FIG. 76) according to the seventh embodiment.
  • the first image cutting unit 540 cuts the first target image data A60 from the first basic image data A50
  • the second image cutting unit 550 cuts the second target image data D60 from the second basic image data D50
  • the mask cutting unit 233 cuts the target mask data B60 from the basic mask data B50.
  • the size of the first target image and the second target image is reduced, so that it is possible to reduce the calculation load in the synthesis system 21G. This is useful for speeding up image synthesis.
  • the target mask and the first target image are not cut out in the supply system 11I. For this reason, the calculation load in the supply system 11I can be reduced.
  • FIG. 87 shows a conceptual diagram of image composition for supply system 11J and composition system 21J according to the tenth embodiment.
  • the supply system 11J and the composition system 21J can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • FIG. 87 shows a conceptual diagram of image composition for supply system 11J and composition system 21J according to the tenth embodiment.
  • the supply system 11J and the composition system 21J can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • FIG. 87 shows a conceptual diagram of image composition for supply system 11J and composition system 21J according to the tenth embodiment.
  • the supply system 11J and the composition system 21J can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • FIG. 87 shows a conceptual diagram of image composition for supply system 11J and composition system 21J according to the
  • the supply system 11J cuts out the first target image from the entire image and outputs the first target image data A60 as the first basic image data A50, as in the eighth embodiment (see FIG. 77). Further, the supply system 11J generates and outputs basic mask data B50 for the entire image range, as in the ninth embodiment (see FIG. 82).
  • the synthesis system 21J uses the entire range of the acquired first basic image as the first target image, as in the eighth embodiment (see FIG. 77). Similarly to the ninth embodiment (see FIG. 82), the composition system 21J cuts out the range specified by the cut range specifying data F50 from the acquired basic mask, and uses the cut portion as the target mask.
  • FIG. 88 shows a configuration example of the supply system 11J.
  • the supply system 11J combines an image data output unit 100H (see FIG. 78) according to the eighth embodiment and a mask data output unit 200I (see FIG. 83) according to the ninth embodiment. It is configured.
  • FIG. 89 shows a configuration example of the synthesis unit 500J of the synthesis system 21J.
  • combining section 500J is combined with image decoder 300 and mask decoder 400G (see FIG. 73) according to Embodiment 7 to form combining system 21J.
  • the combining unit 500J has a configuration in which the first image cutout unit 540 is deleted from the combining unit 500I (see FIG. 84) according to the ninth embodiment.
  • FIG. 90 shows a flowchart for explaining the operation of the supply system 11J.
  • the cutting step S151 (see FIG. 80) according to the eighth embodiment is added to the operation flow S100I according to the ninth embodiment (see FIG. 85). That is, in the cutting step S151, the first image cutting unit 540 cuts the first target image as the first basic image from the entire image based on the cutting range specifying data F50 generated in step S143.
  • FIG. 91 shows a flowchart for explaining the operation of the synthesis system 21J.
  • the data acquisition step S410J of the operation flow S400J of FIG. 91 is provided with a cutting step S470J instead of the cutting step S470I in the data acquisition step S410I of the operation flow S400I (see FIG. 86) according to the ninth embodiment.
  • the second image cutting unit 550 cuts the second target image data D60 from the second basic image data D50, and the mask cutting unit 233 cuts the target mask data B60 from the basic mask data B50.
  • the size of the first target image and the second target image is reduced, so that it is possible to reduce the calculation load in the synthesis system 21J. This is useful for speeding up image synthesis.
  • the data size of the first basic image data A50 can be reduced. For this reason, when the image bit stream Abs is transmitted by wired or wireless communication, the amount of communication can be reduced, which is useful for immediate transmission.
  • FIG. 92 shows a conceptual diagram of image composition according to the eleventh embodiment.
  • the synthesis system 21K according to the eleventh embodiment is combined with the supply system 11 according to the first embodiment.
  • the composition system 21K can be applied to the image processing systems 1, 20 and the like instead of the composition system 21 according to the first embodiment.
  • FIG. 93 shows a configuration example of the synthesis unit 500K of the synthesis system 21K.
  • combining section 500J is combined with image decoder 300 and mask decoder 400G (see FIG. 73) according to Embodiment 7 to form combining system 21K.
  • the combining unit 500K has a configuration in which the above-described cut range determining unit 232 is added to the combining unit 500I (see FIG. 84) according to the ninth embodiment.
  • the cutout range determination unit 232 performs cutout range determination processing based on the entire mask data B10 (supplied to the synthesis system 21K as basic mask data B50) and the synthesis control data C50, and specifies the cutout range. Data F50 is generated.
  • the cut range specifying data F50 is supplied to the first image cutout unit 540, the second image cutout unit 550, and the mask cutout unit 233.
  • FIG. 94 shows a flowchart for explaining the operation of the synthesis system 21J.
  • the operation flow S400K in FIG. 94 is basically the same as the operation flow S400I according to the ninth embodiment (see FIG. 86), except for the following points.
  • step S432 for extracting the synthesis control data C50 from the mask bitstream Bbs is provided instead of step S432G. This is because the cut range specifying data F50 is generated inside the synthesis system 21K, and therefore the cut range specifying data F50 is not embedded in the mask bitstream Bbs.
  • step S143 the cut range determination unit 232 performs cut range determination processing, and generates cut range specifying data F50.
  • the same effect as in the ninth embodiment can be obtained.
  • the cutout range determination process is performed in the synthesis system 21K. For this reason, the calculation load in the supply system 11 is not increased.
  • the eleventh embodiment can be combined with the second embodiment.
  • FIG. 95 shows a conceptual diagram of image composition for supply system 11L and composition system 21L according to the twelfth embodiment.
  • the supply system 11L and the composition system 21L can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • the mask is cut and developed in the supply system 11L.
  • FIG. 96 shows a configuration example of the mask data output unit 200L of the supply system 11L.
  • the mask data output unit 200L is combined with the above-described image data output unit 100 to constitute the supply system 11L.
  • the mask data output unit 200L includes a mask generation unit 210L and the mask encoder 250 described above.
  • the mask generation unit 210L includes an original mask generation unit 220L and a mask development unit 521, similarly to the mask generation unit 210F (see FIG. 50) according to the sixth embodiment.
  • the original mask generation unit 220L according to the twelfth embodiment has the same configuration as the mask generation unit 210G (see FIG. 72) according to the seventh embodiment. That is, the original mask generation unit 220L includes an overall mask generation unit 211, a cut range determination unit 232, and a mask cut unit 233. Thereby, the original mask generation unit 220L generates an original mask for the range of the first target image based on the entire image.
  • the mask development unit 521 generates a development mask for the range of the first target image by performing mask development processing on the original mask.
  • the generated expanded mask data B21 is output from the mask data output unit 200L as the basic mask data B50 by the mask bit stream Bbs.
  • the mask cutout unit 233 cuts out the data within the cutout range 83 from the entire mask data B10 for the basic mask data, as in the seventh embodiment.
  • the cut data is subjected to the mask development process as described above, and thus the cut data is not directly used as the basic mask data B50.
  • FIG. 97 shows a configuration example of the synthesis unit 500L of the synthesis system 21L.
  • combining section 500L is combined with image decoder 300 and mask decoder 400G (see FIG. 73) according to Embodiment 7 to form combining system 21L.
  • the combining unit 500L is basically the same as the combining unit 500G (see FIG. 74) according to Embodiment 7, but differs in the following points.
  • a synthesis execution unit 520F (see FIG. 51 according to Embodiment 6) that does not have the mask development unit 521 is provided.
  • FIG. 98 shows a flowchart for explaining the operation of the supply system 11L.
  • the operation flow S100L in FIG. 98 is basically the same as the operation flow S100G (see FIG. 75) according to the seventh embodiment, except for the following points.
  • a basic mask generation step S122L is provided instead of the basic mask generation step S122G.
  • the basic mask generation step S122L is roughly divided into an original mask generation step S131L and a mask development step S132, similarly to the basic mask step S122F according to the sixth embodiment (see FIG. 53).
  • the original mask generation step S131L according to the twelfth embodiment includes an overall mask generation step S141 and a mask cutout step S142 (see FIG. 75) according to the seventh embodiment.
  • the mask development step S132 is as described above.
  • FIG. 99 shows a flowchart for explaining the operation of the synthesis system 21L.
  • the operation flow S400L of FIG. 99 is basically the same as the operation flow S400G (see FIG. 76) according to the seventh embodiment, except for the following points.
  • a synthesis execution step S450L is provided.
  • the mask development step S451 is deleted from the synthesis execution step S450.
  • the same effects as in the sixth and seventh embodiments can be obtained. Note that although an example in which the seventh embodiment is combined with the sixth embodiment has been described in the twelfth embodiment, the sixth embodiment can be combined with other embodiments.
  • the second basic image is larger than the first target image and the second target image.
  • the second basic image itself may be the same size as the first target image and the second target image.
  • the second image cutout unit 550 may be deleted.
  • the lowest band component LL1 at the decomposition level 1 can provide an image with a reduction ratio of 1/2 with respect to the original image (in other words, an image with an image size of 1/4).
  • the lowest band component LLm at the decomposition level m can provide an image having a reduction ratio of 1/2 m with respect to the original image (see FIG. 100).
  • m is an integer equal to or greater than 0 by associating an original image in a state where wavelet transformation is not performed with the decomposition level 0 as described above.
  • the image size when the original image is reduced by 1/2 m is equal to the image size provided by the lowest band component LLm when the original image is decomposed to the decomposition level m.
  • the size of a 1/2 m reduced image may be expressed as an image size corresponding to the decomposition level m. Note that this representation of image size can also be used when a reduced image is compared with another image having the same image size as the original image.
  • FIG. 101 is a block diagram illustrating an inverse wavelet transform unit 530M according to the thirteenth embodiment.
  • the inverse wavelet conversion unit 530M is applicable to Embodiments 1 to 6 instead of the inverse wavelet conversion unit 530 (see FIG. 13 and the like).
  • the inverse wavelet transform unit 530M performs inverse wavelet transform on the synthesized coefficient data E61 until the decomposition level reaches a predetermined end level.
  • the lowest band component LL is set in the composite image data E80.
  • the end level of the inverse wavelet conversion in other words, the image size of the composite image is instructed to the inverse wavelet conversion unit 530M by the image size control data C60.
  • the image size control data C60 is input to the composition system by the user of the composition system.
  • the image size control data C60 may be supplied from the supply system in the same manner as the composition control data C50 (see FIGS. 3, 38, 45, and 49).
  • the image size control data C60 is, for example, a numerical value that directly indicates the end level of the inverse wavelet conversion.
  • the image size control data C60 may be data that can derive the end level of the inverse wavelet transform. Examples of data from which the end level of the inverse wavelet transform can be derived include a numerical value indicating the number of inverse wavelet conversions, a numerical value indicating a reduction ratio with respect to the original image size, and the like.
  • the end level of the inverse wavelet transform can be set within a range of the decomposition level 0 or more and the decomposition level of the synthesized coefficient data E61 (that is, the decomposition level specified by the synthesis control data C50).
  • the end level of inverse wavelet conversion is set to decomposition level 0
  • a composite image of the original image size can be obtained as in the first to sixth embodiments.
  • Embodiments 1 to 6 are examples in which the end level of inverse wavelet conversion is fixed to 0, and Embodiment 13 is an example in which the end level of inverse wavelet conversion is variable.
  • the image size of the composite image can be controlled, and a composite image not only having the same size as the original image but also smaller than the original image can be obtained.
  • FIG. 102 shows a conceptual diagram of image composition according to the fourteenth embodiment.
  • the first basic image is directly used as the first target image, and the first wavelet coefficient data is generated from the first target image.
  • the decomposition level of the first wavelet coefficient data is 3.
  • the entire second basic image is reduced, and the reduced image is used as the second target image.
  • the second basic image has the same size and shape as the first basic image (in other words, the second basic image is congruent with the first basic image), and the entire second basic image. Is reduced with a reduction ratio of 1/2. That is, the reduction ratio of the second target image to the second basic image and the first basic image is 1 ⁇ 2. In other words, the image size of the second target image is 1/4 with respect to the second basic image and the first basic image.
  • second wavelet coefficient data is generated from the second target image.
  • the first wavelet coefficient data and the second wavelet coefficient data are synthesized in the same manner as in the first embodiment.
  • synthesis is performed according to the range of the second target image having a small image size, in other words, according to the range of the second wavelet coefficient data.
  • a part of the first wavelet coefficient data (the part corresponding to the second wavelet coefficient data) and the whole of the second wavelet coefficient data are used for the coefficient synthesis process.
  • the part of the first wavelet coefficient data is the most decomposed band components (that is, the highest band components) LL3, HL3, LH3, HH3 in the first wavelet coefficient data.
  • the bands HL2, LH2, and HH2 are one level lower than the highest band component.
  • synthesized coefficient data having the same decomposition level as the second wavelet coefficient data (here, decomposition level 2) is generated.
  • composite image data is generated by performing inverse wavelet transform to the decomposition level 0 on the combined coefficient data.
  • a composite image having the same size and shape as the second target image in other words, a composite image having a reduction ratio of 1/2 with respect to the second basic image and the first basic image is provided.
  • the first wavelet coefficient data (in the example of FIG. 103, the highest band components LL3, HL3, LH3, and HH3 among the first wavelet coefficient data) and the second wavelet coefficient data are combined. .
  • synthesized coefficient data having the same decomposition level (here, decomposition level 1) as the second wavelet coefficient data is generated.
  • composite image data is generated by performing inverse wavelet transform to the decomposition level 0 on the combined coefficient data.
  • FIG. 104 shows an example in which the decomposition level of the first wavelet coefficient is 4, and the reduction ratio of the second target image to the second basic image is 1/2.
  • the data in the range from the highest band components LL4, HL4, LH4, HH4 to the bands HL2, LH2, HH2 that are two levels lower than the highest band component Used for synthesis.
  • the composite image is provided with the same image size as the second target image, in other words, with a reduction ratio of 1/2 with respect to the second basic image and the first basic image.
  • the image size of the composite image can be controlled by controlling the image size of the second target image.
  • the following knowledge is obtained from the examples of FIGS.
  • the second target image has a similar shape to the first target image, and the similarity ratio of the second target image to the first target image is less than 1.
  • a similarity ratio of less than 1 may be referred to as a reduction ratio.
  • P1, P2, and P3 are natural numbers.
  • the coefficient synthesis process is performed in accordance with the range of the second wavelet coefficient data. For this reason, as the first wavelet coefficient data, band component data within a predetermined number of levels counted from the most significant band component is used, and the predetermined number of levels represents the decomposition level of the second wavelet coefficient data. Given as a number.
  • the decomposition level of the synthesized coefficient data is the same as that of the second wavelet coefficient data.
  • a composite image having the same image size as that of the second target image is obtained by performing inverse wavelet transform on the combined coefficient data up to the decomposition level 0.
  • FIG. 105 shows a block diagram of the synthesis unit 500N for an example in which the above knowledge is applied to the first embodiment.
  • the composition unit 500N has a configuration in which an image reduction unit 570 is added to the composition unit 500 (see FIG. 13) according to the first embodiment.
  • the image reduction unit 570 acquires the second basic image data D50 and reduces the second basic image to generate a second target image that is similar to the first target image.
  • the generated second target image data D60 is supplied to the wavelet conversion unit 510.
  • the image reducing unit 570 decomposes the wavelet transform for the second basic image data D50 by a method of recursively decomposing the lowest band component LL. Repeat until level P3. Then, the image reduction unit 570 extracts the lowest band component LL at the decomposition level P3 as the second target image data D60.
  • the image reduction unit 570 may be configured to generate the second target image by a general image reduction process using an average filter. In that case, 1/2 reduction may be performed P3 times, or 1/2 P3 may be reduced at a time.
  • the image size of the second target image in other words, the image size of the composite image is instructed to the image reduction unit 570 by the image size control data C70.
  • the image size control data C70 is input to the composition system by the user of the composition system.
  • the image size control data C70 may be supplied from the supply system in the same manner as the composition control data C50 (see FIGS. 3, 38, 45, and 49).
  • the image size control data C70 is set to a value of P3 when the similarity ratio of the second target image to the first target image is expressed as 1/2 P3 as described above.
  • the image size control data C70 may be data from which the value of P3 can be derived. Examples of data from which the value of P3 can be derived include the value of the similarity ratio (1/2 P3 ).
  • the image size control data C70 is also supplied to the wavelet conversion unit 510.
  • the wavelet conversion unit 510 performs wavelet conversion on the second target image data D60 up to the decomposition level indicated by the obtained P2.
  • the image size control data C70 is also supplied to the synthesis execution unit 520, and is used to specify a range for the coefficient synthesis process in the first wavelet coefficient data A61.
  • the range of the first wavelet coefficient data A61 is determined according to the numerical value indicating the decomposition level P2 of the second wavelet coefficient data as described above.
  • the composition execution unit 520 calculates the value of P2.
  • the synthesis execution unit 520 may acquire the calculated value of P2 from the wavelet conversion unit 510. In that case, the supply of the image size control data C70 to the composition execution unit 520 can be omitted.
  • the image size control data C70 is also supplied to the inverse wavelet conversion unit 530 and used to set the number of inverse wavelet conversions, in other words, to know the decomposition level of the combined coefficient data E61.
  • FIG. 106 is a flowchart for explaining the operation of the synthesis system when the synthesis unit 500N is applied to the first embodiment.
  • the operation flow S400N of FIG. 106 is basically the same as the operation flow S400 (see FIG. 25) according to the first embodiment, but instead of the data acquisition step S410, a data acquisition step S410N to which step S490 is added is performed. Is provided.
  • the image reduction unit 570 generates the second target image by reducing the second basic image.
  • the other steps of the operation flow S400N are basically the same as those of the operation flow S400 of FIG.
  • step S490 is executed in parallel with steps S411, S422, S432, and S433, unlike the example of FIG. It is also possible.
  • the image size of the composite image can be controlled, and a composite image having a size smaller than that of the original image can be obtained.
  • the synthesizing unit 500N can be applied to the second to sixth and thirteenth embodiments instead of the synthesizing unit 500 and the like.
  • the second target image is generated by reducing the entire second basic image.
  • the second target image can be generated by reducing a part of the second basic image.
  • the second basic image before reduction may not be similar to the first target image.
  • the horizontal and vertical reduction ratios of the second basic image may be made different.
  • the similarity ratio of the second target image to the first target image is less than 1. That is, when the similarity ratio is expressed as 1/2 P3 , P3 is a natural number. However, even when the similarity ratio is 1 (P3 is 0 at this time), that is, when the second target image is congruent with the first target image, the synthesis unit 500N can be used.
  • the image reduction unit 570 displays the second basic image. May be supplied to the wavelet conversion unit 510 without being reduced.
  • a composite image of the original image size can be obtained as in the first to sixth embodiments.
  • Embodiments 1 to 6 are examples specialized when the similarity ratio of the second target image to the first target image is 1.
  • Embodiment 15 a second target image having a similar shape to the first target image is generated by reducing at least a part of the second basic image (see FIGS. 102 to 104).
  • FIG. 107 an example will be described in which a part of the second basic image is cut out as a second target image having a similar shape to the first target image. That is, as long as the requirement that the second target image is similar to the first target image is satisfied, the second target image can be generated by cutting.
  • FIG. 108 shows a block diagram of the synthesis unit 500P according to the fifteenth embodiment.
  • the combining unit 500P has a configuration in which a second image cutout unit 570P is provided instead of the image reducing unit 570 in the combining unit 500N (see FIG. 105) according to the fourteenth embodiment.
  • Other configurations of the combining unit 500P are the same as those of the combining unit 500N according to the fourteenth embodiment.
  • the second image cutout unit 570P acquires the second basic image data D50, and uses the first target image in the second basic image with the similarity ratio (that is, the similarity ratio of the second target image to the first target image).
  • a similar shape range that forms a similar shape is set, and data within the similar shape range is cut out as second target image data D60 from the second basic image data D50.
  • the generated second target image data D60 is supplied to the wavelet conversion unit 510.
  • the image size of the second target image in other words, the image size of the composite image is instructed to the second image cutout unit 570P by the image size control data C70.
  • the image size control data C70 includes data indicating the position of the similar shape range cut out as the second target image.
  • FIG. 109 is a flowchart for explaining the operation of the synthesis system when the synthesis unit 500P is applied to the first embodiment.
  • the data acquisition step S410P is provided instead of the data acquisition step S410N, as can be seen from the operation flow S400N according to the fourteenth embodiment (see FIG. 106).
  • Step S490P is provided.
  • the second image cutout unit 570P cuts the second target image data D60 from the second basic image data D50 as described above.
  • the other steps of the operation flow S400P are the same as the operation flow S400N of FIG.
  • step S490P is executed in parallel with steps S411, S422, S432, and S433, unlike the example of FIG. It is also possible.
  • the image size of the composite image can be controlled, and a composite image having a size smaller than that of the original image can be obtained.
  • the synthesizing unit 500P can be applied to the second to sixth and thirteenth embodiments instead of the synthesizing unit 500 and the like.
  • the synthesis unit 500P can be used not only when the similarity ratio is less than 1 but also when it is 1 or less.
  • the second image cutout unit 570P has the second basic image.
  • the image may be supplied to the wavelet transform unit 510 without being cut out. In this case, a composite image of the original image size can be obtained as in the first to sixth embodiments.
  • FIG. 110 shows a conceptual diagram of image composition according to the sixteenth embodiment.
  • supply system 11Q according to the sixteenth embodiment is combined with synthesis system 21 according to the first embodiment.
  • the supply system 11Q can be applied to the image processing systems 1, 10 and the like instead of the supply system 11 according to the first embodiment.
  • FIG. 111 shows a block diagram of mask data output unit 200Q according to the sixteenth embodiment.
  • the mask data output unit 200Q includes a mask generation unit 210Q and the mask encoder 250 described above.
  • the mask generation unit 210Q includes a small image generation unit 241, a small mask generation unit 242, and a mask enlargement unit 243.
  • the small image generation unit 241 acquires the entire image data A10 and reduces the entire image to generate a small image (see FIG. 110).
  • a reduction ratio of 1/2 R of the entire image R is a natural number. Note that the reduction ratio 1/2 R may be fixed or may be changed.
  • the small image generation unit 241 performs wavelet transformation on the entire image data A10 up to the decomposition level R by a method of recursively decomposing the lowest band component LL. Then, the small image generation unit 241 extracts the lowest band component LL at the decomposition level R as small image data A30 that is data of small image data.
  • the small image generation unit 241 may be configured to generate a small image by a general image reduction process using an average filter. In that case, 1/2 reduction may be performed R times, or 1/2 R may be performed at a time.
  • the small mask generation unit 242 acquires the small image data A30 from the small image generation unit 241 and generates a small mask that is a mask for the small image (see FIG. 110).
  • the small mask generation unit 242 can be configured in the same manner as the overall mask generation unit 211 (see FIG. 3) described above. That is, since the image size of the mask generation target is different, the same mask generation technique can be used.
  • the mask enlarging unit 243 acquires the small mask data B31, which is data of the small mask, from the small mask generating unit 242, and generates an enlarged mask by enlarging the small mask to the same image size as the entire image (see FIG. 110).
  • enlarged mask data B32 which is enlarged mask data, is output as basic mask data B50 from the mask generator 210Q.
  • FIG. 112 shows a flowchart for explaining such mask enlargement processing.
  • an enlargement process with an enlargement ratio of 2 is executed in step S602.
  • Step S602 is executed until it is determined in step S601 that the enlargement process of the enlargement ratio 2 has been executed a predetermined number of times (here, R times).
  • FIG. 113 shows an explanatory diagram of an enlargement process with an enlargement ratio of 2.
  • the i-th data i is an integer
  • the ⁇ 2i ⁇ 1 ⁇ -th to ⁇ 2i + 1 ⁇ -th in the mask after enlargement with the enlargement ratio of 2 Estimate that the data corresponds to ROI.
  • the estimation rule of FIG. 113 is developed two-dimensionally, as shown in FIG. 114, the estimation rule of FIG. 113 is applied to each of the vertical direction and the horizontal direction of the mask image. Specifically, when the data of the point of coordinates (I, J) corresponds to the ROI in the mask before enlargement, the coordinates (2I, 2J) are centered in the mask after enlargement by one step (ie, enlargement ratio 2 1 ). It is estimated that the data in the range of 3 ⁇ 3 points corresponding to the ROI. In other words, data in a rectangular range having the coordinates (2I-1, 2J-1) and the coordinates (2I + 1, 2J + 1) as diagonal points are associated with the ROI.
  • FIG. 115 illustrates a more specific diagram.
  • a smoothing process may be performed on the mask image.
  • the mask image is a binary image
  • smoothing processing of the binary image can be used.
  • a morphology process which is a process that is performed by combining dilation and erosion several times.
  • the expansion is a process of replacing a target pixel with white if even one pixel is present in the peripheral area centered on the target pixel.
  • contraction is a process of replacing a target pixel with black if there is even a black pixel in the peripheral area. Ranges such as 3 ⁇ 3, 5 ⁇ 5, and 7 ⁇ 7 are set as the peripheral region.
  • the mask image can be smoothed (reducing the jagged shape and smoothing), and further effective for removing isolated points (in other words, filling in holes).
  • the mask enlargement unit 243 performs a smoothing process on the enlarged mask at the end of at least one of the R enlargement processes.
  • the smoothing processing step S603 is performed after the R-th (that is, the last) enlargement processing.
  • the smoothing processing step S603 is performed after each enlargement processing.
  • the execution timing and the number of executions of the smoothing processing step S603 are not limited to the examples of FIGS. 116 and 117.
  • FIG. 118 illustrates an enlarged mask that is enlarged to the same image size as the entire image when the smoothing process is not performed (see FIG. 112).
  • FIG. 119 shows an enlarged mask when the expansion process is performed twice on the enlarged mask of FIG.
  • FIG. 120 shows an enlarged mask when the shrinking process is performed twice on the enlarged mask of FIG. 119 and 120 are enlarged masks generated by the processing flow S600B of FIG. 116, and a 7 ⁇ 7 range is set as the peripheral region of the expansion process and the reduction process. 120, it can be seen that the jagged shape is smoothed compared to FIG.
  • FIG. 121 shows a flowchart for explaining the operation of the supply system 11Q. According to the operation flow S100Q of FIG. 121, in the operation flow S100 of FIG. 9, a basic mask generation step S122Q is provided instead of the basic mask generation step S122.
  • the small image generation unit 241 generates a small image by reducing the entire image in step S161, and the small mask generation unit 242 generates a small mask for the small image in step S162.
  • the mask enlargement unit 243 enlarges the small mask to the same image size as the entire image, and generates an enlarged mask. Note that a smoothing process may be employed in step S163.
  • the enlarged mask generated by the mask enlargement unit 243 is then processed as a basic mask. That is, in the next step S123, the expanded mask data B32 is encoded, and a mask bit stream Bbs is generated.
  • the entire image is output as the first basic image to the combining system 21, and the small image is not output to the combining system 21.
  • the supply system 11Q is combined with the synthesis system 21 according to the first embodiment. That is, the synthesis system 21 operates according to the operation flow S400 (see FIG. 25) according to the first embodiment.
  • the mask is generated not for the entire image but for the small image. For this reason, the amount of calculation required for mask generation can be reduced as compared with mask generation for the entire image. Note that, unlike the above example, the reduction ratio can be set to a value other than 1/2 R , and the same effect can be obtained in this case.
  • the smoothing process it is possible to smooth the jagged shape of the contour of the ROI-corresponding portion that occurs with the enlargement of the small mask. For example, when the ROI in the entire image is set as a curve like a human outline, the ROI corresponding part of the mask can be formed as a smooth curve. As a result, a smooth outer shape of the ROI can be displayed in the composite image.
  • the wavelet plane that is, the image synthesis is performed using the first wavelet coefficient data and the second wavelet coefficient data. For this reason, even if the ROI and the ROI corresponding part have a jagged shape, a better synthesized image can be obtained as compared with the case where the first target image data and the second target image data are synthesized as they are. Therefore, a better composite image can be obtained by adopting the smoothing process. Whether or not to use the smoothing process may be determined in consideration of the amount of calculation of the system, the smoothness required for the composite image, and the like.
  • the same effects as in the first embodiment can be obtained with respect to the same configuration as in the first embodiment. Further, the sixteenth embodiment can be combined with the second to fifteenth embodiments.
  • FIG. 122 shows a conceptual diagram of image composition for supply system 11R and composition system 21R according to the seventeenth embodiment.
  • the supply system 11R and the composition system 21R can be applied to the image processing systems 1, 10, 20 and the like instead of the supply system 11 and the composition system 21 according to the first embodiment.
  • the supply system 11R outputs the small mask as the basic mask, and the synthesis system 21R enlarges the small mask to generate the target mask.
  • 123 and 124 are block diagrams of the mask data output unit 200R and the synthesis unit 500R according to the seventeenth embodiment.
  • mask data output unit 200R is applied in place of mask data output unit 200 in supply system 11 according to the first embodiment.
  • synthesis unit 500R is applied instead of the synthesis unit 500 in the synthesis system 21 according to the first embodiment.
  • the mask data output unit 200R includes a mask generation unit 210R and the mask encoder 250 described above.
  • the mask generation unit 210R has a configuration in which the mask enlargement unit 243 is deleted from the mask generation unit 210Q (see FIG. 111) according to the sixteenth embodiment. Accordingly, the small mask data B31 is output from the mask generation unit 210R as basic mask data B50.
  • the synthesis unit 500R has a configuration in which a mask enlargement unit 243 is added to the above-described synthesis unit 500 (see FIG. 13). More specifically, the mask enlarging unit 243 acquires the small mask data B31 as the basic mask data B50, and generates an enlarged mask by enlarging the small mask to the image size of the first target image. Then, the mask enlargement unit 243 supplies the enlarged mask data B32 as the target mask data B60 to the synthesis execution unit 520.
  • FIG. 125 shows a flowchart for explaining the operation of the supply system 11R.
  • a basic mask generation step S122R is provided instead of the basic mask generation step S122Q.
  • the basic mask generation step S122R has a configuration in which the mask enlargement step S163 is deleted from the basic mask generation step S122Q of FIG.
  • the small mask generated in step S162 is supplied to the mask encoder 250 as a basic mask, and is encoded in step S123 to form a mask bitstream Bbs.
  • the data of the reduction ratio of the small mask is supplied to the synthesis system 21R together with the small mask data B31.
  • the present invention is not limited to this example.
  • the entire image is output as the first basic image.
  • the small mask which is a basic mask is a mask corresponding to the reduced image of the first basic image.
  • the small image is not output to the synthesis system 21R.
  • FIG. 126 shows a flowchart for explaining the operation of the synthesis system 21R.
  • a data acquisition processing step S410R is provided instead of the data acquisition processing step S410.
  • the data acquisition processing step S410R has a configuration in which a mask expansion step S163 is added after step S433 in the data acquisition processing step S410 of FIG.
  • step S163 the mask enlargement unit 243 enlarges the small mask acquired as the basic mask to the image size of the first target image, and supplies the enlarged mask to the synthesis execution unit 520 as the target mask.
  • the basic mask is a small mask
  • the data amount of the basic mask can be reduced, and thus the communication amount can be reduced.
  • the mask is a binary image
  • 1 bit is assigned to each pixel of the mask
  • the data size can be reduced to 1 ⁇ 4, 24 Kbytes.
  • the data size according to the reduction ratio of 1/2 2 can in 6 Kbytes
  • the data size according to the reduction ratio of 1/2 3 can be the 1.5Kbyte.
  • the image size of the small mask becomes 128 ⁇ 96 pixels.
  • the data size of the basic mask can be reduced in this way, it is realistic to transmit the basic mask data without encoding. In that case, the amount of calculation related to encoding and decoding can be reduced.
  • FIG. 127 shows a configuration example of the supply system 11RB and the synthesis system 21RB when the basic mask data is transmitted without being encoded.
  • the mask data output unit 200RB of the supply system 11RB has a configuration in which a mask bit stream generation unit 250RB is provided instead of the mask encoder 250 in the mask data output unit 200R of FIG.
  • a mask data restoration unit 400RB is provided instead of the mask decoder (see the mask decoder 400 in FIG. 3).
  • the mask bitstream generation unit 250RB generates a bitstream of the basic mask data B50 (here, the small mask data B31) as the mask bitstream Bbs according to the bitstream generation specification that does not perform encoding.
  • the mask data restoration unit 400RB restores the basic mask data B50 (here, the small mask data B31) from the mask bitstream Bbs according to the data restoration specification that does not perform decoding.
  • mask bit stream generation unit 250RB embeds synthesis control data C50 in mask bit stream Bbs
  • mask data restoration unit 400RB performs synthesis control from mask bit stream Bbs.
  • Data C50 is extracted.
  • the same effects as in the first embodiment can be obtained with respect to the same configuration as in the first embodiment.
  • the seventeenth embodiment can be combined with any of the second to fifth, seventh, eighth, and thirteenth to fifteenth embodiments.
  • ⁇ Embodiment 18> a combination example of the seventeenth embodiment and the seventh embodiment will be described.
  • the seventeenth embodiment as shown in FIG. 122, not the entire mask but a small mask is supplied to the synthesis system.
  • the seventh embodiment as shown in FIG. 55, only a part of the entire mask including the ROI corresponding part is supplied to the synthesis system.
  • the eighteenth embodiment as shown in the conceptual diagram of FIG. 128, only a part of the small mask including the ROI corresponding part is supplied to the synthesis system.
  • 129 and 130 are block diagrams of mask data output unit 200S and synthesis unit 500S according to the eighteenth embodiment.
  • mask data output unit 200S is applied in place of mask data output unit 200G (see FIG. 72) in supply system 11G according to the seventh embodiment, whereby supply system 11S according to the eighteenth embodiment (see FIG. 128).
  • the synthesizing unit 500S is applied in the synthesizing system 21G according to the seventh embodiment in place of the synthesizing unit 500G (see FIGS. 73 and 74), and thereby the synthesizing system 21S according to the eighteenth embodiment (see FIG. 128). Is configured.
  • the mask data output unit 200S includes a mask generation unit 210S and the mask encoder 250 described above.
  • the mask generation unit 210S basically includes a small image generation unit 241, a small mask generation unit 242, and a mask enlargement unit 243 instead of the overall mask generation unit 211 in the mask data output unit 200G (see FIG. 72).
  • a cutout range determination unit 232S modified to determine a cutout range for the small mask is provided instead of the cutout range determination unit 232 described above.
  • the synthesis unit 500S has a configuration in which a mask enlargement unit 244 is added to the synthesis unit 500G (see FIG. 74).
  • the mask enlargement unit 244 operates in the same manner as the mask enlargement unit 243 according to the seventeenth embodiment. Specifically, the mask enlargement unit 244 acquires data of a partial mask (that is, a part of a small mask cut out by the supply system 11S) as basic mask data B50. The mask enlargement unit 244 generates an enlarged mask by enlarging the partial mask to the image size of the first target image. Then, the mask enlargement unit 244 supplies the enlarged mask data B32 to the synthesis execution unit 520 as target mask data B60.
  • the cutout range determination unit 232S determines the cutout range 83 (see FIG. 57), which is the range of the first target image in the entire image, by executing steps S501 to S503.
  • the minimum range 81 is determined based on the whole mask 70 (see FIG. 58), but in the eighteenth embodiment, an enlarged mask is used instead of the whole mask.
  • the enlarged mask has the same image size as the whole mask (that is, the same image size as the whole image), is generated by the small image generation unit 241, the small mask generation unit 242, and the mask enlargement unit 243, and is supplied to the cutout range determination unit 232S. Is done.
  • the cut range determination processing flow S500S further includes step S504.
  • the cutout range determination unit 232S performs a small mask required range specifying process that is a process of specifying which range of the small mask corresponds to the cutout range 83 specified by the required range specifying process in step S503.
  • step S503 since the entire mask has the same image size as the entire image, the cutout area 83 specified in step S503 is common to the entire mask and the entire image.
  • the small mask necessary range specifying process in step S504 is necessary.
  • FIG. 132 shows a conceptual diagram of the small mask necessary range specifying process.
  • 132 illustrates the small mask 70S having a reduction ratio of 1/2, but is not limited to this example.
  • reference numeral 70aS indicates a ROI corresponding part
  • reference numeral 70bS indicates a non-ROI corresponding part.
  • Reference numeral 83S indicates a small mask cutout range corresponding to the cutout range 83 in the entire image.
  • the cropping range 83 specified in step S503 may be referred to as an image cropping range 83 below.
  • AX4, AY4, BX4, and BY4 are obtained by multiplying AX3, AY3, BX3, and BY3 by the reduction ratio of the small mask 70S with respect to the entire image.
  • the multiplication can be realized by a bit shift. That is, if the value of AX3 (more specifically, a bit string indicating the value of AX3) is shifted to the right by R bits, the value of AX4 is obtained. The same applies to the values of AY3, BX3, and BY3.
  • step S571 AX3, AY3, BX3, BY3 of the image clipping range 83 are set as the parameter t.
  • t AX3 is set first.
  • step S572 t is multiplied by the reduction ratio of the small mask 70S.
  • the reduction ratio of the small mask 70S is 1/2 R
  • the bit string of t is right-shifted by R bits.
  • t after the calculation is set to AX4 of the small mask cutout range 83S in step S573.
  • AY3, BX3, BY3 to t in step S571, AY4, BX4, BY4 are obtained in step S573.
  • the small mask cutout area specifying data F50S for specifying the small mask cutout area 83S is supplied to the mask cutout section 233.
  • the mask cutout unit 233 acquires the small mask data B31, and based on the small mask cutout area specifying data F50S, the mask cutout area 83S among the small mask data B31 is used as the basic mask data. Cut for B50.
  • the cutout area specifying data F50 for specifying the image cutout area 83 is supplied to the mask encoder 250 and embedded in the mask bitstream BbS, as in the seventh embodiment.
  • the small mask cut-out range specifying data F50S may also be embedded in the mask bitstream BbS and supplied to the synthesis system 21S.
  • FIG. 134 shows a flowchart for explaining the operation of the supply system 11S.
  • a basic mask generation step S122S is provided instead of the basic mask generation step S122G.
  • the above-described small image generation step S161, small mask generation step S162, and mask enlargement step S163 are performed in this order instead of the entire mask generation step S141 in the basic mask generation step S122G (see FIG. 75). It has the structure provided in.
  • FIGS. 135 and 136 are flowcharts for explaining the operation of the synthesis system 21S. 135 and 136 are connected by a connector C.
  • a data acquisition processing step S410S is provided instead of the data acquisition processing step S410G.
  • the data acquisition processing step S410S has a configuration in which a mask expansion step S164 is added after step S433 in the data acquisition processing step S410G of FIG.
  • step S164 as described above, the mask enlargement unit 244 enlarges the partial mask acquired as the basic mask (that is, a part of the small mask cut out by the supply system 11S) to the image size of the first target image.
  • the enlarged mask is supplied to the synthesis execution unit 520 as a target mask.
  • the data amount of the basic mask can be further reduced, and the communication amount can be further reduced. Therefore, similarly to FIG. 127, the mask bit stream generation unit 250RB may be used instead of the mask encoder, and the mask data restoration unit 400RB may be used instead of the mask decoder. In this case, the amount of calculation related to encoding and decoding can be reduced.
  • the first image cutout unit 540 (see FIG. 130) is deleted from the combining unit 500S of the combining system 21S and added to the image data output unit of the supply system 11S (the eighth embodiment shown in FIG. 78). (Refer to the image data output unit 100H).
  • the portion of the image clipping range 83 in the entire image can be supplied as the first basic image from the supply system 11SB to the synthesis system 21SB.
  • the other configurations of the supply system 11SB and the synthesis system 21SB are the same as those of the supply system 11S and the synthesis system 21S described above.
  • FIG. 138 shows a block diagram of an image processing system according to the nineteenth embodiment.
  • the supply system 11T according to the nineteenth embodiment is combined with the synthesis system 21 according to the first embodiment.
  • the supply system 11T can be applied to the image processing systems 1, 10 and the like instead of the supply system 11 according to the first embodiment.
  • Supply system 11T has a configuration in which combining unit 700 is added to supply system 11 according to the first embodiment.
  • the synthesizing unit 700 has the same configuration as the synthesizing unit 500 of the synthesizing system 21, and includes first basic image data A50 (which is the same as the whole image data A10 in the first embodiment), and synthesis control data C50.
  • the second basic image data D50 is acquired. Note that the second basic image data D50 can be acquired from the synthesis system 21, for example. Further, the synthesizer 700 acquires basic mask data B50 from the mask generator 210. Thereby, the synthesizing unit 700 operates in the same manner as the synthesizing unit 500 and can generate the composite image data E100.
  • the display unit 31 (see FIG. 2) on the supply system 11T side performs a display operation, whereby the ROI and the second target image in the first target image are displayed. And a composite image can be displayed. For this reason, for example, before supplying the mask bitstream Bbs to the synthesis system 21, the synthesized image can be confirmed on the supply system 11T side. In particular, it is possible to confirm the degree of composition according to the composition control data C50.
  • the supply-side device 30 (see FIG. 2) can be applied alone as an image processing device having an image synthesis function.
  • a hardware configuration example of such an image processing device 30T is shown in FIG.
  • the image processing apparatus 30T includes an image processing system 90, a display unit 31, and an operation unit 32.
  • the image processing apparatus 30T may include one or both of the I / F 33 and the image input unit 34, following the supply-side apparatus 30.
  • the image processing system 90 includes an image composition system 91, and the image composition system 91 includes the mask generation unit 210 described above, the composition unit 700, and a semiconductor memory (hereinafter also referred to as a memory) 800.
  • Mask generation unit 210, composition unit 700, and memory 800 are connected via a bus (which is an example of a wired communication medium).
  • the image processing system 90 can be formed as a single semiconductor integrated circuit, in other words, as a single chip.
  • the image processing system 90 may be configured only by the image composition system 91 or may further include another processing system.
  • the memory 800 stores the entire image data A10 and the second basic image data D50 supplied to the image composition system 91.
  • the mask generation unit 210 reads the entire image data A10 from the memory 800, and generates basic mask data B50 based on the entire image data A10.
  • the synthesizing unit 700 reads the entire image data A10 and the second basic image data D50 from the memory 800 and uses them for image synthesis.
  • the composition control data C50 is supplied to the composition unit 700 when the user inputs it to the operation unit 32.
  • the display unit 31 displays a composite image based on the composite image data E100 generated by the composite unit 700.
  • composition unit 700 may employ the configuration of the composition units 500F to 500L, 500N, 500P, 500R, and 500S according to Embodiments 6 to 12, 14, 15, 17, and 18.
  • the configuration formed by the memory 800 and the mask generation unit 210 can be regarded as a data supply system that supplies the entire image data A10 and the basic mask data B50.
  • the synthesizing unit 700 is further regarded as an image synthesizing system
  • the configuration in FIG. 139 is such that the data supply system and the image synthesizing system are connected via a bus (an example of a wired communication medium as described above). It is possible to grasp that
  • a combination of the data supply system 11 and the synthesis system 21 can be formed by a single semiconductor integrated circuit.
  • the data supply system 11 and the synthesis system 21 are connected via a bus.
  • the data supply system 11 and the synthesizing system 21 communicate without going through the external I / Fs 33 and 43, unlike FIG. 2, but communicate through the bus corresponding to the medium 50.
  • encoding and decoding of the transfer data can be omitted. .
  • the image encoder 150, the image decoder 300, the mask encoder 250, and the mask decoder 400 may not be used.
  • the image encoder 150, the image decoder 300, the mask encoder 250, and the mask decoder 400 can be omitted.
  • other combinations of the data supply system 11B and the like and the synthesis system 21B and the like can be formed by a single semiconductor integrated circuit.
  • Image processing system (overall system) 10, 20, 90 Image processing system 11, 11B to 11J, 11L, 11Q, 11R, 11RB, 11S, 11SB, 11T Data supply system 21, 21B to 21L, 21R, 21RB, 21S, 21SB, 91 Image composition system 32, 42 Operation unit (operation medium) 50 Medium 60 Whole image 60a ROI 60b non-ROI 61 to 63 Wavelet plane 70 Whole mask 70S Small mask 71, 72, 73 Development mask 70a, 71a, 72a, 73a, 70aS ROI corresponding part 70b, 71b, 72b, 73b, 70bS Non-ROI corresponding part 81 Minimum range 82 Tracking minimum Range 83 Cutout range 83S Small mask cutout range 100, 100B, 100H Image data output unit 150, 150B, 150s to 150v Image encoder 200, 200C, 200F, 200G, 200I, 200L, 200Q, 200R, 200RB, 200S Mask data output unit 210, 210F, 210G,

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Discrete Mathematics (AREA)
  • Editing Of Facsimile Originals (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)
  • Studio Circuits (AREA)
PCT/JP2016/057721 2015-03-31 2016-03-11 画像処理システムおよび画像処理方法 Ceased WO2016158310A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/712,881 US10410079B2 (en) 2015-03-31 2017-09-22 Image processing system and image processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2015-071365 2015-03-31
JP2015071365A JP6471022B2 (ja) 2015-03-31 2015-03-31 画像処理システムおよび画像処理方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/712,881 Continuation US10410079B2 (en) 2015-03-31 2017-09-22 Image processing system and image processing method

Publications (1)

Publication Number Publication Date
WO2016158310A1 true WO2016158310A1 (ja) 2016-10-06

Family

ID=57004911

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/057721 Ceased WO2016158310A1 (ja) 2015-03-31 2016-03-11 画像処理システムおよび画像処理方法

Country Status (3)

Country Link
US (1) US10410079B2 (https=)
JP (1) JP6471022B2 (https=)
WO (1) WO2016158310A1 (https=)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018182568A (ja) * 2017-04-14 2018-11-15 株式会社メガチップス 画像処理装置、画像処理システム、情報処理システム及び画像処理方法
CN108989704B (zh) * 2018-07-27 2021-03-12 创新先进技术有限公司 图像生成方法和装置、终端设备
WO2021130918A1 (ja) * 2019-12-25 2021-07-01 富士通株式会社 画像処理システム及び画像処理プログラム
CN111708082B (zh) * 2020-05-29 2022-04-12 成都理工大学 一种随深度变化的深度域地震子波的提取方法
JP6906823B1 (ja) * 2020-12-07 2021-07-21 レーザーテック株式会社 マスク検査方法及びマスク検査装置
WO2025074812A1 (ja) * 2023-10-05 2025-04-10 日本電気株式会社 映像符号化装置、映像復号装置、映像符号化方法、および映像復号方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002094991A (ja) * 2000-09-19 2002-03-29 Mega Chips Corp 関心領域符号化方法
JP2006203409A (ja) * 2005-01-19 2006-08-03 Megachips Lsi Solutions Inc 圧縮符号化装置、圧縮符号化方法およびプログラム
JP2007104645A (ja) * 2005-09-06 2007-04-19 Megachips Lsi Solutions Inc 圧縮符号化装置、圧縮符号化方法およびプログラム
JP2007180801A (ja) * 2005-12-27 2007-07-12 Fujitsu Ltd 画像符号化方法および画像符号化装置
JP2007251476A (ja) * 2006-03-15 2007-09-27 Ricoh Co Ltd 画像圧縮装置および画像圧縮方法、ならびにそのためのプログラム

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SE9800088D0 (sv) 1997-10-10 1998-01-16 Ericsson Telefon Ab L M Long filter lossless region of interest coding
JP4245123B2 (ja) 2002-02-28 2009-03-25 株式会社メガチップス ウエーブレット処理装置及びウエーブレット処理方法
JP2005165688A (ja) 2003-12-02 2005-06-23 Fuji Xerox Co Ltd 複数対象物追跡方法及びシステム
CN1655620B (zh) * 2004-02-09 2010-09-22 三洋电机株式会社 图像显示装置
JP4390671B2 (ja) 2004-09-21 2009-12-24 三洋電機株式会社 画像処理装置
JP4301193B2 (ja) * 2005-03-31 2009-07-22 ソニー株式会社 画像比較装置及び方法、画像検索装置及び方法、並びにプログラム及び記録媒体
US7916961B2 (en) 2005-09-06 2011-03-29 Megachips Corporation Compression encoder, compression encoding method and program
JP4407600B2 (ja) 2005-09-22 2010-02-03 日本ビクター株式会社 画像処理装置
JP4991595B2 (ja) 2008-02-21 2012-08-01 株式会社東芝 パーティクルフィルタを使用する追跡システム
US20110222787A1 (en) * 2008-02-28 2011-09-15 Stefan Thiemert Frame sequence comparison in multimedia streams
US8406297B2 (en) * 2008-10-17 2013-03-26 Futurewei Technologies, Inc. System and method for bit-allocation in video coding
JP6076623B2 (ja) 2012-06-06 2017-02-08 株式会社メガチップス 動体検出装置、動体検出方法およびプログラム
JP6345961B2 (ja) * 2014-03-28 2018-06-20 株式会社メガチップス 画像復号化装置および画像復号化方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002094991A (ja) * 2000-09-19 2002-03-29 Mega Chips Corp 関心領域符号化方法
JP2006203409A (ja) * 2005-01-19 2006-08-03 Megachips Lsi Solutions Inc 圧縮符号化装置、圧縮符号化方法およびプログラム
JP2007104645A (ja) * 2005-09-06 2007-04-19 Megachips Lsi Solutions Inc 圧縮符号化装置、圧縮符号化方法およびプログラム
JP2007180801A (ja) * 2005-12-27 2007-07-12 Fujitsu Ltd 画像符号化方法および画像符号化装置
JP2007251476A (ja) * 2006-03-15 2007-09-27 Ricoh Co Ltd 画像圧縮装置および画像圧縮方法、ならびにそのためのプログラム

Also Published As

Publication number Publication date
US20180012098A1 (en) 2018-01-11
JP2016192054A (ja) 2016-11-10
US10410079B2 (en) 2019-09-10
JP6471022B2 (ja) 2019-02-13

Similar Documents

Publication Publication Date Title
JP6471022B2 (ja) 画像処理システムおよび画像処理方法
JP3814456B2 (ja) 画像処理装置及びその方法
JP6345961B2 (ja) 画像復号化装置および画像復号化方法
JP2008258994A (ja) 画像処理装置
JP2002176553A (ja) 画像処理装置、及びその方法、並びにプログラム、記憶媒体
JP6471023B2 (ja) 画像処理システムおよび画像処理方法
JP2004236299A (ja) 画像処理装置、画像形成装置、画像処理方法、プログラム及び記憶媒体
JP2006197074A (ja) 画像処理装置、画像処理方法及びそのプログラム
JP2016192669A5 (https=)
JP3704515B2 (ja) 画像伸張装置及びその方法
JP4412688B2 (ja) 画像処理装置及びその方法
JP2007005844A (ja) 符号化処理装置、符号化処理方法、プログラム及び情報記録媒体
JP2010098743A (ja) 画像処理装置、画像読取装置、及び画像処理方法
JP4726040B2 (ja) 符号化処理装置、復号処理装置、符号化処理方法、復号処理方法、プログラム及び情報記録媒体
JP6577855B2 (ja) 画像処理システムおよび画像処理方法
JP4584115B2 (ja) 画像符号化装置、画像符号化方法、画像符号化プログラムおよび画像符号化プログラムを記録したコンピュータ読み取り可能な記録媒体
JP2002135781A (ja) 画像符号化装置、画像復号装置、及びそれらの方法並びに記憶媒体
JP2006340300A (ja) 信号処理方法及び信号処理装置、並びに信号処理プログラム及び情報記録媒体
JP3897253B2 (ja) 画像処理方法、画像処理装置、撮像装置、プログラム及び記録媒体
JP4090352B2 (ja) 画像処理装置、画像処理方法、画像形成装置、プログラム及び記憶媒体
KR20110139519A (ko) Ｄｃｔ 처리를 이용한 이미지 스케일링 처리 장치 및 이미지 스케일링 처리 방법
JP4250304B2 (ja) 画像復号装置、画像復号方法
JP4194311B2 (ja) 動画像符号化装置及び動画像復号装置並びにそれらの方法
JPH11284840A (ja) 画像形成装置
JP2001285865A (ja) 画像処理装置及びその方法、コンピュータ可読メモリ

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16772194

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16772194

Country of ref document: EP

Kind code of ref document: A1