Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/142—Detection of scene cut or scene change
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/109—Selection of coding mode or of prediction mode among a plurality of temporal predictive coding modes
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/136—Incoming video signal characteristics or properties
  • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/136—Incoming video signal characteristics or properties
  • H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation

Definitions

• the present invention relates, in general, to an apparatus and method for analyzing images, and, more particularly, to an apparatus and method for analyzing images, which estimates spatial complexity and temporal complexity by analyzing image frames included in image data, detects scene change points of the image data using the spatial complexity, and detects whether a specific image corresponds to specific image classification, thereby estimating the complexity of the images.
• JVT Joint Video Team
• ISO International Data Communication
• ITU International Telecommunication Union
• H.264 has a higher compression rate with regard to the same image quality and excellently higher image quality than H.263 but the degree of the complexity thereof has been increased.
• an H.263 decoding algorithm can be implemented using merely software by an existing embedded processor.
• an H.264 encoding system has encoded image data at a fixed bit rate allocated regardless of the image complexity of video sources. In this case, the same bit rate is allocated regardless of the attributes of the image data. Therefore, there are problems in that one or more null packets are generated in the case in which the complexity of images is low, so that all of given bands cannot be utilized, and in that high bit rate cannot be allocated in the case in which the complexity of the images is high even though the high bit rate must be allocated.
• an object of the present invention is to provide an apparatus and method for analyzing images which analyzes the complexity of images by estimating the spatial complexity of image frames.
• Another object of the present invention is to analyze the complexity of images by estimating the temporal complexity of image frames.
• a further object of the present invention is to analyze the complexity of images by estimating the characteristics of image frames.
• the present invention provides an apparatus for analyzing images, the apparatus being connected to an image capture apparatus for capturing input image data, the apparatus for analyzing images including a spatial complexity estimation unit for, when image data and timestamp information indicative of capture time for a specific image is received from the image capture apparatus, calculating spatial complexity for the single frame of the image data by adding edge sizes within the frame, which are calculated through filtering using a Sobel mask; and a temporal complexity estimation unit for estimating motion using the two successive frames of the image data, and then calculating temporal complexity using errors calculated by compensating for a motion vector generated during the estimation.
• a spatial complexity estimation unit for, when image data and timestamp information indicative of capture time for a specific image is received from the image capture apparatus, calculating spatial complexity for the single frame of the image data by adding edge sizes within the frame, which are calculated through filtering using a Sobel mask
• a temporal complexity estimation unit for estimating motion using the two successive frames of the image data, and then calculating temporal complexity using errors calculated by compensating for a motion vector generated
• the present invention provides an apparatus for analyzing images, the apparatus being connected to an image capture apparatus for capturing input image data, the apparatus for analyzing images including a spatial complexity estimation unit for, when image data and timestamp information indicative of capture time for a specific image is received from the image capture apparatus, calculating spatial complexity for the single frame of the image data by adding edge sizes within the frame, which are calculated through filtering using a Sobel mask; and a temporal complexity estimation unit for estimating motion using the two successive frames of the image data, and then calculating temporal complexity using errors calculated by compensating for a motion vector generated during the estimation; and an image characteristic detection unit for detecting image characteristics by analyzing the basic data of the image data based on preset criteria.
• a spatial complexity estimation unit for, when image data and timestamp information indicative of capture time for a specific image is received from the image capture apparatus, calculating spatial complexity for the single frame of the image data by adding edge sizes within the frame, which are calculated through filtering using a Sobel mask
• a temporal complexity estimation unit for estimating motion using
• the image characteristic detection unit includes scene change detection means for determining a scene change point using the spatial complexity calculated by the spatial complexity estimation unit, and then detecting scene change; image classification means for selecting any one from among preset image types using the spatial complexity, the temporal complexity and the scene change, and then determining the image type; and frame type determination means for determining a frame type depending on the existence of the scene change of the image data.
• the image characteristic detection unit further include interest area detection means for extracting an interest area that matches the image type detected by the image classification means from interest areas for the respective preset image types.
• the image characteristic detection unit further include caption area detection means for detecting a caption area based on the spatial complexity, the temporal complexity, and location information within the frame.
• the spatial complexity estimation unit calculate edge sizes, including a vertical edge size and a horizontal edge size, for each pixel of the frame of the image using a mask, measure the edge sizes by adding the calculated values, generate an edge map at a Macro-Block (MB) level by adding the absolute values of all the edge sizes within a single MB, and then calculate the spatial complexity for the single frame by adding all the edge sizes in MB units.
• edge sizes including a vertical edge size and a horizontal edge size
• the temporal complexity estimation unit estimate motion for the two successive frames of the image data, and then calculate the temporal complexity using a difference between a motion size calculated through compensation and a compensated image.
• the present invention provides a method of analyzing images using an apparatus for analyzing images, the method including a) when the apparatus for analyzing images receives image data and timestamp information indicative of capture time for a specific image, calculating spatial complexity for the single frame of the image data by adding edge sizes within the frame, which are calculated through filtering using a Sobel mask; b) estimating motion using the two successive frames of the image data, and then calculating temporal complexity using errors calculated by compensating for a motion vector generated during the estimation; and c) detecting image characteristics by analyzing the basic data of the image data based on preset criteria.
• step c) includes c-1) determining a scene change point using the spatial complexity calculated at step a), and then detecting scene change; c-2) selecting any one from among preset image types using the spatial complexity, the temporal complexity and the scene change, which are calculated at respective steps a), b) and c- 1), and then determining the image type; and c-3) determining a frame type depending on existence of scene change of the image data.
• the method further include extracting an interest area that matches the image type detected at step c-3) from interest areas for the respective preset image types.
• the method further include detecting a caption area based on the spatial complexity, the temporal complexity, and location information within the frame.
• the apparatus and method for analyzing images according to the present invention analyzes the complexity of images by estimating the spatial complexity of image frames, there is an advantage in that compression can be performed while considering the complexity of images when the images are compressed.
• the present invention analyzes the complexity of images by estimating the temporal complexity of image frames, there is an advantage in that compression can be differentially performed based on temporal complexity when the images are compressed.
• the present invention analyzes the complexity of images by estimating the characteristics of image frames, there is an advantage in that compression can be differentially performed based on the characteristics of the image frames when the images are compressed. Therefore, when the images are provided to a user side over a broadcasting network, there is an advantage in that a larger amount of resources can be allocated to a section which requires relatively wider bandwidth, and with the result that image quality can be improved on an image playback terminal.
• FIG. 1 is a diagram showing the configuration of an apparatus for analyzing images according to the present invention
• FIG. 2 is a view showing a method of estimating spatial complexity according to the present invention
• FIG. 3 is a diagram showing the configuration of an image characteristic detection unit according to the present invention.
• FIG. 4 is a view showing a method of detecting scene change according to the present invention.
• FIG. 5 is a flowchart showing a method of analyzing images according to the present invention.
• FIG. 6 is a flowchart showing a method of detecting image characteristics according to the present invention.
• FIG. 1 is a diagram showing the configuration of an apparatus for analyzing images according to the present invention.
• an apparatus 100 for analyzing images includes a spatial complexity estimation unit 110, a temporal complexity estimation unit 130, and an image characteristic detection unit 150.
• the spatial complexity estimation unit 110 receives image data and timestamp information indicative of capture time for a specific image from an image capture apparatus 230, it calculates spatial complexity for the single frame of the image data by adding edge sizes within the frame, which are calculated through filtering using a Sobel mask.
• the spatial complexity estimation unit 110 receives the image data and the timestamp information indicative of capture time for a specific image from the image capture apparatus 230, it calculates edge sizes, including a vertical edge size (refer to (a) of FIG. 2) and a horizontal edge size (refer to (b) of FIG. 2), for each pixel of the frame of the image data using a mask, measures the edge sizes by adding the calculated values, generates an edge map at a Macro-Block (MB) level by adding the absolute values of all the edge sizes within a single MB, and then calculates the spatial complexity for the single frame by adding all the edge sizes designated in MB units, as shown in FIG. 2.
• edge sizes including a vertical edge size (refer to (a) of FIG. 2) and a horizontal edge size (refer to (b) of FIG. 2)
• the image capture apparatus 230 captures the image data transmitted from the outside and then received by an information transmission/reception apparatus 210.
• the temporal complexity estimation unit 130 estimates motion using two successive frames of the image data, and then calculates temporal complexity using errors calculated by compensating for a motion vector generated during the estimation of the motion.
• the temporal complexity estimation unit 130 estimates motions for the two successive frames of the image data, and then calculates the temporal complexity using difference between a motion size calculated through the compensation and a compensated image.
• the image characteristic detection unit 150 analyzes the basic data of the image data based on preset criteria, and then detects image characteristics.
• FIG. 3 is a diagram showing the configuration of an image characteristic detection unit according to the present invention.
• the image characteristic detection unit 150 includes scene change detection means
• image classification means 153 image classification means 153
• interest area detection means 155 interest area detection means 155
• frame type determination means 157 caption area detection means 159.
• the scene change detection means 151 determines a scene change point and detects scene change using the spatial complexity calculated by the spatial complexity estimation unit 110 shown in FIG. 1.
• the image classification means 153 selects any one from among preset image types using spatial complexity, temporal complexity, and scene change, and then determines the image type.
• the interest area detection means 155 extracts an interest area, which matches the image type detected by the image classification means 153, from interest areas for the respective preset image types.
• Each of the interest areas for the respective preset image types is an area in a specific frame in which it is expected that people who will watch image data will have interest, for example, if the image data corresponds to a soccer game, the interest area in a frame is not the seats for the audience but a green ground where soccer players are playing the game.
• the interest area is referred to so that the images of a corresponding area can be displayed in a more clear state than areas which do not correspond to the interest area by allocating higher bit rate to the interest area when the bit rate is allocated to the corresponding image data.
• the frame type determination means 157 determines a frame type based on the existence of the scene change of the image data.
• the frame types are divided into an Index-frame (I-frame) picture and a
• Predicted-frame (P-frame) picture is basically determined based on an Instant Decoder Refresh (IDR) period, the frame type is reset based on a scene change frame when the scene change occurs.
• IDR Instant Decoder Refresh
• the I-frame is a frame which becomes the index of the following P-frame, and contains complete data about a single frame.
• the P-frame has data about the motion trajectories of respective data, which does not overlap, and data, which does overlap, between a current frame and a frame immediately before the current frame.
• the caption area detection means 159 detects a caption area based on the spatial complexity, the temporal complexity, and location information within a frame.
• the characteristics of the caption area are that the edge size thereof is larger than that of other areas, the location thereof is generally the bottom of a screen, and the motion thereof is regular. Therefore, when an area having a large edge size is continuously detected on the bottom of a screen in regular speed, the area is determined as the caption area.
• the caption area is encoded so that little deterioration occurs by lowering the quantization level thereof, high bit rate is generated after a frame having the caption area is encoded, and this should be considered when a band is allocated.
• FIG. 5 is a flowchart showing a method of analyzing images according to the present invention.
• the spatial complexity estimation unit 110 of the apparatus 100 receives image data and timestamp information indicative of capture time for a specific image from the image capture apparatus 230 at step SlOl, it calculates spatial complexity for the single frame of the image data by adding the edge sizes within the frame, which are calculated through filtering using a Sobel mask at step S 103.
• Step S 103 will be described in greater detail.
• the spatial complexity estimation unit 110 receives the image data and timestamp information indicative of capture time for a specific image from the image capture apparatus 230, it calculates edge sizes, including a vertical edge size (refer to (a) of FIG. 2) and a horizontal edge size (refer to (b) of FIG. 2) using a mask, measures the edge sizes by adding the calculated values, generates an edge map at a MB level by adding the absolute values of all the edge sizes within a single MB, and then calculates the spatial complexity for the single frame by adding all the edge sizes designated in MB units, as shown in FIG. 2.
• the temporal complexity estimation unit 130 estimates motion using two successive frames of the image data, and then calculates temporal complexity using errors calculated by compensating for a motion vector generated during the estimation of the motion at step S 105.
• the temporal complexity estimation unit 130 estimates motions for the two successive frames of the image data, and then calculates the temporal complexity using a difference between a motion size calculated through compensation and a compensated image.
• the image characteristic detection unit 150 analyzes the basic data of the image data based on preset criteria, and then detects image characteristics at step S 107.
• the apparatus 100 for analyzing images provides image analysis information detected at steps S 103 to S 107 to other apparatuses each for providing or encoding images through the information transmission/reception apparatus 210 at step S 109.
• FIG. 6 is a flowchart showing a method of detecting image characteristics according to the present invention, and the image characteristic detection step, shown in FIG. 5, will be described in greater detail.
• step 150 determines a scene change point using the spatial complexity calculated at step S 103 shown in FIG. 5, and then detects scene change at step s201.
• the image classification means 153 selects any one from among preset image types using the spatial complexity, the temporal complexity and the scene change, calculated at respective steps S 103, S 105 and S201, and then determines the image type at step s203.
• the frame type determination means 157 determines the frame type depending on the existence of the scene change of the image data at step s205.
• the frame types are divided into an I-frame and a P-frame.
• the frame type is basically determined based on an IDR period, the frame type is reset based on a scene change frame when the scene change occurs.
• the I-frame is a frame which becomes the index of the following P-frame, and contains complete data about a single frame.
• the P-frame has data about the motion trajectories of respective data, which does not overlap, and data, which does overlap, between a current frame and a frame immediately before the current frame.
• the interest area detection means 155 of the image characteristic detection unit 150 extracts an interest area, which matches the image type detected at step S203, from interest areas for the respective preset image types.
• Each of the interest areas for the respective preset image types is an area in a specific frame in which it is expected that people who will watch image data will have interest, for example, if the image data corresponds to a soccer game, the interest area in a frame is not the seats for the audience but a green ground where soccer players are playing the game.
• the interest area is referred to so that the images of a corresponding area can be displayed in a more clear state than areas which do not correspond to the interest area by allocating higher bit rate to the interest area when the bit rate is allocated to the corresponding image data.
• the caption area detection means 159 of the image characteristic detection unit 150 detects a caption area based on the spatial complexity, the temporal complexity, and location information within a frame.
• the characteristics of the caption area are that the edge size thereof is larger than that of other areas, the location thereof is generally the bottom of a screen, and the motion thereof is regular. Therefore, when an area having a large edge size is continuously detected on the bottom of a screen in regular speed, the caption area detection means 159 determines the area as the caption area.
• a higher bit rate is allocated to the caption area detected by the caption area detection means 150 than is allocated to other areas, so that it is referred to so that people can watch a clearer caption.
• the present invention analyzes the complexity of images by estimating the space, time, and characteristics of image frames, compression can be differentially performed based on results gathered from analysis when the images are compressed. Therefore, when the images are sent to a user side over a broadcasting network, the present invention is suitable for allocating a larger number of resources in a section which requires relatively wide bandwidth.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=40591236

Family Applications (1)

Country Status (2)

Cited By (1)

Families Citing this family (1)

Citations (3)

Family Cites Families (3)

• 2007
  • 2007-10-29 KR KR1020070108723A patent/KR100939435B1/ko active IP Right Grant
• 2008
  • 2008-10-02 WO PCT/KR2008/005805 patent/WO2009057898A1/en active Application Filing

Patent Citations (3)

Also Published As

Similar Documents

Legal Events