TWI673652B - Image processing device and image processing method - Google Patents
Image processing device and image processing method Download PDFInfo
- Publication number
- TWI673652B TWI673652B TW107127425A TW107127425A TWI673652B TW I673652 B TWI673652 B TW I673652B TW 107127425 A TW107127425 A TW 107127425A TW 107127425 A TW107127425 A TW 107127425A TW I673652 B TWI673652 B TW I673652B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- block
- unit
- images
- calculation unit
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/142—Detection of scene cut or scene change
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/87—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving scene cut or scene change detection in combination with video compression
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Analysis (AREA)
Abstract
設置評估值算出部(3),根據特徵量算出部(2)算出的2個圖像內包含的區塊單位的特徵量,算出判定有無場景轉換(scene change)使用的評估值,有無判定部(4),藉由比較評估值算出部(3)算出的評估值與臨界值,判定有無2個圖像間的場景轉換。於是,特徵量算出部(2),在圖像選擇部(1)選擇的圖像的每一區塊,根據上述區塊的編碼量或上述區塊內包含的分割區塊最深的階層數,算出上述區塊的特徵量,構成影像處理裝置。 An evaluation value calculation unit (3) is provided, and an evaluation value for determining the presence or absence of a scene change is calculated based on the feature quantity of each block unit included in the two images calculated by the feature quantity calculation unit (2). (4) The presence or absence of scene transition between the two images is determined by comparing the evaluation value and the critical value calculated by the evaluation value calculation unit (3). Then, the feature calculation unit (2) selects each block of the image selected by the image selection unit (1) according to the coding amount of the block or the deepest number of divisions included in the block. The feature quantities of the blocks are calculated to constitute an image processing device.
Description
此發明,係關於判定有無場景的變化點的場景轉換之影像處理裝置及影像處理方法。 This invention relates to an image processing device and an image processing method for determining the presence or absence of a change point in a scene.
以下的專利文件1中,揭示判定有無場景轉換的影像處理裝置。 The following Patent Document 1 discloses an image processing device that determines the presence or absence of a scene change.
此影像處理裝置,從編碼視訊串流,取得複數的圖像的編碼參數之一的移動向量,根據複數的圖像之間的移動向量的差異,判定有無場景轉換。 This image processing device obtains a motion vector, which is one of the encoding parameters of a plurality of images, from a coded video stream, and determines the presence or absence of a scene transition based on a difference between the motion vectors between the plurality of images.
[專利文件1]國際公開第2010/125757號 [Patent Document 1] International Publication No. 2010/125757
因為習知的影像處理裝置如以上所構成,編碼視訊串流中,圖像的編碼參數,只要包含移動向量的話,就可以判定有無場景轉換。但是,因為圖像的編碼模式例如是畫面內編碼模式,上述圖像的編碼參數,有時不包含移動向量。作為編碼參數,關於不包含移動向量的圖像,具有不能判定有無場景轉換的課題。 Because the conventional image processing device is configured as described above, in the encoded video stream, as long as the encoding parameters of the image include a motion vector, it can determine whether there is a scene transition. However, since the encoding mode of an image is, for example, an intra-frame encoding mode, the encoding parameters of the above-mentioned image do not sometimes include a motion vector. As a coding parameter, an image that does not include a motion vector has a problem that it is impossible to determine the presence or absence of a scene transition.
因為此發明係為了解決上述的課題而形成,目的在於即使關於不包含移動向量的圖像,也得到可以判定有無場景轉換的影像處理裝置及影像處 理方法。 This invention is made in order to solve the above-mentioned problems, and an object thereof is to obtain an image processing device and an image processing method capable of determining the presence or absence of a scene transition even for an image that does not include a motion vector.
根據此發明的影像處理裝置,包括:圖像選擇部,從複數的圖像中,選擇2個圖像作為評估有無場景變化點的場景轉換的對象圖像;特徵量算出部,分別關於圖像選擇部選擇的2個圖像,分別算出圖像內包含的區塊單位的特徵量;評估值算出部,根據特徵量算出部算出的2個圖像內包含的區塊單位的特徵量,算出判定有無場景轉換使用的評估值;以及有無判定部,比較評估值算出部算出的評估值與臨界值,判定有無2個圖像之間的場景轉換;特徵量算出部,在圖像選擇部選擇的圖像的每一區塊,根據上述區塊的編碼量或上述區塊內包含的分割區塊最深的階層數,算出上述區塊的特徵量。 An image processing device according to the present invention includes: an image selecting section that selects two images from a plurality of images as a target image for scene transition for evaluating whether or not a scene change point is present; and a feature amount calculating section for each of the images The two images selected by the selection unit calculate the feature quantity of each block unit included in the image; the evaluation value calculation unit calculates the feature quantity of the block unit included in the two images calculated by the feature quantity calculation unit. The presence or absence of an evaluation value used for scene conversion; and the presence or absence of a judgment section that compares the evaluation value calculated by the evaluation value calculation section with a critical value to determine the presence or absence of a scene transition between two images; the feature quantity calculation section selects the image selection section For each block of the image, the feature amount of the block is calculated according to the coding amount of the block or the deepest level of the divided block contained in the block.
根據本發明,設置評估值算出部,根據特徵量算出部算出的2個圖像內包含的區塊單位的特徵量,算出判定有無場景轉換使用的評估值,有無判定部,藉由比較評估值算出部算出的評估值與臨界值,判定有無2個圖像間的場景轉換。於是,特徵量算出部,在圖像選擇部選擇的圖像的每一區塊,根據上述區塊的編碼量或上述區塊內包含的分割區塊最深的階層數,算出上述區塊的特徵量,構成影像處理裝置。因此,根據此發明的影像處理裝置,即使關於不包含移動向量的圖像,也可以判定有無場景轉換。 According to the present invention, an evaluation value calculation unit is provided, and an evaluation value for determining the presence or absence of a scene change is calculated based on the feature quantity of a block unit included in the two images calculated by the feature quantity calculation unit, and the presence / absence determination unit compares the evaluation values. The evaluation value and the critical value calculated by the calculation unit determine whether there is a scene transition between the two images. Then, the feature calculation unit calculates the feature of the block in each block of the image selected by the image selection unit based on the coding amount of the block or the deepest level of the divided block included in the block. The amount constitutes an image processing device. Therefore, according to the image processing apparatus of the present invention, it is possible to determine the presence or absence of scene transition even for an image that does not include a motion vector.
1‧‧‧圖像選擇部 1‧‧‧Image Selection Department
2‧‧‧特徵量算出部 2‧‧‧ Feature quantity calculation section
3‧‧‧評估值算出部 3‧‧‧Evaluation value calculation section
4‧‧‧有無判定部 4‧‧‧ Presence or absence
5‧‧‧多工器 5‧‧‧ Multiplexer
11‧‧‧圖像選擇電路 11‧‧‧Image selection circuit
12‧‧‧特徵量算出電路 12‧‧‧Characteristic calculation circuit
13‧‧‧評估值算出電路 13‧‧‧Evaluation value calculation circuit
14‧‧‧有無判定電路 14‧‧‧ presence or absence of judgment circuit
21‧‧‧記憶體 21‧‧‧Memory
22‧‧‧處理器 22‧‧‧Processor
[第1圖]係顯示根據此發明的第一實施形態的影像處理裝置的構成圖;[第2圖]係顯示根據此發明的第一實施形態的影像處理裝置的硬體構成圖;[第3圖]係影像處理裝置以軟體或韌體等實現時的電腦硬體構成圖; [第4圖]係顯示影像處理裝置以軟體或韌體等實現時的處理程序的影像處理方法流程圖;[第5圖]係顯示從編碼視訊串流內包含的複數的圖像中,選擇評估對象圖像的實例說明圖;[第6圖]係顯示從編碼視訊串流內包含的複數的圖像中,選擇評估對象圖像的實例說明圖;[第7圖]係顯示編碼方式是AVC/H.264或HEVC/H.265時的編碼視訊串流的構成例說明圖;[第8圖]係顯示處理區塊單位的編碼量的一實例說明圖;[第9圖]係顯示根據此發明的第一實施形態的另一影像處理裝置的構成圖;[第10圖]係顯示評估對象圖像內包含的處理區塊中,明確指定編碼量的處理區塊說明圖;[第11圖]係顯示評估對象圖像內包含的處理區塊中,明確指定編碼量的處理區塊說明圖;[第12圖]係顯示評估對象圖像內包含的處理區塊中,明確指定編碼量的處理區塊說明圖;[第13圖]係顯示編碼區塊中的分割區塊階層數的說明圖;[第14圖]係顯示處理區塊與編碼區塊的實例的說明圖;[第15圖]係顯示處理區塊與編碼區塊的實例的說明圖;[第16圖]係顯示處理區塊的水平方向與垂直方向的座標說明圖;[第17圖]係顯示每一處理區塊的加權係數wn的一實例說明圖;以及[第18圖]係顯示每一處理區塊的加權係數wn的一實例說明圖。 [FIG. 1] A block diagram showing an image processing apparatus according to a first embodiment of the present invention; [FIG. 2] A block diagram showing a hardware configuration of an image processing apparatus according to a first embodiment of the present invention; Figure 3] is a diagram of the computer hardware when the image processing device is implemented in software or firmware; [Figure 4] is a flowchart of the image processing method showing the processing procedure when the image processing device is implemented in software or firmware; [Fig. 5] is a diagram showing an example of selecting an evaluation target image from a plurality of images included in the encoded video stream; [Fig. 6] is a picture showing a plurality of images included in the encoded video stream In the figure, an example of selecting an evaluation target image is shown; [FIG. 7] is a diagram illustrating an example of the structure of an encoded video stream when the encoding method is AVC / H.264 or HEVC / H.265; [FIG. 8] FIG. 9 is an explanatory diagram showing an example of a coding amount of a processing block unit; [FIG. 9] is a diagram showing a configuration of another image processing apparatus according to the first embodiment of the present invention; [FIG. 10] is a diagram showing an evaluation target Among the processing blocks included in the image, the processing block that explicitly specifies the encoding amount Figure; [Fig. 11] shows the processing block description diagram with the code amount explicitly specified among the processing blocks contained in the evaluation target image; [Fig. 12] shows the processing blocks contained in the evaluation target image , Which clearly specifies the processing block explanatory diagram of the coding amount; [Fig. 13] is an explanatory diagram showing the number of divided block levels in the encoding block; [Fig. 14] is an example showing the processing block and the encoding block Explanatory diagram; [Fig. 15] is an explanatory diagram showing an example of a processing block and a coding block; [Fig. 16] is an explanatory diagram showing the horizontal and vertical coordinates of a processing block; [Fig. 17] is a diagram An explanatory diagram showing an example of the weighting coefficient w n of each processing block; and [FIG. 18] An explanatory diagram showing an example of the weighting coefficient w n of each processing block.
以下,為了更詳細說明此發明,關於用以實施此發明的形態,根據附加的圖面說明。 Hereinafter, in order to explain this invention in more detail, the form for implementing this invention is demonstrated based on the attached drawing.
第一實施形態 First embodiment
第1圖係顯示根據此發明的第一實施形態的影像處理裝置的構成圖。第2圖係顯示根據此發明的第一實施形態的影像處理裝置的硬體構成圖。 Fig. 1 is a block diagram showing an image processing apparatus according to a first embodiment of the present invention. Fig. 2 is a diagram showing a hardware configuration of an image processing apparatus according to a first embodiment of the present invention.
第1圖中,圖像選擇部1,例如以第2圖所示的圖像選擇電路11實現。 In FIG. 1, the image selection unit 1 is implemented by, for example, an image selection circuit 11 shown in FIG. 2.
圖像選擇部1,實施取得包含複數的圖像的編碼資料的編碼視訊串流之處理。 The image selection unit 1 performs processing for obtaining an encoded video stream including encoded data of a plurality of images.
圖像選擇部1,從具有編碼區塊單位(區塊單位)編碼的編碼資料的複數的圖像中,決定評估有無場景變化點的場景轉換之候補圖像,從候補圖像中實施選擇2個圖像作為評估有無場景轉換的對象圖像的處理。但是,候補圖像的決定方法,係預先定義。 The image selection unit 1 determines a candidate image for scene transition for evaluating the presence or absence of a scene change point from a plurality of images having encoded data encoded in encoding block units (block units), and performs selection 2 from the candidate images. Each image is processed as a target image for evaluating whether there is a scene transition. However, the method for determining a candidate image is defined in advance.
圖像的編碼資料,例如,係稱作大區塊(macroblock)的編碼區塊單位,或者稱作CTU(Coding Tree Unit(編碼樹單位))的編碼區塊單位編碼的資料。 The image encoding data is, for example, data encoded by a coding block unit called a macroblock, or a coding block unit called a CTU (Coding Tree Unit).
作為大區塊單位中編碼的編碼方式規格,例如,在以下的非專利文件1中揭示AVC/H.264規格。 As a coding method specification for coding in a large block unit, for example, the AVC / H.264 specification is disclosed in the following Non-patent Document 1.
又,作為CTU單位編碼的編碼方式規格,例如,在以下的非專利文件2中揭示HEVC/H.265。 Further, as a coding method specification for CTU unit coding, for example, HEVC / H.265 is disclosed in the following Non-patent Document 2.
[非專利文件1]ISO/IEC 14496-10/ITU-T H. 264規格 [Non-Patent Document 1] ISO / IEC 14496-10 / ITU-T H. 264 Specification
[非專利文件2]ISO/IEC 23008-2/ITU-T H. 265規格 [Non-Patent Document 2] ISO / IEC 23008-2 / ITU-T H. 265 Specification
特徵量算出部2,例如以第2圖所示的特徵量算出電路12實現。 The feature quantity calculation unit 2 is implemented by, for example, a feature quantity calculation circuit 12 shown in FIG. 2.
特徵量算出部2,在圖像選擇部1選擇的每一圖像中,根據上述圖像中的編碼區塊單位的編碼資料,實施分別算出處理區塊單位的特徵量的處理。處理區塊,以1以上的編碼區塊構成,預先定義其構成。處理區塊,因為以1以上的編 碼區塊構成,可定義處理區塊的最小單位成為編碼區塊(處理區塊=編碼區塊)。 The feature amount calculation unit 2 executes processing for calculating the feature amount of each processing block unit based on the coded block unit coded data in each of the images selected by the image selection unit 1. The processing block is composed of one or more coding blocks, and its structure is defined in advance. Since the processing block is composed of one or more coding blocks, the smallest unit that can define a processing block becomes a coding block (processing block = coding block).
第14圖係顯示以4個編碼區塊構成處理區塊的實例。又,如第15圖所示的實例,構成為構成處理區塊的縱方向與橫方向的編碼區塊數不同也可以。 Fig. 14 shows an example in which a processing block is constituted by 4 coding blocks. Further, as shown in the example shown in FIG. 15, the number of coding blocks in the vertical direction and the horizontal direction constituting the processing block may be different.
處理區塊單位的特徵量,係根據處理區塊內的各個編碼區塊的編碼資料算出。此第一實施形態中,解碼各個編碼區塊的編碼資料,根據編碼資料的解碼結果明確指定各個編碼區塊的編碼量,以屬於各個處理區塊的編碼區塊的編碼量總和作為處理區塊單位的特徵量。 The feature quantity of the processing block unit is calculated based on the coding data of each coding block in the processing block. In this first embodiment, the encoding data of each encoding block is decoded, the encoding amount of each encoding block is explicitly specified according to the decoding result of the encoding data, and the sum of the encoding amounts of the encoding blocks belonging to each processing block is used as the processing block. The characteristic quantity of the unit.
又,使圖像選擇部1選擇的2個圖像的處理區塊尺寸(預先定義的處理區塊中的編碼區塊的構成)相同。 In addition, the processing block sizes (the structure of the coding blocks among the predefined processing blocks) of the two images selected by the image selection unit 1 are made the same.
評估值算出部3,例如以第2圖所示的評估值算出電路13實現。 The evaluation value calculation unit 3 is implemented by, for example, an evaluation value calculation circuit 13 shown in FIG. 2.
評估值算出部3,根據特徵量算出部2算出的2個圖像內包含的處理區塊單位的特徵量,實施算出判定有無場景轉換使用的評估值的處理。 The evaluation value calculation unit 3 performs a process of calculating an evaluation value for determining the presence or absence of a scene change based on the feature values of the processing block units included in the two images calculated by the feature value calculation unit 2.
有無判定部4,例如以第2圖所示的有無判定電路14實現。 The presence / absence determination unit 4 is implemented by, for example, the presence / absence determination circuit 14 shown in FIG. 2.
有無判定部4,藉由比較評估值算出部3算出的評估值與臨界值,實施判定有無2個圖像間的場景轉換之處理。 The presence / absence determination unit 4 compares the evaluation value calculated by the evaluation value calculation unit 3 with a critical value, and executes a process of determining the presence or absence of a scene transition between two images.
第1圖中,影像處理裝置的構成要素圖像選擇部1、特徵量算出部2、評估值算出部3以及有無判定部4分別假設如第2圖所示的專用硬體實現。即,假設以圖像選擇電路11、特徵量算出電路12、評估值算出電路13以及有無判定電路14實現。 In FIG. 1, the constituent elements of the image processing device, the image selection unit 1, the feature quantity calculation unit 2, the evaluation value calculation unit 3, and the presence / absence determination unit 4 are assumed to be implemented by dedicated hardware as shown in FIG. 2. That is, it is assumed that the image selection circuit 11, the feature quantity calculation circuit 12, the evaluation value calculation circuit 13, and the presence / absence determination circuit 14 are implemented.
在此,圖像選擇電路11、特徵量算出電路12、評估值算出電路13以及有無判定電路14,例如是單一電路、複合電路、程式化的處理器、並聯程式化的處理器、ASIC(特殊應用積體電路)、FPGA(現場可編程邏輯閘陣列),或相當於這些的組合。 Here, the image selection circuit 11, the feature quantity calculation circuit 12, the evaluation value calculation circuit 13, and the presence / absence determination circuit 14 are, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (special Application integrated circuit), FPGA (field programmable logic gate array), or a combination of these equivalent.
影像處理裝置的構成要素,不限於以專用的硬體實現,影像處理 裝置以軟體、韌體或軟體與韌體的組合實現也可以。 The constituent elements of the image processing device are not limited to being implemented by dedicated hardware, and the image processing device may be implemented by software, firmware, or a combination of software and firmware.
軟體或韌體作為程式被收納在電腦的記憶體內。電腦,意味實行程式的硬體,例如,CPU(中央處理單元)、中央處理裝置、處理裝置、演算裝置、微處理器、微電腦、處理器、DSP(數位信號處理器)等適用。 Software or firmware is stored as a program in the computer's memory. A computer means hardware that implements a program, such as a CPU (Central Processing Unit), a central processing unit, a processing device, a calculation device, a microprocessor, a microcomputer, a processor, a DSP (Digital Signal Processor), and the like.
電腦的記憶體,例如,RAM(隨機存取記憶體)、ROM(唯讀記憶體)、快閃記憶體、EPROM(可抹除可編程唯讀記憶體)、EEPROM(電氣可抹除可編程唯讀記憶體)等的非揮發性或揮發性的半導體記憶體或磁碟、軟碟、光碟(Optical disc)、光碟(Compact disc)、迷你光碟、DVD(數位通用光碟)等適用。 Computer memory, such as RAM (Random Access Memory), ROM (Read Only Memory), Flash Memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Non-volatile or volatile semiconductor memory or magnetic disks, floppy disks, optical discs, compact discs, mini-discs, DVDs (digital versatile discs), etc. are applicable.
第3圖係影像處理裝置以軟體或韌體實現時的電腦硬體構成圖。 Figure 3 is a diagram of the computer hardware configuration when the image processing device is implemented in software or firmware.
影像處理裝置以軟體或韌體等實現時,用以使電腦實行圖像選擇部1、特徵量算出部2、評估值算出部3以及有無判定部4的處理程序的程式收納在記憶體21內,電腦的處理器22只要實行記憶體21內收納的程式即可。 When the image processing device is implemented by software or firmware, a program for causing a computer to execute a processing program of the image selection section 1, the feature amount calculation section 2, the evaluation value calculation section 3, and the presence / absence determination section 4 is stored in the memory 21 The processor 22 of the computer only needs to execute the programs stored in the memory 21.
第4圖係顯示影像處理裝置以軟體或韌體實現時的處理程序的影像處理方法流程圖。 FIG. 4 is a flowchart of an image processing method showing a processing program when the image processing device is implemented in software or firmware.
又,第2圖中,顯示影像處理裝置的構成要素分別以專用硬體實現的實例,第3圖中,顯示影像處理裝置以軟體、韌體等實現的實例,但影像處理裝置中的一部分的構成要素以專用硬體實現,剩下的構成要素以軟體、韌體等實現也可以。 Fig. 2 shows an example in which the constituent elements of the image processing device are implemented by dedicated hardware, and Fig. 3 shows an example in which the image processing device is implemented by software, firmware, etc. The constituent elements are implemented by dedicated hardware, and the remaining constituent elements may be implemented by software or firmware.
其次說明關於動作。 Next, the operation will be described.
圖像選擇部1,取得包含複數的圖像的編碼資料的編碼視訊串流。 The image selection unit 1 obtains an encoded video stream including encoded data of a plurality of images.
圖像選擇部1,從取得的編碼視訊串流,根據預先定義的候補圖像的決定方法決定候補圖像,從候補圖像中,選擇2個圖像,作為評估有無場景變化點的場景轉換的對象圖像(第4圖的步驟ST1)。 The image selection unit 1 determines a candidate image from the obtained encoded video stream based on a predetermined candidate image determination method, and selects two images from the candidate images as a scene transition for evaluating the presence or absence of a scene change point. Target image (step ST1 in FIG. 4).
在此,選擇的2個圖像中,稱播放順序中後面的圖像為「評估對象圖像」, 播放順序中前面的圖像為「參照圖像」。所謂「評估有無2個圖像間的場景轉換」,係指在播放順序中比評估參照圖像後一個的圖像到評估對象圖像(包含評估對象圖像本身)是否發生場景轉換。 Here, among the two selected images, the latter image in the playback order is referred to as the “evaluation target image”, and the previous image in the playback order is referred to as the “reference image”. The "evaluation of the presence or absence of scene transition between two images" refers to whether or not a scene transition has occurred in the playback sequence from the image subsequent to the evaluation reference image to the evaluation target image (including the evaluation target image itself).
第5及6圖,係顯示從包含編碼資料在編碼視訊串流內的圖像中,決定成為評估對象圖像的候補圖像,並從候補圖像中,選擇評估對象圖像與參照圖像的實例說明圖。 Figures 5 and 6 show the candidate images that are to be evaluated from the images containing the encoded data in the encoded video stream, and select the evaluation target image and the reference image from the candidate images. The illustration illustrates the figure.
第5圖中,顯示包含編碼資料在編碼視訊串流內的複數圖像全部作為候補圖像,從候補圖像中,依播放順序選擇評估對象圖像,最後,選擇編碼視訊串流內包含的全部圖像作為評估對象圖像之實例。 In Fig. 5, all the plural images containing the encoded data in the encoded video stream are displayed as candidate images. From the candidate images, the evaluation target image is selected according to the playback order. Finally, the encoded video stream is selected. All images are examples of the evaluation target images.
第6圖中,顯示包含編碼資料在編碼視訊串流內的複數的圖像具有GOP(Group Of Pictures(圖像組))構造,各個GOP中的隨機存取點的幀內圖像(intra picture)作為候補圖像,從候補圖像中,依播放順序選擇作為評估對象圖像之實例。 In Fig. 6, a picture showing a plurality of coded data in a coded video stream has a GOP (Group Of Pictures) structure. Intra pictures of random access points in each GOP (intra picture ) As a candidate image, an example of an evaluation target image is selected from the candidate images in the playback order.
AVC/H.264中,IDR(Instantaneous Decoding Refresh(解碼立即刷新))圖像係隨機存取點的幀內圖像。 In AVC / H.264, an IDR (Instantaneous Decoding Refresh) image is an intra-frame image of a random access point.
HEVC/H.265中,IRAP(Intra Radom Access Point(幀內隨機存取點))圖像係隨機存取點的幀內圖像。 In HEVC / H.265, an IRAP (Intra Radom Access Point) image is an intra-frame image of a random access point.
參照圖像,在第5圖及第6圖中都是在播放順序中比評估對象圖像前一個的候補圖像。因此,第5圖的例中,評估有無鄰接的2個圖像間的場景轉換,即,評估對象圖像本身是否是場景轉換點。第6圖的例中,評估有無鄰接的GOP中的隨機存取點的幀內圖像間的場景轉換。 The reference image is a candidate image that precedes the evaluation target image in the playback order in both FIG. 5 and FIG. 6. Therefore, in the example of FIG. 5, whether or not there is a scene transition between two adjacent images is evaluated, that is, whether or not the target image itself is a scene transition point. In the example of FIG. 6, scene transitions between intra-frame images with and without random access points in adjacent GOPs are evaluated.
又,上述之外也以N個(N是0以上的整數)的圖像間隔選擇評估對象圖像(N=0時,與第5圖相同)等,可以任意設定候補圖像的定義。 In addition to the above, an evaluation target image is selected at an image interval of N (N is an integer of 0 or more) (when N = 0, the same as in FIG. 5), etc., and the definition of the candidate image can be arbitrarily set.
第7圖係顯示編碼方式是AVC/H.264或HEVC/H.265時的編碼視 訊串流的構成例說明圖。 Fig. 7 is an explanatory diagram showing a configuration example of an encoded video stream when the encoding method is AVC / H.264 or HEVC / H.265.
第7圖中,複數的存取單元依編碼順序排列,1個存取單元表示1個圖像的資料。 In FIG. 7, a plurality of access units are arranged in an encoding order, and one access unit represents data of one image.
各個存取單元,以複數的NAL(Network Abstraction Layer(網路抽象層))單元構成。 Each access unit is composed of a plurality of NAL (Network Abstraction Layer) units.
各個NAL單元,分類為標題資訊(non-VCL)與圖像的編碼資料的圖像資料(VCL)。 Each NAL unit is classified into title information (non-VCL) and picture data (VCL) of the coded data of the picture.
標題資訊(non-VCL),具有存取單元限定器、序列等級標題以及圖像等級標題。 Title information (non-VCL), which has an access unit qualifier, a sequence level header, and a picture level header.
AVC/H.264或HEVC/H.265中,構成1個存取單元的1個以上的NAL單元連續,為了清楚串流上鄰接的存取單元間的段落位置,定義存取單元的前頭位置的識別方法。作為其中一例,定義表示存取單元前頭的NAL單元的存取單元限定器。 In AVC / H.264 or HEVC / H.265, one or more NAL units constituting one access unit are continuous. In order to clarify the position of the paragraph between adjacent access units on the stream, the head position of the access unit is defined. Identification method. As one example, an access unit qualifier is defined which represents the NAL unit in front of the access unit.
又,第7圖所示的NAL單元是一例,NAL單元的構成模型,以各個規格的方法為標準。此時,構成存取單元的NAL單元中有時也不存在存取單元限定器,即使在那情況下,按照以規格定義的存取單元的前頭位置的識別方法,也可以識別存取單元的前頭位置。 The NAL unit shown in FIG. 7 is an example. The configuration model of the NAL unit is based on the method of each standard. At this time, the access unit limiter may not exist in the NAL unit constituting the access unit. Even in that case, the access unit can be identified according to the identification method of the front position of the access unit defined by the specification. Front position.
因此,圖像選擇部1,按照以規格定義的存取單元的前頭位置的識別方法,藉由識別存取單元的前頭位置,可以識別圖像的存取單元的段落位置。 Therefore, the image selection unit 1 can identify the paragraph position of the access unit of the image by identifying the front position of the access unit according to the identification method of the front position of the access unit defined by the specification.
圖像選擇部1,因為可以識別存取單元的段落位置,從複數的存取單元中,可以選擇2個圖像,輸出選擇的2個圖像的編碼資料至特徵量算出部2。 The image selection unit 1 can identify the paragraph position of the access unit. From the plurality of access units, two images can be selected, and the encoded data of the two selected images are output to the feature quantity calculation unit 2.
特徵量算出部2,在圖像選擇部1選擇的每一圖像,根據編碼區塊單位的編碼資料,分別算出處理區塊單位的特徵量(第4圖的步驟ST2)。 The feature quantity calculation unit 2 calculates, for each image selected by the image selection unit 1, the feature quantity in the processing block unit based on the coding data in the coding block unit (step ST2 in FIG. 4).
以下,具體說明根據特徵量算出部2的處理區塊單位的特徵量的算出處理。 Hereinafter, the calculation processing of the feature quantity in the processing block unit by the feature quantity calculation unit 2 will be specifically described.
在此,假設圖像選擇部1選擇的圖像號碼t的圖像Pt內包含的處理區塊的特徵量為Cn,t,參照圖像的圖像號碼為t=tp,評估對象圖像的圖像號碼為t=tc。 Here, it is assumed that the feature amount of the processing block included in the image P t of the image number t selected by the image selection unit 1 is C n, t , and the image number of the reference image is t = tp. The image number is t = tc.
n,係明確指定圖像內的特徵量算出對象的處理區塊的區塊號碼,從左上的處理區塊開始,依光域掃描順序編號1,2,…,NUM_BLt。在此,NUM_BLt,表示圖像號碼t的圖像Pt中的特徵量算出對象的處理區塊總數。處理區塊尺寸是64×64畫素,圖像的解析度是3840×2160畫素的情況下,以1畫面的全部為對象時,成為NUM_BLt=60×34=2040。但是,最下端列的區塊的區塊尺寸為60×48畫素。NUM_BLt,如上述為畫面全體的處理區塊數也可以,為了降低處理負荷限制至比畫面全體的處理區塊數更小的值也可以。 n is the block number of the processing block that explicitly specifies the feature quantity calculation object in the image, starting from the processing block on the upper left, and numbered 1,2, ..., NUM_BL t according to the scanning order of the optical domain. Here, NUM_BL t represents the total number of processing blocks to be calculated by the feature amount calculation in the image P t of the image number t. When the processing block size is 64 × 64 pixels and the resolution of the image is 3840 × 2160 pixels, when all of one screen is targeted, NUM_BL t = 60 × 34 = 2040. However, the block size of the block in the bottom row is 60 × 48 pixels. NUM_BL t may be the number of processing blocks for the entire screen as described above, and may be limited to a value smaller than the number of processing blocks for the entire screen in order to reduce the processing load.
因此,根據下述分別算出t=tp以及t=tc中的特徵量Cn,tp,、Cn,tc。 Therefore, the feature quantities C n, tp , and C n, tc in t = tp and t = tc are calculated from the following.
特徵量算出部2,實施特徵量算出對象圖像Pt中的語法(syntax)解碼處理,根據語法解碼處理結果,分別明確指定處理區塊單位的編碼量Sn,t(n=1,2,…,NUM_BLt)。 The feature amount calculation unit 2 performs syntax decoding processing on the feature amount calculation target image P t , and explicitly specifies the coding amounts Sn , t (n = 1,2) for each processing block unit based on the syntax decoding process result. , ..., NUM_BL t ).
例如,特徵量算出部2,分別解碼屬於特徵量算出對象的處理區塊的各個編碼區塊的編碼資料內包含的全部編碼參數,明確指定解碼的全部編碼參數的編碼量總和作為編碼量Sn,t。 For example, the feature amount calculating unit 2, respectively for decoding all the coded data for each coding parameters coded blocks included in the processing block belonging to the object feature amount calculation, the total code amount of all coding parameters explicitly specified as the decoded code amount S n , t .
編碼參數,係產生編碼區塊的解碼圖像所需要的參數,例如,表示編碼模式的參數、幀內預測參數、互相預測參數、預測差異編碼參數、移動向量等適用。 The coding parameters are parameters required to generate a decoded image of a coded block, for example, parameters indicating a coding mode, intra prediction parameters, mutual prediction parameters, prediction difference coding parameters, motion vectors, and the like are applicable.
其次,特徵量算出部2,明確指定特徵量算出對象圖像Pt的編碼量SALL,t,如以下的式(1)所示,以特徵量算出對象圖像Pt的編碼量SALL,t,除處理區塊單位的編碼量Sn,t,藉此標準化,使標準化的處理區塊單位的編碼量Sn,t為M倍,作為處理區塊單位的特徵量Cn,t。M係預先設定的常數,例如,比0大的實數。 Next, the feature quantity calculation unit 2 explicitly specifies the feature quantity calculation target image P t encoding amount S ALL, t , as shown in the following formula (1), calculates the feature quantity calculation target image P t encoding amount S ALL with the feature quantity. , t , divided by the coding amount Sn , t of the processing block unit, thereby standardizing, so that the standardized coding amount Sn , t of the processing block unit is M times, as the feature amount C n, t of the processing block unit . M is a constant set in advance, for example, a real number greater than 0.
在此,顯示特徵量算出部2明確指定特徵量算出對象圖像Pt的編碼量SALL,t的實例,但考慮算出特徵量算出對象圖像Pt內包含的全部處理區塊的編碼量Sn,t的總和,作為特徵量算出對象圖像Pt的編碼量SALL,t的實例。 Here, examples of the display 2 to explicitly specify the code amount feature quantities calculation unit calculates the target image P t S ALL, t, but considering the calculated code amount calculated by the feature amount in all processing blocks included in the target image P t The total of S n, t is an example in which the amount of coding S ALL, t of the target image P t is calculated as a feature amount.
但是,這不過是一例,作為特徵量算出對象圖像Pt的編碼量SALL,t,明確指定特徵量算出對象圖像Pt的圖像資料(VCL)的尺寸也可以。圖像資料(VCL)的尺寸,不實施特徵量算出對象圖像Pt中的語法解碼處理,而可以明確指定。其他還有,作為特徵量算出對象圖像Pt的編碼量SALL,t,明確指定特徵量算出對象圖像Pt的存取單元尺寸也可以。存取單元尺寸,可以只藉由識別上述的存取單元的段落位置計算。 However, this is only an example, and it is also possible to explicitly specify the size of the image data (VCL) of the feature quantity calculation target image P t as the encoding amount S ALL, t of the feature quantity calculation target image P t . The size of the image data (VCL) can be explicitly specified without performing syntax decoding processing on the feature amount calculation target image P t . Other well as the feature amount calculation target amount of coding image P t S ALL, t, explicitly specify the size of the access unit calculates a feature amount of an object image P t may be. The size of the access unit can be calculated only by identifying the position of the paragraph of the above-mentioned access unit.
式(1)算出的處理區塊單位的特徵量Cn,t,表示各個處理區塊的編碼量比,係抑制每一圖像的總編碼量的大小不均影響的特徵量。 The feature amount C n, t in the processing block unit calculated by the formula (1) represents the coding amount ratio of each processing block , and is a feature amount that suppresses the influence of the size variation of the total coding amount of each image.
第8圖係顯示處理區塊單位的編碼量的一實例說明圖。 FIG. 8 is an explanatory diagram showing an example of a coding amount in a processing block unit.
第8圖,處理區塊以2×2個編碼區塊構成,表示編碼區塊是大區塊(macroblock)或CTU的實例,以數字表示各個編碼區塊的編碼量。又,各處理區塊中央,表示屬於上述處理區塊的編碼區塊的編碼量總和,作為上述處理區塊的編碼量。 In FIG. 8, the processing block is composed of 2 × 2 coding blocks, indicating that the coding block is an example of a macroblock or a CTU, and the coding amount of each coding block is represented by numbers. The center of each processing block indicates the sum of the coding amounts of the coding blocks belonging to the processing block as the coding amount of the processing block.
第8圖的例中,以byte(字組)表示編碼區塊的編碼量單位,但只不過是一例,以bit(字元)單位表示也可以。 In the example of FIG. 8, the coding unit of the coding block is represented by byte, but it is only an example, and the unit may be expressed by bit.
評估值算出部3,根據特徵量算出部2算出的參照圖像Ptp中的處理區塊單位的特徵量Cn,tp以及評估對象圖像Ptc中的處理區塊單位的特徵量Cn,tc,算出判定有無場景轉換(scene change)使用的評估值J(第4圖的步驟ST3)。 Evaluation value calculating unit 3, the feature amount of the feature amount calculating unit C n P tp 2 calculated by the reference image in the processing block units, tp and an evaluation of the object image P tc processing block units of the feature amount C n , tc , calculates an evaluation value J used to determine the presence or absence of a scene change (step ST3 in FIG. 4).
例如,評估值算出部3,作為判定有無場景轉換(scene change)使用的評估值J,如以下的式(2)所示,算出參照圖像Ptp與評估對象圖像Ptc之間的處理區塊單位 的特徵量的差異絕對值和。 For example, the evaluation value calculation unit 3 calculates the processing between the reference image P tp and the evaluation target image P tc as the evaluation value J used to determine the presence or absence of a scene change, as shown in the following formula (2). The sum of the absolute values of the differences in the feature amounts of the block units.
評估值算出部3算出的評估值J,係參照圖像Ptp與評估對象圖像Ptc之間的圖案變化越大,參照圖像Ptp與評估對象圖像Ptc的處理區塊單位的編碼量分佈的差異越大,值越大。 The evaluation value J calculated by the evaluation value calculation unit 3 is the larger the pattern change between the reference image P tp and the evaluation target image P tc , the larger the processing block unit of the reference image P tp and the evaluation target image P tc is. The larger the difference in the code amount distribution, the larger the value.
又,MIN_NUM_BL係NUM_BLtp與NUM_BLtc的最小值。即,參照圖像Ptp與評估對象圖像Ptc的處理區塊數不同時,只有兩圖像都算出特徵量的處理區塊成為評估值J的算出對象。 In addition, MIN_NUM_BL is the minimum value of NUM_BL tp and NUM_BL tc . That is, when the number of processing blocks of the reference image P tp and the evaluation target image P tc is different, only the processing blocks for which the feature amount is calculated for both images become the calculation target of the evaluation value J.
例如,參照圖像Ptp與評估對象圖像Ptc的編碼資料尺寸大不同時,解碼處理時間也傾向大不同。如此的情況下,一律限制各圖像的編碼資料的解碼處理、處理區塊單位的特徵量的算出所分的時間時,可處理的處理區塊數在兩圖像成為不同的結果。如此的情況下,只有兩圖像都算出特徵量的處理區塊作為對象才求出評估值J。又,如上述由於決定各個圖像中可以消耗的處理時間T,根據所希望的處理速度(圖框率1/T)的特徵量算出處理變得可能。 For example, when the reference image P tp and the coded data size of the evaluation target image P tc are greatly different, the decoding processing time also tends to be significantly different. In such a case, when the time required for the decoding process of the encoded data of each image and the calculation of the feature amount of the processing block unit are uniformly restricted, the number of processable blocks can be different for the two images. In this case, the evaluation value J is obtained only when the processing block for which the feature amount is calculated in both images is the target. In addition, since the processing time T that can be consumed in each image is determined as described above, it is possible to calculate the processing based on the feature amount of the desired processing speed (frame rate 1 / T).
有無判定部4,比較評估值算出部3算出的評估值J與事先設定的臨界值Th(第4圖的步驟ST4)。 The presence / absence determination unit 4 compares the evaluation value J calculated by the evaluation value calculation unit 3 with a threshold value Th set in advance (step ST4 in FIG. 4).
有無判定部4,如果評估值算出部3算出的評估值J在臨界值Th以上的話(第4圖的步驟ST4:YES時),判定參照圖像Ptp與評估對象圖像Ptc之間有場景轉換(第4圖的步驟ST5)。 The presence / absence determination unit 4 determines whether there is a gap between the reference image P tp and the evaluation target image P tc if the evaluation value J calculated by the evaluation value calculation unit 3 is equal to or greater than the critical value Th (when step ST4 in FIG. 4: YES). Scene change (step ST5 in FIG. 4).
有無判定部4,如果評估值算出部3算出的評估值J未滿臨界值Th的話,(第4圖的步驟ST4:NO時),判定參照圖像Ptp與評估對象圖像Ptc之間無場景轉換(第4圖的步驟ST6)。 The presence / absence determination unit 4 determines whether the reference value P tp and the evaluation target image P tc are between the reference image P tp and the evaluation target image P tc if the evaluation value J calculated by the evaluation value calculation unit 3 is less than the critical value Th (when step ST4 in FIG. 4: NO). No scene change (step ST6 in FIG. 4).
有無判定部4,輸出有無場景轉換的判定結果。 The presence / absence determination unit 4 outputs a determination result of the presence or absence of a scene transition.
又,臨界值Th,是預先設定的固定值也可以,根據特定的條件又轉換又變化也可以。例如,考慮每一內容的種類(戲劇、新聞、運動等)準備臨界值Th轉換的方法、每一廣播台準備臨界值Th轉換的方法、產生串流的每一編碼器準備臨界值Th轉換的方法、根據產生串流的編碼器的設定順應算出臨界值Th(準備以編碼器的設定值作為變數的臨界值Th算出式)的方法、根據串流的編碼量推移順應算出臨界值Th(準備以編碼量的時間變化(時間方向的1~n次微分,n是1以上的整數)為變數的臨界值Th算出式)的方法、形成與作為對象的處理區塊數即MIN_NUM_BL成比例的值(Th=MIN_NUM_BL×Th_Base,Th_Base係比成為臨界值Th基準值的預先設定的0大的常數)的方法等。又,根據形成與MIN_NUM_BL成比例的值的方法,設定加上每一評估對象圖像Ptc的MIN_NUM_BL(或MIN_NUM_BL’)的變動產生的評估值J能取得的最大值的變動之臨界值,可以實現高精度的場景轉換檢出處理。 The threshold value Th may be a fixed value set in advance, or may be changed or changed according to a specific condition. For example, consider a method of preparing a threshold Th conversion for each type of content (drama, news, sports, etc.), a method of preparing a threshold Th conversion for each broadcast station, and a method of preparing a threshold Th conversion for each encoder that generates a stream. Method, method of calculating threshold value Th according to the setting of the encoder that generates the stream (preparation of the threshold value Th using the setting value of the encoder as a variable), and calculating the threshold value of Th according to the change of the encoding amount of the stream (preparation The method of calculating the time change of the code amount (differential in the time direction from 1 to n times, n is an integer of 1 or more) as the critical value of the variable, and forming a value proportional to the number of target processing blocks, that is, MIN_NUM_BL (Th = MIN_NUM_BL × Th_Base, Th_Base is a constant that is larger than a preset value of 0 that becomes the threshold Th reference value), and the like. In addition, according to a method of forming a value proportional to MIN_NUM_BL, a threshold value for a change in the maximum value that can be obtained by adding a change in MIN_NUM_BL (or MIN_NUM_BL ') for each evaluation target image P tc is set, and Achieve high-precision scene transition detection processing.
又,這些方法的組合方法也可以。例如,根據串流的編碼量推移順應算出臨界值Th的方法以及形成與MIN_NUM_BL成比例的值的方法的組合方法等。此時,根據串流的編碼量推移順應算出的臨界值Th,藉由置換為Th_Base,可以實現。 A combination of these methods is also possible. For example, a combination method of a method of calculating a critical value Th in accordance with a change in the encoding amount of a stream, and a method of forming a value proportional to MIN_NUM_BL. In this case, the threshold value Th calculated in accordance with the change in the encoding amount of the stream can be realized by replacing it with Th_Base.
圖像選擇部1決定的候補圖像中,留下還沒選作評估對象圖像的圖像的話(第4圖的步驟ST7:YES時),回到步驟ST1的處理,重複步驟ST1~ST7的處理。 If there is an image that has not been selected as an evaluation target image among the candidate images determined by the image selection unit 1 (step ST7 in FIG. 4: YES), the process returns to step ST1 and repeats steps ST1 to ST7 Processing.
又,回到步驟ST1的處理,圖像選擇部1選擇的2個圖像中,關於已算出特徵量的圖像,省略特徵量算出部2的特徵量算出處理也可以。以此方式,不用實施特徵量的再計算,可以縮短處理時間。例如,如第5、6圖所說明地,從候補圖像中,依播放順序選擇評估對象圖像,以參照圖像作為在播放順序中比評估對象圖像前1個候補圖像時,評估對象圖像中的特徵量Cn,tc,成為選擇在播放順序 中前1個候補圖像為評估對象圖像之際的參照圖像中的特徵量Cn,tp。因此,先依序保存算出的特徵量Cn,tc,由於特徵量Cn,tp從保存的特徵量讀出,可以省略特徵量的算出處理。 Returning to the process of step ST1, the feature amount calculation process of the feature amount calculation unit 2 may be omitted from the two images selected by the image selection unit 1 for the images whose feature amounts have been calculated. In this way, it is possible to shorten the processing time without performing recalculation of feature quantities. For example, as illustrated in Figs. 5 and 6, when the evaluation target image is selected in the playback order from the candidate images, and the reference image is used as the candidate image before the evaluation target image in the playback order, evaluation The feature amount C n, tc in the target image becomes the feature amount C n, tp in the reference image when the first candidate image in the playback order is selected as the evaluation target image. Therefore, the calculated feature amounts C n, tc are sequentially saved first , and since the feature amounts C n, tp are read from the stored feature amounts, the calculation process of the feature amounts can be omitted.
又,解碼順序與播放順序不同時,依解碼順序選擇評估對象圖像,以參照圖像作為在解碼順序中比評估對象圖像前面而且在播放順序中最接近評估對象圖像的候補圖像,評估有無場景轉換也可以。在此情況下,已算出特徵量的圖像,挪用特徵量算出結果,省略處理也可以。 In addition, when the decoding order is different from the playback order, the evaluation target image is selected according to the decoding order, and the reference image is used as a candidate image that precedes the evaluation target image in the decoding order and is closest to the evaluation target image in the playback order. It is also possible to evaluate whether there is a scene change. In this case, the image in which the feature amount has been calculated, the result of the feature amount calculation may be misapplied, and the process may be omitted.
圖像選擇部1決定的候補圖像中,沒留下還沒選擇作為評估對象圖像的候補圖像的話(第4圖的步驟ST7:NO時),結束一連串的處理。 If there is no candidate image selected as the evaluation target image among the candidate images determined by the image selection unit 1 (when step ST7 in FIG. 4 is NO), the series of processing ends.
根據以上很清楚地,根據此第一實施形態,設置評估值算出部3,根據特徵量算出部2算出的2個圖像內包含的區塊單位的特徵量,算出判定有無場景轉換(scene change)使用的評估值,有無判定部4,藉由比較評估值算出部3算出的評估值與臨界值,判定有無2個圖像間的場景轉換。於是,特徵量算出部2,在圖像選擇部1選擇的圖像的每一區塊,根據上述區塊的編碼量算出上述區塊的特徵量,構成影像處理裝置。因此,影像處理裝置,即使關於不包含移動向量的圖像,也可以判定有無場景轉換。 As is clear from the above, according to this first embodiment, an evaluation value calculation unit 3 is provided, and the presence or absence of a scene change is calculated based on the feature quantity of a block unit included in the two images calculated by the feature quantity calculation unit 2. The evaluation value used by the presence / absence determination unit 4 determines the presence or absence of a scene transition between two images by comparing the evaluation value calculated by the evaluation value calculation unit 3 with a critical value. Then, the feature quantity calculation unit 2 calculates the feature quantity of the block for each block of the image selected by the image selection unit 1 based on the coding amount of the block, and configures an image processing device. Therefore, the image processing apparatus can determine whether or not there is a scene transition even for an image that does not include a motion vector.
此第一實施形態中,顯示給予影像處理裝置的圖像選擇部1編碼視訊串流的實例,但有時給予影像處理裝置收納編碼視訊串流的媒體傳送串流。 In this first embodiment, an example is shown in which the video selection device 1 encodes a video stream to the image processing device, but the image processing device may be given a media transmission stream that stores the encoded video stream.
在此情況下,如第9圖所示,也可以備置多工器5,從媒體傳送串流取出編碼視訊串流,輸出編碼視訊串流至圖像選擇部1。 In this case, as shown in FIG. 9, a multiplexer 5 may be provided to take out the encoded video stream from the media transmission stream and output the encoded video stream to the image selection unit 1.
第9圖係顯示根據此發明的第一實施形態的另一影像處理裝置的構成圖。 FIG. 9 is a configuration diagram showing another image processing apparatus according to the first embodiment of the present invention.
作為媒體傳送串流,例如,假設以MPEG-2 TS(ISO/IEC13818-1/ITU-T H.222.0的Transport Stream(傳輸流))或MMT(ISO/IEC 23008-1)等媒體傳送格式封包化的媒體傳送串流等。 As a media transmission stream, for example, it is assumed that a packet is transmitted in a media transmission format such as MPEG-2 TS (Transport Stream (ISO / IEC13818-1 / ITU-T H.222.0)) or MMT (ISO / IEC 23008-1) Stream, etc.
多工器5,根據給予的媒體傳送串流的標題,可以知道各個存取單元間的段落、NAL單元的種類及尺寸。 The multiplexer 5 can know the type of the paragraph between the access units, the type and size of the NAL unit based on the given title of the media transmission stream.
多工器5,如果給予圖像選擇部1表示各個存取單元間段落的資訊,圖像選擇部1根據多工器5給予的資訊,可以從編碼視訊串流中選擇所希望的圖像。 If the multiplexer 5 gives the image selection unit 1 information indicating the paragraphs between the access units, the image selection unit 1 can select a desired image from the encoded video stream based on the information given by the multiplexer 5.
又,多工器5,如果給予特徵量算出部2指示NAL單元的種類及尺寸的資訊,特徵量算出部2不實施評估對象圖像中的語法解碼處理,可以明確指定上述評估對象圖像的編碼量。 In addition, if the multiplexer 5 gives information indicating the type and size of the NAL unit to the feature quantity calculation unit 2, the feature quantity calculation unit 2 does not perform syntax decoding processing on the evaluation target image, and can explicitly specify the evaluation target image. Encoding amount.
此第一實施形態中顯示,特徵量算出部2在處理區塊的編碼量Sn,t的算出處理中,以圖像全體的處理區塊為對象的實例。 This first embodiment shows an example in which the feature amount calculation unit 2 targets the entire processing block of the image in the calculation process of the coding amount Sn , t of the processing block.
這只不過是一例,為了減輕特徵量算出部2的處理負荷,處理區塊的編碼量Sn,t的算出處理中,只有以圖像內的一部分的處理區塊作為對象也可以。 This is just an example. In order to reduce the processing load of the feature amount calculation unit 2 , in the calculation processing of the coding amount Sn , t of the processing block, only a part of the processing blocks in the image may be targeted.
具體而言,如第10圖所示,分別在行方向及列方向中,只以隔一個的處理區塊作為對象也可以。 Specifically, as shown in FIG. 10, in the row direction and the column direction, only another processing block may be targeted.
第10圖,在構成圖像的處理區塊中,指示明確指定編碼量的處理區塊。 FIG. 10 indicates a processing block that explicitly specifies a coding amount among processing blocks constituting an image.
第10圖中,記述”1”的處理區塊,係明確指定編碼量的處理區塊,記述”0”的處理區塊,係未明確指定編碼量的處理區塊。 In FIG. 10, a processing block describing "1" is a processing block that explicitly specifies a coding amount, and a processing block describing "0" is a processing block that does not explicitly specify a coding amount.
又,如第11圖所示,只以隔1行的處理區塊或隔1列的處理區塊作為對象也可以。 In addition, as shown in FIG. 11, only one processing block in one row or one processing block in one column may be targeted.
第11圖係顯示構成圖像的處理區塊中,明確指定編碼量的處理區塊說明圖。 FIG. 11 is an explanatory diagram showing a processing block that explicitly specifies a coding amount among processing blocks constituting an image.
第11圖中,記述”1”的處理區塊,係明確指定編碼量的處理區塊,記述”0”的處理區塊,係未明確指定編碼量的處理區塊。 In FIG. 11, a processing block describing “1” is a processing block that explicitly specifies a coding amount, and a processing block describing “0” is a processing block that does not explicitly specify a coding amount.
如第12圖所示,處理區塊的編碼量Sn,t的算出處理中,以圖像左上的處理區塊開始依光域掃描順序到特定的處理區塊作為對像,剩下的處理區塊不作為對象也可以。 As shown in FIG. 12 , in the calculation process of the coding amount Sn , t of the processing block, the processing block in the upper left of the image is started to the specific processing block as the object according to the scanning order of the light domain, and the remaining processing Blocks are not required as objects.
第12圖係顯示構成圖像的的處理區塊中,明確指定編碼量的處理區塊說明圖。 FIG. 12 is an explanatory diagram showing a processing block that explicitly specifies an encoding amount among processing blocks constituting an image.
第12圖中,記述”1”的處理區塊,係明確指定編碼量的處理區塊,記述”0”的處理區塊,係未明確指定編碼量的處理區塊。 In FIG. 12, a processing block describing “1” is a processing block that explicitly specifies a coding amount, and a processing block describing “0” is a processing block that does not explicitly specify a coding amount.
又,第10圖到第12圖中,記述”1”的處理區塊與記述”0”的處理區塊,相反也可以。 In FIGS. 10 to 12, the processing block describing “1” and the processing block describing “0” may be reversed.
第二實施形態 Second embodiment
上述第一實施形態中,顯示特徵量算出部2解碼編碼區塊的編碼資料,根據編碼資料的解碼結果明確指定處理區塊的編碼量,根據處理區塊的編碼量,算出處理區塊的特徵量之實例。 In the first embodiment described above, the display feature amount calculation unit 2 decodes the encoded data of the encoding block, explicitly specifies the encoding amount of the processing block based on the decoding result of the encoded data, and calculates the characteristics of the processing block based on the encoding amount of the processing block. Amount of examples.
上述第二實施形態中,特徵量算出部2,在圖像選擇部1選擇的圖像號碼t的圖像Pt的每一處理區塊,也可以使用上述處理區塊內包含的分割區塊個數作為上述處理區塊的特徵量Cn,t。 In the second embodiment described above, the feature amount calculation unit 2 may use the divided blocks included in the processing blocks for each processing block of the image P t of the image number t selected by the image selection unit 1. The number is used as the feature amount C n, t of the processing block.
上述處理區塊內包含的分割區塊個數,藉由加上屬於上述處理區塊的各編碼區塊內包含的分割區塊個數而算出。 The number of divided blocks included in the processing block is calculated by adding the number of divided blocks included in each coding block belonging to the processing block.
編碼區塊內包含的分割區塊個數,例如,編碼方式是AVC/H.264的話,相當於以大區塊型(mb_type)決定的大區塊內的區塊數。大區塊內的區塊數,藉由實施評估對象圖像中的語法解碼處理得到。 The number of divided blocks included in the encoding block. For example, if the encoding method is AVC / H.264, it is equivalent to the number of blocks in the large block determined by the large block type (mb_type). The number of blocks in a large block is obtained by performing a syntax decoding process on an evaluation target image.
編碼方式是HEVC/H.265的話,相當於編碼區塊的CTU內的CU(Coding Unit(編碼單元))數。CTU內的CU數,藉由實施評估對象圖像中的語法解碼處理得到。 If the encoding method is HEVC / H.265, it is equivalent to the number of CUs (Coding Units) in the CTU of the encoding block. The number of CUs in the CTU is obtained by performing syntax decoding processing on the image to be evaluated.
作為處理區塊的特徵量,明確指定上述處理區塊內包含的分割區塊個數時,特徵量算出部2,在算出處理區塊的特徵量之際,不用標準化編碼量的處理,減輕特徵量算出部2的處理負荷。 When the number of divided blocks included in the processing block is explicitly specified as the characteristic amount of the processing block, the characteristic amount calculation unit 2 does not need to process the standardized coding amount to reduce the characteristics when calculating the characteristic amount of the processing block. The processing load of the amount calculation unit 2.
第三實施形態 Third embodiment
上述第一實施形態中顯示,特徵量算出部2,解碼編碼區塊的編碼資料,根據編碼資料的解碼結果明確指定處理區塊的編碼量,根據處理區塊的編碼量,算出處理區塊的特徵量之實例。 The above-mentioned first embodiment shows that the feature amount calculation unit 2 decodes the encoded data of the encoding block, explicitly specifies the encoding amount of the processing block based on the decoding result of the encoded data, and calculates the Examples of feature quantities.
上述第三實施形態中,特徵量算出部2,在圖像選擇部1選擇的圖像號碼t的圖像Pt的每一處理區塊,也可以使用上述處理區塊中的分割區塊的最深階層數作為上述處理區塊的特徵量Cn,t。 In the third embodiment described above, the feature quantity calculation unit 2 may use each of the processing blocks of the image P t of the image number t selected by the image selection unit 1. The number of deepest levels is used as the feature amount C n, t of the processing block.
上述處理區塊中的分割區塊的最深階層數,藉由求出屬於上述處理區塊的各編碼區塊中的分割區塊的最深階層數最大值而算出。 The number of the deepest levels of the divided blocks in the processing block is calculated by obtaining the maximum number of the deepest levels of the divided blocks in each coding block belonging to the processing block.
編碼區塊中的分割區塊的最深階層數,例如,編碼方式是HEVC/H.265的話,如第13圖所示,相當於編碼區塊的CTU內包含的複數的CU中分割階層最深的數量。 The number of the deepest level of the divided block in the coded block. For example, if the encoding method is HEVC / H.265, as shown in Figure 13, it corresponds to the deepest level of the divided CU in the complex CU included in the CTU of the coded block. Quantity.
第13圖係顯示編碼區塊中的分割區塊階層數的說明圖。 FIG. 13 is an explanatory diagram showing the number of divided block levels in a coded block.
第13圖的例中,因為CU depth的最大值是3,編碼區塊中的分割區塊的最深階層數是3。 In the example of FIG. 13, since the maximum value of the CU depth is 3, the number of the deepest levels of the divided blocks in the coding block is 3.
分割區塊的最深階層數,藉由實施評估對象圖像中的語法解碼處理得到。 The number of the deepest levels of the divided blocks is obtained by performing a syntax decoding process on the evaluation target image.
作為處理區塊的特徵量,明確指定上述處理區塊中的分割區塊的最深階層數時,特徵量算出部2,在算出處理區塊的特徵量之際,不用標準化編碼量的處理,減輕特徵量算出部2的處理負荷。 When the deepest level of the divided block in the processing block is explicitly specified as the feature amount of the processing block, the feature amount calculation unit 2 does not need to process the normalized coding amount when calculating the feature amount of the processing block. The processing load of the feature amount calculation unit 2.
第四實施形態 Fourth Embodiment
上述第一實施形態中顯示,評估值算出部3,根據特徵量算出部2算出的參照圖像Ptp中的處理區塊單位的特徵量Cn,tp、評估對象圖像Ptc中的處理區塊單位的特徵量Cn,tc,算出評估值J之實例。 The first embodiment described above shows the processing in the evaluation value calculation unit 3, the feature amount C n, tp in the processing block unit in the reference image P tp calculated by the feature amount calculation unit 2 , and the evaluation target image P tc . An example of calculating the evaluation value J by the feature quantity C n, tc of a block unit.
此第四實施形態中說明,評估值算出部3,根據圖像號碼t=tp的參照圖像Ptp 中的處理區塊單位的特徵量Cn,tp、圖像號碼t=tc的評估對象圖像Ptc中的處理區塊單位的特徵量Cn,tc、以及候補圖像中在播放時刻比參照圖像Ptp前1個的圖像號碼t=tp’的二次參照圖像Ptp’中的處理區塊單位的特徵量Cn,tp’,算出評估值J的實例。 In this fourth embodiment, it is explained that the evaluation value calculation unit 3 evaluates objects based on the feature amount C n, tp of the processing block unit in the reference image P tp of the image number t = tp and the image number t = tc. The feature quantity C n, tc of the processing block unit in the image P tc and the secondary reference image P of the candidate image at the playback time which is an image number t = tp 'one before the reference image P tp An example of calculating the evaluation value J by processing the feature amount C n, tp ' of a block unit in tp' .
又,二次參照圖像Ptp’係參照圖像Ptp的參照圖像,即參照圖像Ptp是評估對象圖像Ptc時的參照圖像Ptp。因此,3個圖像的播放時刻位置關係是tp’<tp<tc。但是,一般,每一GOP的圖像數,因為可變也可以,第6圖的候補圖像的實例中,tp’、tp、tc的各候補圖像間隔(tc-tp、tp-tp’),有可能不同。 The secondary reference picture P tp ′ is a reference picture of the reference picture P tp , that is, the reference picture P tp is a reference picture P tp when the evaluation target picture P tc is used. Therefore, the positional relationship between the playback times of the three images is tp '<tp <tc. However, in general, the number of pictures per GOP may be variable. In the example of the candidate picture in FIG. 6, the interval between the candidate pictures of tp ', tp, and tc (tc-tp, tp-tp' ), May be different.
假設二次參照圖像Ptp’是播放順序中第1號圖像時,參照圖像Ptp是播放順序中第2號圖像,評估對象圖像Ptc是播放順序中第3號圖像。 When the secondary reference image P tp ' is the first image in the playback order, the reference image P tp is the second image in the playback order, and the evaluation target image P tc is the third image in the playback order. .
其次說明關於動作。 Next, the operation will be described.
此第四實施形態中,與上述第一實施形態相同,根據第4圖的流程圖處理,處理內容變更為下述說明的處理。 This fourth embodiment is the same as the first embodiment described above, and the processing content is changed to the processing described below according to the flowchart processing of FIG. 4.
圖像選擇部1,與上述第一實施形態相同,取得包含複數的圖像的編碼資料的編碼視訊串流。 The image selection unit 1 acquires an encoded video stream including encoded data of a plurality of images in the same manner as the first embodiment.
圖像選擇部1,從編碼視訊串流內包含編碼資料的複數的圖像中,根據預先定義的候補圖像的決定方法決定候補圖像,從候補圖像中,與上述第一實施形態相同,選擇參照圖像Ptp以及評估對象圖像Ptc。 The image selection unit 1 determines a candidate image from a plurality of images including encoded data in the encoded video stream according to a predetermined candidate image determination method. The candidate image is the same as the first embodiment described above. , Select a reference image P tp and an evaluation target image P tc .
又,圖像選擇部1,從候補圖像中,選擇二次參照圖像Ptp’。即,第4圖的步驟ST1中,除了評估對象圖像與參照圖像之外,也選擇二次參照圖像。 The image selection unit 1 selects a secondary reference image P tp ′ from among the candidate images. That is, in step ST1 of FIG. 4, in addition to the evaluation target image and the reference image, a secondary reference image is also selected.
特徵量算出部2,與上述第一實施形態相同,在圖像選擇部1選擇的每一圖像,根據上述評估對象圖像內包含的編碼區塊單位的編碼資料,分別算出處理區塊單位的特徵量(第4圖的步驟ST2)。 The feature quantity calculation unit 2 is similar to the first embodiment described above. Each image selected by the image selection unit 1 calculates a processing block unit based on the coding data of the coding block unit included in the evaluation target image. (Step ST2 in FIG. 4).
即,特徵量算出部2,算出參照圖像Ptp內包含的處理區塊單位的特徵量Cn,tp以及評估對象圖像Ptc內包含的處理區塊單位的特徵量Cn,tc。 That is, the feature quantity calculation unit 2 calculates the feature quantity C n, tp of the processing block unit included in the reference image P tp and the feature quantity C n, tc of the processing block unit included in the evaluation target image P tc .
又,特徵量算出部2,算出二次參照圖像Ptp’內包含的處理區塊單位的特徵量Cn,tp’。 In addition, the feature amount calculating section 2 calculates the second reference image P tp 'feature amount C n processing block units contained therein, tp'.
評估值算出部3,根據特徵量算出部2算出的參照圖像Ptp內包含的處理區塊單位的特徵量Cn,tp、評估對象圖像Ptc內包含的處理區塊單位的特徵量Cn,tc以及二次參照圖像Ptp’內包含的處理區塊單位的特徵量Cn,tp’,算出評估值J(第4圖的步驟ST3)。 The evaluation value calculation unit 3 calculates the feature quantity C n, tp of the processing block unit included in the reference image P tp calculated by the feature quantity calculation unit 2 and the feature amount of the processing block unit included in the evaluation target image P tc . C n, tc and the feature amount C n, tp ' of the processing block unit included in the secondary reference image P tp ′ are used to calculate the evaluation value J (step ST3 in FIG. 4).
具體而言,評估值算出部3,算出參照圖像Ptp與評估對象圖像Ptc之間的處理區塊單位的特徵量的差異絕對值和。 Specifically, the evaluation value calculation unit 3 calculates the sum of the absolute values of the differences in the feature amounts of the processing block units between the reference image P tp and the evaluation target image P tc .
又,評估值算出部3,算出參照圖像Ptp與二次參照圖像Ptp’之間的處理區塊單位的特徵量的差異絕對值和。 Further, the evaluation value calculation unit 3 calculates the sum of the absolute values of the differences in the feature amounts of the processing block units between the reference image P tp and the secondary reference image P tp ′ .
評估值算出部3,如以下的式(3)所示,算出算出的雙方的差異絕對值和的差異,作為評估值J。 The evaluation value calculation unit 3 calculates the calculated difference between the absolute value of the difference between both sides as the evaluation value J as shown in the following formula (3).
在此,MIN_NUM_BL’係NUM_BLtp’、NUM_BLtp、NUM_BLtc的3個值的最小值。即,二次參照圖像Ptp’與參照圖像Ptp的處理區塊數不完全相同時,只有3個圖像共同算出特徵量的處理區塊成為評估值J的算出對象。 Here, MIN_NUM_BL 'is the minimum of three values of NUM_BL tp' , NUM_BL tp , and NUM_BL tc . That is, when the number of processing blocks of the secondary reference image P tp ′ and the reference image P tp are not exactly the same, only the processing blocks for which the three images collectively calculate the feature amount become the calculation target of the evaluation value J.
評估值算出部3算出的評估值J,二次參照圖像Ptp’與參照圖像Ptp之間圖案不怎麼變化,處理區塊單位的特徵量變化變小,又,參照圖像Ptp與評估對象圖像Ptc之間圖案大變化而處理區塊單位的特徵量變化變大時,值變大。 The evaluation value J calculated by the evaluation value calculation unit 3 does not change the pattern between the secondary reference image P tp ′ and the reference image P tp much , the change in the feature amount of the processing block unit becomes smaller, and the reference image P tp The value becomes larger when the pattern and the evaluation target image P tc change greatly and the feature amount of the processing block unit changes.
因此,上述第一~三實施形態中,評估參照圖像Ptp與評估對象圖像Ptc之間的圖案變化大小,此第四實施形態中,因為評估參照圖像Ptp中的圖案變化程度與評估對象圖像Ptc中的圖案變化程度之差異大小,降低將不是場景轉換的圖像間的緩和圖案變化誤檢出為畫面全體圖案大變化的場景轉換的可能性,比上述 第一~三實施形態,可以更提高場景轉換的檢出精度。 Therefore, in the first to third embodiments, the magnitude of the pattern change between the reference image P tp and the evaluation target image P tc is evaluated. In this fourth embodiment, the degree of the pattern change in the reference image P tp is evaluated. The magnitude of the difference from the pattern change degree in the evaluation target image P tc reduces the possibility of erroneously detecting a mode change between images that are not a scene change as a scene change that is a large change in the overall pattern of the screen, than the first ~ The three implementation forms can further improve the detection accuracy of scene transitions.
又,成為處理區塊單位特徵量算出對象之處理區塊的定義方法,與上述第一實施形態相同(圖像內全部處理區塊、如第10~12圖的一部分的處理區塊等)可以任意定義。 In addition, the definition method of the processing block that is the target of calculating the feature quantity per processing block is the same as the first embodiment (all processing blocks in the image, such as a part of the processing blocks in Figs. 10 to 12). Arbitrary definition.
有無判定部4,與上述第一實施形態相同,比較評估值算出部3算出的評估值J與事先設定的臨界值Th(第4圖的步驟ST4)。 The presence / absence determination unit 4 compares the evaluation value J calculated by the evaluation value calculation unit 3 with a threshold value Th set in advance, as in the first embodiment (step ST4 in FIG. 4).
有無判定部4,與上述第一實施形態相同,如果評估值算出部3算出的評估值J在臨界值Th以上的話,判定參照圖像Ptp與評估對象圖像Ptc之間有場景轉換(第4圖的步驟ST5)。 The presence / absence determination unit 4 is the same as the first embodiment described above. If the evaluation value J calculated by the evaluation value calculation unit 3 is greater than or equal to the threshold value Th, it is determined that there is a scene transition between the reference image P tp and the evaluation target image P tc ( Step ST5 in FIG. 4).
有無判定部4,與第一實施形態相同,如果評估值算出部3算出的評估值J未滿臨界值Th的話,判定參照圖像Ptp與評估對象圖像Ptc之間無場景轉換(第4圖的步驟ST6)。 The presence / absence determination unit 4 is the same as the first embodiment. If the evaluation value J calculated by the evaluation value calculation unit 3 is less than the critical value Th, it is determined that there is no scene transition between the reference image P tp and the evaluation target image P tc (the Step ST6 in FIG. 4).
有無判定部4,輸出有無場景轉換的判定結果。 The presence / absence determination unit 4 outputs a determination result of the presence or absence of a scene transition.
又,臨界值Th的設定方法,與上述第一實施形態相同。 The method of setting the threshold value Th is the same as that in the first embodiment.
圖像選擇部1決定的候補圖像中,留下還沒選擇作為評估對象圖像的圖像的話(第4圖的步驟ST7:YES時),回到步驟ST1的處理,重複步驟ST1~ST7的處理。 If the candidate image determined by the image selection unit 1 has no image selected as the evaluation target image (step ST7 in FIG. 4: YES), the process returns to step ST1 and repeats steps ST1 to ST7 Processing.
又,回到步驟ST1的處理,圖像選擇部1選擇的3個圖像中,關於已算出特徵量的圖像,省略特徵量算出部2的特徵量算出處理也可以。以此方式,不用實施特徵量的再計算,可以縮短處理時間。 Further, returning to the processing of step ST1, among the three images selected by the image selection unit 1, the feature quantity calculation process of the feature quantity calculation unit 2 may be omitted for the image for which the feature quantity has been calculated. In this way, it is possible to shorten the processing time without performing recalculation of feature quantities.
圖像選擇部1決定的候補圖像中,沒留下還沒選擇作為評估對象圖像的圖像的話(第4圖的步驟ST7:NO時),結束一連串的處理。 If the candidate image determined by the image selection unit 1 does not include an image that has been selected as the evaluation target image (step ST7 in FIG. 4: NO), the series of processing ends.
第五實施形態 Fifth Embodiment
上述第一~四的實施形態中顯示,評估值算出部3,利用式(2)或式(3),根據 特徵量算出部2算出的處理區塊單位的特徵量Cn,t,算出評估值J之實例。 The first to fourth embodiments show that the evaluation value calculation unit 3 calculates the evaluation by using the formula (2) or formula (3) based on the feature amount C n, t of the processing block unit calculated by the feature amount calculation unit 2. Example of value J.
此第五實施形態中說明,評估值算出部3算出評估值J之際,利用處理區塊單位的特徵量Cn,t以及對應上述處理區塊的加權係數wn,算出評估值J之實例。 In the fifth embodiment, when the evaluation value calculation unit 3 calculates the evaluation value J, an example of calculating the evaluation value J using the feature amount C n, t of the processing block unit and the weighting coefficient w n corresponding to the processing block is described. .
此第五實施形態中,變更式(2)為以下的式(4)。或,變更式(3)為以下的式(5)。 In this fifth embodiment, the modified expression (2) is the following expression (4). Alternatively, the formula (3) is changed to the following formula (5).
評估值算出部3,代入處理區塊單位的特徵量Cn,t之特徵量Cn,tc以及特徵量Cn,tp與加權係數wn至式(4),算出評估值J。 The evaluation value calculation unit 3 substitutes the characteristic amounts C n, tc , the characteristic amounts C n, tc , the characteristic amounts C n, tp, and the weighting coefficients w n to Expression (4) of the processing block unit to calculate the evaluation value J.
或者,評估值算出部3,代入處理區塊單位的特徵量Cn,t之特徵量Cn,tc、特徵量Cn,tp以及特徵量Cn,tp’與加權係數wn至式(5),算出評估值J。 Alternatively, the evaluation value calculating unit 3 is substituted into processing block units of the feature amount C n, t of feature amount C n, tc, the feature amount C n, tp and the feature amount C n, tp 'weighting coefficients w n to ( 5) Calculate the evaluation value J.
式(4)及式(5)中,特徵量Cn,tc,係評估對象圖像Ptc中的區塊號碼n的處理區塊的特徵量,特徵量Cn,tp,係評估參照圖像Ptp中的區塊號碼n的處理區塊的特徵量。 In the expressions (4) and (5), the feature amount C n, tc is the feature amount of the processing block of the block number n in the evaluation target image P tc , and the feature amount C n, tp is the evaluation reference diagram. A feature quantity of a processing block like the block number n in P tp .
又,特徵量Cm,tp’係二次參照圖像Ptp’中的圖像號碼n的處理區塊的特徵量。 The feature amount C m, tp ′ is a feature amount of the processing block of the picture number n in the secondary reference picture P tp ′ .
wn,係乘以特徵量Cn,tc與特徵量Cn,tp的差異絕對值的加權係數,或者乘以特徵量Cn,tp與特徵量Cn,tp’的差異絕對值的加權係數。 w n is the weighting coefficient multiplied by the absolute value of the difference between the characteristic amount C n, tc and the characteristic amount C n, tp , or the weighted value of the absolute value of the difference between the characteristic amount C n, tp and the characteristic amount C n, tp ' coefficient.
式(4),可以如以下的式(6)變形。又,式(5),可以如以下的式(7)變形。 Expression (4) can be modified as shown in the following expression (6). The expression (5) can be modified as shown in the following expression (7).
因此,評估值算出部3,利用式(6)或式(7),可以算出評估值J。 Therefore, the evaluation value calculation unit 3 can calculate the evaluation value J using Expression (6) or Expression (7).
評估值算出部3,利用式(6),算出評估值J時,取代評估值算出部3將差異絕對值乘以加權係數wn,特徵量算出部2將特徵量Cn,tc及特徵量Cn,tp分別乘以加權係數wn。 When the evaluation value calculation unit 3 calculates the evaluation value J using Equation (6), instead of the evaluation value calculation unit 3 multiplying the absolute value of the difference by the weighting coefficient w n , the characteristic value calculation unit 2 multiplies the characteristic amounts C n, tc and the characteristic amounts. C n, tp is multiplied by the weighting factor w n , respectively.
評估值算出部3,利用式(7),算出評估值J時,取代評估值算出部3將差異絕對值乘以加權係數wn,特徵量算出部2將特徵量Cn,tc、特徵量Cn,tp及特徵量Cn,tp’分別乘以加權係數wn。 2 the feature amount C n, tc, the evaluation value feature quantity calculating unit 3, using the formula (7), when the calculated evaluation value J, substituted evaluation value calculating section 3 the absolute value of the difference by weighting coefficients w n, the feature amount calculating section C n, tp and feature quantities C n, tp ' are multiplied by a weighting coefficient w n , respectively.
具體而言,評估值算出部3,利用式(6),算出評估值J時,特徵量算出部2把式(1)的常數M換成以下的式(8)所示的變數M’n之後,利用式(1),分別算出特徵量Cn,tc及特徵量Cn,tp。 Specifically, when the evaluation value calculation unit 3 calculates the evaluation value J using the formula (6), the feature quantity calculation unit 2 replaces the constant M of the formula (1) with a variable M ′ n shown in the following formula (8). Then, using the formula (1), the characteristic amounts C n, tc and the characteristic amounts C n, tp are calculated, respectively.
特徵量算出部2,將算出的特徵量Cn,tc與特徵量Cn,tp輸出至評估值算出部3。 The feature quantity calculation unit 2 outputs the calculated feature quantities C n, tc and the feature quantities C n, tp to the evaluation value calculation unit 3.
從特徵量算出部2輸出的特徵量Cn,tc,相當於式(6)所示的wnCn,tc,從特徵量算出部2輸出的特徵量Cn,tp,相當於式(6)所示的wnCn,tp。 The feature quantity C n, tc output from the feature quantity calculation unit 2 corresponds to w n C n, tc shown in Equation (6) , and the feature quantity C n, tp output from the feature quantity calculation unit 2 corresponds to formula ( 6) as shown by w n C n, tp .
評估值算出部3,利用式(7),算出評估值J時,特徵量算出部2把式(1)的常數M換成以下的式(8)所示的變數M’n之後,利用式(1),分別算出特徵量Cn,tc、特徵量Cn,tp以及特徵量Cn,tp’。 When the evaluation value calculation unit 3 calculates the evaluation value J using equation (7), the characteristic quantity calculation unit 2 replaces the constant M of equation (1) with the variable M ′ n shown in the following equation (8), and then uses the equation (1) Calculate the feature amounts C n, tc , the feature amounts C n, tp, and the feature amounts C n, tp 'respectively .
特徵量算出部2,將算出的特徵量Cn,tc、特徵量Cn,tp以及特徵量Cn,tp’輸出至評估值算出部3。 The feature quantity calculation unit 2 outputs the calculated feature quantity C n, tc , the feature quantity C n, tp, and the feature quantity C n, tp ′ to the evaluation value calculation unit 3.
從特徵量算出部2輸出的特徵量Cn,tc,相當於式(7)所示的wnCn,tc,從特徵量算出部2輸出的特徵量Cn,tp,相當於式(7)所示的wnCn,tp。又,從特徵量算出部2輸出的特徵量Cn,tp’,相當於式(7)所示的wnCn,tp’。 The feature quantity C n, tc output from the feature quantity calculation unit 2 corresponds to w n C n, tc shown in Equation (7) , and the feature quantity C n, tp output from the feature quantity calculation unit 2 corresponds to formula ( 7) w n C n, tp shown . The feature quantity C n, tp ' output from the feature quantity calculation unit 2 corresponds to w n C n, tp' shown in Equation (7).
評估值算出部3,將特徵量算出部2輸出的特徵量Cn,tc作為wnCn,tc、特徵量算出部2輸出的特徵量Cn,tp作為wnCn,tp,代入式(6),算出評估值J。 The evaluation value calculation unit 3 uses the feature quantities C n, tc output by the feature quantity calculation unit 2 as w n C n, tc and the feature quantities C n, tp output by the feature quantity calculation unit 2 as w n C n, tp , and substitutes them into Equation (6) calculates an evaluation value J.
或者,評估值算出部3,將特徵量算出部2輸出的特徵量Cn,tc作為wnCn,tc、特 徵量算出部2輸出的特徵量Cn,tp作為wnCn,tp、特徵量算出部2輸出的特徵量Cn,tp’作為wnCn,tp’,代入式(7),算出評估值J。 Alternatively, the evaluation value calculation unit 3 sets the feature quantity C n, tc output by the feature quantity calculation unit 2 as w n C n, tc and the feature quantity C n, tp output by the feature quantity calculation unit 2 as w n C n, tp 2. The feature quantity C n, tp ' output by the feature quantity calculation unit 2 is taken as w n C n, tp' and substituted into equation (7) to calculate the evaluation value J.
在此,每一處理區塊的加權係數wn,指示處理區塊的重要度,加權係數wn的值越大的處理區塊,檢出場景轉換之際,意味越重要的處理區塊。 Here, the weighting coefficient w n of each processing block indicates the importance of the processing block. A processing block with a larger value of the weighting coefficient w n means a more important processing block when a scene change is detected.
因此,每一處理區塊的加權係數wn,在想要檢出的場景的變化類型中,越是圖案變化不連續或變化容易變陡峭的處理區塊,只要設計越大的值即可。 Therefore, the weighting coefficient w n of each processing block, in the type of scene change to be detected, is the processing block with a more discontinuous pattern change or a steeper change, as long as a larger value is designed.
以下,舉出加權係數wn的具體例。 Specific examples of the weighting coefficient w n are given below.
如以下的式(9)所示,以處理區塊的2次元座標(Xn,Yn)為變數的函數h(Xn,Yn),定義加權係數wn。 As shown in the following formula (9), a function h (X n , Y n ) is defined as a function h (X n , Y n ) with a variable coordinate (X n , Y n ) of a processing block as a variable, and a weighting coefficient w n is defined.
w n =h(X n ,Y n ) (9) w n = h ( X n , Y n ) (9)
例如,圖像內包含的複數的處理區塊,如第16圖所示時,(Xn,Yn),表示區塊號碼n的處理區塊的2次元座標(Xn:水平方向(x)成分,Yn:垂直方向(y)成分)。 For example, when a plurality of processing blocks included in an image are shown in FIG. 16, (X n , Y n ) represents a 2D coordinate (X n : horizontal direction (x ) Component, Y n : vertical (y) component).
第16圖係顯示處理區塊的水平方向與垂直方向的座標說明圖。 FIG. 16 is an explanatory diagram showing the horizontal and vertical coordinates of the processing block.
第16圖的例中,以左上的處理區塊(0,0)作為基準的處理區塊,基準的處理區塊的2次元座標係(X1,Y1)。 In the example of FIG. 16, the processing block using the upper left processing block (0,0) as a reference, and the reference processing block's 2-dimensional coordinate system (X 1 , Y 1 ).
例如,處理區塊(5,0)的2次元座標,係(X6,Y6),處理區塊(2,3)的2次元座標,係(X21,Y21)。 For example, the 2-dimensional coordinate of the processing block (5,0) is (X 6 , Y 6 ), and the 2-dimensional coordinate of the processing block (2, 3) is (X 21 , Y 21 ).
例如,定義場景轉換為「轉移至畫面的中央部分的拍攝對象不連續變化」時,以下的式(10)所示的函數h(x,y)表示的加權係數wn,設定為越接近畫面中央的處理區塊,值變得越大。 For example, when the scene transition is defined as "discontinuous changes in the subject moving to the center of the screen", the weighting coefficient w n represented by the function h (x, y) shown in the following formula (10) is set to be closer to the screen The central processing block, the larger the value becomes.
式(10)中,W係以處理區塊分割圖像全體時的水平方向的處理區塊數,H係以處理區塊分割圖像全體時的垂直方向的處理區塊數,Ax、Ay、Lx、Ly分別是正的常數。又,Ax、Ay分別具有x成分、y成分的加權縮放比例值的作用,值越大,離畫面中央的距離越隔開時的加權降低比例變高。 In Equation (10), W is the number of processing blocks in the horizontal direction when the entire image is divided by the processing block, and H is the number of processing blocks in the vertical direction when the entire image is divided by the processing block. A x , A y , L x , and L y are positive constants, respectively. In addition, A x and A y have the function of weighted scaling ratio values of the x component and the y component, respectively. The larger the value is, the higher the distance from the center of the screen becomes when the weight reduction ratio increases.
fi(x),係任意的單調增加函數。作為fi(x)例,考慮以下的式(11)或式(12)表示的函數。 f i (x) is an arbitrary monotonic increasing function. As an example of f i (x), a function represented by the following formula (11) or (12) is considered.
f i (x)=a i x (11) f i ( x ) = a i x (11)
f i (x)=b ix 2 (12) f i ( x ) = b ix 2 (12)
式(11)或式(12)中,ai及bi,分別是正的常數。 In formula (11) or formula (12), a i and b i are positive constants, respectively.
例如,W=6,H=4,圖像內包含的複數的處理區塊,如第16圖所示,單調增加函數fi(x),為式(11)所示的函數。 For example, if W = 6 and H = 4, the complex number of processing blocks contained in the image are monotonically increasing the function f i (x) as shown in FIG. 16, which is a function shown in equation (11).
此時,例如,a1=a2=1、Ax=1、Ay=1、Lx=(W/2)-1=2、Ly=(H/2)-1=1的話,如第17圖所示,畫面中心的處理區塊的加權係數為最大的wn=1,越遠離畫面中心的處理區塊的加權係數,成為越小的加權係數。另一方面,使Lx、Ly的值為小的值。例如,a1=a2=1、Ax=1、Ay=1、Lx=1、Ly=1的話,各個處理區塊的加權係數wn,如第18圖所示,在畫面中央部分加權係數為wn=1,加權係數大的處理區塊數,比第17圖的情況更增加。 At this time, for example, if a 1 = a 2 = 1, A x = 1, A y = 1, L x = (W / 2) -1 = 2, and L y = (H / 2) -1 = 1, As shown in FIG. 17, the weighting coefficient of the processing block in the center of the screen is the largest w n = 1, and the weighting coefficient of the processing block further away from the center of the screen becomes a smaller weighting coefficient. On the other hand, the values of L x and L y are made small. For example, if a 1 = a 2 = 1, A x = 1, A y = 1, L x = 1, and L y = 1, the weighting coefficient w n of each processing block is shown in FIG. 18 on the screen. The weighting coefficient of the central part is w n = 1, and the number of processing blocks with a large weighting coefficient is larger than that in the case of FIG. 17.
第17圖及第18圖,係顯示每一處理區塊的加權係數wn的一實例說明圖。 17 and 18 are explanatory diagrams showing an example of the weighting coefficient w n of each processing block.
第17圖中,例如,處理區塊(5,0)的加權係數wn是1/√6,處理區塊(1,1)的加權係數wn是1/√2,處理區塊(3,1)的加權係數wn是1。 FIG. 17, for example, processing block (5,0) is the weighting coefficient w n 1 / √6, processing blocks (1,1) is the weighting coefficient w n 1 / √2, processing block (3 , 1) The weighting factor w n is 1.
第18圖中,例如,處理區塊(5,0)的加權係數wn是1/√3,處理區塊(1,1)以及處理區塊(3,1)的加權係數wn是1。 FIG. 18, for example, processing block (5,0) is the weighting coefficient w n 1 / √3, processing blocks (1,1) and a processing block (3,1) is a weighting coefficient w n .
又,關於式(10),fi(x)係以式(11)表示的函數,a1=a2=1、Lx=(W/2)-1、Ly=(H/2)-1時,式(10),可以如同式(13)變形。藉此,不用分依照x、y值的情況。 Regarding formula (10), f i (x) is a function represented by formula (11), a 1 = a 2 = 1, L x = (W / 2) -1, and L y = (H / 2) At -1, equation (10) can be transformed like equation (13). Thereby, it is not necessary to distinguish the cases according to the x and y values.
又,加權係數wn,代替式(10)所示的函數h(x,y),為以下的式(14)所示的h(x,y)也可以。 The weighting coefficient w n may be h (x, y) shown in the following formula (14) instead of the function h (x, y) shown in the formula (10).
加權係數wn以式(14)所示的函數h(x,y)表示時,因為沒有平方的計算以及平方根的計算,比加權係數wn以式(10)所示的函數h(x,y)表示時,可以更降低計算負荷。 When the weighting coefficient w n is expressed by the function h (x, y) shown in Equation (14), because there is no calculation of square and the calculation of the square root, the weighting coefficient w n is expressed by the function h (x, y) can reduce the calculation load even more.
與式(10)的例相同,例如,a1=a2=1、Lx=(W/2)-1、Ly=(H/2)-1的話,畫面中心的處理區塊的加權係數,成為最大的wn=1,越遠離畫面中心的處理區塊的加權係數,成為越小的加權係數。另一方面,使Lx、Ly的值為小的值。例如,a1=a2=1、Lx=1、Ly=1的話,各個處理區塊的加權係數wn,在畫面中央部分加權係數為wn=1,加權係數大的處理區塊數增加。 It is the same as the example of equation (10). For example, if a 1 = a 2 = 1, L x = (W / 2) -1, and L y = (H / 2) -1, the weight of the processing block in the center of the screen is weighted. The coefficient becomes the largest w n = 1, and the weight coefficient of the processing block farther from the center of the screen becomes the smaller weight coefficient. On the other hand, the values of L x and L y are made small. For example, if a 1 = a 2 = 1, L x = 1, and L y = 1, the weighting coefficient w n of each processing block is the weighting coefficient w n = 1 in the center of the screen, and the processing block with the larger weighting coefficient is The number increases.
又,關於式(14),fi(x)係以式(11)表示的函數,a1=a2=1、Lx=(W/2)-1、Ly=(H/2)-1時,式(14),可以如式(15)變形。藉此,不用分依照x、y值的情況。 Regarding formula (14), f i (x) is a function represented by formula (11), a 1 = a 2 = 1, L x = (W / 2) -1, and L y = (H / 2) At -1, equation (14) can be transformed as equation (15). Thereby, it is not necessary to distinguish the cases according to the x and y values.
又,為了削減式(10)及式(14)的計算負荷,意味x成分、y成分的加權的縮放比例值之Ax、Ay,分別如式(16)及式(17)所示,可以是位移演算。藉此方式,縮放比例的精度,成為整數精度,但縮放比例處理,變成可以以位元位移演算。 In addition, in order to reduce the calculation load of the expressions (10) and (14), Ax and Ay, which are weighted scaling values of the x component and the y component, are shown in formulas (16) and (17), respectively, and can be Displacement calculation. In this way, the accuracy of the scaling ratio becomes integer precision, but the processing of the scaling ratio can be calculated by bit shift.
在此,Bx、By,表示位移量,值越大,離畫面中央的距離越隔開時的加權降低比例變高。又,式(16)以及式(17)中,定義左位移,而如式(18)及式(19)所示,分別採用右位移也可以。 Here, B x, B y, represents the amount of displacement, the larger the value, the more spaced from the weights when the distance from the center of the screen to reduce the proportion increases. In formulas (16) and (17), the left displacement is defined. As shown in formulas (18) and (19), the right displacements may be used.
式(18)以及式(19),與式(16)以及式(17)相同,Bx、By,表示位移量,值越大,但值越大,離畫面中央的距離越隔開時的加權降低比例變低方面,與式(16)以及式(17)不同。 , Formula (16) and the same formula (18) and (19) Formula (17), B x, B y, represents the amount of displacement, the larger the value, the larger the value, when the spaced distance from the center of the screen The ratio of the weight reduction is lower than that of equations (16) and (17).
又,上述說明的「根據Bx、By的位移演算Ax、Ay」。也可以適用於式(13)以及式(15)。 Also, the above described "according to the displacement calculation Ax B x, B y's, Ay." It is also applicable to Formula (13) and Formula (15).
又,如第12圖所示,處理區塊的編碼量Sn,t的算出處理中,構成圖像的處理區塊中,從圖像左上的處理區塊開始依光域掃描順序到特定的處理區塊作為算出對像。於是,關於剩下的處理區塊不作為算出對象時,或者,以解碼處理時間等的限制能進行編碼量Sn,t的算出處理的處理區塊數在每一圖像變化時,Lx與Ly,成為評估值J的算出單位可變也可以。 In addition, as shown in FIG. 12 , in the calculation process of the coding amount Sn , t of the processing block, among the processing blocks constituting the image, the processing block in the upper left of the image starts from the processing block in the light area to the specific The processing block is used as a calculation object. Therefore, when the remaining processing blocks are not the object of calculation, or the number of processing blocks that can be used to calculate the encoding amount S n, t is limited by the limitation of the decoding processing time, etc., when each image changes, L x and L y may be a variable calculation unit for the evaluation value J.
例如,可以實施編碼量Sn,t的算出處理的最後處理區塊的2次元座標為(XMIN_NUM_BL,YMIN_NUM_BL)(或者(XMIN_NUM_BL’,YMIN_NUM_BL’))時,Lx固定在(W/2)-1。另一方面,Ly,在YMIN_NUM_BL(或者YMIN_NUM_BL’)比(H/2)-1小的情況下,成為Ly=YMIN_NUM_BL(或者YMIN_NUM_BL’),此外,Ly=(H/2)-1。藉此,算出評估值J之際的MIN_NUM_BL(或者MIN_NUM_BL’)小的情況下,比起以Ly=(H/2)-1固定的情況,因為各個處理區塊的加權係數wn變大而評估值J的值變大,可以提高有無判定部4的場景轉換檢出感度。 For example, when the 2nd-dimensional coordinate of the last processing block that can perform calculation processing of the encoding amount Sn , t is (X MIN_NUM_BL , Y MIN_NUM_BL ) (or (X MIN_NUM_BL ' , Y MIN_NUM_BL' )), L x is fixed at (W / 2) -1. On the other hand, L y becomes L y = Y MIN_NUM_BL (or Y MIN_NUM_BL ' ) when Y MIN_NUM_BL (or Y MIN_NUM_BL' ) is smaller than (H / 2) -1. In addition, L y = (H / 2) -1. With this, when the MIN_NUM_BL (or MIN_NUM_BL ') when the evaluation value J is calculated, the weighting coefficient w n of each processing block becomes larger than the case where it is fixed with Ly = (H / 2) -1. On the other hand, the value of the evaluation value J becomes large, and the scene transition detection sensitivity of the presence / absence determination unit 4 can be increased.
上述中,定義場景轉換為「轉移至畫面的中央部分的拍攝對象不連續變化」,但也可以定義場景轉換為「畫面的背景不連續變化」。 In the above, it is defined that the scene is converted into "discontinuous changes in the subject moving to the center of the screen", but the scene can also be defined as "discontinuous changes in the background of the screen".
定義場景轉換為「畫面的背景不連續變化」時,設定加權係數wn,越接近背景容易映照的畫面端的處理區塊,值變越大。 When the definition scene is changed to "the background of the screen changes discontinuously", the weighting coefficient w n is set, and the value becomes larger as it approaches the processing block on the screen side where the background is easily reflected.
定義場景轉換為「畫面的背景不連續變化」時,加權係數wn,取代式(10)所示的函數h(x,y),可以用以下的式(20)所示的函數h(x,y)表示。 When the scene is converted to "the background of the screen changes discontinuously", instead of the function h (x, y) shown in equation (10), the weighting coefficient w n can be used as the function h (x) shown in the following equation (20) , y) said.
或者,加權係數wn,取代式(14)所示的函數h(x,y),可以用以下的式(21)所示的函數h(x,y)表示。 Alternatively, instead of the function h (x, y) represented by the formula (14), the weighting coefficient w n can be expressed by the function h (x, y) represented by the following formula (21).
有無判定部4中的臨界值Th的設定方法,與上述第一實施形態相同也可以。又,作為另外的例,MIN_NUM_BL(或者MIN_NUM_BL’)每一評估對象圖像Ptc更換時,根據加權係數wn以可以求出的式(22)算出臨界值Th也可以。以此方式,設定臨界值,加進每一評估對象圖像Ptc的MIN_NUM_BL(或者MIN_NUM_BL’)的變動產生的評估值J能取得的最大值的變動,可以實現高精度的場景轉換檢出處理。 The method of setting the threshold value Th in the presence / absence determination unit 4 may be the same as that in the first embodiment. As another example, when the MIN_NUM_BL (or MIN_NUM_BL ') is changed for each evaluation target image P tc , the critical value Th may be calculated based on the weighting coefficient w n in the formula (22) which can be obtained. In this way, by setting the critical value and adding the change in the maximum value of the evaluation value J generated by the change in the MIN_NUM_BL (or MIN_NUM_BL ') of each evaluation target image P tc , it is possible to achieve a highly accurate scene transition detection process .
但是,Th_Base,係比成為臨界值Th的基準值之預先設定的0大的常數。又, 評估值J的算出使用MIN_NUM_BL’時,替換式(22)的MIN_NUM_BL為MIN_NUM_BL’。 However, Th_Base is a constant larger than 0, which is a preset value that becomes a reference value of the threshold value Th. When MIN_NUM_BL 'is used for the calculation of the evaluation value J, MIN_NUM_BL of the replacement formula (22) is MIN_NUM_BL'.
又,將本方法與上述第一實施形態中記載的方法組合的方法也可以。例如,將本方法與隨著串流的編碼量推移順應算出臨界值Th的方法組合的方法等。此時,藉由將隨著串流的編碼量推移順應算出的臨界值Th替換為Th_Base可以實現。 A method of combining this method with the method described in the first embodiment may be used. For example, a method of combining the present method with a method of calculating the threshold value Th in accordance with the change in the amount of encoding of the stream. At this time, it can be realized by replacing the threshold value Th calculated according to the encoding amount of the stream with Th_Base.
第六實施形態 Sixth embodiment
上述第五實施形態,顯示利用每一區塊的加權係數wn,算出評估值J的實例。 The fifth embodiment described above shows an example in which the evaluation value J is calculated using the weighting coefficient w n of each block.
此第六實施形態,顯示利用每一區塊的加權係數wm,算出評估值J的實例。 This sixth embodiment shows an example in which the evaluation value J is calculated using the weighting coefficient w m of each block.
首先,特徵量算出部2,取代每一區塊的加權係數wn,取得每一編碼區塊的加權係數wm。m係表示依光域掃描順序編號的編碼區塊的區塊號碼的變數,wm,係關於區塊號碼m的編碼區塊的加權係數。 First, the feature quantity calculation unit 2 obtains the weighting coefficient w m of each coding block instead of the weighting coefficient w n of each block. m is a variable representing a block number of a coded block numbered according to the scanning order of the optical domain, and w m is a weighting coefficient for a coded block with a block number m.
特徵量算出部2,替換式(1)的常數M為以下的式(23)所示的變數M’m之後,利用式(1),分別算出區塊號碼m的編碼區塊中的特徵量Cm,tc以及特徵量Cm,tp,或者分別算出特徵量Cm,tc、特徵量Cm,tp以及特徵量Cm,tp’。 The feature quantity calculation unit 2 calculates the feature quantity in the coded block of the block number m using formula (1) after replacing the constant M of the formula (1) with the variable M ′ m shown in the following formula (23). C m, tc and feature quantity C m, tp , or calculate feature quantity C m, tc , feature quantity C m, tp and feature quantity C m, tp 'respectively .
M' m =w m M (23) M ' m = w m M (23)
特徵量算出部2,利用變數M’m,算出特徵量Cm,t(Cm,tc、Cm,tp或Cm,tp’)時,取代式(1)中的區塊號碼n的處理區塊編碼量Sn,t,使用區塊號碼m的編碼區塊的編碼量。 When the feature quantity calculation unit 2 calculates the feature quantity C m, t (C m, tc , C m, tp, or C m, tp ' ) using the variable M ′ m , it replaces the block number n in the formula (1). The block encoding amount Sn , t is processed using the encoding amount of the encoding block of the block number m.
特徵量算出部2,將算出的特徵量Cm,tc以及特徵量Cm,tp輸出至評估值算出部3。 The feature quantity calculation unit 2 outputs the calculated feature quantities C m, tc and the feature quantities C m, tp to the evaluation value calculation unit 3.
或者,特徵量算出部2,將算出的特徵量Cm,tc、特徵量Cm,tp以及特徵量Cm,tp’輸出至評估值算出部3。 Alternatively, the feature quantity calculation unit 2 outputs the calculated feature quantity C m, tc , the feature quantity C m, tp, and the feature quantity C m, tp ′ to the evaluation value calculation unit 3.
評估值算出部3,在式(6)中,取代wnCn,tc,使用Cm,tc,取代wnCn,tp,使用Cm, tp,算出評估值J。 The evaluation value calculation unit 3 calculates the evaluation value J in formula (6) in place of w n C n, tc , uses C m, tc , instead of w n C n, tp , and uses C m, tp .
或者,評估值算出部3,在式(7)中,取代wnCn,tc,使用Cm,tc,取代wnCn,tp,使用Cm,tp,取代wnCn,tp’,使用Cm,tp’,算出評估值J。 Alternatively, the evaluation value calculation unit 3 replaces w n C n, tc in formula (7), uses C m, tc , replaces w n C n, tp , uses C m, tp , and replaces w n C n, tp ' , Use C m, tp' to calculate the evaluation value J.
此第六實施形態中,因為使用每一編碼區塊的加權係數wm,算出評估值J,利用比處理區塊更小的編碼區塊單位,可以調節各區域的重要度,並可以提高場景轉換的檢出性能。 In this sixth embodiment, because the weighting coefficient w m of each coding block is used to calculate the evaluation value J, the coding block unit smaller than the processing block can be used to adjust the importance of each region and improve the scene. Conversion checkout performance.
又,本申請發明在其發明範圍內,各實施形態的自由組合或各實施形態的任意構成要素的變形或者各實施形態中任意的構成要素的省略是可能的。 In addition, within the scope of the invention of the present application, it is possible to freely combine each embodiment or to modify any constituent element of each embodiment or to omit any constituent element in each embodiment.
此發明,適於判定有無場景變化點的場景轉換之影像處理裝置及影像處理方法。 The invention is suitable for an image processing device and an image processing method for determining whether there is a scene change point of a scene change point.
Claims (11)
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2017/038339 WO2019082268A1 (en) | 2017-10-24 | 2017-10-24 | Image processing device, and image processing method |
??PCT/JP2017/038339 | 2017-10-24 | ||
PCT/JP2018/013789 WO2019082413A1 (en) | 2017-10-24 | 2018-03-30 | Image processing device, and image processing method |
??PCT/JP2018/013789 | 2018-03-30 |
Publications (2)
Publication Number | Publication Date |
---|---|
TW201917631A TW201917631A (en) | 2019-05-01 |
TWI673652B true TWI673652B (en) | 2019-10-01 |
Family
ID=66247342
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW107127425A TWI673652B (en) | 2017-10-24 | 2018-08-07 | Image processing device and image processing method |
Country Status (3)
Country | Link |
---|---|
JP (1) | JP6782789B2 (en) |
TW (1) | TWI673652B (en) |
WO (2) | WO2019082268A1 (en) |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101088280A (en) * | 2004-12-23 | 2007-12-12 | 英特尔公司 | Method and algorithm for detection of scene cuts or similar images in video images |
TW200822739A (en) * | 2006-11-10 | 2008-05-16 | Benq Corp | Method for chaptering an image datum according to scene change |
US20080259218A1 (en) * | 2007-04-20 | 2008-10-23 | Sony Corporation | Apparatus and method of processing image as well as apparatus and method of generating reproduction information |
TW200915866A (en) * | 2007-09-25 | 2009-04-01 | Himax Tech Ltd | Display parameter adjusting method and apparatus for scene change compensation |
CN101681518A (en) * | 2007-05-21 | 2010-03-24 | 三菱电机株式会社 | Image difference detection method and apparatus, scene change detection method and apparatus, as well as image difference value detection method and apparatus |
CN101873495A (en) * | 2009-04-22 | 2010-10-27 | 三菱电机株式会社 | Scene transition detecting device and image recording device |
CN101909511A (en) * | 2008-01-09 | 2010-12-08 | 奥林巴斯株式会社 | Scene change detection device and scene change detection program |
US8170269B2 (en) * | 2006-03-07 | 2012-05-01 | Sony Corporation | Image processing apparatus, image processing method, and program |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0636026A (en) * | 1992-07-13 | 1994-02-10 | Toshiba Corp | Moving picture processor |
JP3315766B2 (en) * | 1992-09-07 | 2002-08-19 | 富士通株式会社 | Image data encoding method, image data encoding device using the method, image data restoring method, image data restoring device using the method, scene change detecting method, scene change detecting device using the method, scene change recording Device and image data scene change recording / reproducing device |
JPH07152779A (en) * | 1993-11-29 | 1995-06-16 | Toshiba Corp | Processing method for detecting moving picture index and moving picture processor having moving picture index detection processing function |
JP3175632B2 (en) * | 1997-04-18 | 2001-06-11 | 松下電器産業株式会社 | Scene change detection method and scene change detection device |
JP2001251630A (en) * | 2000-03-06 | 2001-09-14 | Nec Corp | Image signal coder and its method |
JP2012142645A (en) * | 2009-04-28 | 2012-07-26 | Mitsubishi Electric Corp | Audio/video reproducing apparatus, audio/video recording and reproducing apparatus, audio/video editing apparatus, audio/video reproducing method, audio/video recording and reproducing method, and audio/video editing apparatus |
US10110916B2 (en) * | 2014-04-22 | 2018-10-23 | Sony Corporation | Encoding device and encoding method |
JP6373681B2 (en) * | 2014-07-30 | 2018-08-15 | Kddi株式会社 | Objective image quality evaluation apparatus, objective image quality evaluation method, and program |
JP6588745B2 (en) * | 2015-06-24 | 2019-10-09 | 日本放送協会 | Scene change detection device and program thereof |
-
2017
- 2017-10-24 WO PCT/JP2017/038339 patent/WO2019082268A1/en active Application Filing
-
2018
- 2018-03-30 WO PCT/JP2018/013789 patent/WO2019082413A1/en active Application Filing
- 2018-03-30 JP JP2018550470A patent/JP6782789B2/en active Active
- 2018-08-07 TW TW107127425A patent/TWI673652B/en not_active IP Right Cessation
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101088280A (en) * | 2004-12-23 | 2007-12-12 | 英特尔公司 | Method and algorithm for detection of scene cuts or similar images in video images |
US8170269B2 (en) * | 2006-03-07 | 2012-05-01 | Sony Corporation | Image processing apparatus, image processing method, and program |
TW200822739A (en) * | 2006-11-10 | 2008-05-16 | Benq Corp | Method for chaptering an image datum according to scene change |
US20080259218A1 (en) * | 2007-04-20 | 2008-10-23 | Sony Corporation | Apparatus and method of processing image as well as apparatus and method of generating reproduction information |
CN101681518A (en) * | 2007-05-21 | 2010-03-24 | 三菱电机株式会社 | Image difference detection method and apparatus, scene change detection method and apparatus, as well as image difference value detection method and apparatus |
TW200915866A (en) * | 2007-09-25 | 2009-04-01 | Himax Tech Ltd | Display parameter adjusting method and apparatus for scene change compensation |
CN101909511A (en) * | 2008-01-09 | 2010-12-08 | 奥林巴斯株式会社 | Scene change detection device and scene change detection program |
CN101873495A (en) * | 2009-04-22 | 2010-10-27 | 三菱电机株式会社 | Scene transition detecting device and image recording device |
Also Published As
Publication number | Publication date |
---|---|
WO2019082413A1 (en) | 2019-05-02 |
WO2019082268A1 (en) | 2019-05-02 |
TW201917631A (en) | 2019-05-01 |
JP6782789B2 (en) | 2020-11-11 |
JPWO2019082413A1 (en) | 2019-11-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108182421B (en) | Video segmentation method and device | |
KR101779313B1 (en) | Method for depth lookup table signaling | |
JP6016332B2 (en) | Image processing apparatus and image processing method | |
TW201220855A (en) | Image encoding method and apparatus, image decoding method and apparatus, and programs therefor | |
CN103873865A (en) | Moving image coding apparatus, method and program | |
CN109194955B (en) | Scene switching detection method and system | |
KR20170030968A (en) | Method and apparatus for processing image | |
JP2006270301A (en) | Scene change detecting apparatus and scene change detection program | |
TWI673652B (en) | Image processing device and image processing method | |
JP5178616B2 (en) | Scene change detection device and video recording device | |
KR101982258B1 (en) | Method for detecting object and object detecting apparatus | |
US8520742B2 (en) | Moving image compression-coding device, method of compression-coding moving image, and H.264 moving image compression-coding device | |
US20110261878A1 (en) | Bit rate control method and apparatus for image compression | |
US20150154759A1 (en) | Method, image processing device, and computer program product | |
KR102098322B1 (en) | Method and Device of Motion Estimation for Depth Video Coding by Plane Modeling, and NON-TRANSITORY COMPUTER READABLE RECORDING MEDIUM | |
TW201208385A (en) | Code amount control method and apparatus | |
Chan et al. | Improve load balancing and coding efficiency of tiles in high efficiency video coding by adaptive tile boundary | |
JP4154459B2 (en) | Digital video processing method and apparatus | |
EP2109047A1 (en) | Video characterization, identification and search system | |
KR102329383B1 (en) | Method, Device And Non-Transitory Computer Readable Recording Medium For Depth Picture Encoding Using Plane Modeling Coefficient Prediction | |
US20160071281A1 (en) | Method and apparatus for segmentation of 3d image data | |
KR101492060B1 (en) | Method for overall transforming color of video | |
KR102262030B1 (en) | VIDEO PROCESSING Device and Method For Depth Video by Spherical Modeling, and Non-Transitory COMPUTER READABLE RECORDING MEDIUM | |
KR102224321B1 (en) | Coding Method and Device for Depth Video Plane Modeling | |
KR102224315B1 (en) | Intra Prediction Coding Method and Device for Depth Image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
MC4A | Revocation of granted patent |