JP6782789B2 - Image processing device and image processing method - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method for determining the presence or absence of a scene change, which is a scene change point.

The following Patent Document 1 discloses an image processing device that determines the presence or absence of a scene change.
This image processing device acquires a motion vector, which is one of the coding parameters of a plurality of pictures, from the coded video stream, and determines whether or not there is a scene change from the difference of the motion vector between the plurality of pictures. I have to.

International Publication No. 2010/125757

Since the conventional image processing device is configured as described above, if a motion vector is included as a picture coding parameter in the coded video stream, it is possible to determine whether or not there is a scene change. .. However, since the coding mode of the picture is, for example, the in-screen coding mode, the motion vector may not be included as the coding parameter of the picture. There is a problem that it is not possible to determine the presence or absence of a scene change for a picture that does not include a motion vector as a coding parameter.

The present invention has been made to solve the above problems, and it is desired to obtain an image processing device and an image processing method capable of determining the presence or absence of a scene change even for a picture that does not include a motion vector. The purpose.

The image processing apparatus according to the present invention, from among a plurality of pictures that have been processed coded in a manner that coded marks Goka blocks, as a target picture to assess the presence or absence of a scene change is a change point of scenes, For each of the picture selection unit that selects two pictures and the two pictures selected by the picture selection unit, one or more coding blocks indicating the coding processing units included in the pictures are independently included between the two pictures. The presence or absence of a scene change is determined from the feature amount calculation unit that calculates the feature amount of each processing block unit defined in the same size and the feature amount of the processing block unit included in the two pictures calculated by the feature amount calculation unit. The evaluation value calculation unit that calculates the evaluation value used for the above, and the presence / absence judgment unit that determines the presence / absence of a scene change between the two pictures by comparing the evaluation value calculated by the evaluation value calculation unit with the threshold value. The feature amount calculation unit prepares the processing block for each processing block of the picture selected by the picture selection unit from the code amount of the processing block or the number of layers of the deepest division block in the coding block included in the processing block. It is designed to calculate the feature amount of.

According to the present invention, an evaluation value calculation unit for calculating an evaluation value used for determining the presence / absence of a scene change is provided from a block-based feature amount included in two pictures calculated by the feature amount calculation unit. However, by comparing the evaluation value calculated by the evaluation value calculation unit with the threshold value, it is determined whether or not there is a scene change between the two pictures. Then, the feature amount calculation unit calculates the feature amount of the block from the code amount of the block or the deepest number of layers of the divided block included in the block for each block of the picture selected by the picture selection unit. The image processing device was configured to do so. Therefore, the image processing apparatus according to the present invention can determine the presence or absence of a scene change even for a picture that does not include a motion vector.

It is a block diagram which shows the image processing apparatus according to Embodiment 1 of this invention. It is a hardware block diagram which shows the image processing apparatus according to Embodiment 1 of this invention. It is a hardware block diagram of a computer when an image processing apparatus is realized by software or firmware. It is a flowchart which shows the image processing method which is the processing procedure when the image processing apparatus is realized by software or firmware. It is explanatory drawing which shows the example which selects the evaluation target picture from a plurality of pictures included in a coded video stream. It is explanatory drawing which shows the example which selects the evaluation target picture from a plurality of pictures included in a coded video stream. The coding method is AVC / H. 264 or HEVC / H. It is explanatory drawing which shows the structural example of the coded video stream in the case of 265. It is explanatory drawing which shows an example of the code amount in the processing block unit. It is a block diagram which shows the other image processing apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the processing block which specifies the code amount among the processing blocks included in the evaluation target picture. It is explanatory drawing which shows the processing block which specifies the code amount among the processing blocks included in the evaluation target picture. It is explanatory drawing which shows the processing block which specifies the code amount among the processing blocks included in the evaluation target picture. It is explanatory drawing which shows the number of layers of the division block in a coded block. It is explanatory drawing which shows the example of the processing block and the coding block. It is explanatory drawing which shows the example of the processing block and the coding block. It is explanatory drawing which shows the coordinates in a horizontal direction and a vertical direction of a processing block. Is an explanatory view showing an example of weighting coefficient w _n for each processing block. Is an explanatory view showing an example of weighting coefficient w _n for each processing block.

Hereinafter, in order to explain the present invention in more detail, a mode for carrying out the present invention will be described with reference to the accompanying drawings.

Embodiment 1.
FIG. 1 is a configuration diagram showing an image processing apparatus according to the first embodiment of the present invention. FIG. 2 is a hardware configuration diagram showing an image processing apparatus according to the first embodiment of the present invention.
In FIG. 1, the picture selection unit 1 is realized by, for example, the picture selection circuit 11 shown in FIG.
The picture selection unit 1 performs a process of acquiring a coded video stream containing coded data of a plurality of pictures.
The picture selection unit 1 determines candidate pictures for evaluating the presence or absence of a scene change, which is a change point of a scene, from a plurality of pictures having coded data encoded in a coded block unit (block unit). A process of selecting two pictures from the candidate pictures as target pictures for evaluating the presence or absence of a scene change is performed. However, it is assumed that the method for determining the candidate picture is defined in advance.

The coded data of the picture is, for example, data encoded in a coded block unit called a macro block or a coded block unit called a CTU (Coding Tree Unit).
As a standard of a coding method for coding in macroblock units, for example, AVC / H. The 264 standard is disclosed in Non-Patent Document 1 below.
Further, as a standard of a coding method for encoding in units of CTU, for example, HEVC / H. The 265 standard is disclosed in Non-Patent Document 2 below.
[Non-Patent Document 1] ISO / IEC 14496-10 / ITU-TH. 264 Standard [Non-Patent Document 2] ISO / IEC 23008-2 / ITU-TH. 265 standard

The feature amount calculation unit 2 is realized by, for example, the feature amount calculation circuit 12 shown in FIG.
The feature amount calculation unit 2 performs a process of calculating the feature amount of each processing block unit from the coded data of the coded block unit in the picture for each picture selected by the picture selection unit 1. The processing block is composed of one or more coded blocks, and the configuration is defined in advance. Since the processing block is composed of one or more coded blocks, the smallest definable unit of the processing block is the coded block (processing block = coded block).
FIG. 14 shows an example in which a processing block is composed of four coding blocks. Further, as in the example shown in FIG. 15, the number of coded blocks in the vertical direction and the horizontal direction constituting the processing block may be different.
The feature amount of each processing block is calculated from the coded data of each coded block in the processing block. In the first embodiment, the coded data of each coded block is decoded, the code amount of each coded block is specified from the decoding result of the coded data, and the coded blocks belonging to each processing block The sum of the code amounts is used as the feature amount for each processing block.
The processing block sizes of the two pictures selected by the picture selection unit 1 (the configuration of the coded block in the predefined processing block) are the same.

The evaluation value calculation unit 3 is realized by, for example, the evaluation value calculation circuit 13 shown in FIG.
The evaluation value calculation unit 3 executes a process of calculating an evaluation value used for determining the presence or absence of a scene change from the feature amount of each processing block included in the two pictures calculated by the feature amount calculation unit 2.

The presence / absence determination unit 4 is realized by, for example, the presence / absence determination circuit 14 shown in FIG.
The presence / absence determination unit 4 performs a process of determining the presence / absence of a scene change between the two pictures by comparing the evaluation value calculated by the evaluation value calculation unit 3 with the threshold value.

In FIG. 1, each of the picture selection unit 1, the feature amount calculation unit 2, the evaluation value calculation unit 3, and the presence / absence determination unit 4, which are the components of the image processing device, is realized by the dedicated hardware as shown in FIG. I'm assuming something. That is, it is assumed that the picture selection circuit 11, the feature amount calculation circuit 12, the evaluation value calculation circuit 13, and the presence / absence determination circuit 14 are realized.
Here, the picture selection circuit 11, the feature amount calculation circuit 12, the evaluation value calculation circuit 13, and the presence / absence determination circuit 14 are, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, and an ASIC (Application Specific). Integrated Circuit), FPGA (Field-Programmable Gate Array), or a combination thereof is applicable.

The components of the image processing device are not limited to those realized by dedicated hardware, and the image processing device may be realized by software, firmware, or a combination of software and firmware.
The software or firmware is stored as a program in the memory of the computer. A computer means hardware for executing a program, and corresponds to, for example, a CPU (Central Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, a DSP (Digital Signal Processor), and the like. ..
The memory of the computer is, for example, RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Online Memory), EEPROM (Electrically Molecular Memory) such as EEPROM (Electrically Molecular Memory). This includes semiconductor memories, magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs (Digital Versaille Disc), and the like.

FIG. 3 is a hardware configuration diagram of a computer when the image processing device is realized by software, firmware, or the like.
When the image processor is realized by software or firmware, the memory 21 is provided with a program for causing the computer to execute the processing procedures of the picture selection unit 1, the feature amount calculation unit 2, the evaluation value calculation unit 3, and the presence / absence determination unit 4. It may be stored and the processor 22 of the computer may execute the program stored in the memory 21.
FIG. 4 is a flowchart showing an image processing method which is a processing procedure when the image processing device is realized by software or firmware.

Further, FIG. 2 shows an example in which each of the components of the image processing device is realized by dedicated hardware, and FIG. 3 shows an example in which the image processing device is realized by software, firmware, or the like. Some components of the image processing apparatus may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, or the like.

Next, the operation will be described.
The picture selection unit 1 acquires a coded video stream containing coded data of a plurality of pictures.
The picture selection unit 1 determines candidate pictures from the acquired encoded video stream according to a predetermined method for determining candidate pictures, and evaluates the presence or absence of a scene change, which is a change point of the scene, from the candidate pictures. Two pictures are selected as the pictures of (step ST1 in FIG. 4).
Here, of the two selected pictures, the later picture in the reproduction order is referred to as "evaluation target picture", and the earlier picture in the reproduction order is referred to as "reference picture". "Evaluating the presence or absence of a scene change between two pictures" means that a scene change has occurred from the picture one after the reference picture to the picture to be evaluated (including the picture to be evaluated itself) in the playback order. It indicates that it is evaluated whether or not it is.

In FIGS. 5 and 6, a candidate picture to be evaluated is determined from the pictures whose coded data is included in the coded video stream, and the evaluation target picture and the reference picture are selected from the candidate pictures. It is explanatory drawing which shows the example.
In FIG. 5, all of a plurality of pictures whose coded data is included in the coded video stream are set as candidate pictures, and the evaluation target pictures are finally selected from the candidate pictures in the order of reproduction. , An example is shown in which all the pictures included in the encoded video stream are selected as the pictures to be evaluated.
In FIG. 6, a plurality of pictures in which the coded data is included in the coded video stream have a GOP (Group Of Pictures) structure, and the intra picture of the random access point in each GOP is used as a candidate picture and is a candidate. An example is shown in which the pictures are selected as evaluation target pictures in order of reproduction.
AVC / H. In 264, the IDR (Instantaneus Decoding Refresh) picture is an intra picture of a random access point.
HEVC / H. In 265, the IRAP (Intra Random Access Point) picture is an intra picture of a random access point.
The reference picture is a candidate picture immediately preceding the evaluation target picture in the reproduction order in both FIGS. 5 and 6. Therefore, in the example of FIG. 5, it is evaluated whether or not there is a scene change between two adjacent pictures, that is, whether or not the evaluation target picture itself is a scene change point. In the example of FIG. 6, the presence or absence of a scene change between the intra pictures of random access points in the adjacent GOP is evaluated.
In addition to the above, the definition of candidate pictures can be arbitrarily set, such as selecting evaluation target pictures at N picture intervals (N is an integer of 0 or more) (when N = 0, it is the same as in FIG. 5). can do.

In FIG. 7, the coding method is AVC / H. 264 or HEVC / H. It is explanatory drawing which shows the structural example of the coded video stream in the case of 265.
In FIG. 7, a plurality of access units are arranged in the order of coding, and one access unit shows data of one picture.
Each access unit is composed of a plurality of NAL (Network Abst rection Layer) units.
Each NAL unit is classified into header information (non-VCL) and picture data (VCL) which is picture coding data.
The header information (non-VCL) includes an access unit delimiter, a sequence level header, and a picture level header.
AVC / H. 264 or HEVC / H. In 265, one or more NAL units constituting one access unit are continuous, and a method of identifying the head position of the access unit is defined so that the delimiter position between adjacent access units on the stream can be known. There is. As one example, an access unit delimiter, which is a NAL unit indicating the head of the access unit, is defined.
The NAL unit shown in FIG. 7 is an example, and the configuration pattern of the NAL unit conforms to the specifications of the respective standards. At this time, the access unit delimiter may not exist in the NAL unit that constitutes the access unit, and even in that case, the head position of the access unit can be identified according to the method for identifying the head position of the access unit defined in the standard. It is possible.

Therefore, the picture selection unit 1 can identify the delimiter position of the access unit which is a picture by identifying the head position of the access unit according to the method of identifying the head position of the access unit defined in the standard.
Since the picture selection unit 1 can identify the delimiter position of the access unit, two pictures can be selected from a plurality of access units, and the coded data of the two selected pictures is calculated as a feature amount. Output to part 2.

The feature amount calculation unit 2 calculates the feature amount of each processing block unit from the coded data of the coded block unit for each picture selected by the picture selection unit 1 (step ST2 in FIG. 4).
Hereinafter, the feature amount calculation process for each processing block by the feature amount calculation unit 2 will be specifically described.
Here, the feature amounts of the processing blocks included in the picture P _t of the picture number t selected by the picture selection unit 1 are set to C _{n and t} , the picture number of the reference picture is t = tp, and the picture number of the evaluation target picture is set. Let t = tc.
n is a block number for specifying the processing block for which the feature amount is calculated in the picture, and is numbered 1, 2, ..., NUM_BL _{t in the} order of raster scan from the upper left processing block. Here, NUM_BL _t indicates the total number of processing blocks for which the feature amount is to be calculated in the picture P _t of the picture number t. When the processing block size is 64 × 64 pixels and the resolution of the picture is 3840 × 2160 pixels, NUM_BL _t = 60 × 34 = 2040, assuming that the entire screen is targeted. However, the block in the lowermost column has a block size of 64 × 48 pixels. NUM_BL _t may be the number of processing blocks of the entire screen as described above, or may be limited to a value smaller than the number of processing blocks of the entire screen in order to reduce the processing load.
Therefore, the feature quantities C _{n, tp} , C _n, and tk at t = tp and t = tk are calculated according to the following, respectively.

The feature amount calculation unit 2 executes the syntax decoding process in the feature amount calculation target picture P _t , and based on the syntax decoding process result, the code amount _{Sn, t} (n = 1, 2, ... ･ ･, NUM_BL _t ) are specified respectively.
For example, the feature amount calculation unit 2 decodes all the coding parameters included in the coding data of each coding block belonging to the processing block for which the feature amount is calculated, and of all the decoded coding parameters. The sum of the code amounts is specified as the code amount _{Sn, t} .
The coding parameter is a parameter necessary for generating a decoded image of the coded block, and corresponds to, for example, a parameter indicating a coding mode, an intra prediction parameter, an inter prediction parameter, a prediction difference coding parameter, a motion vector, and the like.

Next, the feature amount calculating unit 2, the code amount S _ALL feature quantity calculation target picture P _t, identifies _t, as shown in the following equation (1), the feature quantity calculation target picture P _t code amount S _ALL, in _t, the code amount S _n of the processing _blocks, normalized by dividing _t, feature amount C _n of the processing _block, as _t, the code amount S _n of normalized processing block _units, the _t M Double. M is a preset constant, for example, a real number greater than 0.

Here, the feature amount calculating section 2, the code amount _{S ALL} feature quantity calculation target picture _{P t,} is shown an example of specifying a _t, the feature quantity calculation target picture _{P t} code amount _{S ALL,} as _t, An example is considered in which the sum of the code amounts _{Sn and t} of all the processing blocks included in the feature amount calculation target picture P _t is calculated.
However, this is only an example, the code amount S _ALL feature quantity calculation target picture P _t, as _t, the size of the picture data (VCL) of the feature quantity calculation target picture P _t may be specified. The size of the picture data (VCL), without performing the decoding process of syntax in the feature quantity calculation target picture P _t, can be identified. Additional code amount S _ALL feature quantity calculation target picture P _t, as _t, may be specified the size of the access unit of the feature amount calculation target picture P _t. The size of the access unit can be calculated only by identifying the partition position of the access unit described above.

The feature quantities C _{n and t} of each processing block calculated by the equation (1) represent the ratio of the code amounts of each processing block, and the influence of the variation in the size of the total code amount for each picture is suppressed. It is a feature quantity.
FIG. 8 is an explanatory diagram showing an example of the code amount in the processing block unit.
FIG. 8 shows an example in which the processing block is composed of 2 × 2 coded blocks and the coded block is a macro block or a CTU, and the code amount of each coded block is represented by a number. Further, in the center of each processing block, the total code amount of the coded blocks belonging to the processing block is represented as the code amount of the processing block.
In the example of FIG. 8, the unit of the code amount of the coded block is represented by byte, but it is only an example and may be represented by bit unit.

評価値算出部３により算出された評価値Ｊは、参照ピクチャＰ_ｔｐと評価対象ピクチャＰ_ｔｃとの間での絵柄の変化が大きい程、参照ピクチャＰ_ｔｐと評価対象ピクチャＰ_ｔｃの処理ブロック単位の符号量の分布の差異が大きくなり、値が大きくなる。
また、ＭＩＮ＿ＮＵＭ＿ＢＬは、ＮＵＭ＿ＢＬ_ｔｐとＮＵＭ＿ＢＬ_ｔｃの最小値である。即ち、参照ピクチャＰ_ｔｐと評価対象ピクチャＰ_ｔｃの処理ブロック数が異なる場合、両ピクチャ共に特徴量を算出している処理ブロックのみが評価値Ｊの算出対象となる。
例えば、参照ピクチャＰ_ｔｐと評価対象ピクチャＰ_ｔｃの符号化データのサイズが大きく異なる場合、復号処理時間も大きく異なる傾向にある。そのようなケースにて、各ピクチャの符号化データの復号処理や処理ブロック単位の特徴量の算出に割かれる時間が一律に制限されている場合、処理可能な処理ブロック数が両ピクチャで異なる結果となる。このような場合、評価値Ｊは、両ピクチャ共に特徴量を算出している処理ブロックのみを対象として求められる。なお、上記のように各々のピクチャで消費できる処理時間Ｔを決定することで、所望の処理速度（フレームレート１／Ｔ）での特徴量算出処理が可能となる。Evaluation value calculation unit 3, the feature amount C _n of the processing block in the reference picture P _tp calculated by the feature amount calculation unit _2, and _tp, the feature amount C _n of the processing block in the evaluation object picture P _tc, and _tc From, the evaluation value J used for determining the presence or absence of the scene change is calculated (step ST3 in FIG. 4).
For example, the evaluation value calculation unit 3 sets the evaluation value J used for determining the presence or absence of a scene change as a processing block between the reference picture P _{t t} and the evaluation target picture P _tc as shown in the following equation (2). Calculate the sum of the absolute values of the features of the unit.

Evaluation value J calculated by the evaluation value calculating section 3, as the pattern of change between the reference picture P _tp evaluated picture P _tc is large, the reference process block units of picture P _tp evaluated picture P _tc The difference in the distribution of the code amount of is large, and the value is large.
Further, MIN_NUM_BL is the minimum value of NUM_BL _tp and NUM_BL _tc. That is, when the number of processing blocks of the reference picture P _tp and the evaluation target picture P _tk are different, only the processing block for which the feature amount is calculated for both pictures is the calculation target of the evaluation value J.
For example, when the sizes of the encoded data of the reference picture P _tp and the evaluation target picture P _ct are significantly different, the decoding processing time tends to be significantly different. In such a case, if the time devoted to decoding the coded data of each picture and calculating the feature amount in each processing block is uniformly limited, the number of processing blocks that can be processed differs between the two pictures. It becomes. In such a case, the evaluation value J is obtained only for the processing block for which the feature amount is calculated for both pictures. By determining the processing time T that can be consumed in each picture as described above, the feature amount calculation processing at a desired processing speed (frame rate 1 / T) becomes possible.

The presence / absence determination unit 4 compares the evaluation value J calculated by the evaluation value calculation unit 3 with the preset threshold value Th (step ST4 in FIG. 4).
If the evaluation value J calculated by the evaluation value calculation unit 3 is equal to or greater than the threshold value Th (in the case of step ST4: YES in FIG. 4), the presence / absence determination unit 4 is between the reference picture P _tp and the evaluation target picture P _tk. It is determined that there is a scene change in (step ST5 in FIG. 4).
If the evaluation value J calculated by the evaluation value calculation unit 3 is less than the threshold value Th (in the case of step ST4: NO in FIG. 4), the presence / absence determination unit 4 is between the reference picture P _tp and the evaluation target picture P _tk. It is determined that there is no scene change in (step ST6 in FIG. 4).
The presence / absence determination unit 4 outputs a determination result of presence / absence of a scene change.
The threshold value Th may be a fixed value set in advance, or may be switched or changed according to a specific condition. For example, a method of preparing a threshold Th for each type of content (drama, news, sports, etc.) and switching, a method of preparing a threshold Th for each broadcasting station and switching, and each encoder that generates a stream. A method of preparing a threshold Th and switching, a method of adaptively calculating the threshold Th according to the setting of the encoder that generates the stream (preparing a calculation formula of the threshold Th using the setting value of the encoder as a variable), a method of the stream A method of adaptively calculating the threshold Th according to the transition of the code amount (preparing a calculation formula of the threshold Th with the time change of the code amount (1st to nth differential in the time direction, n is an integer of 1 or more) as a variable). A method of setting the number of target processing blocks, that is, a value proportional to MIN_NUM_BL (Th = MIN_NUM_BL × Th_Base, Th_Base is a preset value larger than 0 which is a reference value of the threshold Th) can be considered. In the method of a value proportional to MIN_NUM_BL, becomes the threshold in consideration of the variation of the maximum possible value of the evaluation value J of changes in MIN_NUM_BL of each evaluation object picture _{P tc} (or MIN_NUM_BL ') is set, the high Accurate scene change detection processing can be realized.
Further, a method in which these methods are combined may be used. For example, there is a method that combines a method of adaptively calculating the threshold value Th according to the transition of the code amount of the stream and a method of setting the value proportional to MIN_NUM_BL. In this case, it can be realized by replacing the threshold value Th, which is adaptively calculated according to the code amount transition of the stream, with Th_Base.

If a picture that has not yet been selected as an evaluation target picture remains among the candidate pictures determined by the picture selection unit 1 (step ST7 in FIG. 4: YES), the process returns to step ST1 and step ST1 The process of ~ ST7 is repeated.
Of the two pictures selected by the picture selection unit 1 in the returned step ST1, the feature amount calculation process by the feature amount calculation unit 2 is omitted for the picture whose feature amount has already been calculated. You may do so. By doing so, it is not necessary to recalculate the feature amount, and the processing time can be shortened. For example, as described in FIGS. 5 and 6, when the evaluation target picture is selected from the candidate pictures in the playback order and the reference picture is set to the previous candidate picture in the playback order from the evaluation target picture, the evaluation target is The feature amounts C _{n, tk} in the picture are the feature amounts C _{n, tp} in the reference picture when the candidate picture one ahead in the reproduction order is selected as the evaluation target picture. Therefore, the calculated feature quantities C _{n and tk} are sequentially stored, and the feature quantities C _{n and tp} are read out from the stored feature quantities, so that the feature quantity calculation process can be omitted.
If the decoding order and the playback order are different, the evaluation target picture is selected in the decoding order, and the presence or absence of a scene change is evaluated as the reference picture before the evaluation target picture in the decoding order and as the candidate picture closest to the evaluation target picture in the playback order. You may. Even in this case, for the picture for which the feature amount has already been calculated, the calculation result of the feature amount may be diverted and the processing may be omitted.
If there is no candidate picture that has not yet been selected as the evaluation target picture in the candidate pictures determined by the picture selection unit 1 (step ST7: NO in FIG. 4), a series of processes is completed.

As is clear from the above, according to the first embodiment, the evaluation value used for determining the presence or absence of a scene change is calculated from the feature amount of each block included in the two pictures calculated by the feature amount calculation unit 2. The evaluation value calculation unit 3 is provided, and the presence / absence determination unit 4 determines the presence / absence of a scene change between the two pictures by comparing the evaluation value calculated by the evaluation value calculation unit 3 with the threshold value. Then, the image processing device is configured so that the feature amount calculation unit 2 calculates the feature amount of the block from the code amount of the block for each block of the picture selected by the picture selection unit 1. Therefore, the image processing device can determine whether or not there is a scene change even for a picture that does not include a motion vector.

In the first embodiment, an example in which the coded video stream is given to the picture selection unit 1 of the image processing device is shown, but a media transmission stream for storing the coded video stream may be given to the image processing device. ..
In this case, as shown in FIG. 9, a demultiplexer 5 may be provided which takes out the coded video stream from the media transmission stream and outputs the coded video stream to the picture selection unit 1.
FIG. 9 is a configuration diagram showing another image processing apparatus according to the first embodiment of the present invention.
The media transmission stream may be, for example, in a media transmission format such as MPEG-2 TS (ISO / IEC 13818-1 / ITU-TH.222.0 Transport Stream) or MMT (ISO / IEC 23008-1). A packetized media transmission stream or the like is assumed.

From the header information of the given media transmission stream, the demultiplexer 5 can know the delimiter between each access unit, the type and size of the NAL unit.
If the demultiplexer 5 gives the picture selection unit 1 information indicating the delimiter between the access units, the picture selection unit 1 will use the coded video stream based on the information given by the demultiplexer 5. A desired picture can be selected from the above.
Further, if the demultiplexer 5 provides the feature amount calculation unit 2 with information indicating the type and size of the NAL unit, the feature amount calculation unit 2 does not perform the syntax decoding process in the evaluation target picture. , The code amount of the evaluation target picture can be specified.

In the first embodiment, the feature amount calculation unit 2 shows an example in which the processing block of the entire picture is targeted in the calculation processing of the code amounts _{Sn and t} of the processing block.
This is only an example, and in order to reduce the processing load of the feature amount calculation unit 2 _{, in} the calculation process of the code amounts _{Sn and t} of the processing blocks, only a part of the processing blocks in the picture are targeted. You may.

Specifically, as shown in FIG. 10, only every other processing block may be targeted in each of the row direction and the column direction.
FIG. 10 is an explanatory diagram showing a processing block for specifying a code amount among the processing blocks constituting the picture.
In FIG. 10, the processing block in which "1" is described is a processing block for specifying the code amount, and the processing block in which "0" is described is a processing block for which the code amount is not specified.

Further, as shown in FIG. 11, only the processing blocks every other row or the processing blocks every other column may be targeted.
FIG. 11 is an explanatory diagram showing a processing block for specifying a code amount among the processing blocks constituting the picture.
In FIG. 11, the processing block in which "1" is described is a processing block for specifying the code amount, and the processing block in which "0" is described is a processing block for which the code amount is not specified.

As shown in FIG. 12, in the calculation process of the code amounts _{Sn and t} of the processing blocks, the processing blocks from the upper left of the picture to the specific processing blocks in the order of raster scan are targeted, and the remaining processing blocks are not targeted. You may.
FIG. 12 is an explanatory diagram showing a processing block for specifying a code amount among the processing blocks constituting the picture.
In FIG. 12, the processing block in which "1" is described is a processing block for specifying the code amount, and the processing block in which "0" is described is a processing block for which the code amount is not specified.
In addition, in FIGS. 10 to 12, the processing block in which "1" is described and the processing block in which "0" is described may be reversed.

Embodiment 2.
In the first embodiment, the feature amount calculation unit 2 decodes the coded data of the coded block, specifies the code amount of the processing block from the decoding result of the coded data, and processes from the code amount of the processing block. An example of calculating the feature amount of a block is shown.
In the second embodiment, the feature amount calculating section 2, for each processing block of the picture P _t of the picture number t selected by the picture selecting unit 1, the feature amount of the processing block C _n, as _t, the processing block The number of division blocks included in may be used.

The number of divided blocks included in the processing block is calculated by adding the number of divided blocks included in each coded block belonging to the processing block.
The number of divided blocks included in the coded block is determined by, for example, the coding method being AVC / H. If it is 264, it corresponds to the number of blocks in the macro block determined by the macro block type (mb_type). The number of blocks in the macroblock can be obtained by performing a syntax decoding process on the picture to be evaluated.
The coding method is HEVC / H. If it is 265, it corresponds to the number of CUs (Coding Units) in the CTU which is a coding block. The number of CUs in the CTU is obtained by performing a syntax decoding process on the picture to be evaluated.
When specifying the number of divided blocks included in the processing block as the feature amount of the processing block, the feature amount calculation unit 2 does not need to normalize the code amount when calculating the feature amount of the processing block. Therefore, the processing load of the feature amount calculation unit 2 is reduced.

Embodiment 3.
In the first embodiment, the feature amount calculation unit 2 decodes the coded data of the coded block, specifies the code amount of the processing block from the decoding result of the coded data, and processes from the code amount of the processing block. An example of calculating the feature amount of a block is shown.
In the third embodiment, the feature amount calculating section 2, for each processing block of the picture P _t of the picture number t selected by the picture selecting unit 1, the feature amount of the processing block C _n, as _t, the processing block You may use the number of layers of the deepest division block in.

The number of the deepest layers of the divided block in the processing block is calculated by obtaining the maximum value of the number of the deepest layers of the divided block in each coded block belonging to the processing block.
The number of layers in the deepest division block in the coding block is, for example, the coding method is HEVC / H. If it is 265, as shown in FIG. 13, among the plurality of CUs included in the CTU which is the coding block, the division hierarchy corresponds to the deepest number.
FIG. 13 is an explanatory diagram showing the number of layers of the divided blocks in the coded block.
In the example of FIG. 13, since the maximum value of CU depth is 3, the number of layers in the deepest division block in the coded block is 3.
The number of layers at the deepest layer of the divided block can be obtained by performing a syntax decoding process on the picture to be evaluated.
When specifying the number of layers of the division block in the processing block as the feature amount of the processing block, the feature amount calculation unit 2 does not need to normalize the code amount when calculating the feature amount of the processing block. Therefore, the processing load of the feature amount calculation unit 2 is reduced.

Embodiment 4.
In the first embodiment, the evaluation value calculating unit 3, the processing block in the reference picture P _tp calculated by the feature amount calculation unit 2 feature quantity C _n, and _tp, the processing block in the evaluation object picture P _tc An example of calculating the evaluation value J from the feature quantities C _{n and tk} is shown.
In the fourth embodiment, the evaluation value calculation unit 3, the feature amount _{C n} of the processing block in the reference picture _{P tp} the picture number t = _tp, and _tp, the processing in the evaluation object picture _{P tc} the picture number t = tc Features of block units C _n, tk and features of processing block units in the secondary reference picture P _tp'with picture number t = tp'one before the reference picture P _tp in the candidate pictures An example of calculating the evaluation value J from the quantities C _{n and tp'will be} described.
Incidentally, the secondary reference picture _{P tp 'is} the reference picture of the reference picture _{P tp,} i.e., a reference picture _{P tp} when the reference picture _{P tp} was evaluated picture _{P tc.} Therefore, the reproduction time positional relationship of the three pictures is tp'<tp <tk. However, in general, since the number of pictures for each GOP may be variable, in the example of the candidate pictures of FIG. 6, the intervals (tc-tp, tp-tp') of the candidate pictures of tp', tp, and tk are set. It can be different.
With secondary reference picture P _{tp 'is} assumed to be the first picture in the reproduction order, the reference picture P _tp is the second picture in the reproduction order, the evaluation object picture P _tc is the third picture in the reproduction order ..

Next, the operation will be described.
In the fourth embodiment, the processing is performed according to the flowchart of FIG. 4 as in the first embodiment, but the processing content is changed to the processing described below.

Similar to the first embodiment, the picture selection unit 1 acquires a coded video stream including coded data of a plurality of pictures.
The picture selection unit 1 determines a candidate picture from a plurality of pictures whose coded data is included in the coded video stream according to a predetermined candidate picture determination method, and performs the above operation from the candidate pictures. The reference picture P _t p and the evaluation target picture P _tc are selected in the same manner as in Form 1 of the above.
Further, the picture selection unit 1 selects the secondary reference picture P _tp'from the candidate pictures. That is, in step ST1 of FIG. 4, in addition to the evaluation target picture and the reference picture, the secondary reference picture is also selected.

Similar to the first embodiment, the feature amount calculation unit 2 performs features of the processing block unit from the coded data of the coded block unit included in the evaluation target picture for each picture selected by the picture selection unit 1. Each amount is calculated (step ST2 in FIG. 4).
That is, the feature amount calculation unit 2 calculates the feature amount C _n of the processing blocks included in the reference picture P _tp, and _tp, the feature amount C _n of the processing blocks included in the evaluation object picture P _tc, and _tc ..
Further, the feature amount calculation unit 2 calculates the feature amount C _{n, tp'in the} processing block unit included in the secondary reference picture P _tp' .

ここで、ＭＩＮ＿ＮＵＭ＿ＢＬ’は、ＮＵＭ＿ＢＬ_ｔｐ’、ＮＵＭ＿ＢＬ_ｔｐ、ＮＵＭ＿ＢＬ_ｔｃの３つの値の最小値である。即ち、二次参照ピクチャＰ_ｔｐ’と参照ピクチャＰ_ｔｐと評価対象ピクチャＰ_ｔｃの処理ブロック数が全て同じでない場合、３つのピクチャ共通で特徴量を算出している処理ブロックのみが評価値Ｊの算出対象となる。
評価値算出部３により算出された評価値Ｊは、二次参照ピクチャＰ_ｔｐ’と参照ピクチャＰ_ｔｐの間で絵柄があまり変化せずに処理ブロック単位の特徴量の変化が小さくなり、さらに、参照ピクチャＰ_ｔｐと評価対象ピクチャＰ_ｔｃの間で絵柄が大きく変化して処理ブロック単位の特徴量の変化が大きくなる場合、値が大きくなる。
したがって、上記実施の形態１〜３では、参照ピクチャＰ_ｔｐと評価対象ピクチャＰ_ｔｃの間の絵柄の変化の大きさを評価していたが、この実施の形態４では、参照ピクチャＰ_ｔｐにおける絵柄の変化の程度と評価対象ピクチャＰ_ｔｃにおける絵柄の変化の程度の差異の大きさを評価するようにしたので、シーンチェンジではないピクチャ間での緩やかな絵柄の変化を、画面全体の絵柄が大きく変化するシーンチェンジと誤検出する可能性を低減し、上記実施の形態１〜３よりも、シーンチェンジの検出精度を高めることができる。
また、処理ブロック単位の特徴量の算出対象となる処理ブロックの定義方法は、上記実施の形態１と同様（ピクチャ内全ての処理ブロック、図１０〜１２のような一部の処理ブロック等）に任意に定義できる。The evaluation value calculation unit 3 includes the feature amounts C _{n and tp of the} processing block unit included in the reference picture P _tp calculated by the feature amount calculation unit 2 and the feature amount C of the processing block unit included in the evaluation target picture P _{tc. The} evaluation value J is calculated from _{n, tk} and the feature amounts C _{n, tp'in the} processing block unit included in the secondary reference picture P _tp' (step ST3 in FIG. 4).
Specifically, evaluation value calculation unit 3 calculates a difference absolute value sum of the feature processing block between the reference picture P _tp evaluated picture P _tc.
Further, the evaluation value calculation unit 3 calculates the sum of the difference absolute values of the feature amounts in the processing block units between the reference picture P _tp and the secondary reference picture P _tp' .
The evaluation value calculation unit 3 calculates the difference between the calculated absolute values of the differences as the evaluation value J, as shown in the following equation (3).

Here, MIN_NUM_BL 'is, NUM_BL _tp', the minimum value of the three values of _{NUM_BL} tp, _{NUM_BL tc.} That is, when the number of processing blocks of the secondary reference picture P _tp' , the reference picture P _tp, and the evaluation target picture P _tk are not all the same, only the processing block for which the feature amount is calculated for all three pictures has the evaluation value J. It is a calculation target.
Evaluation value evaluation value J calculated by the calculating section 3, changes in the characteristics of the processing block without pattern so much change is reduced between the secondary reference picture P _{tp 'and} the reference picture P _tp, further, If the change of the feature of the picture is greatly changed and the processing block unit between the reference pictures P _tp evaluation object picture P _tc increases, the value becomes larger.
Therefore, in the first to third embodiments, reference had been evaluating the size of the pattern of change between pictures P _tp evaluated picture P _tc, in the fourth embodiment, a picture in the reference picture P _tp Since the magnitude of the difference between the degree of change in the picture and the degree of change in the picture in the evaluation target picture _PTC is evaluated, the picture on the entire screen becomes large due to the gradual change in the picture between pictures that are not scene changes. It is possible to reduce the possibility of erroneous detection of a changing scene change and improve the detection accuracy of the scene change as compared with the above-described first to third embodiments.
Further, the method of defining the processing block for which the feature amount of each processing block is calculated is the same as that of the first embodiment (all processing blocks in the picture, some processing blocks as shown in FIGS. 10 to 12 and the like). It can be defined arbitrarily.

The presence / absence determination unit 4 compares the evaluation value J calculated by the evaluation value calculation unit 3 with the preset threshold value Th, as in the first embodiment (step ST4 in FIG. 4).
Similar to the first embodiment, the presence / absence determination unit 4 has a scene between the reference picture P _tp and the evaluation target picture P _tk when the evaluation value J calculated by the evaluation value calculation unit 3 is equal to or higher than the threshold value Th. It is determined that there is a change (step ST5 in FIG. 4).
Similar to the first embodiment, if the evaluation value J calculated by the evaluation value calculation unit 3 is less than the threshold value Th, the presence / absence determination unit 4 has a scene between the reference picture P _tp and the evaluation target picture P _tk. It is determined that there is no change (step ST6 in FIG. 4).
The presence / absence determination unit 4 outputs a determination result of presence / absence of a scene change.
Further, the method of setting the threshold value Th is the same as that of the first embodiment.

If a picture that has not yet been selected as an evaluation target picture remains among the candidate pictures determined by the picture selection unit 1 (step ST7 in FIG. 4: YES), the process returns to step ST1 and step ST1 The process of ~ ST7 is repeated.
Of the three pictures selected by the picture selection unit 1 in the returned step ST1, the feature amount calculation process by the feature amount calculation unit 2 is omitted for the pictures for which the feature amount has already been calculated. You may do so. By doing so, it is not necessary to recalculate the feature amount, and the processing time can be shortened.
If there is no picture that has not yet been selected as the evaluation target picture among the candidate pictures determined by the picture selection unit 1 (step ST7: NO in FIG. 4), a series of processes is completed.

Embodiment 5.
In the first to fourth embodiments, the evaluation value calculation unit 3 uses the formula (2) or the formula (3) to calculate the feature amount Cn _{, t of} the processing block unit calculated by the feature amount calculation unit 2. An example of calculating the evaluation value J is shown.
In the fifth embodiment, when the evaluation value calculation unit 3 calculates the evaluation value J, the feature amounts C _{n and t} of the processing block unit and the weighting coefficient w _n corresponding to the processing block are used. An example of calculating the evaluation value J will be described.

In the fifth embodiment, the equation (2) is changed to the following equation (4). Alternatively, the equation (3) is changed to the following equation (5).
The evaluation value calculation unit 3 substitutes the feature quantities C _{n, tk} and the feature quantities C _{n, tp ,} which are the feature quantities C _{n, t} in the processing block unit _, and the weighting coefficient w _n into the equation (4). The evaluation value J is calculated.
Alternatively, evaluation value calculation unit 3, the feature amount _{C n} of the processing _block, a _t feature amount _{C n, tc,} the feature amount _{C n, tp} and the feature _{C n,} and _{tp ',} and a weighting coefficient _{w n} The evaluation value J is calculated by substituting into the equation (5).

In the formulas (4) and (5), the feature quantities C _{n and tk} are the feature quantities of the processing block of the block number n in the evaluation target picture P _tk , and the feature quantities C _{n and tp} are the feature quantities in the reference picture P _tp . It is a feature amount of the processing block of block number n.
Further, the feature quantities C _{n and tp'are} the feature quantities of the processing block of the block number n in the secondary reference picture P _tp' .
w _n is a weighting coefficient that multiplies the absolute value of the difference between the feature C _{n, tk} and the feature C _{n, tp} , or the absolute value of the difference between the feature C _{n, tp} and the feature C _{n, tp'.} The weighting factor to multiply.

Equation (4) can be modified as in equation (6) below. Further, the equation (5) can be modified as the following equation (7).

Therefore, the evaluation value calculation unit 3 can calculate the evaluation value J by using the formula (6) or the formula (7).
When the evaluation value calculation unit 3 calculates the evaluation value J using the equation (6), the evaluation value calculation unit 3 uses the feature amount calculation unit 2 instead of multiplying the difference absolute value by the weighting coefficient w _n. , Each of the feature quantities C _{n and tk} and the feature quantities C _{n and tp} are multiplied by the weighting coefficient w _n .
When the evaluation value calculation unit 3 calculates the evaluation value J using the equation (7), the evaluation value calculation unit 3 uses the feature amount calculation unit 2 instead of multiplying the difference absolute value by the weighting coefficient w _n. , The feature amount C _{n, tk} , the feature amount C _{n, tp} and the feature amount C _{n, tp'are} each multiplied by the weighting coefficient w _n .

Specifically, when the evaluation value calculation unit 3 calculates the evaluation value J using the equation (6), the feature amount calculation unit 2 shows the constant M of the equation (1) in the following equation (8). after having replaced the variable M _'n, using equation (1), the feature amount _{C n, tc} and the feature _{C n,} calculates the respective _tp.
The feature amount calculation unit 2 outputs the calculated feature amounts C _{n, tk} and the feature amount C _{n, tp} to the evaluation value calculation unit 3.
Amount characteristic output from the feature amount calculation unit 2 _{C n, tc} is _w n _{C n} shown in equation (6) _corresponds to _tc, the feature quantity output from the feature amount calculation unit 2 _{C n, tp} is It corresponds to w _n C _{n, tp} shown in the formula (6).

Evaluation value calculating unit 3 using Equation (7), when calculating the evaluation value J, the feature amount calculation unit 2 was replaced by a variable M _'n indicating the constant M in equation (1) into equation (8) Above, using the formula (1), each of the feature quantities C _{n, tk} , the feature quantities C _{n, tp} and the feature quantities C _{n, tp'is} calculated.
The feature amount calculation unit 2 outputs the calculated feature amounts C _{n, tk} , the feature amount C _{n, tp,} and the feature amount C _{n, tp'to the} evaluation value calculation unit 3.
Feature amount _{C n} outputted from the feature amount calculation unit _{2, tc} is _w n _{C n} shown in equation (7) _corresponds to _tc, the output from the feature amount calculation unit 2 feature quantity _{C n, tp} is It corresponds to w _n C _{n, tp} shown in the formula (7). The feature amount _{C n} outputted from the feature amount calculation unit _{2, tp 'is, w} n _{C n} shown in equation _{(7), tp'} corresponds to.

The evaluation value calculation unit 3 sets the feature amounts C _{n and ct} output from the feature amount calculation unit 2 as w _n C _{n and tk} , _{and sets} the feature amounts C _{n and tp} output from the feature amount calculation unit 2 as w _n C _{n. , Tp} is substituted into the equation (6) to calculate the evaluation value J.
Alternatively, the evaluation value calculation unit 3 sets the feature amounts C _{n, ct} output from the feature amount calculation unit 2 to w _n C _{n, tk} , _{and sets} the feature amounts C _{n, tp} output from the feature amount calculation unit 2 w _n. C _n, by substituting _tp, the feature amount _{C n} outputted from the feature amount calculation unit _{2, 'a w} n _{C n, tp' tp} in equation (7) as to calculate the evaluation value J.

Here, the weighting coefficient w _{n for} each processing block indicates the importance of the processing block, and the larger the value of the weighting coefficient w _n is, the more important the processing block is when detecting a scene change. Means.
Therefore, the weighting coefficient w _n for each processing block at the changing pattern of to be detected scene, the pattern of change in a discontinuous or changes as prone processing block steep, need only be designed to be large value.

例えば、ピクチャに含まれる複数の処理ブロックが、図１６のように表される場合、（Ｘ_ｎ，Ｙ_ｎ）は、ブロック番号ｎの処理ブロックの２次元座標（Ｘ_ｎ：水平方向（ｘ）成分、Ｙ_ｎ：垂直方向（ｙ）成分）を示している。
図１６は、処理ブロックの水平方向と垂直方向の座標を示す説明図である。
図１６の例では、左上の処理ブロック（０，０）を基準の処理ブロックとしており、基準の処理ブロックの２次元座標は、（Ｘ_１，Ｙ_１）である。
例えば、処理ブロック（５，１）の２次元座標は、（Ｘ_６，Ｙ_６）であり、処理ブロック（２，３）の２次元座標は、（Ｘ_２１，Ｙ_２１）である。Hereinafter, specific examples of the weighting coefficient w _n will be given.
As shown in the following equation (9), the weighting coefficient w _n is defined by the function h (X _n , Y _n ) whose variables are the two-dimensional coordinates (X _n , Y _n ) of the processing block.

For example, when a plurality of processing blocks included in the picture are represented as shown in FIG. 16, (X _n , Y _n ) is the two-dimensional coordinates (X _n : horizontal direction (x)) of the processing block having the block number n. Component, Y _n : vertical (y) component) is shown.
FIG. 16 is an explanatory diagram showing the coordinates in the horizontal direction and the vertical direction of the processing block.
In the example of FIG. 16, the upper left processing block (0, 0) is used as the reference processing block, and the two-dimensional coordinates of the reference processing block are (X ₁ , Y ₁ ).
For example, the two-dimensional coordinates of the processing block (5, 1) are (X ₆ , Y ₆ ), and the two-dimensional coordinates of the processing block (2, 3) are (X ₂₁ , Y ₂₁ ).

For example, when a scene change is defined as "a subject moving to the center of the screen changes discontinuously", a weighting coefficient represented by the function h (x, y) shown in the following equation (10) is used. The value of w _n is set so that the processing block closer to the center of the screen has a larger value.

式（１１）及び式（１２）において、ａ_ｉ及びｂ_ｉは、それぞれ正の定数である。In equation (10), W is the number of horizontal processing blocks when the entire picture is divided by processing blocks, and H is the number of vertical processing blocks when the entire picture is divided by processing blocks, A _x , A. _y , L _x , and L _y are positive constants, respectively. Further, A _x and A _y have a role of scaling values of the weights of the x component and the y component, respectively, and the larger the value, the higher the weight reduction ratio when the distance from the center of the screen is increased.
f _i (x) is any monotonically increasing function. As an example of _fi (x), a function represented by the following equation (11) or equation (12) can be considered.

In the formula (11) and Equation (12), _{a i} and _{b i} is a positive constant, respectively.

For example, as W = 6, H = 4, a plurality of processing blocks included in the picture is represented as in FIG. 16, monotonically increasing function _f i (x) is a function expressed by the formula (11) And.
At this time, for example, a ₁ = a ₂ = 1, A _x = 1, A _y = 1, L _x = (W / 2) -1 = 2, _Ly = (H / 2) -1 = 1. For example, as shown in FIG. 17, the weighting coefficient of the processing block at the center of the screen is the largest w _n = 1, and the weighting coefficient of the processing block farther from the center of the screen is smaller. On the other hand, the values of L _x and _Ly are set to small values. For example, if a ₁ = a ₂ = 1, A _x = 1, A _y = 1, L _x = 1, _Ly = 1, the weighting coefficient w _n of each processing block is a screen as shown in FIG. The weighting coefficient is w _n = 1 in the central portion, and the number of processing blocks having a large weighting coefficient increases as compared with the case of FIG.
17 and 18 are explanatory views showing an example of the weight w _n for each processing block.
In Figure 17, for example, the weight coefficient _{w n} of the processing block (5,0) is, 1 / √6, weighting coefficient _{w n} of the processing block (1,1), 1 / √2, processing blocks (3,1 ) Has a weighting coefficient w _n of 1.
In Figure 18, for example, the weight coefficient _{w n} of the processing block (5,0) is 1 / √3, the weight coefficient _{w n} of the processing block (1,1) and the processing block (3,1) is a 1 ..

Further, the equation _{(10), f i (x} ) is a function expressed by the formula _{(11), a 1 = a} 2 = 1, L x = (W / 2) -1, L y = ( In the case of H / 2) -1, the equation (10) can be modified as in the equation (13). As a result, it is not necessary to classify cases according to the values of x and y.

重み係数ｗ_ｎを式（１４）に示す関数ｈ（ｘ，ｙ）で表す場合、二乗の計算及び平方根の計算がないため、重み係数ｗ_ｎを式（１０）に示す関数ｈ（ｘ，ｙ）で表す場合よりも、計算負荷を下げることができる。
式（１０）の例と同様に、例えば、ａ_１＝ａ_２＝１、Ｌ_ｘ＝（Ｗ／２）−１、Ｌ_ｙ＝（Ｈ／２）−１であれば、画面中心の処理ブロックの重み係数が、最も大きいｗ_ｎ＝１となり、画面中心から離れている処理ブロックの重み係数ほど、小さい重み係数となる。一方、Ｌ_ｘ、Ｌ_ｙの値を小さい値とする。例えばａ_１＝ａ_２＝１、Ｌ_ｘ＝１、Ｌ_ｙ＝１とすれば、各々の処理ブロックの重み係数ｗ_ｎは、画面中央部分で重み係数がｗ_ｎ＝１となり、重み係数が大きい処理ブロックの数が増加する。Further, the weighting coefficient w _n may be a function h (x, y) shown in the following equation (14) instead of the function h (x, y) shown in the equation (10).

When the weighting coefficient w _n is expressed by the function h (x, y) shown in the equation (14), the weighting coefficient w _n is expressed by the function h (x, y) shown in the equation (10) because there is no calculation of the square and the square root. ) Can be used to reduce the calculation load.
Similar to the example of the equation (10), for example, if a ₁ = a ₂ = 1, L _x = (W / 2) -1, and _Ly = (H / 2) -1, the processing block at the center of the screen. The weighting coefficient of is the largest w _n = 1, and the weighting coefficient of the processing block farther from the center of the screen is smaller. On the other hand, the values of L _x and _Ly are set to small values. For example, if a ₁ = a ₂ = 1, L _x = 1, and _Ly = 1, the weighting coefficient w _n of each processing block is w _n = 1 in the center of the screen, and the weighting coefficient is large. The number of processing blocks increases.

Further, the equation _{(14), f i (x} ) is a function expressed by the formula _{(11), a 1 = a} 2 = 1, L x = (W / 2) -1, L y = ( In the case of H / 2) -1, the equation (14) can be transformed like the equation (15). As a result, it is not necessary to classify cases according to the values of x and y.

ここで、Ｂ_ｘ、Ｂ_ｙは、シフト量を示し、値が大きいほど、画面中央からの距離が離れたときの重みの低減比率が高くなる。さらに、式（１６）及び式（１７）では、左シフトするように定義したが、式（１８）及び式（１９）に示すように、それぞれ右シフトを用いるようにしてもよい。

式（１８）及び式（１９）は、式（１６）及び式（１７）と同様に、Ｂ_ｘ、Ｂ_ｙは、シフト量を示すが、値が大きいほど、画面中央からの距離が離れたときの重みの低減比率が低くなる点で、式（１６）及び式（１７）と異なる。
また、上記で説明した「Ａ_ｘ、Ａ_ｙをＢ_ｘ、Ｂ_ｙによるシフト演算とすること」は、式（１３）及び式（１５）にも適用できる。Furthermore, in order to reduce the computational load of the formula (10) and (14), x component, _A x, which means the scaling value of the weight of the y component, _{A y} are each formula (16) and (17) As shown, it may be a shift operation. By doing so, the scaling accuracy becomes an integer accuracy, but the scaling process can be calculated by bit shifting.

Here, B _x, B _y represents the shift amount, the larger the value, reducing the ratio of the weight increases when a remote distance from the screen center. Further, in the equations (16) and (17), the shift to the left is defined, but as shown in the equations (18) and (19), the right shift may be used respectively.

Equation (18) and (19), similar to Equation (16) and _(17), B x, _{B y,} while indicating the shift amount, the larger the value, apart distance from the screen center It differs from the equations (16) and (17) in that the weight reduction ratio at the time is low.
Further, as described above, _"A x, to the _{A y B} x, a shift operation by _{B y"} is also applicable to Formula (13) and (15).

Further, as shown in FIG. 12, in the calculation process of the code amounts _{Sn and t} of the processing blocks, among the processing blocks constituting the picture, the processing blocks from the upper left processing block of the picture to the specific processing blocks in the order of raster scan are calculated. And. Then, when the remaining processing blocks are not included in the calculation target, or when the number of processing blocks that can perform the calculation processing of the code amounts _{Sn and t} changes for each picture due to restrictions such as the decoding processing time. L _x and _{L y} may be variable in the calculation unit of the evaluation value J.
For example, when the two-dimensional coordinates of the last processing block that can perform the calculation processing of the code amounts S _{n and t} are (X _{MIN_NUM_BL} , Y _{MIN_NUM_BL} ) (or (X _{MIN_NUM_BL'} , Y _{MIN_NUM_BL'} )), L _x is , (W / 2) -1. On the other hand, _{L y is 'If} (H / 2) -1 _smaller, L _{y = Y MIN_NUM_BL} (or _{Y MIN_NUM_BL Y MIN_NUM_BL} (or _{Y MIN_NUM_BL)'} and), _otherwise, L y = (H / 2 ) -1. Thus, in the case MIN_NUM_BL when calculating the evaluation value J (or MIN_NUM_BL ') is _smaller, than when secured with L y = (H / 2) -1, a large weighting coefficient _{w n} of the respective processing blocks Therefore, since the value of the evaluation value J becomes large, the scene change detection sensitivity in the presence / absence determination unit 4 can be increased.

あるいは、重み係数ｗ_ｎは、式（１４）に示す関数ｈ（ｘ，ｙ）の代わりに、以下の式（２１）に示す関数ｈ（ｘ，ｙ）で表すことができる。

In the above, the scene change is defined as "the subject moving to the center of the screen changes discontinuously", but the scene change is defined as "the background of the screen changes discontinuously". You may try to do it.
When the scene change is defined as "the background of the screen changes discontinuously", the weighting coefficient w _n is set so that the processing block closer to the edge of the screen where the background is easily reflected has a larger value. ..
When the scene change is defined as "the background of the screen changes discontinuously", the weighting coefficient w _n is expressed by the following equation (x, y) instead of the function h (x, y) shown in the equation (10). It can be represented by the function h (x, y) shown in 20).

Alternatively, the weighting coefficient w _n can be expressed by the function h (x, y) shown in the following equation (21) instead of the function h (x, y) shown in the equation (14).

ただし、Ｔｈ＿Ｂａｓｅは、閾値Ｔｈの基準値となる予め設定する０より大きい定数である。また、評価値Ｊの算出にＭＩＮ＿ＮＵＭ＿ＢＬ’を用いる場合は、式（２２）のＭＩＮ＿ＮＵＭ＿ＢＬをＭＩＮ＿ＮＵＭ＿ＢＬ’に置き換える。
さらに、本方法を上記実施の形態１に記載の方法と組み合わせた方法でもよい。例えば、本方法とストリームの符号量推移に従って適応的に閾値Ｔｈを算出する方法とを組み合わせた方法等である。この場合、ストリームの符号量推移に従って適応的に算出する閾値ＴｈをＴｈ＿Ｂａｓｅに置き換えることで実現できる。The method of setting the threshold value Th in the presence / absence determination unit 4 may be the same as that of the first embodiment. Further, as another example, if the MIN_NUM_BL (or MIN_NUM_BL ') is changed for each evaluation object picture _{P tc,} it may be the threshold value Th be calculated by equation (22) obtained from the weighting coefficient _{w n.} In this way, it becomes the threshold in consideration of the variation of the maximum possible value of the evaluation value J of changes in MIN_NUM_BL of each evaluation object picture _{P tc} (or MIN_NUM_BL ') is set, accurate scene change detection Processing can be realized.

However, Th_Base is a preset value larger than 0, which is a reference value of the threshold value Th. When MIN_NUM_BL'is used for the calculation of the evaluation value J, MIN_NUM_BL in the formula (22) is replaced with MIN_NUM_BL'.
Further, a method in which this method is combined with the method described in the first embodiment may be used. For example, it is a method that combines this method and a method of adaptively calculating the threshold value Th according to the code amount transition of the stream. In this case, it can be realized by replacing the threshold value Th, which is adaptively calculated according to the code amount transition of the stream, with Th_Base.

Embodiment 6.
In the fifth embodiment, an example in which the evaluation value J is calculated by using the weighting coefficient w _n for each processing block is shown.
In the sixth embodiment, by using a weighting factor w _m for each code block, an example of calculating the evaluation value J.

特徴量算出部２は、変数Ｍ’_ｍを用いて、特徴量Ｃ_ｍ，ｔ（Ｃ_{ｍ，ｔｃ、}Ｃ_{ｍ，ｔｐ、}あるいは、Ｃ_{ｍ，ｔｐ’}）を算出する場合、式（１）におけるブロック番号ｎの処理ブロックの符号量Ｓ_ｎ，ｔの代わりに、ブロック番号ｍの符号化ブロックの符号量を用いる。First, the feature amount calculating unit 2, instead of the weighting coefficient w _n for each processing block acquires a weighting factor w _m for each encoding block. m is a variable indicating the block number of the coded block numbered in the raster scan order, and w _m is a weighting coefficient for the coded block of the block number m.
Feature amount calculation unit 2, after replacing the variable M _'m shown in equation (23) below the constant M in equation (1), using equation (1), characterized in the coding block of the block number m Calculate each of the quantities C _{m, ct} and the feature C _{m, tp} , or calculate each of the features C _{m, tk} , the features C _{m, tp} and the features C _{m, tp'} .

Feature amount calculation unit 2 'with _m, the feature amount _{C m, t (C m,} tc, C m, tp _{or,, C m, tp'} variables M) when calculating the, in the formula (1) The code amount of the coded block of the block number m is used instead of the code amount _{Sn and t} of the processing block of the block number n.

The feature amount calculation unit 2 outputs the calculated feature amounts C _{m, tk} and the feature amount C _{m, tp} to the evaluation value calculation unit 3.
Alternatively, the feature amount calculation unit 2 outputs the calculated feature amounts C _{m, ct} , feature amount C _{m, tp,} and feature amount C _{m, tp'to the} evaluation value calculation unit 3.
Evaluation value calculation unit 3, in formula _{(6), w n C n} , with _{C m, tc} instead of _tc, with _{w n C n, C m,} tp instead of _tp, the evaluation value J calculate.
Alternatively, evaluation value calculation unit 3, in formula _{(7), C m,} the _tc using w _{n C n,} instead of _{tc, C m,} the _tp using w _{n C n,} instead of _{tp, w} n C _The evaluation value J is calculated by using C _{m, tp'instead of n, tp'} .

In the sixth embodiment, since the evaluation value J is calculated by using the weighting coefficient w _m for each coded block, the importance of each area is adjusted in units of coded blocks finer than the processing block. It is possible to improve the detection performance of the scene change.

In the present invention, within the scope of the invention, it is possible to freely combine each embodiment, modify any component of each embodiment, or omit any component in each embodiment. ..

The present invention is suitable for an image processing apparatus and an image processing method for determining the presence or absence of a scene change, which is a scene change point.

1 Picture selection unit, 2 Feature amount calculation unit, 3 Evaluation value calculation unit, 4 Presence / absence determination unit, 5 Demultiplexer, 11 Picture selection circuit, 12 Feature amount calculation circuit, 13 Evaluation value calculation circuit, 14 Presence / absence determination circuit, 21 Memory , 22 processors.

Claims (23)

From a manner of encoding the marks Goka blocks of the encoding process by a plurality of pictures, as a target picture to assess the presence or absence of a scene change is a change point of scenes, the picture selecting unit for selecting two pictures When,
For each of the two pictures selected by the picture selection unit, a processing block unit defined to the same size including one or more coding blocks indicating a coding processing unit included in the picture independently between the two pictures. A feature amount calculation unit that calculates each feature amount,
An evaluation value calculation unit that calculates an evaluation value used for determining the presence or absence of a scene change from the feature amount of the processing block unit included in the two pictures calculated by the feature amount calculation unit.
A presence / absence determination unit for determining the presence / absence of a scene change between the two pictures is provided by comparing the evaluation value calculated by the evaluation value calculation unit with the threshold value.
The feature amount calculation unit is based on the code amount of the processing block or the number of layers of the deepest division block in the coding block included in the processing block for each processing block of the picture selected by the picture selection unit. , An image processing apparatus characterized by calculating a feature amount of the processing block. The feature amount calculation unit specifies the code amount of the processing block for each of the processing blocks of the picture selected by the picture selection unit, and divides the code amount of the processing block by the code amount of the entire picture. The image processing apparatus according to claim 1, wherein the feature amount of the processing block is calculated. The feature amount calculation unit specifies the number of the deepest layers of the divided blocks included in the processing block for each processing block of the picture selected by the picture selection unit, and determines the number of layers of the processing block. The image processing apparatus according to claim 1, wherein the feature amount is used. The evaluation value calculation unit calculates the sum of the difference absolute values of the feature amounts in the processing block units between the two pictures calculated by the feature amount calculation unit as the evaluation value used for determining the presence or absence of a scene change. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized by the above. The picture selection unit selects three pictures as the pictures to be evaluated for the presence or absence of a scene change.
The feature amount calculation unit has the same size for each of the three pictures selected by the picture selection unit, including one or more coding blocks indicating coding processing units included in the pictures independently among the three pictures. Calculate the feature amount of each processing block of
The evaluation value calculation unit performs the processing between the second picture and the third picture in the reproduction order among the three pictures for which the feature amount in the processing block unit is calculated by the feature amount calculation unit. In addition to calculating the difference absolute value sum of the feature amount in the block unit, the difference absolute value sum of the feature amount in the processing block unit between the first picture and the second picture is calculated in the playback order, and the scene As an evaluation value used to determine the presence or absence of a change, the difference between the calculated absolute values of the differences between the two is calculated.
The presence / absence determination unit determines whether or not there is a scene change between the second picture and the third picture in the playback order by comparing the evaluation value calculated by the evaluation value calculation unit with the threshold value. The image processing apparatus according to claim 1. The evaluation value calculation unit uses the evaluation value as an evaluation value used for determining the presence or absence of a scene change.
The absolute value of the feature amount in the processing block unit between the two pictures calculated by the feature amount calculation unit is multiplied by the weighting coefficient corresponding to the processing block, and each is multiplied by the weighting coefficient. The image processing apparatus according to claim 1, wherein the sum of the absolute differences between the blocks is calculated. The feature amount calculation unit multiplies the feature amount of the processing block unit between the two pictures selected by the picture selection unit by the weighting coefficient corresponding to the processing block, respectively, and the two pictures. As the feature amount of the processing block unit between, the feature amount multiplied by the weighting coefficient is output.
The evaluation value calculation unit calculates the sum of the difference absolute values of the feature amounts in the processing block units between the two pictures output from the feature amount calculation unit as the evaluation values used for determining the presence or absence of a scene change. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized by the above. The picture selection unit selects three pictures as the pictures to be evaluated for the presence or absence of a scene change.
The feature amount calculation unit has the same size for each of the three pictures selected by the picture selection unit, including one or more coding blocks indicating coding processing units included in the pictures independently among the three pictures. Calculate the feature amount of each processing block of
The evaluation value calculation unit is a block unit between the second picture and the third picture in the playback order among the three pictures for which the feature amount of the processing block unit is calculated by the feature amount calculation unit. The absolute difference value of the feature amount of is multiplied by the weighting coefficient corresponding to the processing block to calculate the total difference absolute value of each block multiplied by the weighting coefficient, and the first in the playback order. The processing block obtained by multiplying the absolute value of the difference in the feature amount in the processing block unit between the picture and the second picture by the weighting coefficient corresponding to the processing block and multiplying each by the weighting coefficient. The total difference absolute value of the unit is calculated, and the difference between the calculated total difference absolute values of both is calculated as the evaluation value used for determining the presence or absence of a scene change.
The presence / absence determination unit determines whether or not there is a scene change between the second picture and the third picture in the playback order by comparing the evaluation value calculated by the evaluation value calculation unit with the threshold value. The image processing apparatus according to claim 1. The picture selection unit selects three pictures as the pictures to be evaluated for the presence or absence of a scene change.
The feature amount calculation unit has the same size for each of the three pictures selected by the picture selection unit, including one or more coding blocks indicating the coding processing units included in the pictures independently among the three pictures. The feature amount of each processing block unit is calculated, and the calculated feature amount of the processing block unit is multiplied by the weighting coefficient corresponding to the processing block, respectively, and the feature amount of the processing block unit included in the picture is included. As the feature quantity, the feature quantity multiplied by the weighting coefficient is output.
The evaluation value calculation unit performs the processing between the second picture and the third picture in the reproduction order among the three pictures in which the feature amount in the processing block unit is output from the feature amount calculation unit. In addition to calculating the difference absolute value sum of the feature amount in the block unit, the difference absolute value sum of the feature amount in the processing block unit between the first picture and the second picture is calculated in the playback order, and the scene As an evaluation value used to determine the presence or absence of a change, the difference between the calculated absolute values of the differences between the two is calculated.
The presence / absence determination unit determines whether or not there is a scene change between the second picture and the third picture in the playback order by comparing the evaluation value calculated by the evaluation value calculation unit with the threshold value. The image processing apparatus according to claim 1. Picture selecting unit from among the coding process is a plurality of pictures in a manner that coded marks Goka blocks, as a target picture to assess the presence or absence of a scene change, select two pictures,
The feature amount calculation unit defines, for each of the two pictures selected by the picture selection unit, the same size including one or more coding blocks indicating the coding processing units included in the pictures independently between the two pictures. Calculate the feature amount of each processing block,
The evaluation value calculation unit calculates an evaluation value used for determining the presence or absence of a scene change from the feature amount of the processing block unit included in the two pictures calculated by the feature amount calculation unit.
The presence / absence determination unit determines the presence / absence of a scene change between the two pictures by comparing the evaluation value calculated by the evaluation value calculation unit with the threshold value.
For each processing block of the picture selected by the picture selection unit, the feature amount calculation unit is based on the code amount of the processing block or the number of layers of the deepest division block in the coding block included in the processing block. , An image processing method characterized by calculating a feature amount of the processing block. The picture selection unit selects three pictures as the pictures to be evaluated for the presence or absence of a scene change.
The feature amount calculation unit has the same size for each of the three pictures selected by the picture selection unit, including one or more coding blocks indicating coding processing units included in the pictures independently among the three pictures. Calculate the feature amount of each processing block of
The evaluation value calculation unit performs the processing between the second picture and the third picture in the reproduction order among the three pictures for which the feature amount in the processing block unit is calculated by the feature amount calculation unit. In addition to calculating the difference absolute value sum of the feature amount in the block unit, the difference absolute value sum of the feature amount in the processing block unit between the first picture and the second picture is calculated in the playback order, and the scene As an evaluation value used to determine the presence or absence of a change, the difference between the calculated absolute values of the differences between the two is calculated.
The presence / absence determination unit determines whether or not there is a scene change between the second picture and the third picture in the playback order by comparing the evaluation value calculated by the evaluation value calculation unit with the threshold value. The image processing method according to claim 10, wherein the image processing method is characterized. Of the processing blocks for which the feature amount is calculated for the two pictures selected from the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. The image processing apparatus according to claim 1. The code amount of the entire picture is the size of the access unit.
Of the processing blocks for which the feature amount is calculated for the two pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. 2. The image processing apparatus according to claim 2. Among the processing blocks for which the feature amount is calculated for the three pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. The image processing apparatus according to claim 5. The feature amount calculation unit specifies the code amount of the processing block for each of the processing blocks of the picture selected by the picture selection unit, and divides the code amount of the processing block by the code amount of the entire picture. Then, the feature amount of the processing block was calculated.
The code amount of the entire picture is the size of the access unit.
The claim is characterized in that, of the processing blocks for which the feature amount is calculated for the three selected pictures, only the processing block common to all the pictures is targeted for the evaluation value calculation unit to calculate the evaluation value. Item 5. The image processing apparatus according to item 5. Among the processing blocks for which the feature amount is calculated for the three pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. 8. The image processing apparatus according to claim 8. The feature amount calculation unit specifies the code amount of the processing block for each of the processing blocks of the picture selected by the picture selection unit, and divides the code amount of the processing block by the code amount of the entire picture. Then, the feature amount of the processing block was calculated.
The code amount of the entire picture is the size of the access unit.
Among the processing blocks for which the feature amount is calculated for the three pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. 8. The image processing apparatus according to claim 8. Among the processing blocks for which the feature amount is calculated for the three pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. 9. The image processing apparatus according to claim 9. The feature amount calculation unit specifies the code amount of the processing block for each of the processing blocks of the picture selected by the picture selection unit, and divides the code amount of the processing block by the code amount of the entire picture. Then, the feature amount of the processing block was calculated.
The code amount of the entire picture is the size of the access unit.
Among the processing blocks for which the feature amount is calculated for the three pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. 9. The image processing apparatus according to claim 9. Of the processing blocks for which the feature amount is calculated for the two pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. 10. The image processing method according to claim 10. The feature amount calculation unit specifies the code amount of the processing block for each of the processing blocks of the picture selected by the picture selection unit, and divides the code amount of the processing block by the code amount of the entire picture. Then, the feature amount of the processing block was calculated.
The code amount of the entire picture is the size of the access unit.
Of the processing blocks for which the feature amount is calculated for the two pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. 10. The image processing method according to claim 10. Among the processing blocks for which the feature amount is calculated for the three pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. The image processing method according to claim 11. The feature amount calculation unit specifies the code amount of the processing block for each of the processing blocks of the picture selected by the picture selection unit, and divides the code amount of the processing block by the code amount of the entire picture. Then, the feature amount of the processing block was calculated.
The code amount of the entire picture is the size of the access unit.
Among the processing blocks for which the feature amount is calculated for the three pictures selected by the picture selection unit, only the processing block common to all the pictures is targeted for calculating the evaluation value by the evaluation value calculation unit. The image processing method according to claim 11.

Images