KR20070111968A - A method and apparatus for decoding a video signal - Google Patents

Abstract

A method and an apparatus for decoding a video signal are provided to reduce the number of bits to be transmitted by accurately performing inter-view prediction. Pictures coded before a current picture are stored in a reference picture storage unit(S31). A decoder obtains view information for identifying a view of a picture(S32). A decoded picture buffer unit induces a variable used for generating a reference picture list for inter-view prediction by using the obtained view information(S33). A reference picture list for temporal prediction and the reference picture list for inter-view prediction are generated differently according to a type of a current slice(S34). If the current slice is a P/SP slice, a reference picture list 0 is generated(S35). If the current slice is a B slice, a reference picture 0 and a reference picture list 1 are generated(S36). In order to improve a compression rate, a smaller number is allocated to a frequently referred picture in the initialized reference picture list(S37). The decoded picture buffer unit manages the decoded reference pictures(S38).

Description

A method and apparatus for decoding a video signal

1 is a schematic block diagram of a video signal decoding apparatus to which the present invention is applied.

2 is an embodiment to which the present invention is applied and shows an internal block diagram of the reference picture list generator 630.

3 is a flowchart for generating a reference picture list as an embodiment to which the present invention is applied.

4 is a diagram for describing a method of initializing a reference picture list when a current slice is a P slice according to an embodiment to which the present invention is applied.

FIG. 5 is a diagram for explaining a method of initializing a reference picture list when a current slice is a B slice as an embodiment to which the present invention is applied.

FIG. 6 illustrates an internal block diagram of the reference picture list rearranging unit 640 according to an embodiment to which the present invention is applied.

7 shows an internal block diagram of reference number assignment changing units 643B and 645B as an embodiment to which the present invention is applied.

FIG. 8 is a diagram for explaining a process of rearranging a reference picture list using viewpoint information according to an embodiment to which the present invention is applied.

9 is a block diagram illustrating an internal reference picture list reordering unit 640 according to another embodiment to which the present invention is applied.

FIG. 10 is an embodiment to which the present invention is applied and shows an internal block diagram of a reference picture list rearranger 970 for inter-view prediction.

11 illustrates a syntax for reordering a reference picture list according to an embodiment to which the present invention is applied.

12 illustrates a syntax for reordering a reference picture list according to another embodiment to which the present invention is applied.

The present invention relates to a method and apparatus for decoding a video signal.

The mainstream video broadcasting image is a single view image acquired with one camera. Multi-view video, on the other hand, is a three-dimensional (3D) image processing method that geometrically corrects images taken by more than one camera and provides users with various viewpoints in various directions through spatial synthesis. It is a field. Multi-view video can increase the freedom of view for the user, and has a feature that includes a larger area than the image area that can be acquired using a single camera.

Since such multi-view video images have a high correlation between viewpoints, redundant information can be removed through spatial prediction between viewpoints. Therefore, in order to efficiently perform prediction between viewpoints, it is necessary to manage reference pictures for inter-view prediction.

An object of the present invention is to efficiently code a video signal by providing a method for managing reference pictures used for inter-view prediction.

Another object of the present invention is to efficiently code a video signal by providing a method for generating a reference picture list for inter-view prediction.

Another object of the present invention is to efficiently code a video signal by providing a method of rearranging a reference picture list for inter-view prediction.

In order to achieve the above object, the present invention includes obtaining view information identifying a view of a picture from a video signal and generating information for reference picture management using the view information. It provides a video signal decoding method characterized in that.

The present invention also provides a view information acquisition unit for obtaining view information identifying a view of a picture from a video signal, a variable deriving unit for deriving a first variable ViewNum for generating a reference picture list using the view information; And a reference picture list initializer for generating the reference picture list using the first variable.

The above objects and features of the construction will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In addition, the terms used in the present invention was selected as a general term widely used as possible now, but in some cases, the term is arbitrarily selected by the applicant, in which case the meaning is described in detail in the description of the invention, It is to be understood that the present invention is to be understood as the meaning of terms rather than the names of terms.

MPEG MVC (Multiview Video Coding) deals with compression standards for video images from multiple cameras. The starting point is the modified form for multiview video images with the H.264 / AVC codec, and standardization is underway with AVC amendment. In H.264, three profiles that support specific functions are defined. A profile is a standardization of the technical components that enter the algorithm during video encoding / decoding. In other words, it is a kind of sub-standard that is a set of description elements necessary for decoding a bit string of a compressed image. The three profiles refer to a baseline profile, a main profile, and an extended profile, and the requirements of the encoder and the decoder are defined to be compatible with each profile.

Looking at the structure of the bit stream in H.264 / AVC, the network abstraction between the video coding layer (VCL) that deals with the video encoding process itself and the subsystem that transmits and stores the encoded information Layer, which is defined as a separate hierarchical structure. The output of the encoding process is VCL data and is mapped in units of NAL before transmission or storage. Each NAL unit includes raw video sequence payload (RBSP), which is data corresponding to compressed video data or header information.

The NAL unit basically consists of two parts: the NAL header and the RBSP. The NAL header includes flag information (nal_ref_idc) indicating whether a slice serving as a reference picture of the NAL unit is included and an identifier (nal_unit_type) indicating the type of the NAL unit. The RBSP stores the compressed original data and adds an RBSP trailing bit at the end of the RBSP to express the length of the RBSP in multiples of 8 bits. These NAL unit types include Instantaneous Decoding Refresh (IDR) pictures, Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and Supplemental Enhancement Information (SEI). Etc.

In addition, the specification restricts the product to various profiles and levels so that the target product can be implemented at a reasonable cost. The decoder must satisfy the constraints defined in the profile and level. As such, two concepts, profile and level, have been defined to represent the function or parameter of a compressed video range. Which profile the bitstream is based on may be identified by a profile identifier (profile_idc). The profile identifier means a flag indicating a profile on which the bitstream is based. For example, in H.264 / AVC, 66 means that the profile identifier is based on the baseline profile, 77 means that it is based on the main profile, and 88 means that it is based on the extended profile. The profile identifier may be included in a sequence parameter set.

Therefore, in order to handle a multiview image, a syntax for identifying whether an input bitstream is for a multiview profile, and for transmitting one or more additional information about a multiview if identified as a multiview profile. You need to add The multiview profile here refers to a profile mode that handles multiview video as an additional technique of H.264 / AVC. Since MVC is an additional technology to the existing AVC technology, it may be more efficient to add syntax as additional information for the MVC mode than an unconditional syntax. For example, when the profile identifier of the AVC indicates a multi-view profile, adding information about the multi-view image may increase encoding efficiency.

The sequence parameter set refers to header information that includes information that covers the entire sequence, such as profile and level. Since the entire compressed video, i.e., the sequence, must start from the sequence header, the sequence parameter set corresponding to the header information must arrive at the decoder before the data referring to the parameter set. After all, the sequence parameter set RBSP serves as header information for the result data of the video compression. When the bitstream is input, the profile identifier first identifies which of the plurality of profiles the input bitstream is based on. Thus, by adding a portion on the syntax that determines whether the input bitstream is for a multiview profile (e.g., "If (profile_idc == MULTI_VIEW_PROFILE)"), the input bitstream It is possible to add various attribute information only when it is determined whether or not to be a multi-view profile. For example, the number of all viewpoints, the number of inter-view reference pictures in the case of the anchor picture (List0 / 1), the number of inter-view reference pictures in the case of the non-anchor picture (List0 / 1), etc. may be added. . In addition, the decoded picture buffer may use information about a viewpoint to generate and manage a reference picture list.

1 is a schematic block diagram of a video signal decoding apparatus to which the present invention is applied.

The decoding apparatus includes a parsing unit 100, an entropy decoding unit 200, an inverse quantization / inverse transform unit 300, an intra prediction unit 400, a deblocking filter unit 500, a decoded picture buffer unit 600, Inter prediction unit 700 and the like. The decoded picture buffer unit 600 includes a reference picture storage unit 610, a reference picture list generator 630, a reference picture manager 650, and the like.

 The parsing unit 100 performs parsing on a NAL basis to decode the received video image. In general, one or more sequence parameter sets and picture parameter sets are transmitted to the decoder before the slice header and slice data are decoded. In this case, various attribute information may be included in the NAL header area or the extension area of the NAL header. For example, temporal level information, view level information, anchor picture identification information, view identifier information, and the like may be included.

Here, the temporal level information refers to information on a hierarchical structure for providing temporal scalability from a video signal. Through such temporal level information, it is possible to provide a user with images of various time zones. The viewpoint level information refers to information on a hierarchical structure for providing viewpoint scalability from a video signal. In a multi-view video image, it is necessary to define the levels of time and view in order to provide a user with images of various times and views. When defining the level information in this way, scalability with respect to time and time can be used. Accordingly, the user may view only an image of a desired time and time point, and may view only an image according to another constraint. The level information may be set differently in various ways according to the reference condition. For example, it may be set differently according to the position of the camera, or may be set differently according to the arrangement of the camera. In addition, the level information may be arbitrarily set regardless of a special criterion.

The viewpoint identification information refers to information for distinguishing a picture at a current view from a picture at a different view. When a video image signal is coded, a picture order count (POC) and frame_num are used to identify each picture. In the case of a multiview video image, since an inter-view prediction is performed, an identifier for distinguishing a picture at a current view from a picture at a different view is required. Therefore, it is necessary to define a view identifier indicating the viewpoint of the picture. The view identifier may be used to obtain information about a picture at a different point in time from the current picture, and the video signal may be decoded using the picture information at a different point in time. This view identifier may be applied throughout the encoding / decoding process of the video signal. In addition, the frame_num may be applied to a multi-view video coding as it is, using a frame_num in consideration of a view rather than a specific view identifier.

The parsed bitstream is entropy decoded by the entropy decoding unit 200, and coefficients, motion vectors, and the like of each macroblock are extracted. The inverse quantization / inverse transform unit 300 multiplies the received quantized value by a constant constant to obtain a transformed coefficient value, and inversely transforms the coefficient value to restore the pixel value. The intra prediction unit 400 performs intra prediction from the decoded samples in the current picture by using the reconstructed pixel value. Meanwhile, the deblocking filter unit 500 is applied to each coded macroblock in order to reduce block distortion. The filter smoothes the edges of the block to improve the quality of the decoded frame. The choice of filtering process depends on the boundary strength and the gradient of the image samples around the boundary. The filtered pictures are output or stored in the decoded picture buffer unit 600 for use as a reference picture.

The decoded picture buffer unit 600 stores or opens previously coded pictures in order to perform inter prediction. In this case, in order to store or open the decoded picture buffer unit 600, frame_num and POC (Picture Order Count) of each picture are used. Therefore, some of the previously coded pictures in MVC have pictures that are different from the current picture. Therefore, in order to utilize these pictures as reference pictures, not only the frame_num and the POC but also the view information for identifying the view point of the picture are included. It is available. The decoded picture buffer unit 600 includes a reference picture storage unit 610, a reference picture list generator 630, and a reference picture manager 650. The reference picture storage unit 610 stores pictures that are referenced for coding the current picture. The reference picture list generator 630 generates a list of reference pictures for inter prediction. In multi-view video coding, since inter-view prediction may be performed, it may be necessary to generate a reference picture list for inter-view prediction when the current picture refers to a picture at a different view. In this case, the reference picture list generator 630 may use information about a viewpoint to generate a reference picture list for inter-view prediction. This will be described in detail with reference to FIG. 2.

The reference picture list generator 630 includes a variable derivator 631, a reference picture list initializer 635, and a reference picture list rearranger 640. The variable derivation unit 631 derives variables used for initializing the reference picture list. For example, the variable may be derived by using frame_num, which is a unique identification number of a picture, or using view_id identifying a view point of a picture. The reference picture list initialization unit 635 initializes the reference picture list by using the variables. In this case, the initialization process of the reference picture list may vary depending on the slice type. For example, when decoding a P slice, a reference picture number may be assigned based on a decoding order, and when decoding a B slice, a reference picture number may be assigned based on a picture output order. In addition, when initializing the reference picture list for inter-view prediction, a number may be assigned to the reference picture based on the variable, that is, a variable derived from the viewpoint information. This will be described in detail with reference to FIGS. 3 to 5.

The reference picture list rearranger 640 improves the compression ratio by allocating a smaller number to a picture frequently referenced in the initialized reference picture list. The reference picture number for designating the reference picture is encoded in units of blocks because a smaller number of bits is assigned as the reference picture number for encoding the reference picture number is smaller. This will be described in detail with reference to FIG. 6.

In addition, the reference picture list reordering unit 640 includes a slice type checking unit 642, a reference picture list0 reordering unit 643, and a reference picture list1 reordering unit 645. When the initialized reference picture list is input, the slice type checking unit 642 determines whether to rearrange the reference picture list 0 or the reference picture list 1 by checking the type of the slice to be decoded. Accordingly, in the reference picture list 0/1 rearrangement units 643 and 645, for example, when the slice type is not I slice, the reference picture list 0 is rearranged, and when the slice type is B slice, the reference picture list is used. A rearrangement of 1 is also performed. When the rearrangement process is completed, the reference picture list is generated.

The reference picture list 0/1 rearrangement units 643 and 645 include identification information acquisition units 643A and 645A and reference number assignment changers 643B and 645B, respectively. When the identification information acquisition units 643A and 645A perform rearrangement of the reference picture list according to flag information indicating whether to rearrange the reference picture list, identification information indicating a method of assigning reference numbers (reordering_of_pic_nums_idc) Get input. Reference number assignment changing units 643B and 645B rearrange the reference picture list by changing the assignment of reference numbers according to the identification information. This will be described in detail with reference to FIGS. 7 and 8.

In addition, the reference picture list rearranger 640 may be performed by applying another method. For example, before passing through the slice type checking units 642A and 642B, the type of the NAL transmitted may be checked to be rearranged by dividing it into an MVC NAL and a non-MVC NAL. This will be described in detail with reference to FIGS. 9 to 12.

The reference picture manager 650 manages the reference picture in order to more flexibly implement inter prediction. For example, an adaptive memory management control method and a sliding window method may be used. This is to manage the memory of the reference picture and the non-reference picture into one memory and manage them efficiently with less memory. In multi-view video coding, since pictures in a view direction have the same picture order count, information for identifying the view point of each picture may be used for their marking. Reference pictures managed through this process may be used in the inter predictor 700.

The inter prediction unit 700 performs inter prediction using a reference picture stored in the decoded picture buffer unit 600. Inter-coded macroblocks can be divided into macroblock partitions, where each macroblock partition can be predicted from one or two reference pictures. The inter predictor 700 includes a motion compensator 710 and an illumination compensator 720.

The motion compensator 710 compensates for the motion of the current block by using the information transmitted from the entropy decoding unit 200. A motion vector of blocks neighboring the current block is extracted from the video signal, and a motion vector predictor of the current block is obtained. The motion of the current block is compensated by using the obtained motion vector predictor and the difference vector extracted from the video signal. In addition, such motion compensation may be performed using one reference picture or may be performed using a plurality of reference pictures. In multi-view video coding, when a current picture refers to pictures at different views, motion compensation is performed using information on a reference picture list for inter-view prediction stored in the decoded picture buffer unit 600. Can be done.

In addition, when the input bitstream corresponds to a multi-view image, since the view sequences are images obtained from different cameras, illumination differences may occur due to internal and external factors of the camera. In order to prevent this, the illumination compensation unit 720 performs illumination compensation. In addition, in performing the illumination compensation, information of a neighboring block or information of a block at a different time from the current block may be used to restore the current block, and an offset value of the current block may be used. Here, the offset value of the current block refers to a difference between the average pixel value of the current block and the average pixel value of the reference block corresponding thereto. In this case, when the current block refers to blocks at different views, illumination compensation may be performed using information on a reference picture list for inter-view prediction stored in the decoded picture buffer unit 600.

As such, inter predicted pictures and intra predicted pictures using lighting compensation, motion compensation, and the like are selected according to a prediction mode to reconstruct the current picture. Hereinafter, specific embodiments of encoding / decoding methods for managing a reference picture for the inter prediction will be described.

2 is an embodiment to which the present invention is applied and shows an internal block diagram of the reference picture list generator 630.

The reference picture list generator 630 generates a list of reference pictures for inter prediction. Here, the reference picture list generator may use information about a viewpoint to generate a reference picture list for inter-view prediction. The reference picture list generator 630 includes a variable derivator 631, a reference picture list initializer 635, and a reference picture list rearranger 640.

The variable derivation unit 631 derives variables used for initializing the reference picture list. For example, the variable may be derived using frame_num indicating an identification number of a picture. As a specific example, a variable FrameNum and a variable FrameNumWrap may be used for each short-term reference picture. First, the variable FrameNum is equal to the frame_num value that is a syntax element. The variable FrameNumWrap may be used by the decoded picture buffer unit 600 to allocate a small number for each reference picture, and may be derived from the variable FrameNum. The variable PicNum can be derived using the variable FrameNumWrap thus derived. The variable PicNum may refer to an identification number of a picture used in the decoded picture buffer unit 600. The variable LongTermPicNum may be used when representing a long term reference picture.

In addition, in order to generate a reference picture list for inter-view prediction, a first variable (eg, ViewNum) for generating a reference picture list for inter-view prediction may be derived using information about the viewpoint. For example, the second variable (eg, ViewId) may be derived by using view_id identifying a viewpoint of the picture. First, the second variable may be equal to the view_id value that is a syntax element. The third variable (eg, ViewIdWrap) may be used by the decoded picture buffer unit 600 to allocate a small view identification number for each reference picture, and may be derived from the second variable. Here, the first variable ViewNum may refer to a view identification number of a picture used in the decoded picture buffer unit 600. However, in multi-view video coding, since the number of reference pictures used for inter-view prediction may be relatively smaller than the number of reference pictures used for temporal prediction, a separate variable for indicating the view identification number of the long-term reference picture is defined. You can't.

The reference picture list initialization unit 635 initializes the reference picture list by using the variables. In this case, the initialization process of the reference picture list may vary depending on the slice type. For example, when decoding P slices, reference picture numbers may be allocated based on the decoding order, and when decoding B slices, reference picture numbers may be allocated based on the picture output order. In addition, when initializing a reference picture list for inter-view prediction, a number may be assigned to a reference picture based on the first variable, that is, a variable derived from viewpoint information. This will be described in detail with reference to FIGS. 3 to 5.

3 is a flowchart for generating a reference picture list as an embodiment to which the present invention is applied.

The decoded picture buffer unit 600 stores or opens previously coded pictures in order to perform inter prediction. First, pictures coded before the current picture are stored in the reference picture storage unit 610 to be used as a reference picture (S31). In the multi-view video coding, some of the previously coded pictures may be pictures that are at different views from the current picture. Therefore, in order to use these pictures as reference pictures, view information identifying a view point of a picture may be used. Therefore, the decoder should acquire viewpoint information for identifying the viewpoint of the picture (S32). For example, the viewpoint information may include view_id identifying a viewpoint of a picture. The decoded picture buffer unit 600 needs to derive a variable used in the decoded picture buffer unit 600 to generate a reference picture list. In multi-view video coding, since inter-view prediction may be performed, it may be necessary to generate a reference picture list for inter-view prediction when the current picture refers to a picture at a different view. At this time, the decoded picture buffer unit 600 needs to derive a variable used to generate a reference picture list for inter-view prediction using the obtained view information (S33).

A reference picture list for temporal prediction or a reference picture list for inter-view prediction may be generated in different ways according to the slice type of the current slice (S34). For example, when the slice type is a P / SP slice, a reference picture list 0 is generated (S35). When the slice type is a B slice, a reference picture list 0 and a reference picture list 1 are generated (S36). . In this case, the reference picture list 0 or the reference picture list 1 may include only a reference picture list for temporal prediction, or may include both a reference picture list for temporal prediction and a reference picture list for inter-view prediction. . This will be described in detail with reference to FIGS. 4 and 5.

The initialized reference picture list is subjected to a process of allocating a smaller number to the frequently referenced pictures in order to improve the compression ratio (S37). This is called a rearrangement process of the reference picture list. 6 to 12 will be described in detail. The current picture is decoded using the rearranged reference picture list, and the decoded picture buffer unit 600 needs to manage decoded reference pictures in order to operate the buffer more efficiently (S38). Reference pictures managed through this process are loaded from the inter prediction unit 700 and used for inter prediction. In multiview video coding, since the inter prediction may include inter-view prediction, in this case, a reference picture list for inter-view prediction may be used.

4 and 5 will be described through specific embodiments of how the reference picture list is generated according to the slice type.

4 is a diagram for describing a method of initializing a reference picture list when a current slice is a P slice according to an embodiment to which the present invention is applied.

T ₀ , T ₁ ,... , T _N represents time, and V ₀ , V ₁ ,... , V ₄ represents a time point. For example, the current picture represents a picture at T ₃ time at the time V ₄ . In addition, the slice type of the current picture is a P slice. PN stands for variable PicNum, LPN stands for variable LongtermPicNum, and VN stands for variable ViewNum. The number after each variable means an index indicating a time (for PN or LPN) or a viewpoint (for VN) of each picture. The same applies to FIG. 5.

The reference picture list for temporal prediction or the reference picture list for inter-view prediction may be generated in different ways according to the slice type of the current slice. For example, in FIG. 4, when the slice type is a P / SP slice, a reference picture list 0 is generated in this case. The reference picture list 0 may include a reference picture list for temporal prediction and / or a reference picture list for inter-view prediction. In the present embodiment, a case in which the reference picture list 0 includes both a reference picture list for temporal prediction and a reference picture list for inter-view prediction will be described. There may be various ways to arrange the reference pictures. For example, the reference pictures may be arranged in decoding order, or the reference pictures may be arranged in picture output order. Alternatively, the reference pictures may be arranged based on a variable derived using the viewpoint information, or the reference pictures may be arranged according to view dependency information.

In the case of a reference picture list for temporal prediction, the short-term reference picture and the long-term reference picture may be arranged based on the decoding order. For example, it may be arranged according to the value of a variable (PicNum or LongtermPicNum) derived from a value (eg, frame_num or Longtermframeidx) indicating an identification number of a picture. First, the short-term reference pictures may be initialized prior to the long-term reference picture. The order in which the short-term reference pictures are arranged may be arranged in order from the reference picture having the highest variable PicNum value among the reference pictures to the reference picture having the lowest variable value. For example, starting from PN1 having the highest variable of PN0, PN1, PN2, then PN2, then PN0 having the lowest variable, may be arranged. The order in which the long-term reference picture is arranged may be arranged in the order of the reference picture having the lowest variable (LongtermPicNum) value from the reference picture to the reference picture having the highest variable value. For example, starting from LPN0 having the lowest variable among LPN0, LPN1, it may then be arranged in LPN1 order having the lowest variable.

In the case of a reference picture list for inter-view prediction, the reference picture list may be arranged based on the first variable ViewNum derived using the viewpoint information. For example, the reference picture may be arranged in order from the reference picture having the highest first variable ViewNum value among the reference pictures to the reference picture having the lowest variable value. For example, it may be arranged starting from VN3 having the highest variable of VN0, VN1, VN2, VN3, and then in the order of VN2, VN1, then VN0 having the lowest variable.

As such, the reference picture list for temporal prediction and the reference picture list for inter-view prediction may be managed as one reference picture list or may be managed as separate reference picture lists, respectively. In the case of managing one reference picture list, the information may be initialized in order or may be initialized at the same time. In the case of being initialized in order, for example, the reference picture list for the temporal prediction may be initialized first, and then the reference picture list for the inter-view prediction may be additionally initialized. This concept can also be applied to FIG. 5. Hereinafter, a case in which the slice type of the current picture is a B slice will be described.

FIG. 5 is a diagram for explaining a method of initializing a reference picture list when a current slice is a B slice as an embodiment to which the present invention is applied.

If the slice type is a B slice, reference picture list 0 and reference picture list 1 are generated. In this case, the reference picture list 0 or the reference picture list 1 may include only a reference picture list for temporal prediction, or may include both a reference picture list for temporal prediction and a reference picture list for inter-view prediction. .

First, in the case of a reference picture list for temporal prediction, a method of arranging short-term reference pictures and long-term reference pictures may be different. For example, in the case of a short-term reference picture, reference pictures may be arranged according to a picture output order (hereinafter, referred to as a POC). In the case of a long-term reference picture, reference pictures may be arranged according to a variable (LongtermPicNum) value. Can be. The short-term reference pictures may be initialized before the long-term reference picture.

The order of the short-term reference pictures of reference picture list 0 is arranged in the order of the reference picture having the lowest POC value from the reference picture having the highest POC value among the reference pictures having the lower POC value than the current picture, and then The reference picture having the lowest POC value among the reference pictures having a higher POC value than the current picture may be arranged in order from the reference picture having the highest POC value. For example, starting from PN1 having the highest POC value among the reference pictures PN0 and PN1 having a lower POC value than the current picture, arranged in PN0 order, and then a reference picture having a higher POC value than the current picture. Starting from PN3 having the lowest POC value among PN3 and PN4, they may be arranged in the order of PN4.

The order in which the long-term reference pictures of the reference picture list 0 are arranged may be arranged in the order of the reference picture having the lowest variable (LongtermPicNum) among the reference pictures from the reference picture having the highest variable. For example, starting from LPN0 having the lowest variable among LPN0, LPN1, it may then be arranged in LPN1 order having the lowest variable.

In the case of a reference picture list for inter-view prediction, the reference picture list may be arranged based on the first variable ViewNum derived using the viewpoint information. For example, in the case of the reference picture list 0 for inter-view prediction, the reference picture having the lowest first variable value from the reference picture having the highest first variable value among the reference pictures having the first variable value lower than the current picture is used. Can be arranged in order. The reference picture having the first variable value among the reference pictures having the first variable value higher than the current picture may be arranged in order from the reference picture having the highest first variable value. For example, starting from VN1 having the highest first variable value among the reference pictures VN0 and VN1 having the first variable value lower than the current picture, they may be arranged in the order of VN0 having the lowest first variable value. Then, starting from VN3 having the lowest first variable value among the reference pictures VN3 and VN4 having the first variable value higher than the current picture, they may be arranged in the order of VN4 having the highest first variable value.

In the case of the reference picture list 1, it may be applied similarly to the arrangement of the reference picture list 0 described above.

First, in the case of the reference picture list for temporal prediction, the arrangement order of the short-term reference pictures of the reference picture list 1 has the highest POC value from the reference picture having the lowest POC value among the reference pictures having a higher POC value than the current picture. The reference pictures may be arranged in the order of the reference pictures, and then may be arranged in the order of the reference pictures having the lowest POC value from the reference picture having the highest POC value among the reference pictures having a lower POC value than the current picture. For example, starting from PN3 having the lowest POC value among the reference pictures PN3 and PN4 having a higher POC value than the current picture, arranged in PN4 order, and then being a reference picture having a POC value lower than the current picture. Starting from PN1 having the highest POC value among PN0 and PN1, they may be arranged in the order of PN0.

The order in which the long-term reference pictures of the reference picture list 1 are arranged may be arranged in the order of the reference picture having the lowest variable (LongtermPicNum) among the reference pictures from the reference picture having the highest variable. For example, starting from LPN0 having the lowest variable among LPN0, LPN1, it may then be arranged in LPN1 order having the lowest variable.

In the case of a reference picture list for inter-view prediction, the reference picture list may be arranged based on the first variable ViewNum derived using the viewpoint information. For example, in the case of reference picture list 1 for inter-view prediction, the reference picture having the highest first variable value from the reference picture having the lowest first variable value among the reference pictures having the first variable value higher than the current picture Can be arranged in order. The reference picture having the first variable value among the reference pictures having the first variable value lower than the current picture may be arranged in the order of the reference picture having the lowest first variable value. For example, starting from VN3 having the lowest first variable value among the reference pictures VN3 and VN4 having the first variable value higher than the current picture, they may be arranged in the order of VN4 having the highest first variable value. Then, starting from VN1 having the highest first variable value among the reference pictures VN0 and VN1 having the first variable value lower than the current picture, they may be arranged in the order of VN0 having the lowest first variable value.

The reference picture list initialized through the above process is transmitted to the reference picture list reordering unit 640 to rearrange the initialized reference picture list for more efficient coding. This rearrangement process is to reduce the bit rate by operating a decoded picture buffer and assigning a low number to a reference picture most likely to be selected as a reference picture. 6 to 12, a method of rearranging the reference picture list will be described with reference to various embodiments.

FIG. 6 illustrates an internal block diagram of the reference picture list rearranging unit 640 according to an embodiment to which the present invention is applied.

The reference picture list reordering unit 640 largely includes a slice type checking unit 642, a reference picture list0 reordering unit 643, and a reference picture list1 reordering unit 645. The reference picture list reordering unit 643 includes a first identification information obtaining unit 643A and a first reference number assignment changing unit 643B, and the reference picture list first reordering unit 645 includes second identification information. An acquiring unit 645A and a second reference number assignment changing unit 645B.

The slice type checking unit 642 checks the slice type of the current slice and determines whether to reorder the reference picture list 0 and / or the reference picture list 1 according to the slice type. For example, when the slice type of the current slice is I slice, neither the reference picture list 0 nor the reference picture list 1 perform rearrangement. In case of P slice, rearrangement is performed only for reference picture list 0, and in case of B slice, rearrangement is performed for both reference picture list 0 and reference picture list 1.

The reference picture list 0 rearrangement unit 643 operates when the slice type of the current slice is not an I slice and flag information for reordering the reference picture list 0 is true. The first identification information obtaining unit 643A obtains identification information indicating a reference number assignment method, and the first reference number assignment changing unit 643B assigns each reference picture of the reference picture list 0 according to the identification information. Will change the reference number.

Similarly, the reference picture list reordering unit 645 operates when the slice type of the current slice is a B slice and flag information for reordering the reference picture list 1 is true. The second identification information acquisition unit 645A obtains identification information indicating a reference number assignment method, and the second reference number assignment changer 645B assigns each reference picture of the reference picture list 1 according to the identification information. Will change the reference number.

Reference picture list information used for actual inter prediction is generated through the reference picture list reordering unit 643 and the reference picture list reordering unit 645. In FIG. 7, a method of changing reference numbers assigned to respective reference pictures by the first and second reference number assignment changing units 643B and 645B will be described in detail.

7 shows an internal block diagram of reference number assignment changing units 643B and 645B as an embodiment to which the present invention is applied.

In the present embodiment, the reference picture list 0 rearrangement unit 643 and the reference picture list 1 rearrangement unit 645 shown in FIG. 6 will be described together. The first and second reference number assignment changing units 643B and 645B include a reference number assignment changing unit 644A for temporal prediction, a reference number assignment changing unit 644B for long term reference pictures, and reference number assignment for inter-view prediction. A change section 644C and a reference number assignment change end section 644D. The parts of the first and second reference number assignment changing units 643B and 645B operate according to the identification information obtained by the first and second identification information acquisition units 643A and 645A. This rearrangement process is performed until identification information is received to terminate the reference number assignment change.

For example, when identification information is transmitted from the first and second identification information acquisition units 643A and 645A to change the allocation of the reference number for the temporal prediction, the reference number assignment changing unit 644A for the temporal prediction is transmitted. Will work. The reference number assignment changing unit 644A for temporal prediction obtains a difference value of the picture number according to the identification information. Here, the difference value of the picture number may mean a difference between a picture number of the current picture and a predicted picture number, and the predicted picture number may mean a number of a reference picture allocated immediately before. The allocation of the reference number may be changed by using the difference value of the picture number thus obtained. At this time, the difference value of the picture number may be added or subtracted from the predicted picture number according to the identification information.

As another example, when identification information for allocating a reference number to a designated long-term reference picture is transmitted, the reference number assignment changing unit 644B for the long-term reference picture is operated. The reference number assignment changing unit 644B for the long term reference picture acquires a long term reference picture number of a picture designated according to the identification information.

As another example, when identification information for changing the assignment of the reference number for the inter-view prediction is transmitted, the reference number assignment changing unit 644C for the inter-view prediction operates. The reference number assignment changer 644C for inter-view prediction obtains a difference value of the view information according to the identification information. Here, the difference value of the view information may mean a difference between a view number of the current picture and a predicted view number, and the predicted view number may mean a view number of a reference picture immediately allocated. The allocation of the reference number may be changed using the difference value of the viewpoint information thus obtained. At this time, the difference value of the viewpoint information may be added or subtracted from the predicted viewpoint number according to the identification information.

As another example, when identification information for terminating a reference number assignment change is transmitted, the reference number assignment change end unit 644D operates. The reference number assignment change end unit 644D ends the assignment change of the reference number according to the identification information, and accordingly, the reference picture list rearranger 640 generates the reference picture list information.

As such, reference pictures used for inter-view prediction may be managed together with reference pictures used for temporal prediction. Alternatively, reference pictures used for inter-view prediction may be managed separately from reference pictures used for temporal prediction. In this case, new information for managing reference pictures used for the inter-view prediction may be needed. This case will be described in detail with reference to FIGS. 9 to 12. Hereinafter, the reference number assignment changing unit 644C for inter-view prediction will be described with reference to specific embodiments.

FIG. 8 is a diagram for explaining a process of rearranging a reference picture list using viewpoint information according to an embodiment to which the present invention is applied.

In the present embodiment, when the view number VN of the current picture is 3, the size of the decoded picture buffer is 4, and the slice type of the current slice is P slice, the rearrangement process of the reference picture list 0 will be described. The initial predicted view number is 3, which is the view number of the current picture, and the first array of the reference picture list 0 for inter-view prediction is 4, 5, 6, 2 (1). At this time, the identification information is transmitted to subtract the difference value of the viewpoint information to change the allocation of the reference number for the inter-view prediction, and obtains 1 as the difference information of the viewpoint information according to the identification information. The newly predicted viewpoint number (= 2) is calculated by subtracting the viewpoint information difference value (= 1) from the predicted viewpoint number (= 3). That is, the first index of the reference picture list 0 for inter-view prediction is allocated to the reference picture having the view number 2. The picture assigned to the first first index may be moved to the last part of the reference picture list 0. Thus, the rearranged reference picture list 0 is 2, 5, 6, and 4 (2). Then, identification information is transmitted to subtract the difference value of the viewpoint information again to change the assignment of the reference number for the inter-view prediction, and obtain -2 as the difference information of the viewpoint information according to the identification information. The predicted viewpoint number (= 2) is subtracted from the viewpoint information difference value (= -2) to calculate a newly predicted viewpoint number (= 4). That is, the second index of the reference picture list 0 for inter-view prediction is allocated to the reference picture having the view number 4. Thus, the rearranged reference picture list 0 is 2, 4, 6, 5 (3). Next, when identification information for terminating the reference number assignment change is transmitted, reference picture list 0 is generated after the rearranged reference picture list 0 according to the identification information (4). As a result, the array of the reference picture list 0 for the last generated inter-view prediction is 2, 4, 6, 5.

As another example of rearranging the remaining pictures after allocating the first index of the reference picture list 0 for inter-view prediction, the pictures allocated to the respective indexes may be sequentially moved to the immediately following positions. That is, the second index is assigned to the picture having the viewpoint number 4, the third index is assigned to the picture having the viewpoint number 5 to which the second index has been assigned, and the fourth index is assigned to the picture having the viewpoint number 6 to which the third index has been assigned. It becomes possible. Thus, the rearranged reference picture list 0 becomes 2, 4, 5, 6. The rearrangement process can then be applied equally.

The reference picture list generated through the above process is used for inter prediction. The reference picture list for inter-view prediction may be managed as one reference picture list together with the reference picture list for temporal prediction. In addition, it may be managed as a separate reference picture list, which will be described with reference to FIGS. 9 to 12.

9 is a block diagram illustrating an internal reference picture list reordering unit 640 according to another embodiment to which the present invention is applied.

New information may be needed to manage the reference picture list for inter-view prediction as a separate reference picture list. For example, first, the reference picture list for temporal prediction may be rearranged, and in some cases, the reference picture list for inter-view prediction may be rearranged.

In the present embodiment, the reference picture list rearranger 640 largely includes a reference picture list rearranger 910, a NAL type checker 960, and a reference picture list rearranger 970 for inter-view prediction for temporal prediction. Include. The reference picture list rearranging unit 910 for temporal prediction obtains a slice type checking unit 642, a third identification information obtaining unit 920, a third reference number assignment changing unit 930, and a fourth identification information obtaining unit. A unit 940 and a fourth reference number assignment changer 950 are included. The third reference number assignment changer 930 includes a reference number assignment changer 930A for temporal prediction, a reference number assignment changer 930B for a long term reference picture, and a reference number assignment change terminator 930C. do. Similarly, the fourth reference number assignment changer 950 includes a reference number assignment changer 950A for temporal prediction, a reference number assignment changer 950B for a long term reference picture, and a reference number assignment change terminator 950C. Include.

The reference picture list rearranger 910 for temporal prediction rearranges reference pictures used for temporal prediction. The operation of the reference picture list rearranger 910 for temporal prediction is identical to the operation of the reference picture list rearranger 640 except for information about the reference picture for inter-view prediction described with reference to FIG. 6. . Therefore, since it can be inferred from FIG. 6, description thereof will be omitted.

The NAL type checking unit 960 checks the NAL type of the transmitted bitstream. When the NAL type is NAL for multi-view image coding, the reference pictures list rearranger 970 for inter-view prediction rearranges reference pictures used for inter-view prediction. The reference picture list for the inter-view prediction generated as described above is used for inter prediction together with the reference picture list for temporal prediction generated by the reference picture list rearranging unit 910 for temporal prediction. However, when the NAL type is not NAL for multiview image coding, the reference picture list for inter-view prediction is not rearranged. In this case, only the reference picture list for temporal prediction is generated. The reference picture list rearranging unit 970 for inter-view prediction rearranges reference pictures used for inter-view prediction. This will be described in detail with reference to FIG. 10.

FIG. 10 is an embodiment to which the present invention is applied and shows an internal block diagram of a reference picture list rearranger 970 for inter-view prediction.

The reference picture list rearranging unit 970 for inter-view prediction includes a slice type checking unit 642, a fifth identification information obtaining unit 971, a fifth reference number assignment changing unit 972, and a sixth identification information obtaining unit ( 973) and a sixth reference number assignment changer 974.

The slice type checking unit 642 checks the slice type of the current slice and determines whether to reorder the reference picture list 0 and / or the reference picture list 1 according to the slice type. A detailed description of the slice type checking unit 642 will be omitted since it can be inferred from FIG. 6.

The fifth and sixth identification information acquisition units 971 and 973 acquire identification information indicating a reference number assignment method, and the fifth and sixth reference number assignment changing units 972 and 974 respectively refer to the reference picture list 0, according to the identification information. The reference number assigned to each reference picture of 1 is changed. Here, the reference number may mean only the view number of the reference picture. In addition, the identification information indicating the reference number assignment method may be flag information. For example, if the flag information is true, the allocation of the viewpoint number may be changed, and if the flag information is false, the rearrangement process of the viewpoint number may be terminated. If the flag information is true, the fifth and sixth reference number assignment changing units 972 and 974 may obtain a difference value of the view number according to the flag information. Here, the difference value of the view number may mean a difference between the view number of the current picture and the view number of the predicted picture, and the view number of the predicted picture may mean the view number of the reference picture immediately allocated. The allocation of the viewpoint number may be changed by using the difference value of the viewpoint number thus obtained. In this case, the difference value of the viewpoint number may be added or subtracted from the viewpoint number of the predicted picture according to the identification information.

As such, in order to manage the reference picture list for inter-view prediction as a separate reference picture list, the syntax structure needs to be newly defined. Therefore, as an embodiment of the contents described with reference to FIGS. 9 and 10, the syntaxes of the schemes will be described with reference to FIGS.

11A and 11B illustrate syntax for reordering reference picture lists as an embodiment to which the present invention is applied.

FIG. 11A illustrates the operation of the reference picture list rearranging unit 910 for temporal prediction shown in FIG. 9. Compared with the blocks of FIG. 9, the slice type checking unit 642 corresponds to S1 and S6, and the third identification information obtaining unit 920 is S2 and the fourth identification. The information acquisition unit 940 corresponds to (S7). The internal block diagrams of the third reference number assignment changer 930 correspond to S3, S4, and S5, respectively, and the internal block diagrams of the fourth reference number assignment changer 950 respectively. It corresponds to (S8), (S9), and (S10), respectively.

FIG. 11B is a syntax diagram illustrating operations of the NAL type checking unit 960 and the reference picture list rearranging unit 970 for inter-view prediction shown in FIGS. 9 and 10. 9 and 10, the NAL type checking unit 960 corresponds to S11, and the slice type checking unit 642 corresponds to S12 and S15. The fifth identification information acquisition unit 971 corresponds to S13, and the sixth identification information acquisition unit 973 corresponds to S16. The fifth reference number assignment changer 972 corresponds to S14, and the sixth reference number assignment changer 974 corresponds to S17.

12 illustrates a syntax for reordering a reference picture list according to another embodiment to which the present invention is applied.

12 illustrates an example of another syntax of operations of the NAL type checking unit 960 and the reference picture list rearranging unit 970 for inter-view prediction shown in FIGS. 9 and 10. 9 and 10, the NAL type checking unit 960 corresponds to S21, and the slice type checking unit 642 corresponds to S22 and S25. The fifth identification information acquisition unit 971 corresponds to (S23), and the sixth identification information acquisition unit 973 corresponds to (S26). The fifth reference number assignment changer 972 corresponds to S24, and the sixth reference number assignment changer 974 corresponds to S27.

As mentioned above, preferred embodiments of the present invention are disclosed for purposes of illustration, and those skilled in the art can improve and change various other embodiments within the spirit and technical scope of the present invention disclosed in the appended claims below. , Replacement or addition would be possible.

As described above, the present invention provides a method for managing reference pictures used for inter-view prediction in coding multi-view images acquired by a plurality of cameras, thereby enabling more efficient coding. In addition, by providing a method of initializing and rearranging a reference picture list for inter-view prediction, coding can be performed more efficiently. When performing inter-view prediction using the present invention, the coding speed can be improved by reducing the burden of a decoded picture buffer (DPB), and more accurate prediction can be performed, thereby reducing the number of bits to be transmitted. You can also

Claims (14)

Obtaining viewpoint information identifying a viewpoint of a picture from the video signal; Generating information for reference picture management using the viewpoint information Video signal decoding method comprising a. The method of claim 1, wherein the generating of the reference picture management information comprises: Generating a reference picture list for inter-view prediction using the viewpoint information Video signal decoding method comprising a. The method of claim 2, wherein the generating of the reference picture list for inter-view prediction includes: Deriving a first variable for generating a reference picture list for the inter-view prediction using the viewpoint information, And a reference picture list for generating the inter-view prediction using the first variable. The method of claim 3, wherein the deriving of the first variable comprises: And a second variable set from the viewpoint information and a third variable derived from the second variable. The method of claim 3, wherein the generating of the reference picture list for the inter-view prediction includes: And initializing a reference picture list for the inter-view prediction. The method of claim 5, wherein when initializing the reference picture list for the inter-view prediction, The reference picture list for the inter-view prediction is arranged based on the first variable. The method of claim 5, wherein when initializing the reference picture list for the inter-view prediction, And a reference picture list for inter-view prediction is arranged based on a decoding order. The method of claim 5, wherein when initializing the reference picture list for the inter-view prediction, The reference picture list for the inter-view prediction is arranged in a different manner according to the slice type. The method of claim 8, wherein the generating of the reference picture list for the inter-view prediction includes: When the slice type is a P / SP slice, the reference picture list 0 for inter-view prediction is generated using the first variable, and the reference picture list 0 for the inter-view prediction is the highest first among the reference pictures. And from the reference picture having the variable to the picture having the lowest first variable. The method of claim 8, wherein the generating of the reference picture list for the inter-view prediction includes: When the slice type is a B slice, a reference picture list 0 for inter-view prediction is generated using the first variable, and the reference view list 0 for inter-view prediction is lower than the first variable of the current picture. The first picture is sorted in order from the reference picture having the highest first variable to the reference picture having the lowest first variable, and then the highest first variable is added starting from the reference picture having the lowest first variable that is higher than the first variable of the current picture. And a video signal decoding method arranged in a reference picture order. The method of claim 8, wherein the generating of the reference picture list for the inter-view prediction includes: When the slice type is a B slice, the reference picture list 1 for inter-view prediction is generated using the first variable, and the reference view list 1 for the inter-view prediction is higher than the first variable of the current picture and is the lowest. The first picture is sorted in order from the reference picture having the first variable to the highest first variable, and then the lowest first variable starting from the reference picture having the highest first variable is lower than the first variable of the current picture. And a video signal decoding method arranged in a reference picture order. The method of claim 2, wherein the video signal decoding method comprises: And rearranging the reference picture list for the inter-view prediction using the view information. The method of claim 2, wherein the video signal decoding method comprises: And using the view information, marking a reference picture of a reference picture list for inter-view prediction. A view point information acquisition unit for obtaining view point information identifying a view point of a picture from a video signal; A variable derivation unit for deriving a first variable for generating a reference picture list using the viewpoint information; And A reference picture list initialization unit which generates the reference picture list using the first variable Video signal decoding apparatus comprising a.

Images