JP2018129688A - Video encoding device, video encoding/decoding system, and video encoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video encoding device, a video encoding/decoding system, and a video encoding program capable of preventing an increase in the amount of codes due to an intraframe code and reducing the amount of computation required for decoding an ROI.SOLUTION: A video encoding device X1 includes an ROI (Region Of Interest) setting unit that sets an ROI in a picture and generates ROI information representing the position and size of the ROI in the picture and an encoder unit U1 that executes compression coding processing for an input image signal on the basis of the ROI information in response to the input image signal and the ROI information to generate a bit stream signal. On the basis of the ROI information, the encoder unit executes intra coding as the compression coding processing for an image block included in the ROI, and selects and executes one of intra coding and inter coding as the compression coding processing for image blocks included in the non-ROI.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a video encoding device that encodes a video signal, a video encoding / decoding system that encodes / decodes a video signal, and a video encoding program.

  In recent years, surveillance cameras have become popular, and further higher resolution, higher frame rate, and multi-viewpoint are desired. However, increasing the resolution, increasing the frame rate, and increasing the number of viewpoints cause a significant increase in the amount of moving image data, resulting in increased communication costs and storage costs.

  In order to alleviate this problem, for example, a specific region in which “face” is displayed is detected as a ROI (Region of Interest) from a moving image, and a large number of bits are assigned to this ROI to perform image compression coding. An image processing apparatus that performs processing has been proposed (see, for example, Patent Document 1).

  In the image processing apparatus, the information amount of the area other than the ROI, that is, the background area is reduced from the information amount of the ROI by using the function of the compression unit that sets the parameter for controlling the code amount and the video quality for each block. As described above, the data amount is reduced by controlling the parameters given to the compression unit.

  On the other hand, when using a compression unit that does not have a function of setting a parameter for controlling the code amount and video quality for each block due to circumstances such as price and compatibility, the amount of information is It cannot be reduced.

  Therefore, in order to reduce the amount of information in the background area without depending on the compression unit, the amount of information in the background area is reduced by filtering the background area with a low-pass filter to remove high-frequency component information. An image coding apparatus has been proposed (see, for example, Patent Document 2). In this image encoding device, the generation of artifacts occurring at block boundaries is suppressed while reducing the amount of image signal information by filtering with a low-pass filter.

JP 2009-49979 A JP-A-4-219089

  By the way, the following problems occur in the conventional image processing as described above. In other words, in order to reduce the amount of computation required for decoding, when so-called intraframe coding is performed in which an image signal is compressed within each frame without referring to other pictures, reference to other pictures is performed. Compared with the allowable interframe coding, the code amount is significantly increased. Therefore, it becomes difficult to reduce the code amount significantly.

  On the other hand, when inter-frame coding is performed, the amount of code can be reduced, but there is a problem that the amount of calculation at the time of decoding becomes large. That is, when an arbitrary picture including an ROI is decoded with respect to an encoded image signal subjected to inter-frame coding, if the picture refers to another picture, before decoding the ROI In addition, it is necessary to decode other pictures. Further, when the other picture refers to another picture, it is necessary to decode this other picture first. As described above, when inter-frame coding is performed, when decoding any picture including ROI, one or more pictures must be decoded in advance. Therefore, the amount of calculation required for ROI decoding is small. The problem of becoming larger arises.

  Therefore, an object of the present invention is to provide a video encoding device, a video encoding / decoding system, and a video encoding program capable of achieving both a reduction in code amount and a reduction in calculation amount.

  A video encoding apparatus according to the present invention is a video encoding apparatus that generates a bitstream signal by performing compression encoding processing on an input image signal, and sets an ROI (Region Of Interest) in a picture, and the picture A ROI setting unit that generates ROI information representing the position and size of the ROI within the network, and receives the input image signal and the ROI information, and performs compression encoding processing on the input image signal based on the ROI information An encoder unit that generates a bitstream signal, and the encoder unit performs compression coding processing on an image block included in the ROI in a picture in the input image signal based on the ROI information. Intra coding is performed, and for the image blocks included in the non-ROI, the intra coding and intra coding are performed as the compression coding processing. Applied by selecting one of the over coding.

  A video encoding / decoding system according to the present invention includes: a video encoding device that performs compression encoding processing on an input image signal to generate a bitstream signal; and a video decoding device that performs decoding processing on the bitstream signal. An image encoding / decoding system including: an ROI setting unit configured to set an ROI (Region Of Interest) in a picture and generate ROI information indicating a position and a size of the ROI in the picture An upper syntax generation unit that generates upper syntax information including a parameter set necessary for decoding, and receives the input image signal and the ROI information, and compresses and encodes the input image signal based on the ROI information A bit stream signal obtained by multiplexing an elementary stream signal obtained by processing and the upper syntax information An encoder unit for outputting, based on the ROI information, the encoder unit performs intra coding as the compression coding process on an image block included in the ROI in a picture in the input image signal. For the image blocks included in the non-ROI, one of the intra coding and the inter coding is selected as the compression coding processing, and the video decoding device receives the bit stream signal, A decoded image selection unit that extracts the parameter set included in the higher-level syntax information from the bit stream signal and an access unit corresponding to the elementary stream signal, and a decoding process based on the parameter set for the access unit And a video decoding unit that generates an output image.

  A video encoding program according to the present invention is a video encoding program executed by a control unit of an information processing apparatus that generates a bitstream signal by performing compression encoding processing on an input image signal, and includes a ROI (Region A first step of generating ROI information representing the position and size of the ROI within the picture, receiving the input image signal and the ROI information, and receiving the input based on the ROI information A second step of generating a bitstream signal by performing compression coding processing on the image signal, and in the second step, based on the ROI information, the ROI in the picture in the input image signal Intra coding is performed as the compression coding process for the included image block, and for the image block included in the non-ROI Applied by selecting one of said intra-coded and inter-coded as serial compression coding processing.

  In the present invention, an ROI is set in a picture, ROI information representing the position and size of the ROI in the picture is generated, and an input image signal is compressed and encoded based on the ROI information D01. The following processing is performed. That is, based on the ROI information, the image block included in the ROI is subjected to intra coding as a compression coding process, and one of the intra coding and the inter coding is performed on the image block group included in the non-ROI. Select and apply.

  As a result, it is possible to reduce the code amount as compared with a case where all image blocks in a picture are compression-encoded by intra-encoding. In addition, the amount of calculation can be reduced as compared with the case where all image blocks in a picture are compression-encoded by inter-encoding.

  Therefore, according to the present invention, it is possible to achieve both a reduction in code amount and a reduction in calculation amount.

It is a block diagram which shows an example of a structure of the video encoding apparatus X1. It is a flowchart which shows the video coding process of the image coding part U1. It is a block diagram which shows the structure of ROI identification system RIS as an example of a video processing system. It is a flowchart which shows an example of the decoding image selection process which the decoding image selection apparatus X2 implements. It is a flowchart which shows another example of the decoding image selection process which the decoding image selection apparatus X2 implements. It is a block diagram which shows another example of a structure of the video encoding apparatus X1. It is a flowchart which shows an example of the decoding image selection process which the video coding apparatus X1 which consists of a structure shown in FIG. 6 implements. It is a block diagram which shows the structure of the information processing apparatus which performs a video encoding process by a program.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a video encoding device X1 according to the present invention. As shown in FIG. 1, the video encoding device X1 includes a ROI setting unit 301 and an encoder unit U1. The encoder unit U1 includes an intra prediction mode designating unit 102 and a prediction mode variable encoder unit U2. The prediction mode variable encoder unit U2 includes a prediction mode determination unit 103, an encoder core unit U3, an upper syntax generation unit 106, and a bit stream generation unit 307.

  Hereinafter, the operations of the ROI setting unit 301, the intra prediction mode specifying unit 102, the prediction mode variable encoder unit U2, the prediction mode determining unit 103, the encoder core unit U3, the upper syntax generation unit 106, and the bit stream generation unit 307 will be described. 2 will be described with reference to the flowchart showing the video encoding process shown in FIG.

  First, the ROI setting unit 301 receives an input image signal E1 including a series of pixel values representing the luminance level of each pixel, sets the ROI for the picture represented by the input image signal E1, and sets the ROI thus set. ROI information D01 representing the position and size in the picture is generated (step S101).

  For example, the ROI setting unit 301 detects, as an ROI, an area in which a human face is displayed from the input image signal E1 using a face detection algorithm that detects a human face as a specific object. Then, the ROI setting unit 301 sets ROI information D01 representing the position (coordinates) and size of the ROI in the picture. The ROI setting unit 301 may set the ROI arbitrarily designated by the user without depending on the input image signal E1, and generate the ROI information D01 representing the position and size of the ROI in the picture. . The ROI setting unit 301 supplies the ROI information D01 to the intra prediction mode designating unit 102 and the encoder core unit U3.

  Next, the intra prediction mode designating unit 102 generates an intra prediction mode designating signal D02 that designates the intra prediction mode for the image block included in the ROI based on the ROI information D01 (step S102). For example, the encoder core unit U3 uses H.264 as an image encoding method. When the H.264 / MPEG (Moving Picture Experts Group) -4 AVC is adopted, the intra prediction mode designation signal D02 is, for example, an image block (hereinafter referred to as an intra macroblock or an intra MB) composed of 16 pixels × 16 pixels. Specify the intra prediction mode with. The intra prediction mode designation unit 102 supplies the intra prediction mode designation signal D02 to the prediction mode determination unit 103.

  Next, the prediction mode determination unit 103 determines a prediction mode according to the presence or absence of the intra prediction mode designation signal D02, and generates prediction mode information D03 indicating the determined prediction mode (step S103). That is, the prediction mode determination unit 103 generates prediction mode information D03 indicating the intra prediction mode for the image block designated as the intra prediction mode by the intra prediction mode designation signal D02.

  On the other hand, for an image block that is not designated as the intra prediction mode by the intra prediction mode designation signal D02, the prediction mode determination unit 103 selects an appropriate prediction mode (intra prediction mode or inter prediction mode) from among various prediction modes. ) And prediction mode information D03 indicating the determined prediction mode is generated. For example, the prediction mode determination unit 103 selects one of the intra prediction mode and the inter prediction mode based on the picture represented by the input image signal E1, and predicts the prediction mode information indicating the selected prediction mode. D03 is generated. The prediction mode determination unit 103 supplies the prediction mode information D03 to the encoder core unit U3.

  Next, the encoder core unit U3 outputs an elementary stream signal D05 obtained by subjecting the input image signal E1 to compression coding processing according to the prediction mode information D03 (step S105). The encoder core unit U3 includes a preprocessing unit 305A and a compression unit 305B. The preprocessing unit 305A receives the input image signal E1 and the ROI information D01, and controls the image quality by performing the following preprocessing on the input image signal E1.

  That is, the pre-processing unit 305A determines that the pixel value series included in the input image signal E1 corresponds to a region other than the ROI (hereinafter referred to as a background region or a non-ROI) based on the ROI information D01. The value is subjected to an encoding process such that the code amount is smaller than that of the pixel value determined to correspond to the ROI. Such an encoding process is a process of suppressing the high frequency component of the input image signal E1 by convolving the input image signal E1 using, for example, a Gaussian filter.

  The preprocessing unit 305A supplies a preprocessed image signal obtained by performing the above-described preprocessing on the input image signal E1 to the compression unit 305B. That is, the pre-processing unit 305A performs image quality control for reducing the code amount by suppressing the image quality for the picture including the ROI.

  Based on the prediction mode information D03 and the upper syntax, the compression unit 305B generates an elementary stream signal D05 obtained by subjecting the preprocessed image signal to compression encoding processing in accordance with the above-described image encoding method. 307. As this image encoding method, ROI can be extracted by decoding only pictures including ROI on the decoding side. H.264 / MPEG-4 AVC (Advanced Video Coding), or H.264. H.265 / MPEG-H HEVC (High Efficiency Video Coding) or the like is used.

  With this configuration, the encoder core unit U3 performs compression coding processing on the input image signal E1 based on the ROI information D01, the prediction mode information D03, and higher-order syntax indicating a restricted intra, which will be described later. That is, the encoder core unit U3 performs an intra coding process on the block corresponding to the ROI in the picture represented by the input image signal E1 according to the intra prediction mode indicated by the prediction mode information D03. At this time, in the case of performing intra coding using the pixel values around the image block included in the ROI according to the upper syntax indicating the restricted intra, the encoder core unit U3 uses the surrounding pixel values as the surrounding pixel values. Only pixel values that have already been intra-coded or predetermined values are used.

  On the other hand, for an area that does not correspond to ROI in the picture, that is, a block group that corresponds to a non-ROI (background area), the encoder core unit U3 switches to the intra prediction mode or the inter prediction mode indicated by the prediction mode information D03. The encoding process is performed accordingly. That is, when the prediction mode information D03 indicates the intra prediction mode, the encoder core unit U3 performs an intra coding process on the non-ROI block in the picture represented by the input image signal E1. When the prediction mode information D03 indicates the inter prediction mode, the encoder core unit U3 performs inter coding processing on the non-ROI block in the picture represented by the input image signal E1.

  The encoder core unit U3 supplies the elementary stream signal D05 obtained by the compression encoding process described above to the bit stream generation unit 307.

  Next, the higher-order syntax generation unit 106 generates higher-order syntax that includes information necessary for decoding and indicates a restricted intra (step S106). For example, the image coding method of the encoder core unit U3 is the above-described H.264. H.264 / MPEG-4 AVC or H.264 In the case of H.265 / MPEG-H HEVC, the upper syntax generation unit 106 generates SPS (Sequence Parameter Set) and PPS (Picture Parameter Set) as upper syntax. SPS is a parameter set indicating information related to encoding of the entire sequence, and PPS is a parameter set indicating information related to encoding of the entire picture. For example, the encoder core unit U3 is H.264. H.264 / MPEG-4 AVC or H.264 When the image coding method corresponding to the restricted intra designation such as H.265 / MPEG-H HEVC is adopted, the upper syntax generation unit 106 designates the restricted intra using the PPS. In the intra with restrictions, when the pixel values around the intra MB are inter-coded, the intra MB is not used but is used only with the intra-coded pixel values around the intra MB. Encode. At this time, if there is no intra-coded pixel value around the intra MB, the intra MB is intra-coded using a predetermined fixed value in the restricted intra.

  The upper syntax generation unit 106 supplies the bit stream generation unit 307 with higher syntax information D06 representing the parameter set such as SPS and PPS as the upper syntax.

  In addition, when the encoder core unit U3 employs an image coding method in which the upper syntax used in the compression encoding process can be selected in units of pictures or slices from among a plurality of upper syntaxes, the upper syntax generation unit 106 supplies the upper syntax information D06 to the bitstream generation unit 307 as follows.

  That is, each time the encoder core unit U3 selects an upper syntax for each picture or slice, the upper syntax generation unit 106 transmits upper syntax information D06 indicating the selected upper syntax to the bitstream generation unit 307. Supply. A slice represents a processing unit composed of a plurality of macroblocks in one picture.

  If the encoder core unit U3 employs an image coding method that does not assume a restricted intra, such as MPEG-2, the higher-order syntax generation unit 106 is defined by the image coding method. Generate the normal high-level syntax.

  Note that the upper syntax generation unit 106, the prediction mode determination unit 103, and the encoder core unit U3 are all connected to the working memory M, and share various image information and encoding information via the working memory M. ing.

  Next, the bit stream generation unit 307 generates a bit stream signal D07 obtained by multiplexing the elementary stream signal D05 supplied from the encoder core unit U3 and the upper syntax information D06 (step S107).

  As described above, in the video encoding device X1, the ROI setting unit 301 sets the ROI, and generates the ROI information D01 indicating the position and size of the ROI in the picture. Then, the encoder unit U1 receives the input image signal E1 and the ROI information D01, performs compression coding processing on the input image signal E1 based on the ROI information D01, and generates a bit stream signal D07. At this time, based on the ROI information D01, the encoder unit U1 performs intra coding on the image blocks included in the ROI in the picture based on the input image signal E1 as compression coding processing, and applies the image blocks included in the non-ROI to the image block group. On the other hand, one of intra coding and inter coding is selected and applied.

  As a result, it is possible to reduce the code amount as compared with a case where all image blocks in a picture are compression-encoded by intra-encoding. In addition, the amount of calculation can be reduced as compared with the case where all image blocks in a picture are compression-encoded by inter-encoding.

  Therefore, according to the video encoding device X1, it is possible to achieve both a reduction in code amount and a reduction in calculation amount.

  Hereinafter, a video processing system employing the video encoding device X1 will be described.

  FIG. 3 is a block diagram showing a configuration of an ROI identification system RIS as an example of a video processing system.

  The ROI identification system RIS is a system that receives an input image signal E1 obtained by photographing with a video camera or the like, for example, and identifies a specific ROI from the picture represented by the input image signal E1. The specific ROI is an ROI including a human face registered in advance, for example.

  As shown in FIG. 3, the ROI identification system RIS includes a video code decoding system VCD and an ROI recognition device X4.

  The video code decoding system VCD includes a decoded image selection device X2 and a video decoding device X3 in addition to the video encoding device X1 shown in FIG.

  The video encoding device X1 receives the above-described input image signal E1, and performs the above-described compression encoding processing on the input image signal E1 to generate the bit stream signal D07. The video encoding device X1 supplies the bit stream signal D07 to the decoded image selection device X2 via an arbitrary communication path such as a LAN (Local Area Network) or a public line.

  The decoded image selection device X2 receives the bit stream signal D07 and the activation event E2, and performs the following decoded image selection processing.

  FIG. 4 is a flowchart showing an example of such a decoded image selection process. As shown in FIG. 4, first, the decoded image selection device X2 extracts and holds a parameter set (SPS, PPS) as the upper syntax information D06 from the bit stream signal D07 (step S501).

  Next, the decoded image selection device X2 obtains, from the bit stream signal D07, an access unit sequence obtained by dividing the elementary stream signal D05 included in the bit stream signal D07 for each decoding processing unit (step S5021). Next, the decoded image selection device X2 determines whether or not the ROI is included in the picture corresponding to the access unit for each access unit (step S5022).

  As a method for determining whether or not the picture corresponding to the access unit includes the ROI, the bit stream signal D07 is decoded or only the access unit is decoded and the ROI is detected based on the decoding result. It is determined whether or not the ROI is included. Alternatively, first, the bit stream generation unit 307 of the video encoding device X1 supplies the bit stream signal D07 obtained by multiplexing the ROI information D01 to the decoded image selection device X2.

  As this multiplexing method, for example, H.264. H.264 / MPEG-4 AVC or H.264 In the case of H.265 / MPEG-H HEVC, “User unregistered SEI message” is used. Then, the ROI information D01 is extracted from the bit stream signal D07 on the decoded image selection device X2 side to determine whether or not the ROI is included.

  If it is determined in step S5022 that the ROI is included in the picture corresponding to the access unit, the decoded image selection device X2 receives the parameter set information D08 representing the parameter set held in step S501 described above, and the access unit The access unit sequence D09 representing the sequence is supplied to the video decoding device X3 (step S5023).

  And after completion | finish of this step S5023, the decoding image selection apparatus X2 complete | finishes the decoding image selection process shown in FIG.

  Also, when it is determined in step S5022 that the picture corresponding to the access unit does not include the ROI, the decoded image selection device X2 ends the decoded image selection process shown in FIG. Therefore, at this time, the decoded image selection device X2 does not supply the parameter set information D08 and the access unit sequence D09 to the video decoding device X3 even though it receives the bit stream signal D07.

  The video decoding device X3 supplies the output image signal E3 obtained by subjecting the access unit sequence D09 to decoding processing based on the parameter set information D08, to the ROI recognition device X4. Note that the video decoding device X3 performs a decoding process on the access unit sequence D09 without decoding the reference picture even if the access unit sequence D09 is inter-frame encoded.

  That is, the video decoding device X3 generates an arbitrary virtual picture instead of decoding the reference picture, and performs decoding by regarding the virtual picture as a reference picture. Note that the virtual picture is, for example, a picture in which all pixel values are fixed to a minimum luminance value “0”, an intermediate luminance value “128”, a maximum luminance value “255”, or the like.

  The ROI recognition device X4 receives the output image signal E3 and the registered image signal E4. The registered image signal E4 is an image signal representing, for example, one or more faces registered in advance and identification information (for example, name) for identifying the faces. The ROI recognition device X4 determines whether the face represented by the registered image signal E4 is included in the ROI included in the output image signal E3 by using, for example, a face recognition engine. When the ROI recognition apparatus X4 determines that the face represented by the registered image signal E4 is included in the output image signal E3, the ROI recognition device X4 outputs identification information corresponding to the face.

  As described above, in the video code decoding system VCD, the decoded image selection device X2 selects only the access unit corresponding to the picture including the ROI from the access unit sequence and supplies it to the video decoding device X3. Yes. Therefore, since the video decoding device X3 decodes only the picture including the ROI, the amount of calculation required for decoding the ROI, which is necessary in the downstream system in FIG. 3 using the ROI recognition device X4 as an example, is suppressed. Is possible.

  By the way, in the decoded image selection process shown in FIG. 4, when the ROI is included in the picture corresponding to the access unit, the access unit corresponding to the picture including the ROI is always supplied to the video decoding device X3. However, the access unit corresponding to the picture including the ROI may be supplied to the video decoding device X3 only when the activation event E2 is received.

  FIG. 5 is a flowchart showing another example of the decoded image selection process performed in view of such points. In FIG. 5, the decoded image selection device X2 accumulates the bit stream signal D07 in the database (step S601). Here, when receiving the activation event E2, the decoded image selection device X2 extracts only the parameter unit included in the bit stream signal D07 and the access unit including the ROI from the database. Then, the decoded image selection device X2 outputs parameter set information D08 representing the extracted parameter set, and an access unit sequence D09 including an access unit sequence including the ROI.

  The decoded image selection device X2 searches the database for only the access unit corresponding to the picture including the ROI only when the activation event E2 is issued by performing the decoded image selection process shown in FIG. Is the decryption target. As a result, the amount of calculation required for ROI decoding is reduced as compared with the case where the decoded image selection processing shown in FIG. 4 is adopted. Even when the decoded image selection process shown in FIG. 5 is adopted, only the picture including the ROI is recognized in the ROI recognition device X4 as in the case where the decoded image selection process shown in FIG. 4 is adopted. be able to. Therefore, since past ROIs accumulated in the database can be decoded with a small amount of computation, it is possible to achieve practicality of re-recognition processing.

  FIG. 6 is a block diagram showing another configuration of the video encoding device X1 according to the present invention. In the configuration shown in FIG. 6, other configurations except that a higher syntax specifying unit 204 is newly added and a higher syntax generation unit 106 a is employed instead of the higher syntax generation unit 106 are shown in FIG. 1. Is the same as Therefore, the operation will be described below with a focus on the upper syntax designation unit 204 and the upper syntax generation unit 106a.

  The high-order syntax generation unit 106a generates high-order syntax (referred to as normal syntax) that specifies a normal intra that is not subject to restriction, together with high-order syntax (referred to as restricted high-order syntax) that specifies a restricted intra. To do. The upper syntax generation unit 106 a supplies the upper stream syntax information D06 indicating the upper syntax with restrictions and the normal syntax to the bitstream generation unit 307.

  The upper syntax designating unit 204 receives the ROI information D01, and based on the ROI information D01, an upper syntax designating signal D04 that designates an upper syntax (normal syntax, constrained upper syntax) used for compression encoding processing. Is supplied to the encoder core unit U3.

  The encoder core unit U3 performs compression coding processing on the input image signal E1 based on the upper syntax specified by the ROI information D01, the prediction mode information D03, and the higher syntax specification signal D04. The encoder core unit U3 supplies the elementary stream signal D05 obtained by the compression encoding process to the bit stream generation unit 307.

  FIG. 7 is a flowchart showing a video encoding process performed by the video encoding device X1 having the configuration shown in FIG. In the flow shown in FIG. 7, S105a and S106a are adopted instead of the above-described steps S105 and S106, and other steps (S101 to S103, S107) except that step S104 is added after step S103 are as follows. This is the same as the flow shown in FIG. Therefore, each operation will be described focusing on steps S104, S105a, and S106a.

  That is, after execution of step S103, the higher-level syntax specifying unit 204 specifies a restricted intra for a picture that includes an ROI based on the ROI information D01, and is not subject to a restriction for a picture that does not include an ROI. The higher-level syntax designation signal D04 that designates the intra is generated (step S104). It should be noted that any upper syntax may be specified for a picture composed only of ROI.

  Next, the encoder core unit U3 outputs an elementary stream signal D05 obtained by subjecting the input image signal E1 to compression encoding processing based on the prediction mode information D03 and the upper syntax designation signal D04 (step S105a). Here, when the higher-order syntax designation signal D04 indicates the higher-order syntax designating the restricted intra, the encoder core unit U3 codes the block determined to be coded in the intra prediction mode, with the restricted intra. Then, the encoder core unit U3 includes an index indicating the upper syntax specifying the restricted intra at a predetermined position in the elementary stream signal D05. For example, H.M. In the case of H.264 / MPEG-4 AVC, the slice header has an area for designating the upper syntax to be used. Therefore, an index indicating the upper syntax for designating the restricted intra is included in the area.

  Further, when the upper syntax designation signal D04 indicates the normal higher syntax, the encoder core unit U3 encodes the block determined to be encoded in the intra prediction mode, with the normal intra without restriction. Then, the encoder core unit U3 includes an index indicating a normal higher-order syntax at a predetermined position in the elementary stream signal D05.

  Next, the upper syntax generation unit 106a generates an upper syntax specifying a restricted intra and a normal upper syntax, and generates upper syntax information D06 indicating both (step S106a). Here, the difference between the upper syntax specifying the restricted intra and the normal higher syntax is whether or not the restricted intra is specified. For example, H.M. In the case of H.264 / MPEG-4 AVC, the difference is represented by “constrained_intra_pred_flag” included in the PPS.

  Note that the upper syntax includes not only PPS but also H.264. H.264 may include SPS and the like. In this case, for higher-level syntax other than PPS, a higher-level syntax specifying a constrained intra and a normal higher-level syntax are not generated separately, but a single higher-level syntax is specified as a constrained intra. The higher-level syntax and the normal higher-level syntax may be shared.

  If the same information piece as the information piece included in the information group represented by the normal upper syntax exists in the information group represented by the upper syntax specifying the restricted intra, this same These pieces of information may be shared between the upper syntax specifying the restricted intra and the normal higher syntax. That is, the upper syntax generation unit 106a first generates an upper syntax specifying a restricted intra. Then, a normal upper syntax in which the same piece of information is omitted is generated, and a final normal upper syntax is obtained by including the same information piece in the generated normal upper syntax. As a result, the operation load on the upper syntax generation unit 106a is reduced.

  As described above, in the configuration of the second embodiment, a picture or a slice that does not need a restricted intra because it is configured only by a non-ROI is encoded as a normal intra MB. Here, since a normal intra tends to have higher code efficiency than a constrained intra, a picture or a slice that does not require a constrained intra is encoded by encoding as a normal intra MB. The amount can be reduced.

  In the first and second embodiments described above, for example, by providing the preprocessing unit 305A, processing for reducing the code amount by suppressing the image quality is performed on the picture including the ROI. Therefore, it is not necessary to perform quality control to suppress such image quality. At this time, the prediction mode variable encoder unit U2 does not need to receive the ROI information D01, and the preprocessing unit 305A of the encoder core unit U3 becomes unnecessary.

  Further, one of the two types of ROIs set based on different standards may be set as the encoding target in the restricted intra, and the other may be subjected to the compression encoding process as the quality control target as described above. At this time, the ROI setting unit 301 generates ROI information D01 representing two systems of ROIs set based on different standards. Then, among the ROIs represented by the ROI information D01, the ROI having the maximum usefulness is set as the encoding target of the restricted intra, and the other ROIs are set as the encoding targets with priority on the quality control.

  The ROI having the maximum usefulness is, for example, the ROI having the maximum size. Here, an example of a method for limiting the ROI to be encoded with the restricted intra from among the plurality of ROIs will be described below. That is, it is determined whether or not the ROI of interest is included in the pictures before and after the moving image, and when included, the usefulness (for example, the size of the ROI) is calculated. Then, it is determined whether or not the usefulness is the maximum, and the ROI determined to be the maximum is set as the encoding target of the restricted intra.

  As described above, for example, by using the method of the first and second embodiments, the ROI to be encoded with the restricted intra is set based on a stricter standard than the ROI targeted for quality control, which is necessary for the ROI decoding. It is possible to minimize the increase in the code amount while minimizing the calculation amount.

  In the first and second embodiments, the ROI setting unit 301 sets the region specified by the user as the ROI, or sets the ROI detected from the input image signal E1 as the ROI. When the region designated by the user is set as the ROI, it is not necessary to supply the input image signal E1 to the ROI setting unit 301.

  Further, the method for setting the ROI is not limited to the method described above. For example, the ROI may be set using an image taken with an infrared camera or an image taken with a depth sensor. At this time, an image captured by an infrared camera or a signal representing an image captured by a depth sensor is supplied to the ROI setting unit 301 as an input image signal E1.

  Moreover, although the video encoding process by the video encoding device X1 shown in FIGS. 1 and 6 is realized by hardware, the video encoding process may be realized by software.

  FIG. 8 is a block diagram showing an example of the configuration of an information processing apparatus that implements this video encoding processing by software. The information processing apparatus includes a central processing unit (CPU) 1001, a random access memory (RAM) 1002, a read only memory (ROM) 1003, and an input / output unit 1004 as control units. The ROM 1003 stores a program according to the video encoding process shown in FIG. The CPU 1001 generates a bit stream signal D07 based on the input image signal E1 input via the input / output unit 1004 by executing a program stored in the ROM 1003, and outputs this through the input / output unit 1004. To do.

102 Intra prediction mode designating unit 103 Prediction mode determining unit 106 Upper syntax generating unit 307 Bit stream generating unit U1 Encoder unit U2 Predictive mode variable encoder unit U3 Encoder core unit X1 Video encoding device X2 Decoded image selecting device X3 Video decoding device X4 ROI recognition device VCD video code decoding system

Claims (18)

A video encoding device that generates a bitstream signal by performing compression encoding processing on an input image signal,
An ROI setting unit that sets ROI (Region Of Interest) in a picture and generates ROI information indicating the position and size of the ROI in the picture;
An encoder unit that receives the input image signal and the ROI information, and performs a compression encoding process on the input image signal based on the ROI information to generate a bitstream signal;
The encoder unit is
Based on the ROI information, the image block included in the ROI in the picture in the input image signal is subjected to intra encoding as the compression encoding process, and the image block included in the non-ROI is subjected to the compression. A video encoding apparatus, wherein one of the intra encoding and the inter encoding is selected and applied as an encoding process. The encoder unit is
An intra prediction mode designating unit that generates an intra prediction mode designating signal for designating an intra prediction mode for an image block included in the ROI in a picture in the input image signal based on the ROI information;
When the intra prediction mode designation signal is generated, the intra prediction mode is indicated. When the intra prediction mode designation signal is not generated, the prediction mode information indicating the intra prediction mode or the inter prediction mode is generated. A mode determination unit;
When the prediction mode information indicates the intra prediction mode, the intra coding is performed on the input image signal, and when the prediction mode information indicates the inter prediction mode, the inter coding is performed on the input image signal. An encoder core unit for generating an elementary stream signal,
An upper syntax generation unit that generates upper syntax information representing information necessary for decoding as upper syntax;
The video encoding apparatus according to claim 1, further comprising: a bit stream generation unit that generates a signal obtained by multiplexing the elementary stream signal and the upper syntax information as the bit stream signal.   The video encoding apparatus according to claim 2, further comprising a working memory that shares information among the prediction mode determination unit, the encoder core unit, and the higher syntax generation unit. The upper syntax generation unit generates the upper syntax information representing the upper syntax for designating a restricted intra;
The constrained intra uses only the intra-encoded pixel value or the predetermined value as the peripheral pixel value when performing the intra-encoding using the peripheral pixel value of the image block included in the ROI. 4. The video encoding apparatus according to claim 2, wherein the video encoding apparatus is urged to be used. The encoder core unit selects the upper syntax used in the compression encoding process from a plurality of upper syntaxes in units of pictures or slices,
The high-order syntax generation unit generates the high-order syntax information indicating the selected high-order syntax every time the encoder core unit selects the high-order syntax. Video encoding device. The encoder unit is
Based on the ROI information, the first upper syntax designating the restricted intra is specified for a picture including both the ROI and the non-ROI, and there is no restriction for a picture including only the non-ROI. An upper syntax specifying unit for generating an upper syntax specifying signal indicating a second upper syntax for specifying an intra;
The encoder core unit performs the compression encoding process on the input image signal according to the upper syntax specified by the upper syntax specifying signal,
6. The high-order syntax generation unit generates the high-order syntax information that represents the first high-order syntax and the second high-order syntax, respectively. 6. The video encoding device described. The encoder core part is
When the higher syntax designation signal indicates the first higher syntax, the image block to be subjected to the intra coding is encoded according to the restricted intra and the first higher syntax. An index indicating a tax is included in the elementary stream signal,
When the higher-order syntax designation signal indicates the second higher-order syntax, the image block to be subjected to the intra coding is encoded according to the intra without restriction, and the first The video encoding apparatus according to claim 6, wherein the index indicating a higher-order syntax of 2 is included in the elementary stream signal. When generating the first and second upper syntaxes, the upper syntax generation unit is represented by the second upper syntax in an information group represented by the first upper syntax. If there is an information piece identical to the information piece included in the information group,
The first high-order syntax is generated, and the high-order syntax in which the same piece of information is omitted is generated. The high-order syntax in which the same piece of information is omitted is included in the first high-order syntax. The video encoding apparatus according to claim 6 or 7, wherein the same information piece is included in the second upper syntax. The encoder unit is
The video according to any one of claims 2 to 8, further comprising a processing unit that compresses the ROI and the non-ROI included in the input image signal with different video quality based on the ROI information. Encoding device. The ROI setting unit sets a first ROI and a second ROI different from the first ROI, a position and a size in the picture of the first ROI, and a picture in the second ROI Generating information representing the position and size in the ROI information,
The intra prediction mode designating unit designates the intra prediction mode for an image block included in the first ROI based on the ROI information,
The video encoding device according to claim 9, wherein the processing unit performs the compression on a picture including the second ROI based on the ROI information.   The video encoding apparatus according to claim 10, wherein the size of the first ROI is larger than the size of the second ROI. An image encoding / decoding system including: a video encoding device that performs compression encoding processing on an input image signal to generate a bitstream signal; and a video decoding device that performs decoding processing on the bitstream signal,
The video encoding device includes:
An ROI setting unit that sets ROI (Region Of Interest) in a picture and generates ROI information indicating the position and size of the ROI in the picture;
An upper syntax generation unit that generates upper syntax information including a parameter set necessary for decoding;
A bit stream obtained by multiplexing the elementary stream signal obtained by receiving the input image signal and the ROI information and subjecting the input image signal to compression encoding processing based on the ROI information, and the upper syntax information An encoder unit for outputting a signal,
The encoder unit performs intra coding as the compression coding process on an image block included in the ROI in the picture in the input image signal based on the ROI information, and converts the image block included in the non-ROI to the image block included in the non-ROI For the compression encoding process, one of the intra encoding and inter encoding is selected and applied,
The video decoding device includes:
A decoded image selection unit that receives the bit stream signal and extracts the parameter set included in the higher-level syntax information from the bit stream signal and an access unit corresponding to the elementary stream signal;
And a video decoding unit that generates an output image by performing a decoding process on the access unit based on the parameter set.   The image coding / decoding system according to claim 12, wherein the video decoding unit performs the decoding process on the access unit without decoding a reference picture.   When the access unit is the inter-coded one, the video decoding unit regards a predetermined virtual picture as the reference picture, and performs the decoding process on the access unit. The image encoding / decoding system according to claim 13.   15. The image code decoding according to claim 14, wherein the video decoding unit generates a picture having all pixel values as a minimum luminance value, an intermediate luminance value, or a maximum luminance value as the virtual picture. system. The decoded image selection unit includes:
While holding the parameter set, it is determined whether or not the ROI is included in a picture corresponding to the access unit. If it is determined that the ROI is included, the held parameter and the access are determined. The image code decoding system according to claim 12, wherein a unit is supplied to the video decoding unit. The decoded image selection unit includes a database for storing the bit stream signal,
16. The method according to claim 12, wherein when an activation event is given, only the access unit including the parameter set and the ROI is extracted from the database, and each is supplied to the video decoding unit. The image encoding / decoding system described. A video encoding program executed by a control unit of an information processing apparatus that performs compression encoding processing on an input image signal to generate a bitstream signal,
A first step of setting an ROI (Region Of Interest) in a picture and generating ROI information representing the position and size of the ROI in the picture;
Receiving the input image signal and the ROI information, and performing a compression encoding process on the input image signal based on the ROI information to generate a bitstream signal, and
In the second step, based on the ROI information, an image block included in the ROI in a picture in the input image signal is subjected to intra encoding as the compression encoding process, and an image included in a non-ROI A video encoding program, wherein one of the intra encoding and inter encoding is selected and applied to a block as the compression encoding processing.

Images