JP2015115903A - Imaging apparatus, control method of imaging apparatus, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve image quality in the case of image variation and to improve encoding efficiency even in a compression system where a block size is variable.SOLUTION: An imaging apparatus is configured to divide a moving image that is obtained by imaging a subject, into a plurality of blocks and encode the moving image for each block. The imaging apparatus includes: determination means for determining a block size of dividing the moving image from among a plurality of block sizes; division means for dividing the moving image in accordance with the determined block side; and encoding means for performing encoding processing for each divided block. The determination means determines the block size of dividing the moving image, in accordance with a magnitude of variation of an image in the moving image, while switching between a first pattern that defines a block size for dividing a region of interest of a user of the imaging apparatus in the moving image as a size smaller than a block size for dividing any other region than the region of interest, and a second pattern that divides the moving image in accordance wit a maximum block size among the plurality of block sizes.

Description

The present invention relates to an imaging apparatus, an imaging apparatus control method, and a computer program.
About.

  In an imaging apparatus such as a digital video camera or a digital camera, the MPEG2 system or H.264 is used. Although a method of compressing and recording a moving image by the H.264 method can be used, in recent years, a more efficient compression method, such as the HEVC method, has been developed.

  In the conventional compression method, encoding is processed in units of image divisions called macroblocks, and motion prediction, conversion processing, and quantization processing are performed for each macroblock. On the other hand, in the HEVC method, the encoding efficiency is further improved by changing the block size for performing motion prediction, conversion processing, and quantization processing.

  The coding efficiency can be controlled by a quantization parameter called “quantization characteristic” used in the quantization process. However, when the compression rate is controlled to be high, the image quality generally deteriorates. The quantization parameter can be set for each macroblock, and by changing the quantization parameter for a specific part of the captured image, the image quality can be improved even when the image fluctuates during zoom operation, panning operation, tilt operation, etc. A method for realizing this is proposed in Patent Document 1.

Japanese Patent Laid-Open No. 9-214965

  However, the above proposed method is a proposal of an image quality improvement method in a compression method in which the macroblock size is fixed, and does not consider a compression method in which the block size is variable.

  Therefore, an object of the present invention is to provide a technique that can improve the image quality and the coding efficiency when the image changes even if the compression method has a variable block size.

The present invention for solving the above-described problem is an imaging apparatus that divides a moving image obtained by imaging a subject into a plurality of blocks and encodes each block,
Determining means for determining a block size for dividing the moving image from a plurality of block sizes;
Dividing means for dividing the moving image according to the determined block size;
Encoding means for performing an encoding process for each of the divided blocks,
The determining unit is configured to divide a region other than the region of interest into a block size for dividing the region of interest of the user of the imaging device in the moving image according to the magnitude of image variation in the moving image. The block size for dividing the moving image is determined by switching between the first pattern having a small size and the second pattern for dividing the moving image by the maximum block size among the plurality of block sizes. It is characterized by that.

  According to the present invention, even in a compression scheme with a variable block size, the image quality can be improved while increasing the encoding efficiency when the image changes.

FIG. 4 is a block diagram illustrating an example of a functional configuration corresponding to the first embodiment and (B) corresponding to the second embodiment as the imaging apparatus 100 corresponding to the embodiment of the invention. The figure for demonstrating the example of a block division | segmentation in a HEVC system. 5 is a flowchart showing an example of the operation of the imaging apparatus 100 corresponding to Embodiment 1 of the invention. 9 is a flowchart showing an example of the operation of the imaging apparatus 100 corresponding to Embodiment 2 of the invention. The figure for demonstrating the example of a block division corresponding to Embodiment 1 of invention. The figure for demonstrating the example of a block division corresponding to Embodiment 2 of invention.

  In the embodiment of the invention described below, the coding block division method is determined in accordance with the speed (fluctuation speed) at which a moving image generated by shooting changes.

  As an embodiment of the invention, a coding block division method can be determined based on image fluctuations caused by zoom operations such as a zoom-in operation and a zoom-out operation in a digital camera or digital video camera as an imaging apparatus. Hereinafter, the embodiment will be described in detail as a first embodiment (Embodiment 1) of the invention with reference to the accompanying drawings.

[Embodiment 1]
First, the configuration and operation of the imaging apparatus 100 corresponding to the present embodiment will be described with reference to FIG. FIG. 1A is a diagram illustrating an example of a functional configuration of the imaging apparatus 100 corresponding to the first embodiment. In the imaging apparatus 100 of FIG. 1A, each block may be configured by hardware using a dedicated logic circuit or memory, except for a physical device such as an imaging element or a display element (not shown). Alternatively, the processing program stored in the memory may be configured by software by a computer such as a CPU executing the processing program. In addition to the digital camera and the digital video camera, the imaging device 100 can be any information processing terminal or device having an imaging function, such as a personal computer, a mobile phone, a smartphone, a PDA, or a tablet terminal.

  The sensor 101 is an image sensor such as a CCD or a CMOS. The photographing lens group 100 adjusts the amount of incident light and focusing, photoelectrically converts the formed captured image, and further converts a digital signal obtained by analog / digital conversion. Output. In each pixel of the sensor 100, any one of R (red), G (green), and B (blue) color filters is arranged in a predetermined arrangement, for example, a Bayer arrangement or a honeycomb arrangement. For high-speed output, RGB image signals may be output by performing pixel addition processing for adding several pixels to reduce the output image as one pixel.

  The development processing unit 102 receives an RGB image signal from the sensor 101 and first performs RGB offset adjustment, gain adjustment, and gamma correction processing. The gamma correction is processed with characteristics for generating a recorded image desired by the video camera user in consideration of characteristics of the sensor 101 and the lens group 100. By adjusting the gamma curve, it is possible to generate a recorded image for display on a TV monitor and a recorded image that reproduces the texture and gradation of a movie film. Next, the RGB image signal is converted into a luminance signal (Y) and a color difference signal (Cb, Cr) and output. In addition, correction processing for lens distortion, image stabilization processing, noise reduction processing, and the like of the imaging apparatus 100 are also performed.

  The lens control / zoom amount detection unit 103 performs adjustment control of the amount of incident light and focusing, and also controls zoom magnification. In addition to optical zoom, electronic zoom by so-called image processing can be included. Based on the zoom amount information from the lens control / zoom amount detection unit 103, the block size arrangement determination unit 104 determines the encoding block size and the intra-screen block division. The block division unit 105 divides the screen into first encoded blocks having the same size as shown in FIG. 2A, and follows the instruction of the block size arrangement determination unit 104 as shown in FIG. The first encoded block is further divided into second encoded blocks, and an image signal is output. In HEVC, the first coding block is called a coding tree unit (CTU), and the second coding block is called a coding unit (CU). Note that the block sizes that can be taken by the CTU in HEVC include 64 × 64, 32 × 32, and 16 × 16, and the block sizes that can be taken by the CU include 64 × 64, 32 × 32, 16 × 16, and 8 × 8. In HEVC, motion compensation processing, intra prediction processing, orthogonal transform processing, quantization processing, and entropy coding processing are executed in the second coding block. FIG. 2A shows a state in which a block of 64 × 64 pixel size is divided into four blocks of 32 × 32 pixel size. In FIG. 2B, a block of 32 × 32 pixel size is divided into a block of 16 × 16 pixel size and 8 × 8 pixel size.

  The prediction unit 106 uses an image signal input from the block division 105 and an encoded image signal (described later) stored in the memory 109, which is an optimal prediction method for either intra-screen prediction or inter-screen prediction processing including motion detection. Is selected to generate a prediction difference signal. As shown in FIG. 2C, the prediction process is performed by further dividing the second encoded block into a block size optimal for the prediction process. In HEVC, a unit block for prediction processing is called Prediction Unit (PU). For example, when the size of the CU is 2N × 2N, a block pattern having a size of 2N × 2N, 2N × N, N × 2N, N × N, or the like is selected for motion compensation processing. In the case of intra prediction processing, a block pattern having a size of 2N × 2N and N × N (N = 4) is selected. In FIG. 2C, a 32 × 32 pixel size block is divided into 8 × 8, 16 × 8, and 4 × 4 pixel size blocks.

  As shown in FIG. 2D, the transform / quantization unit 107 further divides the second encoded block into a block size optimal for the transform process, and performs orthogonal transform processing and quantization processing. In HEVC, these processing unit blocks are called transform units (TU). There are four types of block sizes that can be taken by a TU in HEVC: 32 × 32, 16 × 16, 8 × 8, and 4 × 4. In FIG. 2D, a 32 × 32 pixel size block is divided into 16 × 16, 8 × 8, and 4 × 4 pixel size blocks. The orthogonally transformed transform coefficient is quantized with a quantization parameter instructed by a code amount control 111 described later, and the quantized data is output to the entropy coding 110 and the local decoding unit 108.

  The local decoding unit 108 sequentially performs an inverse quantization process and an inverse orthogonal transform process on the input quantized data, adds the predicted image input from the prediction 106, and performs a decoding process. The decoded data is further subjected to loop filter processing and stored in the memory 109.

  The entropy encoding unit 110 is an arithmetic encoding method called CABAC (Context Adaptive Binary Arithmetic Coding) that dynamically updates the occurrence probability of an encoding parameter for input quantized data based on a nearby encoding state. To convert to binary data. The code amount control unit 111 generates a quantization parameter based on the generated code amount after entropy coding, and supplies the quantization parameter to the transform / quantization unit 107. The above processing unit constitutes an image encoding device.

  Next, block size determination processing in the present embodiment will be described with reference to the flowchart of FIG. The processing corresponding to the flowchart can be realized, for example, by executing a corresponding program (stored in a ROM or the like) by the CPU functioning as the block size arrangement determining unit 104.

  First, when shooting is started, in S301, the lens control / zoom amount detection unit 103 detects the presence or absence of a zoom operation based on a change in the value of zoom amount information. If the zoom operation is being performed (“YES” in S301), the process proceeds to S302. On the other hand, when the zoom operation is not performed (“NO” in S301), feature extraction is performed from the image input to the block size arrangement determination unit 104 in S305, and the CU division size and arrangement corresponding to the feature of the image are determined. Thereafter, the process proceeds to S306.

  Regarding the processing in S305, for example, the block size can be set small for an image with many high-frequency components, and the block size can be set large for a flat image with few high-frequency components. Also, if there are information loss areas, light loss areas, strong distortion areas, etc. in the input image in relation to the optical system to be used, the block size should be set large for these areas to reduce the code amount. Can do. In addition, since one motion vector and quantization parameter are included per block, if the block size is increased, the number of motion vectors and quantization parameters can be reduced compared to the case of finely dividing, thus reducing the amount of codes. can do. On the other hand, if the block size is made fine, a motion vector and a quantization width suitable for each block can be set. Thus, by selecting an appropriate block size according to the characteristics of the input image, it is possible to improve the encoding efficiency and generate an encoded stream with high image quality.

    In S302 to S304, the zoom variation speed based on the variation in the value of the zoom amount information is compared with a predetermined threshold to determine the “speed” of the zoom variation speed, and the CU partition size and arrangement are changed according to the determination result. . In general, the zoom operation is an operation of enlarging or reducing an image in a concentric direction around a specific pixel or a specific image region, and the region that is the center of zoom is the region of interest of the user. However, when the region of interest is regarded as a region where a subject that the user is impressed with is captured, the ratio of the subject to the image changes according to the zoom magnification, and the size of the region of interest also changes accordingly. In the generation / recording process of the moving image, the quality of the image of the region of interest may be kept high, and the encoding efficiency may be prioritized over the quality of the image in the peripheral region other than the region of interest. Block size allocation can be determined. However, since it may be difficult for the user to recognize the image quality depending on the zoom fluctuation speed, the above basic block allocation pattern may not necessarily be adopted. In this embodiment, paying attention to this point, a method of adaptively allocating a large block size according to the zoom fluctuation speed is adopted.

  Specifically, when the zoom fluctuation speed is equal to or higher than the threshold and equal to or higher than the predetermined speed, the zoom operation can be determined to be “fast” (“YES” in S302), and in S303, the middle stage of FIG. As shown in the figure, the entire image is divided by the CU having the maximum size (64 × 64 pixel size). This is because when the zoom operation is fast, it is difficult to recognize an image and it is difficult to record a high-quality image, and the CU size is set to be as large as possible in order to increase the encoding efficiency as much as possible.

  On the other hand, when the zoom fluctuation speed is smaller than the threshold, the zoom operation can be “slow” (“NO” in S302). In this case, the block size arrangement determining unit 104 and the block dividing unit 105 perform block division so that the center portion of the image has a fine CU size and the periphery of the image has a coarse CU size as shown in the zoom start point in FIG. Process. For example, the center portion of the image can be the minimum CU size, and the rest can be the remaining CU size. When the zoom operation is slow, the user can recognize the image. Since the zoom center is a meaningful region of interest for the user, the block size is reduced to record a high-quality image. On the other hand, the peripheral area is a part that disappears from the imaging range due to zooming in or is newly included in the imaging range, but the rate of change of the image is high compared to the central part and is less important for the user. It is estimated to be. Therefore, the CU size is set to a large size in order to increase the coding efficiency for the area composed of the peripheral pixels. After block division processing, the process proceeds to S306.

  Referring to FIGS. 5A and 5B in detail, the zoom operation is slow at the start and end of zoom-in or zoom-out, so that the center of the image is a fine CU size and the periphery of the image is a coarse CU. The division is performed with the first pattern having the size. Then, the first pattern is divided as it is according to the zoom fluctuation speed during the zoom operation, or when the zoom fluctuation speed is high, the second pattern is uniformly divided by the maximum size CU. adopt. It should be noted that the second pattern is not limited to a pattern that uniformly divides the whole, for example, the periphery is divided by the maximum size, and the central portion is divided by the second largest block (third pattern). You may do it.

  Such a third pattern may be used for subdividing the threshold used when determining the speed of the zoom operation and switching the pattern stepwise in combination with other patterns. For example, the zoom operation speed is classified into three speed areas, and the first pattern is adopted when belonging to the slowest speed area. Next, in the case of belonging to the second slowest (central) speed region, the third pattern can be adopted. Further, the second pattern can be adopted when belonging to the fastest speed region.

  In addition, since the CU can take four sizes of 64x64, 32x32, 16x16, and 8x8, all four types are used in the first pattern, and the block size increases from the center of the zoom toward the periphery. The pattern may be set as it goes. On the other hand, as the speed of the zoom operation increases, the number of block sizes to be used may be reduced from the smaller size, and the zoom variation speed may be divided into four stages to set the pattern. In this case, all four types of block sizes are used in the speed region in which the zoom fluctuation speed is the slowest, but as the fluctuation speed rises, they are not used in the order of 8 × 8, 16 × 16, and 32 × 32, and finally only 64 × 64.

  In S306, the prediction unit 106 performs prediction processing based on the divided CU. First, one of prediction methods is selected from among inter-screen prediction processes including intra-screen prediction and motion detection, and a PU size is determined. As a selection criterion for the prediction method, for example, a method in which a prediction error in each prediction method becomes smaller can be used. The PU size in the case of intra prediction is the same size as CU or a size divided into four. In the case of one inter-screen prediction, it is possible to improve prediction efficiency when different motion regions are mixed in a CU by dividing the inside of the CU into non-uniform rectangles, so the PU size is determined from the motion vector search result. . Next, a prediction difference signal is generated based on the selected prediction method and PU size.

  In step S307, the transform / quantization unit 107 performs orthogonal transform processing and quantization processing based on the divided CU. The transform / quantization unit 107 determines the TU size so that transform coding efficiency is increased according to the local nature of the input prediction difference signal. In HEVC, there are a mode for determining the TU size based on the CU and a mode for determining the TU size based on the PU size. The transform / quantization unit 107 first performs orthogonal transform processing based on the selected TU size, and calculates transform coefficients. Next, the amount of information is compressed by quantizing the transform coefficient according to the quantization parameter. Next, in S308, the entropy encoding unit 110 generates encoded data by the CABAC method. The above processes from S301 to S308 are repeatedly executed during the zoom operation (“NO” in S309).

  It should be noted that the CU size and the divided arrangement during the zoom operation may be the same at the zoom operation start time and end time as shown in FIGS. 5A and 5B, or in FIGS. 5C and 5D. As shown, the zoom operation start time and end time may be changed.

  In the example shown in FIGS. 5C and 5D, more small blocks are allocated to the center of the image in the divided arrangement after zooming in or before zooming out. This is because the user's region of interest is enlarged and displayed in the center of the image after zooming in and before zooming out, so that a high-quality image is assigned by assigning smaller blocks to the enlarged region of interest for encoding. Is to provide. On the other hand, the number of small blocks allocated is reduced in the divided arrangement before zooming in or after zooming out. This is because the user's region of interest is reduced relative to the center of the image before zooming in or after zooming out, so that the proportion of the region of interest in the entire image is reduced and assigned to the region of interest. The number of fine blocks is reduced, and a block having a larger size is assigned to the surrounding pixels in consideration of encoding efficiency.

  According to the present embodiment described above, when the image capturing apparatus 100 performs predictive encoding of a moving image by the HEVC method and performs image recording, an encoded block division pattern is adaptively set according to the zoom fluctuation speed of the zoom operation. Can be determined. As a result, it is possible to generate a moving image having a quality according to the control state of the zoom operation and improve the image quality, and at the same time improve the encoding efficiency.

[Embodiment 2]
Next, a second embodiment (Embodiment 2) of the invention will be described. In Embodiment 1 described above, when the variation of the moving image generated by shooting is based on the zoom operation, the coding block division method is determined based on the zoom variation speed. On the other hand, in the second embodiment, a case will be described in which the coding block division method is determined according to the moving speed at which the subject moves in the moving image. The movement of the subject is based on the panning operation, tilt operation, or movement of the subject itself of the digital camera or digital video camera as the imaging apparatus 100. In the following description of the present embodiment, the movement of the subject based on the panning operation will be described as an example. The panning operation and the tilt operation are basically the same except that the tilting operation is a vertical operation, whereas the panning operation is a horizontal operation. The movement of the subject itself is the same as the panning operation and the tilt operation in the sense that the relative positional relationship between the imaging device 100 and the subject changes. Therefore, the present embodiment described below can also be applied when the tilting operation or the subject itself moves.

  First, the configuration and operation of the imaging apparatus 100 corresponding to the present embodiment will be described with reference to FIG. FIG. 1B is a diagram illustrating an example of a functional configuration of the imaging apparatus 100 corresponding to the second embodiment. Since only the panning / tilt detector 113 is a functional block different from that of the first embodiment, the description of other constituent blocks is omitted in this embodiment. A panning / tilt detection unit 113 in the imaging apparatus 100 in FIG. 1B detects shaking of the imaging apparatus 100 by a gyro sensor (angular velocity sensor) or the like, and performs a panning operation in the horizontal direction based on the vibration of the imaging apparatus 100 or a vertical direction. It is possible to detect a tilting operation in the direction.

  Next, a block configuration that is a feature of the present embodiment will be described with reference to an operation timing chart of FIG. The processing corresponding to the flowchart can be realized, for example, by executing a corresponding program (stored in a ROM or the like) by the CPU functioning as the block size arrangement determining unit 104.

  First, when shooting is started, in step S401, the panning / tilt detection unit 113 detects the presence / absence of a panning operation and the panning direction. This is substantially synonymous with detecting movement of the subject. At the same time, the panning fluctuation speed (moving speed in the horizontal direction of the imaging apparatus 100) in the panning operation is measured in advance. Note that the panning fluctuation speed can be regarded as a movement speed of the subject in the image. Since a known method may be used as a method for detecting the presence / absence and direction of the panning operation and a method for measuring the fluctuation speed, a detailed description thereof is omitted here. If the panning operation is being performed (“YES” in S401), the process proceeds to S402. On the other hand, when the panning operation is not performed (“NO” in S401), in S405, feature extraction is performed from the image input to the block size arrangement determining unit 104, and the CU partition size and arrangement corresponding to the image feature are determined. . The contents of the block size determination process corresponding to the extracted features are the same as those in S305. Thereafter, the process proceeds to S406.

  In S402 to S404, the panning fluctuation speed is compared with a predetermined threshold to determine the “speed” of the panning fluctuation speed, and the CU partition size and arrangement are changed according to the determination result. In general, the panning operation is an operation in which the entire screen moves in the horizontal direction (right direction or left direction), and a region in which a subject newly appearing in the image is captured is a user's region of interest. Therefore, when panning in the panning direction, that is, when panning to the right, the image quality on the right side of the image is kept high. For the area on the side, encoding efficiency may be prioritized over image quality. Basically, a block size can be assigned in an image based on such a concept. However, since it may be difficult for the user to recognize the image quality depending on the panning fluctuation speed, it is not always necessary to adopt the above basic block allocation pattern. In the present embodiment, paying attention to this point, a method of adaptively allocating a large block size according to the panning fluctuation speed is adopted.

  Specifically, when the panning fluctuation speed is equal to or higher than the threshold and is equal to or higher than the predetermined speed, it is possible to determine that the panning operation is “fast” (“YES” in S402), and in S403, as shown in FIG. Is divided by the CU having the maximum size (64 × 64 pixel size). This is because, when the panning operation is fast, it is difficult to recognize an image and it is difficult to record a high-quality image, and the CU size is set to be as large as possible in order to increase the encoding efficiency as much as possible.

  On the other hand, when the panning fluctuation speed is smaller than the threshold, the panning operation can be determined to be “slow” (“NO” in S402). In this case, in S404, if the panning operation is being performed to the left, the left portion of the image is set to a fine CU size as shown in FIG. 6A, and the CU size is increased toward the right side. Then, the block size arrangement determining unit 104 and the block dividing unit 105 perform block division processing so that the right part of the image disappearing from the screen has a coarser CU size. If the panning operation is being performed to the right, the right portion of the image is set to a fine CU size as shown in FIG. 6B in S404, and the CU size is increased toward the left side. Then, the block size arrangement determining unit 104 and the block dividing unit 105 perform block division processing so that the left portion of the image disappearing from the screen has a coarser CU size. As described above, in the panning operation, a subject image newly appearing (entering) on the screen is a region of interest meaningful to the user, so the block size is reduced to record a high-quality image. On the other hand, since the subject image disappearing from the screen is not important for the user, the CU size is set to a large size in order to increase the encoding efficiency.

  6 (A) and 6 (B) will be further described. In this example, the CU size is changed in three stages as an example, but the CU can take four sizes of 64 × 64, 32 × 32, 16 × 16, and 8 × 8. It may be divided into stages. In this case, for example, when panning in the right direction, the right area of the image can be divided into 8 × 8 CU sizes, and the CU size can be increased to 16 × 16, 32 × 32, and 64 × 64 toward the left area.

  Further, the step of changing the CU size may be varied depending on the panning fluctuation speed, for example. For example, when the panning fluctuation speed is slow, it becomes easier to grasp the contents of the image, so that the block division can be performed using all four kinds of sizes in order to smooth the change of the CU size. On the other hand, since it becomes difficult to grasp the contents of the image as the panning fluctuation speed increases, the types of CU sizes to be used can be reduced. At that time, it may be omitted from the smaller CU size (initially the minimum 8 × 8 pixel size), and finally only the maximum size (64 × 64 pixel size) may be used. This can be realized by subdividing the threshold used when determining the speed of the panning operation and classifying the panning operation speed into a plurality of speed regions. For example, when three threshold values are prepared and four speed regions are set, the number can be set to four types, three types, two types, and one type in order of increasing speed.

  Hereinafter, the processing from S406 to S409 is the same as the processing from S306 to S309 in FIG. The processes from S401 to S408 are repeatedly executed during the panning operation (“NO” in S409).

  Note that the CU division method according to the present embodiment may be executed not only when the video camera detects movement of the imaging apparatus 100 by panning or tilting operation but also according to the movement of the subject itself. For example, when motion detection is performed for motion compensation in HEVC, the division process can be performed by treating the magnitude of the detected motion vector in the same manner as the magnitude of the fluctuation speed of the panning operation. In this case, the CU division pattern shown in FIG. 6A can be applied when the motion of the subject specified by the motion vector is in the left direction. Similarly, when the motion of the subject specified by the motion vector is rightward, the CU division pattern shown in FIG. 6B can be applied.

  According to the above-described embodiment, when the image capturing apparatus 100 performs predictive coding of a moving image by the HEVC method and performs image recording, the image capturing apparatus 100 is adapted according to the fluctuation speed in the panning operation and the tilt operation and the moving speed of the subject. Thus, the coding block division pattern can be determined. Even when the imaging apparatus 100 moves in an oblique direction, the divided block size can be determined according to the moving direction in the same manner as described above. As a result, even when the imaging apparatus 100 or the subject itself moves, it is possible to improve the coding efficiency while improving the image quality.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (18)

An imaging device that divides a moving image obtained by imaging a subject into a plurality of blocks and encodes each block,
Determining means for determining a block size for dividing the moving image from a plurality of block sizes;
Dividing means for dividing the moving image according to the determined block size;
Encoding means for performing an encoding process for each of the divided blocks,
The determining unit is configured to divide a region other than the region of interest into a block size for dividing the region of interest of the user of the imaging device in the moving image according to the magnitude of image variation in the moving image. The block size for dividing the moving image is determined by switching between the first pattern having a small size and the second pattern for dividing the moving image by the maximum block size among the plurality of block sizes. An imaging apparatus characterized by that. The magnitude of the fluctuation of the image is based on the fluctuation speed of the zoom operation when the zoom operation is performed when the subject is imaged.
The determining means includes
Determining the block size based on the second pattern when the fluctuation speed is equal to or higher than a predetermined speed;
The imaging apparatus according to claim 1, wherein the block size is determined based on the first pattern when the fluctuation speed is smaller than the predetermined speed.   The imaging apparatus according to claim 2, wherein the determination unit determines a center of zoom in the zoom operation as the region of interest when determining a block size in the first pattern. In the first pattern, the determining means is
The block size for dividing the region of interest is the minimum block size among the plurality of block sizes,
The block size for dividing the region other than the region of interest is set to a larger block size among the plurality of block sizes excluding the minimum block size as it goes to the periphery of the image. Item 4. The imaging device according to Item 3. In the first pattern, the determining means is
The block size for dividing the region other than the region of interest includes at least the maximum block size among the plurality of block sizes,
The block size for dividing the region of interest is set to a larger block size among block sizes not allocated to regions other than the region of interest among the plurality of block sizes. The imaging device described.   The imaging apparatus according to claim 5, wherein the determining unit sets a block size for dividing an area other than the region of interest to a larger block size as going to the periphery of the image.   In the first pattern, the determining unit sets a block size for dividing the region of interest to a smaller block size among the plurality of block sizes as the fluctuation speed becomes slower. The imaging device according to any one of claims 2 to 6.   The said determination means determines the block size which divides | segments the said moving image based on a said 1st pattern, when the said zoom operation | movement is complete | finished. The imaging device described. When the zoom operation is a zoom-in operation, the determining means sets the size of the region of interest in the first pattern at the end of the zoom-in operation to be larger than the size of the region of interest at the start of the zoom-in operation. Set larger,
In the case where the zoom operation is a zoom-out operation, the determining means determines the size of the region of interest in the first pattern at the start time of the zoom-out operation, and the size of the region of interest at the end time of the zoom-out operation. The imaging apparatus according to claim 2, wherein the imaging apparatus is set to be larger than the size. The magnitude of the fluctuation of the image is based on the moving speed of the subject in the moving image,
In the first pattern, the determining means is
Determining the block size based on the second pattern when the moving speed of the subject is equal to or higher than a predetermined speed;
The imaging apparatus according to claim 1, wherein the block size is determined based on the first pattern when a moving speed of the subject is smaller than the predetermined speed.   The determining means, when determining a block size in the first pattern, sets a region on the side where the subject appears in the moving image based on the moving direction of the subject as the region of interest in the moving image. The imaging device according to claim 10. In the first pattern, the determining means is
The block size for dividing the region of interest is the minimum block size among the plurality of block sizes,
The block size for dividing a region other than the region of interest is set to a larger block size among the plurality of block sizes excluding the minimum block size as the distance from the region of interest increases. Item 12. The imaging device according to Item 11. In the first pattern, the determining means is
The block size for dividing the region other than the region of interest includes at least the maximum block size among the plurality of block sizes,
12. The block size for dividing the region of interest is set to a larger block size among block sizes not assigned to regions other than the region of interest among the plurality of block sizes. The imaging device described in 1.   14. The imaging apparatus according to claim 13, wherein the determining unit sets a block size in a region other than the region of interest to a larger block size away from the region of interest.   11. The determination unit according to claim 10, wherein the block size for dividing the region of interest is set to a smaller block size among the plurality of block sizes as the moving speed of the subject decreases. 15. The imaging device according to any one of 1 to 14.   16. The movement of the subject in the moving image is based on at least one of a panning operation of the imaging device, a tilt operation of the imaging device, and a movement of the subject itself. The imaging apparatus according to item 1. A control method of an imaging apparatus that divides a moving image obtained by imaging a subject into a plurality of blocks and encodes each block,
A determining step for determining a block size for dividing the moving image from a plurality of block sizes;
A dividing step of dividing the moving image by the determined block size;
An encoding means comprising an encoding step of performing an encoding process for each of the divided blocks;
In the determining step, the block size for dividing the region of interest of the user of the imaging device in the moving image is divided into regions other than the region of interest in accordance with the magnitude of image fluctuation in the moving image. A block size for dividing the moving image is changed by switching between a first pattern having a small block size and a second pattern for dividing the moving image by a maximum block size among the plurality of block sizes. A control method for an imaging apparatus, characterized by:   A program that causes a computer to function as each unit of the imaging apparatus according to any one of claims 1 to 16.

Images