JP4310282B2 - Image encoding apparatus and encoding method - Google Patents

Description

  The present invention relates to an image encoding device and an encoding method.

  Conventionally, image coding in which a code amount is uniformly assigned to all macroblocks in one screen, and image coding after controlling the code amount so that the entire screen has a uniform resolution has been performed. It was.

On the other hand, there has been proposed a technique for improving the subjective image quality by improving the image quality of an image portion that is easily noticed by an observer when an encoded image is decoded (see Patent Document 1). The video compression recording apparatus calculates a motion vector in an image frame, and based on the calculated value, encodes a portion with a large motion while suppressing a high-frequency component, and conversely secures a resolution with a portion with a small motion. In addition, a method of encoding without reducing high frequency components is used.
JP 2001-268580 A

  However, in the method of performing uniform processing on the entire image, it is not possible to increase the compression rate of the image while maintaining good image quality.

  Also, in the method of detecting the coordinates of the image portion that is easy for the observer to notice based on the motion vector in the frame of the image and improving the image quality, the location with the large motion vector does not necessarily match the location that the observer is interested in. Therefore, there is a possibility that the image quality improvement effect cannot be obtained.

  In view of such circumstances, the present invention is an image encoding device that performs encoding control of a video signal based on imaging control information acquired from an imaging device. As the imaging control information, in addition to luminance information, focus control information used for focus control, focus evaluation information that is information indicating the degree of focus, and the like are used. When video signal encoding control is performed based on luminance information, the observer pays attention to the location where the difference in luminance between adjacent pixels is small, taking advantage of the fact that the contrast of the image decreases when autofocus is not in focus. It can be determined that it is not a part to do. The portion of the image observed by the observer is finely quantized, and the portion other than the portion of interest by the observer is coarsely quantized so that the apparent image quality can be improved. By enabling flexible control of encoding as described above, it is possible to realize good encoding that achieves both high image quality and high efficiency as a result.

  In addition, when the imaging device acquires a video signal, wobbling control information used for focusing and temporal change information of the imaging control information are acquired as imaging control information, and encoding control is performed based on them. Also good. Furthermore, encoding control may be performed based on position-related imaging control information in which position information in one screen formed by the video signal is associated with imaging control information. In addition, the position-related imaging control information may be weighted according to the position information in the one screen, and the encoding control may be performed based on the second position-related imaging control information obtained as a result. On the other hand, filtering processing may be performed on the imaging control information, and encoding control may be performed based on the filtered imaging control information obtained as a result.

  As described above, the image coding apparatus according to any one or more of the present invention performs coding control of the video signal based on the imaging control information acquired from the imaging apparatus. By enabling flexible control, as a result, it is possible to realize good coding that achieves both high image quality and high efficiency. When image signal encoding control is performed based on focus control information or focus evaluation information as imaging control information, the degree of focus can be immediately recognized, and whether or not it is a part of interest to the observer Judgment can be made easily.

  In addition, when video signal encoding control is performed based on wobbling control information as imaging control information, a portion that is not in focus even based on the wobbling processing result is not a portion that is observed by the observer. Judgment can be made. Furthermore, when video signal encoding control is performed based on temporal change information before and after the wobbling process, whether or not the difference is the difference in the imaging control information that has occurred over time is the part that the observer is interested in. Can be easily determined. Further, in the case of performing video signal encoding control based on position-related imaging control information, it is possible to accurately determine, for each part on the screen, whether or not the part is a part of interest by the observer. In addition, when encoding control is performed based on the second position-related imaging control information obtained as a result of weighting the position-related imaging control information, the center portion of the screen that an observer tends to pay attention to in general Encoding control can be performed accurately by weighting. On the other hand, when encoding control is performed based on filtered imaging control information obtained as a result of performing filtering processing on imaging control information, noise or the like can be removed, and more accurate encoding control processing can be performed. You can expect to do it.

  The best mode for carrying out each invention will be described below. Note that the present invention is not limited to these embodiments, and can be implemented in various modes without departing from the spirit of the present invention.

The first embodiment will mainly describe claims 1, 5, 6 and the like. In the second to fifth embodiments, claims 1 and 5 will be mainly described. The sixth embodiment will mainly describe claim 2 and the like. The seventh embodiment will mainly describe claim 3 and the like. The eighth embodiment will mainly describe claim 4 and the like.

  << Embodiment 1 >>

  <Embodiment 1: Overview> Embodiment 1 will be described. The present embodiment is an image encoding device that performs encoding control of a video signal based on imaging control information acquired from an imaging device.

  FIG. 1 shows an example of the concept of this embodiment. The image encoding apparatus according to the present embodiment can shoot, record, and reproduce video. A scene where a cat is photographed by an image encoding device is illustrated. First, a video signal before encoding is acquired from the imaging device (1). An image of a cat as a subject is converted into a video signal by an image pickup device such as a CCD through a lens. Next, imaging control information is acquired from the imaging apparatus in association with the video signal (2). The imaging control information corresponds to information for controlling brightness, color difference, and the like. It is assumed that these pieces of information are output from a circuit that processes a video signal. Finally, encoding control of the video signal is performed based on the acquired imaging control information (3). The video signal is encoded through discrete cosine transform (DCT) processing, quantization processing, variable length encoding processing, and the like.

  <Embodiment 1: Configuration> FIG. 2 shows an example of functional blocks in the present embodiment. The “image encoding device” (0202) of the present embodiment illustrated in FIG. 2 includes an “imaging device” (0201), a “video signal acquisition unit” (0202), and an “imaging control information acquisition unit” (0203). , “Encoding control unit” (0204).

  Each unit, which is a component of the present invention, is configured by either hardware, software, or both hardware and software. For example, as an example for realizing these, when a computer is used, hardware composed of a CPU, a memory, a bus, an interface, a peripheral device, and the like and software executable on these hardware are listed. be able to. Specifically, the functions of the respective units are realized by sequentially executing the programs developed on the memory, by processing, storing, and outputting data on the memory and data input through the interface. (The same applies throughout this specification.)

  The “image encoding device” (0200) is a device that can shoot, record, and reproduce video, and specifically includes a video camera, a digital camera, a mobile phone with a camera function, and the like.

  The “imaging device” (0201) is a general device used for a series of processes for converting an optical image of a subject into an electric signal. Specifically, the imaging apparatus corresponds to an imaging device such as a charge coupled device (CCD), a video signal processing circuit, and the like. The image of the subject is stored in the image sensor and converted into a video signal (electric signal). This video signal is input to the video signal processing circuit, and imaging control information described later is output.

  The “video signal acquisition unit” (0202) acquires a video signal before encoding from the imaging device (0201). A video signal obtained by converting an image of a subject by an imaging device is acquired.

  The “imaging control information acquisition unit” (0203) acquires imaging control information from the imaging device (0201) in association with the video signal. “Imaging control information” is information for performing imaging control, information for controlling the luminance and color difference of the captured video signal, information for adjusting and controlling the focus, and for controlling the shooting direction. Information indicating whether the image signal is acquired, information indicating whether a video signal is acquired from a plurality of imaging devices, or the like. Note that the luminance information is information relating to only the brightness component of the video signal. The color difference information is information representing the intensity and type of color obtained by removing the brightness component included in the luminance signal from the RGB three color signals.

  In addition, information used for focus control when the imaging device acquires a video signal, information indicating the degree of focusing, and the like may be included. This information will be described in detail in the second and subsequent embodiments.

  The “encoding control unit” (0204) performs encoding control of the video signal based on the imaging control information acquired by the imaging control information acquisition unit (0203). The video signal is encoded through discrete cosine transform (DCT) processing, quantization processing, variable length encoding processing, and the like. The discrete cosine transform (DCT) process is a process for converting an input video signal into a DCT coefficient indicating a frequency component. Since adjacent pixels have similar luminance, the image compression rate can be increased by DCT processing only the difference between the luminance signals in the video signal. Next, the quantization process is a process of changing a continuous numerical value to a discontinuous representative value. By this quantization processing, the number of states that the DCT coefficient can take can be reduced. The variable length coding process is a process for performing efficient coding by applying a short code to a quantized value having a high appearance frequency.

  In addition, as an example of video signal encoding control based on imaging control information, it is assumed that encoding processing based on luminance control information included in imaging control information is performed. In the autofocus mechanism, when the image is not in focus, the contrast of the image is lowered. Therefore, a portion where the difference in luminance between adjacent pixels is small is a portion where the focus is not in focus, and it cannot be said that the observer is not paying attention. . Therefore, the DCT coefficient is finely quantized for the part that is observed by the observer, while the portion other than the part that the observer is paying attention is encoded by coarsely quantizing the DCT coefficient. Can be improved and efficient. Alternatively, encoding may be performed by increasing the code amount allocation for the portion of interest to the observer while decreasing the code amount allocation for portions other than the portion of interest for the observer. Also, the code amount may be assigned in stages according to imaging control information or the like, or may be increased or decreased continuously based on a predetermined assignment function or the like.

  As another example, when an encoding process based on information for controlling the shooting direction included in the imaging control information is performed, dense encoding is performed on a subject that has the highest shooting frequency within a certain period. A technique to perform is conceivable. Further, in a situation where a plurality of imaging devices are provided, the frequency is high when encoding processing based on information for controlling which imaging device the video signal is acquired from is performed. For video signals from the imaging device, a method of performing dense encoding, etc. can be considered.

  <Embodiment 1: Process Flow> FIG. 3 shows an example of the process flow in the first embodiment. First, a video signal before encoding is acquired from the imaging device (video signal acquisition step S0301). Next, imaging control information is acquired from the imaging device in association with the video signal acquired in the video signal acquisition step (S0301). (Imaging control information acquisition step S0302). Finally, encoding control of the video signal is performed based on the imaging control information acquired in the imaging control information acquisition step (S0302) (encoding control step S0303).

  The above processing can be executed by a program for causing a computer to execute, and the program can be recorded on a recording medium readable by the computer. (The same applies throughout the specification.)

  <Embodiment 1: Effect> This embodiment is an image encoding apparatus that performs encoding control of a video signal based on imaging control information acquired from an imaging apparatus. When video signal encoding control is performed based on luminance information included in the imaging control information, the difference in luminance between adjacent pixels is small by taking advantage of the fact that the contrast of the image decreases when autofocus is not in focus. The location is a location that is not in focus and can be determined not to be a portion that the observer is interested in. The portion of the image observed by the observer is finely quantized, and the portion other than the portion of interest by the observer is coarsely quantized so that the apparent image quality can be improved. By enabling flexible control of encoding as described above, it is possible to realize good encoding that achieves both high image quality and high efficiency as a result.

  << Embodiment 2 >>

  <Embodiment 2: Overview> Embodiment 2 will be described. The present embodiment is an image encoding device characterized by acquiring focus control information used for focus control as imaging control information when the imaging device acquires a video signal.

  <Embodiment 2: Configuration> FIG. 4 shows an example of functional blocks in the present embodiment. The “image encoding device” (0400) of the present embodiment illustrated in FIG. 4 includes an “imaging device” (0401), a “video signal acquisition unit” (0402), and an “imaging control information acquisition unit” (0403). The “encoding control unit” (0404), and the imaging control information acquisition unit (0403) further includes “focus control information acquisition unit” (0405).

  "Focus control information acquisition means" (0405) acquires focus control information when the imaging device (0401) acquires a video signal. At this time, the focus control information is acquired in association with the video signal. “Focus control information” is imaging control information used for focus control. This includes information for lens drive control, information for motor drive control, information for focus position movement control, and the like. In addition, a value that can be calculated by passing a video signal (particularly a luminance signal) through a high-pass filter (HPF) and inputting a high-frequency component that is equal to or higher than a predetermined spatial frequency that can be extracted to an integration circuit, It is an example of the information which comprises focus control information. Note that when control is performed to move the lens slightly back and forth so as to be in focus based on the obtained focus control information, the focus control information at the lens position is sequentially calculated.

  The “encoding control unit” (0404) performs encoding control of the video signal based on the focus control information acquired by the focus control information acquisition unit (0405). The contrast method has been described in the first embodiment as an example of a method for performing focusing in the autofocus mechanism. However, as another method, the differential value (high frequency) of the video signal is increased as the subject is focused and the contour of the image becomes clearer. There is an edge detection method using the property that the component is large. For example, when the focus is not in focus, the focus evaluation value included in the focus control information is large. Therefore, a portion where the focus evaluation value of an adjacent pixel is small is a portion that is not in focus, and is a portion that the observer pays attention to. Not so. Therefore, the DCT coefficient is finely quantized for the part that the observer is interested in, while the apparent image quality is improved by performing the coding by coarsely quantizing the DCT coefficient for the part other than the part that the observer is paying attention to. Can be made.

  The processing of each other part is the same as in the first embodiment.

  <Embodiment 2: Process Flow> FIG. 5 shows an example of the process flow in the second embodiment. First, an unencoded video signal is acquired from the imaging device (video signal acquisition step S0501). Next, in association with the video signal acquired in the video signal acquisition step (S0501), focus control is performed from the imaging device. Focus control information that is imaging control information to be used is acquired (focus control information acquisition step S0502). Finally, encoding control of the video signal is performed based on the focus control information acquired in the focus control information acquisition step (S0502) (encoding control step S0503).

  <Embodiment 2: Effect> This embodiment is an image encoding apparatus characterized by acquiring focus control information used for focus control as imaging control information when the imaging apparatus acquires a video signal. is there. When video signal encoding control is performed based on focus control information, the focus evaluation value of an adjacent pixel is taken advantage of the fact that the focus evaluation value included in the focus control information decreases when autofocus is not in focus. It is possible to determine that the portion where is small is not in focus and is not the portion that the observer pays attention to. The portion of the image observed by the observer is finely quantized, and the portion other than the portion of interest by the observer is coarsely quantized so that the apparent image quality can be improved. By enabling various encoding controls in this way, as a result, it is possible to realize good encoding that achieves both high image quality and high efficiency.

  << Embodiment 3 >>

  <Embodiment 3: Overview> Embodiment 3 will be described. The present embodiment is an image encoding device characterized in that the focus control information includes focus evaluation information that is information indicating the degree of focus.

  <Embodiment 3: Configuration> Functional blocks in this embodiment are the same as those in FIG. The “image encoding device” (0400) illustrated in FIG. 4 includes an “imaging device” (0401), a “video signal acquisition unit” (0402), an “imaging control information acquisition unit” (0403), The imaging control information acquisition unit (0403) further includes a “focus control information acquisition unit” (0405).

  "Focus control information acquisition means" (0405) acquires focus control information when the imaging device (0401) acquires a video signal. At this time, the focus control information includes focus evaluation information that is information indicating the degree of focus. It is assumed that the focus evaluation information represents the degree of defocusing in correspondence with the level of a plurality of stages. The focus evaluation information may be calculated according to the image area or the subject. Note that, when control is performed to slightly move the lens back and forth so as to be in focus based on the obtained focus evaluation information, the focus evaluation information at the lens position is sequentially calculated.

  The “encoding control unit” (0404) performs encoding control of the video signal based on the focus control information acquired by the focus control information acquisition unit (0405). In the focus evaluation information included in the focus control information, for example, when the value is “0”, if the focus evaluation information is completely in focus, the focus evaluation information is a value other than “0”, that is, When the subject is not in focus, it can be said that it is not the part that the observer pays attention to. Therefore, the DCT coefficient is finely quantized for the part that the observer is interested in, while the apparent image quality is improved by performing the coding by coarsely quantizing the DCT coefficient for the part other than the part that the observer is paying attention to. Can be made.

  The processing of other parts is the same as that of the second embodiment.

  <Embodiment 3: Effect> This embodiment is an image encoding device characterized by including focus evaluation information, which is information indicating the degree of focus, in focus control information. Since the degree of focus can be recognized from the focus evaluation information, it is possible to easily determine that a portion where autofocus is not in focus is not a portion of interest to the observer.

  << Embodiment 4 >>

  <Embodiment 4: Overview> Embodiment 4 will be described. The present embodiment is an image encoding apparatus characterized by acquiring wobbling control information used for focusing as imaging control information when the imaging apparatus acquires a video signal.

  <Embodiment 4: Configuration> FIG. 6 shows an example of functional blocks in the present embodiment. The “image encoding device” (0600) of the present embodiment illustrated in FIG. 6 includes an “imaging device” (0601), a “video signal acquisition unit” (0602), and an “imaging control information acquisition unit” (0603). , An “encoding control unit” (0604), and an imaging control information acquisition unit (0603) includes a “focus control information acquisition unit” (0605), a “wobbling control information acquisition unit” (0606), Have

  “Wobbling control information acquisition means” (0606) acquires wobbling control information when the imaging device (0601) acquires a video signal. “Wobbling control information” is imaging control information used for focusing. Wobbling refers to control that moves the lens back and forth so that it is in focus. First, after moving the lens at a high speed in a direction in which the differential value of the video signal constituting the focus control information increases, the lens is moved minutely back and forth so that the focus is accurately adjusted. Such processing is repeated until the focus is achieved, and the lens position is determined. The wobbling control information corresponds to information necessary for the wobbling process, for example, information such as the direction in which the differential value of the video signal increases, the lens position, and the result information of the wobbling process. The subject image itself has a wide range of frequency components, but since the lens acts as an optical low-pass filter (LPF), the image signal is blurred as the lens moves away from the focal point. Image.

  The “encoding control unit” (0604) is based on the focus control information acquired by the focus control information acquisition unit (0605) and the wobbling control information acquired by the wobbling control information acquisition unit (0606). Encoding control of the video signal is performed. When it is determined that the subject is out of focus from information such as the result of the wobbling process included in the wobbling control information, it can be said that it is not a portion that the observer pays attention to. Therefore, the DCT coefficient is finely quantized for the part that the observer is interested in, while the apparent image quality is improved by performing the coding by coarsely quantizing the DCT coefficient for the part other than the part that the observer is paying attention to. Can be made. In addition, if there is a wobbling result that the focus target has been changed from the previously focused subject to another subject, the portion that the observer will pay attention to for the subject that is focused later It can be judged that.

  The processing of other parts is the same as in the second and third embodiments.

  <Embodiment 4: Effect> This embodiment is an image encoding apparatus characterized by acquiring wobbling control information used for focusing as imaging control information when the imaging apparatus acquires a video signal. is there. Even if it is based on the result of the wobbling process, it is possible to easily determine that the portion where the focus is not in focus is not the portion that the observer pays attention to.

  << Embodiment 5 >>

  <Embodiment 5: Overview> Embodiment 5 will be described. The present embodiment is an image encoding apparatus characterized by performing encoding control based on temporal change information of imaging control information.

  <Embodiment 5: Configuration> FIG. 7 shows an example of functional blocks in the present embodiment. The “image encoding device” (0700) of the present embodiment illustrated in FIG. 7 includes an “imaging device” (0701), a “video signal acquisition unit” (0702), and an “imaging control information acquisition unit” (0703). , An “encoding control unit” (0704), and the encoding control unit (0704) includes a “time change information acquisition unit” (0705) and a “time change information dependent encoding control unit” (0706). Have. Here, the configuration based on the functional blocks of the first embodiment is shown, but when other embodiments are used as a basis, further configuration requirements may be required.

  The “temporal change information acquisition unit” (0705) acquires temporal change information of the imaging control information based on the imaging control information acquired by the imaging control information acquisition unit (0703). The temporal change information corresponds to a difference in imaging control information before and after the wobbling process.

  A “time-varying information dependent coding control means” (0706) performs the coding control based on the time-varying information. When the temporal change information is large, that is, when the difference between the imaging control information before and after the wobbling process is large, the focus is focused by the wobbling process. Therefore, the DCT coefficient is finely quantized for the part that the observer is interested in, while the apparent image quality is improved by performing the coding by coarsely quantizing the DCT coefficient for the part other than the part that the observer is paying attention to. Can be made.

  The processing of other parts is the same as in the first to fourth embodiments.

  <Embodiment 5: Effect> This embodiment is an image encoding apparatus characterized by performing encoding control based on temporal change information of imaging control information. It can be easily determined whether or not it is a portion of interest by the observer based on the temporal change information, that is, the magnitude of the difference between the imaging control information before and after the wobbling process.

  << Embodiment 6 >>

  <Sixth Embodiment: Overview> A sixth embodiment will be described. In the present exemplary embodiment, position-related imaging control information obtained by associating imaging control information acquired from the imaging apparatus with position information within one screen included in the video signal is acquired, and the encoding is performed based on the position-related imaging control information. It is an image coding apparatus characterized by performing the digitization control.

  <Embodiment 6: Configuration> FIG. 8 shows an example of functional blocks in the present embodiment. The “image encoding device” (0800) of the present embodiment illustrated in FIG. 8 includes an “imaging device” (0801), a “video signal acquisition unit” (0802), and an “imaging control information acquisition unit” (0803). , The “encoding control unit” (0804), the imaging control information acquisition unit (0803) has “position-related imaging control information acquisition means” (0807), and the encoding control unit (0804) “Time-change information acquiring means” (0805), “Time-change information-dependent encoding control means” (0806), and “Position-related imaging control information-dependent encoding control means” (0808).

  “Position-related imaging control information acquisition means” (0807) acquires the positional-related imaging control information from the imaging apparatus. “Position-related imaging control information” is information in which imaging control information is associated with positional information within one screen that is formed by a video signal. FIG. 9 shows an example of position-related imaging control information. In the figure, when the screen is divided into a plurality of parts, the scene where the imaging control information of each part is associated with the position information of the part is shown. The position information corresponds to information indicating a position in one screen, and is indicated by identification information from 1a to 3c here. In addition, as the imaging control information associated with the position information, only the focus evaluation information is shown in the figure, but other information such as focus control information and wobbling control information may be associated.

  The “position-related imaging control information dependent encoding control means” (0808) performs encoding control based on the position-related imaging control information acquired by the position-related imaging control information acquisition means (0807). At this time, the encoding control is performed in consideration of the time change information acquired by the “time change information acquisition means” (0805). If the temporal change information of the screen portion indicated by the position information associated with the imaging control information is large from the position-related imaging control information, that is, if the difference between the imaging control information before and after the wobbling process is large, the wobbling process will focus. Therefore, it can be estimated that this is the part that the observer pays attention to.

  In FIG. 9, the position-related imaging control information acquired in the previous process is schematically shown below the imaging screen divided into nine. Assuming that the focus evaluation information is more focused as the value is smaller, for example, in the portion 3a, the focus evaluation information remains 10 before and after processing, and the temporal change information is indicated as 0. Further, in the portion 3b, the focus evaluation information before processing is 0, but changes to 2 after processing, so that the temporal change information is indicated as +2. Similarly, in the portion 3c, the focus evaluation information before processing is 4, whereas it decreases to 3 after processing, and the temporal change information is indicated as -1. In this way, the portion where the absolute value of the value of the time-dependent change information in each portion is large is the portion where the observer pays attention, and the DCT coefficient is finely quantized, while the portion other than the portion where the observer pays attention is the DCT coefficient. The apparent image quality can be improved by performing encoding by coarse quantization. At this time, a predetermined threshold value or the like may be set for the temporal change information, and it may be determined how to perform encoding by comparing the threshold value with the threshold value. Of course, the determination that the observer is interested may be made for a plurality of parts on the screen. Although FIG. 9 shows an example of dividing into nine parts, the number of divisions is not limited to this, and the division sizes may be different.

  The other processes are the same as those in the fifth embodiment.

  <Sixth Embodiment: Process Flow> FIG. 10 shows an example of a process flow in the sixth embodiment. First, a video signal before encoding is acquired from the imaging device (video signal acquisition step S1001). Next, imaging control information is acquired from the imaging apparatus in association with the video signal acquired in the video signal acquisition step (S1001) (imaging control information acquisition step S1002). In addition, it acquires position-related imaging control information that is information obtained by associating the imaging control information acquired from the imaging apparatus with position information within one screen that is formed by the video signal (position-related imaging control information acquisition step S1003). . Next, the temporal change information of the imaging control information is acquired based on the imaging control information acquired in the imaging control information acquisition step (S1002) (temporal change information acquisition step S1004). Further, the time-varying information is compared with a predetermined value (time-varying information comparison step S1005), and if it is larger than the predetermined value, dense quantization is performed based on the position-related imaging control information and the time-varying information. Alternatively, encoding is performed by assigning a large amount of encoding (fine encoding control step S1006). On the other hand, if the temporal change information is smaller than the predetermined value in the temporal change information comparison step (S1005), sparse quantization or a small amount is performed based on the position-related imaging control information and the temporal change information. Encoding is performed by assigning the amount of encoding (sparse encoding control step S1007).

  <Embodiment 6: Effect> This embodiment acquires imaging control information with position association in which imaging control information acquired from an imaging apparatus is associated with position information within one screen formed by a video signal. The present invention is an image encoding device characterized by performing encoding control based on imaging control information. It is possible to accurately determine for each part on the screen whether or not it is a part that the observer pays attention to.

  << Embodiment 7 >>

  <Embodiment 7: Overview> Embodiment 7 will be described. In the present embodiment, the position-related imaging control information is weighted according to the position information in the one screen, and the encoding control is performed based on the second position-related imaging control information that is obtained as a result. This is an image coding apparatus having a feature.

  <Embodiment 7: Configuration> FIG. 11 shows an example of functional blocks in the present embodiment. The “image encoding device” (1100) of this embodiment shown in FIG. 11 includes an “imaging device” (1101), a “video signal acquisition unit” (1102), and an “imaging control information acquisition unit” (1103). , The “encoding control unit” (1104), and the imaging control information acquisition unit (1103) includes “position-related imaging control information acquisition unit” (1107), and the encoding control unit (1104) “Aging change information acquisition means” (1105), “Aging change information dependent encoding control means” (1106), “Position-related imaging control information dependent encoding control means” (1108), and “Weighting means” ”(1109) and“ second position-related imaging control information dependent encoding control means ”(1110).

  The “weighting means” (1109) weights the position-related imaging control information acquired by the position-related imaging control information acquisition means (1108) according to the position information in the one screen. The weighting means assumes that weighting is performed on the central portion of the screen since the image screen generally pays attention to the central portion of the screen. For example, the imaging control information associated with the position information 2b in FIG.

  The “second position-related imaging control information dependent encoding control means” (1110) performs the encoding control based on the second position-related imaging control information. The “second position-related imaging control information” is information obtained as a result of weighting by the weighting means (1109). In the example using FIG. 9, the value obtained by doubling the imaging control information associated with the position information 2b is the second position-related imaging control information. Based on this second position-related imaging control information, when the temporal change information of the screen portion indicated by the position information associated with the imaging control information is large, that is, when the difference between the imaging control information before and after the wobbling process is large Since the focus is focused by the wobbling process, it can be estimated that the part is the part of the observer's attention.

  The other processes are the same as in the sixth embodiment.

  <Seventh Embodiment: Process Flow> FIG. 12 shows an example of a process flow in the seventh embodiment. First, a video signal before encoding is acquired from the imaging device (video signal acquisition step S1201). Next, imaging control information and position-related imaging control information are acquired from the imaging apparatus in association with the video signal acquired in the video signal acquisition step (S1201) (imaging control information acquisition step S1202). Next, the temporal change information of the imaging control information is acquired based on the imaging control information acquired in the imaging control information acquisition step (S1202) (temporal change information acquisition step S1203). Further, the imaging control information with position association acquired in the imaging control information acquisition step (S1202) is weighted according to the position information in the one screen (weighting step S1204). Further, the time-dependent change information is compared with a predetermined value (time-dependent information comparison step S1205). Based on the imaging control information, the position-related imaging control information, and the time-dependent change information, dense quantization or encoding by assigning a large amount of encoding is performed (fine encoding control step S1206). On the other hand, when the temporal change information is smaller than the predetermined value in the temporal change information comparison step (S1205), based on the second position-related imaging control information, the position-related imaging control information, and the temporal change information. Thus, sparse quantization or encoding by assigning a small amount of encoding is performed (sparse encoding control step S1207).

  <Embodiment 7: Effect> In this embodiment, the position-related imaging control information is weighted according to the position information in the one screen, and the second-position-related imaging control information, which is obtained as a result, is added. The present invention is an image encoding device characterized by performing encoding control based on the above. In general, since the observer often pays attention to the central portion of the screen, weighting is performed on the central portion of the screen, and the portion of interest of the observer can be encoded accurately.

  << Embodiment 8 >>

  <Embodiment 8: Overview> Embodiment 8 will be described. The present embodiment is an image encoding device characterized by performing filtering processing on imaging control information and performing encoding control based on the filtered imaging control information obtained as a result.

  <Embodiment 8: Configuration> FIG. 13 shows an example of functional blocks in this embodiment. The “image encoding device” (1300) of this embodiment shown in FIG. 13 includes an “imaging device” (1301), a “video signal acquisition unit” (1302), and an “imaging control information acquisition unit” (1303). , “Encoding control unit” (1304), and the encoding control unit (1304) further includes “filter processing means” (1305), “filtered imaging control information acquisition means” (1306), “ Filtered imaging control information dependent encoding control means "(1307). Here, the configuration based on the functional blocks of the first embodiment is shown, but when other embodiments are used as a basis, further configuration requirements may be required.

  The “filter processing means” (1305) performs a filter process on the imaging control information. Filtering methods include a method for removing a rapid temporal change such as a low-pass filter (LPS), and a case where there is a prescribed level difference before and after time continuation, and is included in the imaging control information. It is conceivable that the focused evaluation information is not used for calculating the difference, and that the difference calculation of the focused evaluation information is provided with a hysteresis characteristic. In addition to being able to remove noise during shooting, camera shake, changes in the amount of light at the shooting location, etc. from the imaging control information, the purpose is to efficiently encode by omitting unnecessary information for encoding. Is.

  The “filtered imaging control information acquisition unit” (1306) acquires filtered imaging control information obtained as a result of the filtering process. Filtered imaging control information obtained by removing noise and the like from the imaging control information can be acquired by the filtering process.

  The “filtered imaging control information dependent encoding control means” (1307) performs encoding control based on the filtered imaging control information. Based on the brightness information of the filtered imaging control information, etc., the contrast of the image is lowered when it is not in focus. Therefore, the places where the difference in brightness between adjacent pixels is small are places that are not in focus, and the observer Is not the part of interest. Therefore, the DCT coefficient is finely quantized for the part that the observer is interested in, while the apparent image quality is improved by performing the coding by coarsely quantizing the DCT coefficient for the part other than the part that the observer is paying attention to. Can be made. Since the filtered imaging control information is imaging control information from which noise and the like have been removed, more accurate encoding processing can be performed.

  <Eighth Embodiment: Process Flow> FIG. 14 shows an example of a process flow in the eighth embodiment. First, a video signal before encoding is acquired from the imaging device (video signal acquisition step S1401). Next, imaging control information is acquired from the imaging device in association with the video signal acquired in the video signal acquisition step (S1401). (Imaging control information acquisition step S1402). Next, filter processing is performed on the imaging control information acquired in the imaging control information acquisition step (S1402) (filter processing step S1403). Further, the filtered imaging control information obtained as a result of the filtering process is acquired (filtered imaging control information acquisition step S1404). Finally, encoding control of the video signal is performed based on the filtered imaging control information (encoding control step S1405).

  <Embodiment 8: Effect> This embodiment is an image encoding device characterized by performing filtering processing on imaging control information and performing encoding control based on the filtered imaging control information obtained as a result. is there. By performing the filter processing, noise and the like can be removed, and it can be expected that more accurate encoding control processing is performed. Further, it is possible to omit information unnecessary for encoding and perform encoding efficiently.

Conceptual diagram for explaining the first embodiment Functional block diagram for explaining the first embodiment The figure explaining the flow of processing of Embodiment 1. Functional block diagram for explaining the second embodiment The figure explaining the flow of processing of Embodiment 2. Functional block diagram for explaining the fourth embodiment Functional block diagram for explaining the fifth embodiment Functional block diagram for explaining the sixth embodiment FIG. 10 is a diagram for explaining position-related imaging control information in Embodiment 6. The figure explaining the flow of processing of Embodiment 6. Functional block diagram for explaining the seventh embodiment The figure explaining the flow of processing of Embodiment 7. Functional block diagram for explaining the eighth embodiment The figure explaining the flow of processing of Embodiment 8.

Explanation of symbols

0200 Image encoding device 0201 Video signal acquisition unit 0202 Imaging control information acquisition unit 0203 Encoding control unit

Claims (6)

A video signal acquisition unit for acquiring a video signal before encoding from the imaging device;
An imaging control information acquisition unit that acquires imaging control information from the imaging device in association with the video signal;
An encoding control unit that performs encoding control of the video signal based on the imaging control information acquired by the imaging control information acquisition unit,
The imaging control information acquisition unit
Focus control information acquisition means for acquiring focus control information, which is imaging control information used for focus control, including focus evaluation information that is information indicating the degree of focus when the imaging device acquires a video signal When,
Wobbling control information acquisition means for acquiring wobbling control information which is imaging control information used for focusing when the imaging device acquires a video signal;
Have
The encoding control unit
A temporal change information acquisition means for acquiring temporal change information of the imaging control information based on the imaging control information acquired by the imaging control information acquisition unit;
Time-dependent information dependent encoding control means for performing the encoding control based on the time-dependent change information;
An image encoding device having: The imaging control information acquisition unit
Image capturing control information acquired from the image capturing apparatus, and having position-related imaging control information acquisition means for acquiring position-related imaging control information that is information associated with position information in one screen formed by a video signal;
The encoding control unit
The image coding apparatus according to claim 1, further comprising a position-related imaging control information dependent coding control unit that performs coding control based on the position-related imaging control information acquired by the position-related imaging control information acquisition unit. . The encoding control unit
Weighting means for weighting the position-related imaging control information acquired by the position-related imaging control information acquisition means according to the position information in the one screen;
Second position-related imaging control information dependent encoding control means for performing the encoding control based on second position-related imaging control information that is information obtained as a result of weighting by the weighting means;
The image encoding device according to claim 2, comprising: The encoding control unit
Filter processing means for performing filter processing on the imaging control information;
Filtered imaging control information acquisition means for acquiring filtered imaging control information obtained as a result of the filtering process;
Filtered imaging control information dependent encoding control means for performing encoding control based on the filtered imaging control information;
The image encoding device according to claim 1, comprising: A video signal acquisition step of acquiring a video signal before encoding from the imaging device;
An imaging control information acquisition step of acquiring imaging control information from the imaging device in association with the video signal acquired in the video signal acquisition step;
A time-dependent change information acquisition step for acquiring time-dependent change information of the imaging control information based on the imaging control information acquired in the imaging control information acquisition step;
A temporal change information dependent encoding control step for performing encoding control of the video signal based on imaging control information including the temporal change information;
An image encoding method comprising:
The imaging control information acquired in the imaging control information acquisition step is
When the imaging device acquires a video signal, it includes focus evaluation information that is information indicating the degree of focus, is focus control information used for focus control, and
An image encoding method characterized by the wobbling control information used for focusing when the imaging apparatus acquires a video signal. A video signal acquisition step of acquiring a video signal before encoding from the imaging device;
An imaging control information acquisition step of acquiring imaging control information from the imaging device in association with the video signal acquired in the video signal acquisition step;
A time-dependent change information acquisition step for acquiring time-dependent change information of the imaging control information based on the imaging control information acquired in the imaging control information acquisition step;
A temporal change information dependent encoding control step for performing encoding control of the video signal based on imaging control information including the temporal change information;
An image encoding program comprising:
The imaging control information acquired in the imaging control information acquisition step is
When the imaging device acquires a video signal, it includes focus evaluation information that is information indicating the degree of focus, is focus control information used for focus control, and
A recording medium storing an image encoding program characterized by wobbling control information used for focusing when the imaging apparatus acquires a video signal.

Images