JP5219735B2 - Image processing apparatus, method, and program - Google Patents

Description

The present invention relates to an image processing apparatus, method, and program, and more specifically, to a video processing apparatus, method, and program for converting an aspect ratio of a video signal.

  In recent television signals, a conventional standard video signal and a higher-definition high-definition video signal are mixed. In principle, the aspect ratio (aspect ratio of the screen) of the standard video signal SDTV (Standard Definition TeleVision) signal is 4: 3. On the other hand, the aspect ratio of the high definition video signal HDTV (High Definition TeleVision) signal is 16: 9.

  When an SDTV signal image (hereinafter referred to as an SD image) is displayed on an HDTV-compatible video monitor, the SDTV signal is up-converted. On the other hand, when an HDTV signal video (hereinafter referred to as an HD image) is displayed on an SDTV-compatible video monitor, the HDTV signal is down-converted to an SDTV signal. At that time, it is also necessary to convert the aspect ratio.

  There are several methods for aspect ratio conversion. For example, in the case of up-conversion from an SDTV signal to an HDTV signal, an aspect ratio conversion method called anamorphics is known. FIG. 10A shows an example of an SDTV signal with an aspect ratio of 4: 3, and FIG. 10B shows an example of an HDTV signal with an aspect ratio of 16: 9. Anamorphic is simply a method of stretching an image in the horizontal direction so that it fits just on a 16: 9 aspect ratio screen. In this method, all the SD images before aspect ratio conversion can be displayed on the entire screen having an aspect ratio of 16: 9 without omission.

In this method, since the screen is stretched left and right, the image is displayed horizontally, and the appearance is deteriorated. On the other hand, a method for suppressing the enlargement of the central portion of the screen and greatly expanding the peripheral portion has been proposed (see Patent Document 1).
JP 2001-157139 A

  FIG. 11 shows a flowchart of enlargement ratio control at the time of aspect ratio conversion. When the reproduction mode is selected (S31), a preset enlargement rate is detected, and the enlargement rate at the time of up-conversion is determined (S32). When the playback button or the like is pressed and playback is turned on (S33), the image is stretched in the horizontal direction according to the stretch rate acquired in step S32, and the aspect ratio is converted (S34). Step S34 is continued until playback is turned off (S35).

  FIG. 12 shows an example of the screen when the SDTV image is uniformly stretched in the horizontal direction and when the peripheral portion is stretched in the horizontal direction. FIG. 12A shows an original image having an aspect ratio of 4: 3. FIG. 12B shows a screen example when the image shown in FIG. 12A is uniformly stretched in the horizontal method. FIG. 12C shows an example of a screen when the peripheral portion in the horizontal direction of the image shown in FIG. The main subject at the time of shooting often exists in the center of the shooting area as shown in FIG. When the image shown in FIG. 12A is uniformly stretched in the horizontal direction, the subject is crushed and looks fat as shown in FIG. On the other hand, if the peripheral portion in the horizontal direction is mainly stretched, the subject located at the center of the screen is less likely to be crushed as shown in FIG.

  On the other hand, at a school or concert, when an image with an aspect ratio of 4: 3 is displayed on a screen with an aspect ratio of 16: 9, there are times when it is desired to uniformly stretch the image in the horizontal direction. FIG. 13 shows an example of a captured image of a group performance scene such as a school performance. FIG. 13A shows an original image having an aspect ratio of 4: 3. FIG. 13B shows a screen example when the image shown in FIG. 13A is uniformly stretched in the horizontal method. FIG. 13C shows an example of a screen when the peripheral portion in the horizontal direction of the image shown in FIG. Specifically, this is a case where the zoom is set to the wide-angle side and the subject of interest is also located in the peripheral area within a wide field of view. In such a case, the display as shown in FIG. 13B is preferable to the display as shown in FIG.

  When switching from uniform enlargement to peripheral enlargement or vice versa during the same scene (photographing by pressing the record button once), the appearance of the image suddenly changes in the peripheral portion of the screen, and the appearance is poor.

An object of the present invention is to provide an image processing apparatus, a method, and a program capable of appropriately displaying an enlarged image without such inconvenience.

An image processing apparatus according to the present invention includes an acquisition unit that acquires an image signal indicating an image captured by an imaging apparatus , and an image processing unit that performs a process of changing an aspect ratio of the image obtained by the acquisition unit. The image processing means has a larger peripheral position than a center position in the horizontal direction of the image obtained by the acquisition means, as the zoom ratio at the time of imaging of the image obtained by the acquisition means increases. It is characterized by increasing the rate.
An image processing apparatus according to the present invention includes an acquisition unit that acquires an image signal indicating an image captured by an imaging apparatus, and an image processing unit that performs a process of changing an aspect ratio of the image obtained by the acquisition unit. And the image processing means increases the enlargement ratio of the peripheral part in the horizontal direction of the image and increases the enlargement ratio of the peripheral part as the zoom ratio at the time of imaging of the image obtained by the obtaining means increases. Is characterized by narrowing the portion with a low stretch rate.
An image processing apparatus according to the present invention includes an acquisition unit that acquires an image signal indicating an image captured by an imaging apparatus, and an image processing unit that performs a process of changing an aspect ratio of the image obtained by the acquisition unit. And the image processing means reduces the enlargement ratio of the peripheral portion in the horizontal direction of the image as the zoom ratio at the time of imaging of the image obtained by the acquisition means is smaller, and the enlargement ratio of the peripheral portion. Is also characterized by widening the portion of low stretch rate.
An image processing method according to the present invention includes an acquisition step of acquiring an image signal indicating an image captured by an imaging device, and an image processing step of performing a process of changing an aspect ratio of the image obtained by the acquisition step. In the image processing step, the larger the zoom rate at the time of capturing the image obtained in the acquisition step, the larger the peripheral position is stretched than the center position in the horizontal direction of the image obtained in the acquisition step. It is characterized by increasing the rate.
An image processing method according to the present invention includes an acquisition step of acquiring an image signal indicating an image captured by an imaging device, and an image processing step of performing a process of changing an aspect ratio of the image obtained by the acquisition step. And the image processing step increases the enlargement rate of the peripheral portion in the horizontal direction of the image and increases the enlargement rate of the peripheral portion as the zoom rate at the time of capturing the image obtained by the acquisition step increases. Is characterized by narrowing the portion with a low stretch rate.
An image processing method according to the present invention includes an acquisition step of acquiring an image signal indicating an image captured by an imaging device, and an image processing step of performing a process of changing an aspect ratio of the image obtained by the acquisition step. And the image processing step reduces the enlargement rate of the peripheral portion in the horizontal direction of the image as the zoom rate at the time of capturing the image obtained in the acquisition step is smaller, and the enlargement rate of the peripheral portion. Is also characterized by widening the portion of low stretch rate.

  According to the present invention, it is possible to realize automatic aspect ratio conversion suitable for a subject by controlling the horizontal enlargement method with reference to zoom information at the time of shooting. This improves the appearance of the displayed image being crushed when shooting with the zoom set to telephoto on a series of recorded images, and displays the entire image flat when shooting with the zoom set to a wide angle. It becomes possible.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an imaging apparatus incorporating an embodiment of a video processing apparatus according to the present invention. However, it is assumed that the photographed image is recorded on a recording medium as an SDTV signal, converted to an HDTV signal at the time of reproduction output, and output on a monitor.

  The image input device 10 includes a photographic lens and an imaging unit that converts an image captured by the photographic lens into an image signal. The photographing lens has a zoom function and outputs the current zoom value to the control device 12. In this embodiment, the maximum zoom rate is set to x4. The output image signal of the imaging unit, that is, the image signal output from the image input device 10 is input to the image processing device 14.

  The image processing device 14 performs predetermined image processing on the image signal from the image input device 10, and supplies the monitor image signal to the image display device 16 and the recording image signal to the format control device 18.

  On the other hand, the voice input device 20 takes in surrounding sounds and supplies a sound signal indicating the surrounding sounds to the sound processing device 22. The audio processing device 22 performs predetermined processing on the audio signal from the audio input device 20 and supplies it to the format control device 18 for recording.

  The control device 12 continuously supplies the zoom value of the photographing lens of the image input device 10 to the format control device 18.

  The format control device 18 processes and multiplexes the image signal from the image processing device 14, the sound signal from the sound processing device 22, and the zoom value information from the control device 12 for recording, and converts them into a recording format. The recording / reproducing device 24 modulates the signal from the format control device 18 into a form suitable for recording on the recording medium 26 and records it on the recording medium 26. The recording medium 26 is composed of, for example, an optical disk, a magneto-optical disk, a magnetic disk, or a semiconductor nonvolatile memory.

  In this way, the SDTV video signal to which the zoom information at the time of shooting is added is recorded on the recording medium 26.

  FIG. 2 shows the file arrangement of the recording medium 26 and the structure of the image file. Audio information is arranged in the audio folder, and image information is arranged in the image folder. In the image folder, a reproduction control file, an image data file, a backup file, and a user information (User.info) file are stored. The reproduction control file stores information related to reproduction control. The image data file stores the image data itself (or compressed image data in the case of compression encoding). The backup file stores the backed up data. In this embodiment, information (zoom information) indicating the zoom value for each frame at the time of shooting is stored in the user information file.

  Next, a case where the SDTV signal recorded in this manner on the recording medium 26 is up-converted and displayed as an HDTV signal in the playback mode will be described. Here, it is assumed that the image display device 16 also supports HDTV signals. Of course, it is obvious that an HDTV monitor may be connected to the external output terminal and the reproduced video may be displayed on the screen of the HDTV monitor.

  The control device 12 instructs the recording / reproducing device 24 to read out information related to the video designated by the user. The recording / playback device 24 reads the audio file, playback control file, and image data file of the designated video and supplies them to the format control device 18. The recording / reproducing device 24 also reads out the user information file of the designated video from the recording medium 26 and supplies it to the user management information detecting device 28. The user management information detection device 28 detects zoom information for each frame from the user information file and supplies the zoom information to the enlargement ratio control device 30.

  The format control device 18 converts the image information from the recording / reproducing device 24 into a display format and supplies it to the image processing device 14. The image processing device 14 converts the aspect ratio of the reproduced video signal from the format control device 18 at an enlargement rate according to the enlargement rate control device, and converts it into an HDTV signal. Details of the aspect ratio conversion will be described later. The image processing device 14 supplies a playback video signal in the HDTV signal format to the image display device 16.

  On the other hand, the audio processing device 22 processes the reproduced audio signal from the format control device 18 for output and supplies it to the audio output device 32. The audio output device 32 includes, for example, a multi-channel speaker.

  FIG. 3 shows the enlargement ratio characteristic of the present embodiment according to the zoom magnification. The horizontal axis indicates the horizontal position of the screen, and the vertical axis indicates the enlargement rate. A characteristic 40 indicates an enlargement characteristic at a zoom ratio of 1 in the present embodiment. In this embodiment, the same enlargement ratio is set at any horizontal position. That is, the characteristic 40 corresponds to uniform stretching. The characteristics 42, 44, and 46 correspond to the stretching characteristics of peripheral stretching. A characteristic 42 represents an enlargement characteristic when the zoom rate is double. A characteristic 44 indicates an enlargement characteristic when the zoom ratio is three times. A characteristic 46 indicates an enlargement characteristic when the zoom ratio is four times.

  As described above, in the enlargement characteristic, the larger the zoom rate, the larger the enlargement rate of the peripheral portion, the narrower the central portion of non-stretching (or low stretch), and the smaller the zoom rate, the wider the central portion. To be determined. The enlargement ratio control device 30 functions as an enlargement characteristic determination means, an expansion characteristic determination step, or an expansion characteristic determination function that determines the expansion characteristic in this way. Then, the image processing device 14 converts the aspect ratio by enlargement according to the determined enlargement characteristic.

FIG. 4 shows a schematic block diagram of a part of the image processing apparatus 14 for converting an SD image into an HDTV image. A built-in memory 34 that stores at least one frame of SD images from the format control device 18, an up-converter 36, and a frame memory 38 that stores HD images generated by the up-converter 36 are provided. The built-in memory 34 stores a frame image (SD image) of the reproduced video signal from the format control device 18. The enlargement ratio control device 30 determines the enlargement ratio characteristic to be applied in accordance with the zoom information from the user management information detection device 28 and sets it in the up-converter 36. The up-converter 36 stretches the SD image stored in the built-in memory 34 in the horizontal direction in accordance with the stretch rate characteristic set by the stretch rate control device 30. The up-converter 36 further interpolates the image stretched in the horizontal direction in the vertical direction, converts it into an HD image of HD resolution, and stores it in the frame memory 38. The HD images stored in the frame memory 38 are read at a predetermined cycle and supplied to the image display device 16 as HDTV signals.

  FIG. 5 shows a flowchart of aspect ratio conversion control at the time of SD / HD conversion. The user can instruct the control device 12 whether or not to enable the zoom ratio-corresponding enlargement of the present embodiment by an operation means (not shown).

  When the zoom ratio corresponding enlargement is set to be valid (S11), steps S12 to S17 are executed. That is, when a playback mode is selected (S12) and playback start is instructed (S13), zoom information for each frame is detected (S14), and an enlargement ratio corresponding to the zoom information is set (S15). Then, the image is stretched in the horizontal direction according to the set stretch ratio, and the aspect ratio is converted (S16). Steps S14 to S16 are executed until playback is turned off (S17).

  On the other hand, when the zoom ratio corresponding enlargement is not set to be valid (S11), steps S18 to S22 are executed. That is, when the reproduction mode is selected (S18), a preset enlargement rate is detected, and the enlargement rate at the time of up-conversion is determined (S19). When the playback button or the like is pressed and playback is turned on (S20), the image is stretched in the horizontal direction according to the stretch rate acquired in step S19, and the aspect ratio is converted (S21). Step S21 is continued until playback is turned off (S22).

  The characteristic operation of this embodiment will be described with reference to the image examples shown in FIGS. FIG. 6A shows a captured image in the case of 1 × zoom. FIG. 6B shows a captured image in the case of double zoom. FIG. 6C shows a captured image in the case of 3 × zoom. FIG. 6D shows a captured image in the case of 4 × zoom. FIGS. 7A to 7D show examples of reproduced images when the captured images shown in FIGS. 6A to 6D are uniformly stretched in the horizontal direction. FIGS. 8A to 8D each have a fixed enlargement characteristic independent of the zoom value, for example, characteristic 46, and the peripheral portion of the photographed image shown in FIGS. 6A to 6D is horizontally extended. An example of a reproduced image in this case is shown. FIGS. 9A to 9D show examples of reproduced images when the captured images shown in FIGS. 6A to 6D are stretched in the horizontal direction with stretch characteristics 40 to 46 according to the zoom value, respectively.

  In the uniform enlargement aspect ratio conversion, when the zoom ratio is low, as shown in FIG. 7A, a flat feeling can be obtained even if the whole is crushed. On the other hand, when the zoom ratio is high, as shown in FIG.

  In peripheral enlargement with a fixed enlargement characteristic that does not depend on the zoom value, when the zoom ratio is high, the subject at the center of the image is displayed without being crushed as shown in FIG. However, when the zoom rate is low, it cannot be displayed flat as shown in FIG.

  The aspect ratio conversion that changes the enlargement characteristic of the peripheral enlargement according to the zoom value, which is a feature of the present embodiment, can solve the above disadvantages. That is, when the zoom ratio is low, it becomes close to uniform enlargement, and the whole is displayed flat or nearly flat as shown in FIG. On the other hand, when the zoom ratio is high, the center subject can be displayed almost without being crushed as shown in FIG.

  The enlargement ratio characteristic corresponding to the zoom ratio can be arbitrarily set by the user. In the present embodiment, as shown in FIG. 3, it is set in four stages, but it may be changed more or more functionally.

  Although the embodiment in the imaging apparatus has been described, the aspect ratio conversion itself of the present invention can be implemented as a computer program on a computer, that is, a video processing program. A video signal with zoom information is input from the imaging apparatus to the computer, and the aspect ratio is converted on the computer in the same manner as in the above-described embodiment. Then, if necessary, the SDTV signal is converted into an HDTV signal.

1 is a block diagram showing a schematic configuration of an imaging apparatus incorporating an embodiment of a video processing apparatus according to the present invention. It is an example of a file structure of a recording medium. It is an example of a correspondence between zoom ratio and enlargement ratio characteristics. 2 is a block diagram schematically illustrating the configuration of an image processing apparatus 14. FIG. It is a flowchart of enlargement rate control of a present Example. It is an example of a captured image of 1 to 4 times zoom. It is an example of an image of aspect ratio conversion by uniform enlargement of a conventional example. It is an example of an image of aspect ratio conversion by peripheral enlargement of a conventional example. It is an example of an image of aspect ratio conversion by a present Example. It is explanatory drawing of the anamorphic method as an example of the aspect-ratio conversion method. It is a flowchart of the enlargement ratio control in the conventional aspect ratio conversion. It is an example of an image of aspect ratio conversion. It is another example of an image of aspect ratio conversion.

Explanation of symbols

10: Image input device 12: Control device 14: Image processing device 16: Image display device 20: Audio input device 22: Audio processing device 24: Recording / playback device 26: Recording medium 28: User management information detection device 30: Enlargement rate control Device 32: Audio output device 34: Built-in memory 36: Upconverter 38: Frame memory

Claims (13)

Acquisition means for acquiring an image signal indicating an image captured by the imaging device;
Image processing means for performing processing for changing the aspect ratio of the image obtained by the acquisition means
And
Wherein the image processing means, the larger the zoom ratio at the time of imaging of the image obtained by the acquisition unit, greatly towards the peripheral position is stretching rate than the center position in the horizontal direction of the image obtained by the acquisition unit An image processing apparatus characterized by comprising: The image processing means increases the enlargement ratio of the peripheral part in the horizontal direction of the image and enlarges the enlargement ratio lower than the enlargement ratio of the peripheral part as the zoom ratio at the time of capturing the image obtained by the obtaining means increases. The image processing apparatus according to claim 1 , wherein the rate portion is narrowed. The image processing means reduces the enlargement ratio of the peripheral portion in the horizontal direction of the image and reduces the enlargement ratio lower than the enlargement ratio of the peripheral portion as the zoom ratio at the time of capturing the image obtained by the acquisition means is smaller. the image processing apparatus according to claim 1 or 2, characterized in that a wider portion of the rate. Acquisition means for acquiring an image signal indicating an image captured by the imaging device;
Image processing means for performing processing for changing the aspect ratio of the image obtained by the acquisition means
And
The image processing means increases the enlargement ratio of the peripheral part in the horizontal direction of the image and enlarges the enlargement ratio lower than the enlargement ratio of the peripheral part as the zoom ratio at the time of capturing the image obtained by the obtaining means increases. Narrow the rate part
An image processing apparatus. The image processing means reduces the enlargement ratio of the peripheral portion in the horizontal direction of the image and reduces the enlargement ratio lower than the enlargement ratio of the peripheral portion as the zoom ratio at the time of capturing the image obtained by the acquisition means is smaller. The image processing apparatus according to claim 4, wherein the rate portion is widened. Acquisition means for acquiring an image signal indicating an image captured by the imaging device;
Image processing means for performing processing for changing the aspect ratio of the image obtained by the acquisition means
And
The image processing means reduces the enlargement ratio of the peripheral portion in the horizontal direction of the image and reduces the enlargement ratio lower than the enlargement ratio of the peripheral portion as the zoom ratio at the time of capturing the image obtained by the acquisition means is smaller. Widen the rate part
An image processing apparatus. An acquisition step of acquiring an image signal indicating an image captured by the imaging device;
Possess an image processing step of performing processing for changing the aspect ratio of the image obtained by the obtaining step,
In the image processing step, the larger the zoom rate at the time of capturing the image obtained in the acquisition step, the larger the enlargement rate in the peripheral position than in the horizontal center position of the image obtained in the acquisition step. To be
An image processing method. The image processing step increases the enlargement ratio of the peripheral portion in the horizontal direction of the image and enlarges the enlargement rate lower than the enlargement ratio of the peripheral portion as the zoom ratio at the time of capturing the image obtained by the acquisition step increases. 8. The image processing method according to claim 7, wherein the rate portion is narrowed. In the image processing step, the smaller the zoom rate at the time of capturing the image obtained in the acquisition step, the smaller the enlargement rate of the peripheral portion in the horizontal direction of the image, and the lower the enlargement rate of the peripheral portion. 9. The image processing method according to claim 7, wherein the rate portion is widened. An acquisition step of acquiring an image signal indicating an image captured by the imaging device;
An image processing step for performing a process of changing the aspect ratio of the image obtained by the acquisition step
And
The image processing step increases the enlargement ratio of the peripheral portion in the horizontal direction of the image and enlarges the enlargement rate lower than the enlargement ratio of the peripheral portion as the zoom ratio at the time of capturing the image obtained by the acquisition step increases. Narrow the rate part
An image processing method. In the image processing step, the smaller the zoom rate at the time of capturing the image obtained in the acquisition step, the smaller the enlargement rate of the peripheral portion in the horizontal direction of the image, and the lower the enlargement rate of the peripheral portion. The image processing method according to claim 10, wherein the rate portion is widened. An acquisition step of acquiring an image signal indicating an image captured by the imaging device;
An image processing step for performing a process of changing the aspect ratio of the image obtained by the acquisition step
And
In the image processing step, the smaller the zoom rate at the time of capturing the image obtained in the acquisition step, the smaller the enlargement rate of the peripheral portion in the horizontal direction of the image, and the lower the enlargement rate of the peripheral portion. Widen the rate part
An image processing method. As each means according to claims 1 6, a computer program for operating a computer.

Images