JP5371310B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly, to motion blur correction processing of a moving image signal.

Conventionally, a device for displaying a moving image is known.
As this type of device, for example, a television receiver (hereinafter referred to as a television) is known. A television generally has a function of inputting and displaying a moving image signal supplied from an external device such as a video camera in addition to a television program received by a tuner.

  In a video camera, a user often takes a picture while holding the camera at the time of shooting, and screen shots due to camera shake often occur in the shot moving images.

  Therefore, when a moving image including such a screen blur is reproduced by a video camera and displayed on a television, a moving image including the screen blur is displayed.

  Therefore, conventionally, many techniques for preventing camera shake that occurs when an image is taken with a video camera have been proposed.

In addition, a technique for correcting camera shake at the time of playback of a moving image shot with a video camera that does not have a camera shake correction function during shooting has been proposed (see, for example, Patent Document 1).
Japanese Patent Laid-Open No. 10-42233

  As in Japanese Patent Application Laid-Open No. 2004-133620, in the blur correction process at the time of moving image reproduction, a part of the image stored in the memory is cut out and enlarged, so that the image quality may be deteriorated by the interpolation process at the time of enlargement.

  In addition, in a television, the source of moving images to be displayed is not only a video camera but also a broadcast program received by a tuner, a DVD recorder, and the like.

  Therefore, depending on the source (supplier) of the image to be displayed, there is a possibility that the image quality may be deteriorated due to the blur correction process at the time of reproduction.

  An object of the present invention is to solve such problems and to appropriately correct blurring of moving images supplied from a plurality of sources to obtain an image with little deterioration.

  An image processing apparatus according to the present invention includes a shake correction unit that corrects a blur of an image indicated by an input moving image signal, an output unit that outputs a moving image signal output from the shake correction unit to a display device, and the input Based on the added information of the moving image signal, the determining means for determining the supply source of the input moving image signal and the correction processing by the shake correcting means based on the supply source determined by the determining means Control means for controlling whether or not to execute.

  According to the present invention, it is possible to appropriately correct blurring of an input moving image signal.

  Embodiments of the present invention will be described below.

  FIG. 1 is a diagram showing a display device to which the present invention is applied and its peripheral devices.

  In FIG. 1, reference numeral 100 denotes a display device to which the present invention is applied. As will be described later, the display device 100 is a television receiver that receives and displays a television broadcast wave. Reference numeral 101 denotes a video camera. The video camera 101 has a well-known shooting, recording, and playback function. The video camera 101 has a function of recording a shot moving image on a recording medium and playing back the recorded moving image and outputting it to the outside.

  Reference numeral 102 denotes a video recorder. Similar to the display device 100, the video recorder 102 has a known television broadcast receiving function, a moving image of a broadcast program, and a recording / reproducing function of an audio signal. The video recorder 102 records the received television broadcast program image and sound data on a recording medium such as a hard disk, DVD, or memory card, and outputs the recorded image and sound data to the outside.

  In FIG. 1, moving images output from the video camera 101 and the video recorder 102 can be output to the display device 100 and displayed.

  Next, the display device 100 will be described.

  FIG. 2 is a block diagram illustrating a configuration of the display device 100.

  In FIG. 2, reference numeral 201 denotes a tuner that receives a digital television broadcast and outputs a moving image signal and various additional information related to the broadcast program. Reference numeral 202 denotes a screen blur correction unit that corrects blurring of a moving image to be displayed, 203 denotes a display control unit that controls display of a moving image, a menu screen, and the like on the display unit 204, and 204 denotes a display unit including a liquid crystal panel.

  Reference numeral 205 denotes an analog interface (I / F) that receives an analog image signal output from an external device, converts the analog image signal into a digital signal, and outputs the digital signal. Reference numeral 206 denotes a digital I / F for inputting a digital image signal output from an external device and various kinds of additional information. In this embodiment, an HDMI terminal is used as the digital I / F 206. A signal processing unit 207 processes a moving image signal and converts it into a form suitable for display on the display unit 204, and a control unit 208 controls each unit. Reference numeral 209 denotes a user I / F including a remote controller, its receiver, various operation buttons, and the like. 210 is a data bus.

  The display device 100 selects an image related to any one of the broadcast program received by the tuner 201, the analog I / F 205, and the moving image signal input by the digital I / F 206 according to a user instruction, and displays the selected image on the display unit 204.

  The user operates the user I / F 209 to select which of the broadcast program received by the tuner 201, the analog I / F 205, and the image input by the digital I / F 206 is to be displayed.

  First, processing when displaying an image of a broadcast program received by the tuner 201 will be described.

  In response to an instruction from the control unit 208, the tuner 201 receives a broadcast program of a channel designated by the user, outputs a moving image signal of the received broadcast program to the signal processing unit 207, and transmits additional information and the like to the control unit 208. send.

  The signal processing unit 207 decodes the input moving image signal. Then, in response to an instruction from the control unit 208, the decoded video signal of the broadcast program is output to the screen blur correction unit 202. When displaying an image of a broadcast program from the tuner 201, the control unit 203 controls the screen blur correction unit 202 so as to output image data to the display control unit 203 without executing screen blur correction processing.

The display control unit 203 displays an image related to the input moving image signal of the broadcast program on the display unit 204.
In this way, an image relating to the broadcast program of the channel selected by the user is displayed on the display unit 204.

  Next, a process for displaying a moving image input from the analog I / F 205 will be described.

  The user operates the user I / F 209 to instruct switching of the display from the analog I / F 205 to a moving image.

  On the other hand, the analog I / F 205 converts the analog image signal output from the external device into a digital signal and outputs the digital signal to the signal processing unit 207. The analog I / F 205 detects a copy prevention protection signal multiplexed with respect to the input analog image signal. If the protection signal indicates copy prohibition, the control unit 208 is informed of that fact.

  The signal processing unit 207 performs appropriate processing on the image signal from the analog I / F 205 according to an instruction from the control unit 208 and outputs the image signal to the screen blur correction unit 202. When displaying an image from the analog I / F 205, the control unit 203 controls the screen blur correction unit 202 as described later. The display control unit 203 displays an image related to the input analog image signal on the display unit 204.

  In this manner, an image related to the analog image signal input from the analog I / F 205 is displayed on the display unit 204.

  Next, processing for displaying a moving image input from the digital I / F 206 will be described.

  The user operates the user I / F 209 to instruct switching of the display from the digital I / F 206 to a moving image.

  On the other hand, the digital I / F 206 outputs the digital image signal output from the external device to the signal processing unit 207. The digital I / F 206 detects a copy protection protection signal multiplexed with respect to the input digital image signal. If the protection signal indicates copy prohibition, the control unit 208 is informed of that fact.

  Also, the digital I / F 206 detects a connection destination device and outputs the detection result to the control unit 208. Specifically, the digital I / F 206 communicates with the connected device and detects the type of the connected device.

  In addition to this, it is also possible to detect a connected device by detecting device information multiplexed on the output moving image signal.

  The signal processing unit 207 performs an appropriate process on the image signal from the digital I / F 206 according to an instruction from the control unit 208 and outputs the image signal to the screen blur correction unit 202. When displaying an image from the digital I / F 206, the control unit 203 controls the screen blur correction unit 202 as described later. The display control unit 203 displays an image related to the input digital image signal on the display unit 204.

  In this manner, an image related to the digital image signal input from the digital I / F 206 is displayed on the display unit 204.

  Next, the screen blur correction unit 202 will be described.

FIG. 3 is a block diagram illustrating a configuration of the screen blur correction unit 202.
In FIG. 3, the input moving image signal s300 is output to the screen blur detection unit 301, the memory 302, and the selection unit 305.

  The screen blur detection unit 301 has a memory for storing the input moving image signal, and detects the blur amount and the blur direction of the image between two consecutive frames using the input moving image signal. Then, screen blur information indicating the detected blur amount and blur direction is output to the clipping unit 303 for each frame.

  The memory 302 can store an image signal of at least two frames, and stores the input image signal s300 in an area other than the storage area of the image signal being read by the clipping unit 303.

  The cutout unit 303 selects and reads out an image signal of a partial area of the image signal of one frame stored in the memory 302 and outputs the image signal to the enlargement processing unit 304. At this time, the cutout unit 303 changes the read area so as to eliminate screen blur based on the screen blur information detected by the screen blur detection unit 301. In the present embodiment, the area for reading the image signal from the memory 302 is changed depending on the amount of screen blur, but the size is constant.

  The enlargement processing unit 304 enlarges the size of the image signal of each frame selected and read by the cutout unit 303 so as to be the same size as the input image signal s300, and supplies the corrected image signal s304 to the selection unit 305. Output. Note that the enlargement processing by the enlargement processing unit 304 and the interpolation processing at the time of enlargement use known methods.

  The selection unit 305 selects and outputs one of the input image signal s300 and the corrected image signal s304 from the enlargement processing unit 304 based on the shake correction control signal from the control unit 208. Specifically, when a control signal indicating blur correction execution is received from the control unit 208, the corrected image signal s304 is selected. When a control signal indicating blur correction prohibition is received, the control signal is input. The image signal s300 is selected and output.

In this way, when the blur correction is not executed, the input image signal is output as it is, so that there is no image degradation due to the enlargement process during the blur correction.
Here, the state of screen blur correction will be described with reference to FIG.

In FIG. 4A, 401 indicates the state of one frame of image signal stored in the memory 302. Then, out of such an image signal 401, an image signal in a range indicated by a cutout frame 402 of a predetermined size is read out by cutout 303. Further, 403 is an image enlarged by the enlargement processing unit 303 to the same size as the input image signal.
FIG. 4B shows the state of the image signal 404 of the next frame after the frame 401.

  In the frame 404, the solid line is the image of the frame 404, and the dotted line is the image of the previous frame 401. As shown in the figure, the current frame 404 has a screen blur 406 in the downward direction compared to the previous frame 401.

  The screen blur detection unit 301 calculates the screen blur amount and direction.

  As a specific calculation method, for example, the current frame is shifted by a predetermined amount of pixels to obtain a correlation with the previous frame, and when the correlation is the largest value, the pixel shift amount is set as the amount of screen blur.

  In the case of a moving image, since the movement of the subject affects the correlation of the entire screen, it is possible to calculate a highly reliable screen blur amount by calculating without the influence of the moving subject. In addition to Patent Document 1, various known techniques can be used for such a screen blur amount calculation method.

An image 407 is obtained by setting the same position 405 as that of the previous frame as a clipping frame without considering the amount of screen blur generated in this manner, reading out an image signal by the clipping unit 303, and enlarging the image by the enlargement processing unit 304. In 407, the screen blur amount 406 is also enlarged to a blur amount 408.
That is, the screen blur amount is enlarged as it is, and the screen blur amount cannot be corrected.

  Accordingly, a description will be given of the setting process of the cutout frame in consideration of the generated screen blur amount.

  In this case, as shown in FIG. 4C, the cutout frame is moved by the same direction and the same amount as the screen blur 406 with respect to the image signal of the frame 404 in which the screen blur 406 is generated. That is, the clipping frame 409 is moved by 410 compared to the clipping frame 405. Then, when the image signal of the area of the cutout frame 409 is read from the memory 302 and enlarged by the enlargement processing unit 304, a moving image in which the amount of screen blur of the previous frame is corrected is generated as indicated by 411. Through the above operation, the amount of screen blur is corrected.

  Next, blur correction control according to the selection result of the image signal to be displayed will be described with reference to the flowchart of FIG. The processing in FIG. 5 is executed by the control unit 208.

  When the user I / F 209 issues a power-on instruction and a switching instruction of a moving image signal to be displayed, the flow in FIG. 5 starts.

  First, it is determined whether or not display of a broadcast program from the tuner 201 is selected (S501). When display of a broadcast program from the tuner 201 is selected, a control signal indicating prohibition of screen blur correction is output to the screen blur correction unit 202 (S502).

  If the display of the broadcast program from the tuner 201 is not selected, it is determined whether or not the digital I / F 206 is selected and the image signal output from the video camera 102 is input (S503).

  If the input image signal is output from the video camera 102, a control signal indicating execution of blur correction is output to the screen blur correction unit 202 (S505).

  If the analog I / F 205 is selected or the image signal from the digital I / F 206 is not output from the video camera 102, a protection signal for copy prevention is confirmed (S504).

  If the protection signal indicates copy prohibition, a control signal indicating prohibition of blur correction is output to the screen blur correction unit 202 (S502). If the protection signal does not indicate copy prohibition, a control signal indicating execution of blur correction is output to the screen blur correction unit 202 (S505).

  Since an image of a broadcast program is broadcast after editing work by a broadcast station, the frequency of screen blurring is low. Therefore, in the present embodiment, when an image of a broadcast program is displayed, screen blur correction is prohibited and image quality deterioration due to the screen blur correction operation is avoided. Therefore, it is possible to watch while maintaining the image quality of the broadcast wave.

  On the other hand, in the case of a moving image taken by a video camera, screen blur often occurs. Therefore, by executing the screen blur correction process, it is possible to correct the screen blur and display a good image.

  Furthermore, since the moving image prohibited from copying is often an edited moving image, as represented by content such as a movie recorded on a DVD, the frequency of screen blurring is low. Therefore, when the copy is prohibited, the screen blur correction is prohibited. As a result, it is possible to avoid image quality deterioration in the screen blur correction operation and to watch while maintaining the image quality of the broadcast wave.

  Finally, since an image source cannot be specified for an externally input moving image that is not copy-prohibited, screen blur correction operation is executed with emphasis on eliminating discomfort caused by screen blur.

  As described above, according to the present embodiment, whether to perform the image blur correction is automatically switched depending on the source of the selected moving image signal, that is, the supply source of the moving image signal.

  For this reason, it is possible to display a less deteriorated image by appropriately correcting the blurring of the moving image without the user's operation.

  In the present embodiment, the size of the cutout frame is a fixed size. However, the user can arbitrarily change the size. For example, the smaller the size of the cutout frame compared to the size of the input image, the larger the screen blur can be corrected. However, it is conceivable that the degradation during enlargement increases accordingly.

  In this embodiment, the screen blur is detected by using the moving image signal input by the screen blur correction unit 202. In addition to this, for example, when screen blur information is added to the input moving image signal, the screen blur information can be used.

  Further, in the present embodiment, when a moving image signal from a connected external device is selected and displayed, it is determined whether or not to perform blur correction according to the type of the connected device.

  In addition to this, for example, when displaying an image of a broadcast program from a tuner, blur correction is prohibited, and other than that, for example, when displaying an image from an external device, the blur correction is uniformly performed. Also good.

  Next, a second embodiment will be described.

  In the first embodiment, in a display device that displays a moving image signal, whether or not to perform shake correction is controlled by the moving image signal supply source. In this case, the image signal supply source is a moving image signal output circuit or an output device such as a tuner or an external device.

  On the other hand, in the second embodiment, a case will be described in which the present invention is applied to a playback device that plays back a moving image signal from a recording medium and outputs the same to a display device. In this embodiment, it is determined whether or not to perform blur correction depending on the source of the moving image signal to be reproduced. In the present embodiment, the source of a moving image signal is a device that has acquired a moving image signal recorded on a recording medium.

  FIG. 6 is a diagram showing a configuration of an image reproduction device 600 to which the present invention is applied.

  6 has a function of playing back a moving image signal recorded on a recording medium 601 and outputting it to an external display device.

  In FIG. 6, reference numeral 601 denotes a recording medium in which moving images obtained by various image sources such as moving images taken by a video camera and moving images of a television broadcast program are recorded. As the recording medium 601, a memory card, an optical disk, a hard disk, or the like can be used.

  Reference numeral 602 denotes a reproduction unit that reproduces a moving image signal from the recording medium 601, and reference numeral 603 denotes a screen blur correction unit 603 that corrects a screen blur of the reproduced moving image signal. Reference numeral 604 denotes a display control unit that displays a reproduced moving image and various types of information such as menus on the display unit 605, and reference numeral 605 denotes a display unit including a liquid crystal panel. Reference numeral 606 denotes an output unit that outputs the reproduced moving image signal to an external display device.

  Reference numeral 607 denotes an image source detection unit that detects an image source based on additional information of the reproduced moving image signal, and reference numeral 608 denotes a protection signal detection that detects a copy prevention protection signal added to the reproduced moving image signal. Part. A signal processing unit 609 performs necessary processing on the reproduced moving image signal and converts it into a form suitable for output. Reference numeral 610 denotes a control unit for controlling each unit, and 611 denotes a user I / F including a remote controller, its receiver, various operation buttons, and the like. Reference numeral 612 denotes a data bus.

  The user operates the user I / F 209 to select a desired moving image from a large number of moving images recorded on the recording medium 601 and instruct reproduction. When there is a reproduction instruction, the control unit 610 controls the reproduction unit 602 to reproduce the designated moving image signal from the recording medium 601. The reproduced moving image signal is processed by the signal processing unit 609 and output to the screen blur correction unit 603.

  Further, the image source detection unit 607 detects the image source of the reproduced image signal based on the additional information added to the reproduced moving image signal and notifies the control unit 610 of the detected image source. When the additional information indicating the image source is not added, the control unit 610 is notified of that fact.

  In addition, the protection signal detection unit 608 determines whether the reproduced moving image signal is a moving image whose copy is prohibited or copy is restricted based on the protection signal added to the reproduced moving image signal. Determine. Then, the determination result is notified to the control unit 610.

  The screen blur correction unit 603 has a configuration similar to that of the screen blur correction unit 202 in FIG. 3, and corrects and outputs screen blur of a moving image signal reproduced based on the blur correction control signal from the control unit 610. The moving image signal output from the screen blur correction unit 603 is output to the display control unit 604 and the output unit 606.

  The display control unit 604 changes the size of the moving image signal from the screen blur correction unit 603 according to the size of the display unit 605 and outputs the change to the display unit 605. In the present embodiment, the display unit 605 has a smaller number of display pixels than the reproduced moving image signal, and the display control unit 604 displays the reproduced image signal with a reduced size.

  Further, the output unit 606 outputs the moving image signal from the screen blur correction unit 603 to an external display device in accordance with an instruction from the control unit 610. At this time, if copy restriction is applied to the reproduced moving image signal, a protection signal indicating copy restriction is added and output. The output unit 606 can output a moving image signal in either an analog signal format or a digital signal format.

  The user can switch whether or not to output the reproduced moving image signal by the output unit 606 by operating the user I / F 209.

  Next, blur correction control in the present embodiment will be described with reference to the flowchart of FIG. The processing in FIG. 7 is executed by the control unit 610.

  When the user I / F 611 issues a power-on instruction and a playback moving image signal output destination switching instruction, the flow in FIG. 7 starts.

  First, it is determined whether or not the output of the playback video signal by the output unit 606 is set (S701). When the output of the playback video signal to the outside is not set, a control signal indicating that the blur correction is prohibited is output to the screen blur correction unit 603 (S705).

  If the output of the reproduced moving image signal to the outside is set, it is determined whether the image source of the reproduced moving image signal is a video camera based on the detection result of the image source detecting unit 607. (S702). When the image source is a video camera, a control signal indicating execution of blur correction is output to the screen blur correction unit 603 (S703).

  Further, when the image source is not a video camera or when additional information is not detected, it is determined based on the detection result of the protection signal detection unit 608 whether or not copying of the reproduction image signal is restricted. (S704). If copying is restricted, a control signal indicating prohibition of shake correction is output to the screen shake correction unit 603 (S705). If the protection signal does not indicate copy restriction, a control signal indicating execution of blur correction is output to the screen blur correction unit 603 (S703).

  As described above, in the present embodiment, when a moving image signal is reproduced from a recording medium and output to an external device, whether to perform blur correction processing is switched depending on the image source of the reproduced moving image signal. ing.

  For this reason, it is possible to appropriately correct blurring of a moving image and output an image with little deterioration without a user operation.

  Specifically, in the case of a moving image taken by a video camera, screen blur often occurs. Therefore, by executing the screen blur correction process, it is possible to correct the screen blur and display a good image.

  On the other hand, since an image of a broadcast program is broadcast through editing work by a broadcasting station, the frequency of screen blurring is low. Also, most digital television broadcast programs have a limit on the number of copies.

  Therefore, when the reproduced moving image signal is subject to copy restrictions, screen blur correction is prohibited. As a result, it is possible to avoid image quality deterioration in the screen blur correction operation and to watch while maintaining the image quality of the broadcast wave.

  For moving images that are not copy-restricted, the source of the image cannot be specified. Therefore, emphasis is placed on eliminating discomfort due to screen blur, and screen blur correction operation is executed.

  Further, in the present embodiment, when the reproduced image signal is not output to the external device and the reproduced image is displayed only on the display unit 605, screen blur correction is prohibited. This is because the resolution of the display unit 605 is smaller than that of the playback moving image, and it is considered that deterioration due to screen blur is not noticeable.

  In the present embodiment, a case where the present invention is applied to an image reproduction apparatus has been described. In addition to this, the present invention can be similarly applied to any apparatus having a moving image reproduction function, such as an imaging apparatus such as a video camera or a digital camera, or a portable terminal such as a mobile phone.

It is a figure which shows the display apparatus and external apparatus in embodiment of this invention. It is a figure which shows the structure of the display apparatus in embodiment of this invention. It is a figure which shows the structure of a screen shake correction | amendment part. It is a figure which shows the mode of the blur correction by a screen blur correction part. It is a flowchart which shows the blurring correction process of the display apparatus in embodiment of this invention. It is a figure which shows the structure of the reproducing | regenerating apparatus in embodiment of this invention. It is a flowchart which shows the blurring correction process of the reproducing | regenerating apparatus in embodiment of this invention.

Claims (6)

Blur correction means for correcting blur of an image indicated by the input moving image signal;
Output means for outputting a moving image signal output from the shake correction means to a display device;
Discrimination means for discriminating a supply source of the input moving image signal based on the additional information of the input moving image signal;
An image processing apparatus comprising: control means for controlling whether or not to perform correction processing by the shake correction means based on the supply source determined by the determination means . The image processing apparatus according to claim 1, wherein the control unit causes the correction process to be executed when the supply source determined by the determination unit is a video camera . Input means for inputting a moving image signal output from an external device;
A blur correction unit that corrects a blur of an image indicated by the moving image signal input by the input unit;
Output means for outputting a moving image signal output from the shake correction means to a display device;
Image processing comprising: a control unit that detects the type of the external device by communicating with the external device, and controls whether to perform correction processing by the shake correction unit according to the detected type apparatus. The image processing apparatus according to claim 3, wherein the control unit causes the correction process to be executed when the external device is a video camera. A blur correction step for correcting blur of an image indicated by the input moving image signal;
An output step of outputting the moving image signal output from the blur correction step to a display device;
A discriminating step for discriminating a supply source of the inputted moving image signal based on the additional information of the inputted moving image signal;
An image processing method comprising: control means for controlling whether or not to perform the correction process in the blur correction step based on the supply source determined in the determination step . An input step for inputting a moving image signal output from an external device;
A blur correction step of correcting blur of an image indicated by the moving image signal input in the input step;
An output step of outputting the moving image signal output from the blur correction step to a display device;
Image processing comprising: detecting a type of the external device by communicating with the external device, and controlling whether or not to perform correction processing by the blur correction step according to the detected type Method.

Images