JP5203115B2 - Moving picture reproducing apparatus and moving picture reproducing method - Google Patents

Description

  The present invention relates to a moving image reproduction apparatus and a moving image reproduction method that perform correction according to a user's preference on a moving image based on a feature of correction of a captured image performed by the user.

A technology for providing a moving image by dynamically correcting a moving image (video) received by a television receiver by a digital broadcasting technology represented by terrestrial digital broadcasting (terrestrial digital broadcasting; ISDB-T), which has recently become popular. It became possible to realize. As such a conventional technique, a technique for correcting the image quality of a moving image to the user's preference (for example, see Patent Document 1) has been proposed.
On the other hand, a digital camera that acquires a subject as digital data can correct the image quality of the captured image to the user's preference. Some electronic devices such as mobile phones in recent years have various functions, have functions of receiving digital terrestrial broadcasts and playing back moving images, and functions of digital cameras.
JP 2004-7215 A

  In the image correction, although the user's preference for the image quality does not change between the moving image and the still image, the electronic device has to perform the correction of the moving image and the still image separately.

  The present invention provides a technique (moving image reproducing apparatus and moving image reproducing method) capable of performing correction according to a user's preference on a moving image based on a feature of correction of a captured image performed by the user. The purpose is to do.

  In order to achieve the above object, a moving image reproducing apparatus according to claim 1 of the present invention corrects a captured image captured by a display unit that displays a moving image, an imaging unit that captures a subject, and the imaging unit. And a control unit that controls to correct the moving image displayed on the display unit based on the correction information indicating the correction feature at the time.

  A moving image reproduction apparatus according to a second aspect of the present invention includes a storage unit that stores a plurality of correction information of corrections performed by the control unit, and the control unit stores a plurality of correction information stored in the storage unit. Based on the above, control is performed so as to correct the moving image displayed on the display unit.

  A moving image reproduction apparatus according to a third aspect of the present invention includes a storage unit that stores a captured image captured by the imaging unit and correction information for correction of the captured image, and the control unit includes the display unit First image information representing image characteristics for each frame of the moving image displayed on the second image information and second image information representing image characteristics of the captured image stored in the storage unit are acquired, and the first image information is acquired. When the image information and the second image information substantially match, an image of a moving image frame corresponding to the first image information is stored in the storage unit corresponding to the second image information. Control is performed so as to perform correction based on correction information for correction of a captured image.

  In the moving image reproducing apparatus according to claim 4 of the present invention, the control unit corrects the moving image by changing a parameter set for correcting the moving image displayed on the display unit. And when correcting the captured image captured by the imaging unit, the correction of the moving image by changing the parameters set for correcting the moving image, Control is performed so that the correction of the moving image based on the correction information of the correction of the captured image is given priority.

  To achieve the above object, a moving image reproduction method according to claim 5 of the present invention corrects a captured image captured by the imaging step, a display step for displaying a moving image, an imaging step for capturing an image of a subject. And a control step for performing control so as to correct the moving image displayed in the display step based on correction information representing the correction feature at that time.

  According to the present invention, it is possible to provide a moving image reproduction apparatus and a moving image reproduction method capable of performing correction according to the user's preference on a moving image based on a feature of correction of a captured image performed by the user. Therefore, the user does not need to perform a complicated operation only for correcting the moving image, and can view a favorite moving image without being aware of the necessity of correction.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

[Configuration of video playback device]
FIG. 1 is a diagram illustrating the configuration of a moving image playback apparatus according to the present invention. The moving image reproduction apparatus 100 of the present invention includes, for example, a mobile phone terminal in which an imaging unit (camera), a captured image correction apparatus, and a moving image content (digital broadcast content) reproduction apparatus are provided integrally. And includes an antenna 110, a tuner unit 120, a control unit 130, a display unit (display) 140, an imaging unit (camera) 150, a storage unit 160, an operation unit 170, and the like. The control unit 130 includes a demodulation unit 180, a demultiplexing unit 190, a moving image decoding unit 200, a moving image correction unit 210, an image information acquisition unit 220, and a captured image correction unit 230.

The antenna 110 receives broadcast waves and transmits them to the tuner unit.
The tuner unit 120 converts the broadcast wave into a digital signal and transmits the digital signal to the demodulation unit 180.

  The control unit 130 (1) performs control so as to correct the moving image displayed on the display unit 140 based on the correction information representing the correction feature when the captured image captured by the imaging unit 150 is corrected. (2) Based on the plurality of correction information stored in the storage unit 160, control is performed so as to correct the moving image displayed on the display unit 140, and (3) the frame of the moving image displayed on the display unit 140. First image information representing the feature of each image and second image information representing the feature of the image of the captured image stored in the storage unit 160 are obtained, and the first image information and the second image information are obtained. When the image information substantially matches, the image of the moving image frame corresponding to the first image information is used as correction information for correcting the captured image stored in the storage unit 160 corresponding to the second image information. Control to correct based on ( ) Control is performed so as to correct the moving image by changing a parameter set for correcting the moving image displayed on the display unit 140, and the captured image captured by the imaging unit 150 is controlled. When the correction is performed, the correction of the moving image based on the correction information of the correction of the captured image has priority over the correction of the moving image by changing the parameters set for correcting the moving image. Various arithmetic processes such as control are performed.

The display unit 140 outputs (displays) the moving image and characters corrected by the control unit 130.
The imaging unit (camera) 150 captures a captured image (still image) of the subject.
The storage unit 160 temporarily stores data necessary for various types of processing, stores a plurality of pieces of correction information for correction performed by the control unit 130, and also takes a captured image (second image information) captured by the imaging unit 150. ), The image (first image information) for each frame of the moving image displayed on the display unit 140 is stored.
The operation unit 170 inputs various instructions according to user operations to the control unit 130, the display unit 140, and the imaging unit 150.

The demodulator 180 demodulates the digital signal into a packet and transmits the packet to the demultiplexer 190.
The demultiplexing unit 190 refers to the ID included in the packet and separates the multiplexed packet for each type such as a voice packet or a moving image packet.
The moving image decoding unit 200 restores a moving image from the moving image packet.
The moving image correction unit 210 automatically corrects the moving image decoded by the moving image decoding unit 200 based on the correction information stored in the storage unit 160.
The image information acquisition unit 220 includes first image information representing image characteristics for each frame of the moving image displayed on the display unit 140 and second image information representing the image characteristics of the captured image captured by the imaging unit 150. Get image information.
The captured image correction unit 230 performs correction by the user on the captured image captured (acquired) by the imaging unit 150, and acquires correction information of the correction.

  The configuration shown in FIG. 1 is assumed to be a one-seg broadcasting receiver that is transmitted by the ISDB-T system. However, even if the receiver is a MediaFLO or DVB-H receiver, the configuration shown in FIG. It is possible to satisfy the mounting requirement of the moving image reproducing apparatus in the invention. Moreover, when there exists an alternative means which can implement | achieve each function, you may use it and you may make it provide each function outside a terminal. In addition, the moving image playback apparatus and the moving image playback method of the present invention are not limited to broadcast content, and can be applied to a moving image service or the like that is distributed by IP streaming. Furthermore, since the moving image playback apparatus and the moving image playback method of the present invention relate to moving images, components for playback other than moving images, such as audio and subtitles, are omitted. It is also assumed that components necessary for playback of subtitles and subtitles are implemented.

  Next, correction of a captured image (hereinafter also simply referred to as an image) in the moving image reproduction apparatus and the moving image reproduction method of the present invention will be described. Corrections that can be used in the moving image reproduction apparatus and moving image reproduction method of the present invention include “sharpening”, “saturation correction”, “contrast correction”, and “overwriting a new image”.

ここで、ｋは、０より大きな値（例えば０〜１の間の値）であり、ｋの値が大きいほど鮮鋭化効果は大きくなる。 [Sharpness]
Sharpening is a process that emphasizes the outline of an image, and an image that has undergone the sharpening process has a clearer impression. The sharpening is generally performed by applying a sharpening filter represented by Expression (1) to an image.

Here, k is a value larger than 0 (for example, a value between 0 and 1), and the sharpening effect increases as the value of k increases.

ただし、ＭＡＸ、ＭＩＮはそれぞれＲＧＢ成分の最大値および最小値である。 [Saturation correction]
Saturation correction is a process of adjusting the vividness of an image, and corrects the image to a more vivid impression or corrects the image to a more calm impression. Here, the saturation is a component representing the intensity of the vividness of the pixels included in the image. The image obtained by the imaging unit 150 and the moving image decoding unit 200 is composed of three components of red (R), green (G), and blue (B). ) Saturation (S) Value of the saturation component when converted to lightness (V) format. The conversion from the RGB format to the HSV format is performed by the equation (2), and the inverse conversion is performed by the equation (3).

However, MAX and MIN are the maximum and minimum values of the RGB components, respectively.

式（４）において、出力画像Ｏは、γ＞１のとき入力画像より鮮やかになり、γ＜１のとき入力画像より落ち着いた印象の画像になる。 Here, it is possible to change only the saturation of the image by converting the RGB format image into the HSV format image once, changing the saturation component, and then converting back to the RGB format.
For example, when the saturation at the coordinates (x, y) of the input image I is S (I (x, y)), the output image O is expressed by Expression (4).

In Expression (4), the output image O becomes brighter than the input image when γ> 1, and becomes a calmer image than the input image when γ <1.

ここで、ＲＧＢ形式の画像を一旦ＹＣｒＣｂ形式の画像に変換し、Ｙの値を変更してからＲＧＢ形式へ逆変換することにより、画像の輝度のみを変更することができる。 [Contrast correction]
Contrast correction is a process for changing the contrast of an image. Generally, when the contrast is increased, a clear image is obtained, and when the contrast is reduced, an image having a calm impression is obtained. The brightness level of the pixel is used to change the contrast.
The luminance level is a value of a luminance component when RGB format data is converted into a luminance (Y) color difference (Cr, Cb) format. Conversion from the RGB format to the YCrCb format is performed according to Equation (5), and inverse conversion is performed according to Equation (6).

Here, it is possible to change only the brightness of the image by converting the RGB format image into the YCrCb format image once, changing the value of Y, and then converting back to the RGB format.

ここで、
Ａ＝ａ２−ａ１
Ｂ＝ｂ２−ｂ１ ・・・（８）
とすると、（Ｂ／Ａ）＞１のとき出力画像はよりコントラストが強い印象の画像になり、（Ｂ／Ａ）＜１のとき出力画像はよりコントラストが弱い印象の画像になる。 FIG. 2 is a diagram illustrating a histogram of the input image I before contrast correction, and FIG. 3 is a diagram illustrating a histogram of the output image O after contrast correction. In the histogram of FIG. 2, a1 and a2 indicate a lower limit and an upper limit at which the input image frequency is 1 or more. In order to increase the contrast of the input image I, b1 and b2 satisfying b2> b1 are determined in the histogram of FIG. 3, and the coordinates (x, y) of the output image O are determined by the expression (7) of the contrast conversion function. A luminance level Y (O (x, y)) can be calculated.

here,
A = a2-a1
B = b2-b1 (8)
Then, when (B / A)> 1, the output image becomes an image with a higher contrast, and when (B / A) <1, the output image becomes an image with an impression with a lower contrast.

[Overwrite new image]
One example of a correction method for overwriting a captured image with a new image is “stamping”.
Stamp pasting is a correction method for overwriting an image of a predetermined pattern on an input image. The shape and color of the pattern can be freely selected by the user. The user can freely determine the place where the stamp is pasted and the number of times the stamp is pasted.

When performing the four types of correction described above, the user can perform desired corrections on the image by operating the user interface as shown in FIGS. For example, when performing “brightness correction” of FIG. 4A corresponding to saturation correction, the image is corrected more vividly every time the left direction key on the operation unit 170 is pressed, and the right direction key is pressed. Each time the image is corrected, the image is corrected more conservatively. Here, when the user determines that the correction for the image is appropriate and ends the correction, the correction information indicating the correction feature and the image information (second image information) indicating the image feature of the captured image are processed. It is stored in the storage unit 160 as information. In FIGS. 4A to 4D, entry of information irrelevant to the image processing is omitted from the display information on the display unit 140 of the moving image reproducing apparatus.
FIG. 5 is a diagram illustrating one record of the processing content information stored in the storage unit 160. The information stored in the record is not limited to the illustrated example, and other information may be added as necessary. The image information stored in the storage unit 160 may be image information of the entire image or image information for each divided part of the image.

[First Embodiment]
FIG. 6 is a flowchart showing the moving image correction processing of the first embodiment performed in the moving image reproducing apparatus of the present invention. In the present embodiment, a case where the sharpening process is performed as a correction for each frame of a moving image decoded by the moving image decoding unit 200 will be described as an example. In the following description, a frame to be corrected is referred to as a “current frame”. In addition, when a correction value of correction information is set by a user or the like to correct a moving image, the correction value is referred to as a “specified value”. In the present embodiment, when the correction information is searched from the storage unit 160, the correction information of the correction performed by the captured image correction unit 230 is searched.

  First, in step S100 in FIG. 6, the image information of the current frame is searched. In step S101, the moving image correction unit 210 searches the storage unit 160 for correction information for sharpening processing. In the next step S102, it is determined whether or not there is one or more sharpening correction information. If there is no correction information, the process proceeds to step S103. If there is one or more, the process proceeds to step S104. In step S103, the image information of the current frame is corrected (image sharpening process) using the specified value. On the other hand, in step S104, an average value of the correction values of the sharpening process acquired from the corresponding correction information is calculated. For example, when the image is corrected by equation (1), the average value of k in equation (1) is calculated. In the next step S105, the moving image correction unit 210 performs correction (image sharpening processing) by applying the formula (1) to the image information of the current frame using the calculated average value of k. . As a result, sharpening correction can be performed on the current frame with the strength preferred by the user.

[Second Embodiment]
FIG. 7 is a flowchart showing a moving image correction process according to the second embodiment performed in the moving image reproducing apparatus of the present invention. In the present embodiment, a case where saturation correction is performed on the current frame will be described as an example.

  First, in step S110 of FIG. 7, the moving image correction unit 210 acquires the main value of the hue that is the image information (first image information) of the current frame from the storage unit 160. The main value of the hue is a hue value that occupies a large weight in the image. For example, the hue value has the largest number of pixels having the same hue. In the next step S111, the moving image correcting unit 210 causes the main value of the hue of the image information (second image information) of the captured image from the storage unit 160 to substantially match or similar to the main value of the hue of the current frame. Search for. In the next step S112, saturation correction correction information is searched from the imaging information. In the next step S113, it is determined whether or not there is one or more correction information for saturation correction. If there is no correction information, the process proceeds to step S114. If there is one or more, the process proceeds to step S115. In step S114, the image information of the current frame is corrected (image saturation correction) using the specified value. On the other hand, in step S115, an average value of saturation correction values acquired from the corresponding correction information is calculated. For example, when saturation correction is performed using Expression (4), the average value of γ values in Expression (4) is calculated. In the next step S116, the moving image correction unit 210 uses the calculated average value of γ to apply correction (image saturation correction) to the image information of the current frame by applying Equation (4). . Thereby, it is possible to perform saturation correction with the intensity preferred by the user for the current frame.

  It should be noted that the comparison of the main values of hue in the selection of image information is, for example, that the user has a vividness depending on the color, for example, “Blue likes a bright color but red prefers a calm color” It takes into account that preferences are different. Here, as the image information to be compared, if necessary, the average luminance of the image may be included.

[Third Embodiment]
FIG. 8 is a flowchart showing the moving image correction processing of the third embodiment performed in the moving image reproducing apparatus of the present invention. In the present embodiment, a case where contrast correction is performed on the current frame will be described as an example.

  First, in step S120 of FIG. 8, image information of the current frame (in this case, a histogram of the current frame) is acquired. In step S121, the moving image correction unit 210 searches the storage unit 160 for correction information for contrast correction. In the next step S122, it is determined whether or not there is one or more contrast correction information. If there is no correction information, the process proceeds to step S123, and if there is one or more, the process proceeds to step S124. In step S123, the image information of the current frame is corrected (image contrast correction) using the specified value. On the other hand, in step S124, an average value of contrast correction values acquired from the corresponding correction information is calculated. For example, when the contrast correction of the image is performed by the expression (7) of the contrast conversion function, the average value of (B / A) in the expression (8) is calculated, and this value is expressed as M (B / A). To do.

  In the next step S125, the average value of the contrast correction values is superimposed on the specified contrast correction value. In the superimposition process, a new correction value is calculated using the average value of the contrast correction values and the specified contrast correction value. In the present embodiment, the average value of the contrast correction values and the specified contrast correction value are used as new correction values.

In the next step S126, Expression (7) is applied using the superimposed correction value to correct the current frame (image contrast correction). Specifically, first, the moving image correction unit 210 calculates values corresponding to a1 and a2 for the acquired histogram shown in FIG. 2, and then corresponds to b1 and b2 in the histogram illustrated in FIG. The value is calculated in association with the value of M (B / A). For example, when the association is performed as shown in the following formula (9), when M (B / A)> 1, b1 and b2 are values located outside a1 and a2, and M (B / A) <1 In this case, b1 and b2 are values located inside a1 and a2. In this way, the user's preference for contrast is reflected in the values of b1 and b2. Next, assuming that the parameter corresponding to b1 in the contrast correction performed on the frame immediately before the current frame by the moving image correcting unit 210 is b1 ′, | b1′−b1 with respect to the contrast variation limiting parameter p. B1 is corrected so that | ≦ p. Here, p is a parameter for suppressing the fluctuation of the correction amount from becoming too large in the previous frame and the current frame, and has an effect of preventing flickering in the moving image due to the fluctuation of the correction amount. can get. Also for b2, the parameter variation is limited by p in the same manner as described above. Next, the moving image correction unit 210 corrects the contrast of the image information of the current frame using Equation (7) using the calculated b1 and b2. As a result, contrast correction can be performed on the current frame with the intensity preferred by the user.

[Fourth Embodiment]
FIG. 9 is a flowchart showing the moving image correction process of the fourth embodiment performed in the moving image reproducing apparatus of the present invention. In the present embodiment, a case where saturation correction is performed on the current frame will be described as an example.

  First, in step S130 of FIG. 9, image information of the current frame is acquired. Next, in step S131, the moving image correcting unit 210 causes the stamp pasting correction information (in this case, the main value of the hue of the image information (second image information) of the captured image and the stamp pasting from the storage unit 160). Search for correction value information). In the next step S132, it is determined whether or not there is one or more correction information for stamp pasting correction. If there is no correction information, the process proceeds to step S133. If there is one or more, the process proceeds to step S134. In step S133, the image information of the current frame is corrected (image saturation correction) using the specified value. On the other hand, in step S134, a correction value is calculated by comparing the stamp pasting correction value acquired from the corresponding correction information with the main value of the hue of the captured image. In the next step S135, an average value of the calculated correction values is calculated.

  Hue relates to the types of colors such as red, blue, yellow, and green. In the hue table illustrated in FIG. 10, each color is assigned to an angle of 0 to 360 degrees. When the hues of the two colors are compared, those having similar angles in the hue table of FIG. 10 are the same color system, and the contrast between the colors is small, so that a conservative impression is obtained when the two colors are arranged. When the hues of two colors are compared, those with an angle of 180 degrees in the hue table in FIG. 10 are complementary colors, and the contrast between the colors is large, so a vivid impression is obtained when the two colors are arranged. receive. Therefore, the closer the difference value between the main hue value and the hue of the stamp color is to 0, the user tends to use a modest color usage, and the closer the difference value is to 180, the more the user tends to use a vivid color usage. is there. Therefore, the hue difference value and the γ value in the equation (4) are set to a smaller value as the difference value is closer to 0, and set to a larger value as the difference value is closer to 180, so that the user's preference is met. Saturation is obtained.

  In FIG. 9, in the next step S136, the moving image correction unit 210 applies correction using the average value of the calculated correction values (γ values) and applies the equation (4) (image saturation correction). I do. Thereby, it is possible to perform saturation correction with the intensity preferred by the user for the current frame.

  By combining the first to fourth embodiments described above, it is possible to obtain better effects.

If the user has not yet corrected the image by the time of correcting the moving image, correction information necessary for correcting the moving image cannot be obtained. In such a case, the correction is determined in advance. Values are used or no correction is made.
It is more effective to calculate the average value by adding some weight to the correction information. Examples of weighting include: “Increase the weight of the one with the new correction date / time”, “Increase the weight of the one with a large captured image size”, “Increase the weight of the one that took time for the correction work”, “External memory The weight of the image copied or transmitted by e-mail is increased, and the weight of the deleted image is decreased without use. "
In addition, correction processing other than sharpening correction, saturation correction, and contrast correction can be performed on a captured image (still image), and image information and correction information of the captured image can be defined. The present invention can be applied as long as possible correction processing.

It is a figure which illustrates the structure of the moving image reproducing device of this invention. It is a figure which illustrates the histogram of the input image before contrast correction in the moving image reproducing device of this invention. It is a figure which illustrates the histogram of the output image after contrast amendment in the moving picture reproducing device of the present invention. (A)-(d) is a figure which illustrates the user interface used for four types of moving image correction processes in the moving image reproducing apparatus of this invention, respectively. It is a figure which illustrates 1 record of the processing content information memorize | stored in the memory | storage part of the moving image reproducing device of this invention. It is a flowchart which shows the moving image correction process of 1st Embodiment implemented in the moving image reproducing device of this invention. It is a flowchart which shows the moving image correction process of 2nd Embodiment implemented in the moving image reproducing device of this invention. It is a flowchart which shows the moving image correction process of 3rd Embodiment implemented in the moving image reproducing device of this invention. It is a flowchart which shows the moving image correction process of 4th Embodiment implemented in the moving image reproducing device of this invention. It is a figure which illustrates the hue table used for the moving image correction process of 4th Embodiment implemented in the moving image reproducing device of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Moving image reproducing apparatus 110 Antenna 120 Tuner part 130 Control part 140 Display part (display)
150 Imaging unit (camera)
160 storage unit 165 correction information storage unit 170 operation unit 180 demodulation unit 190 demultiplexing unit 200 moving image decoding unit 210 moving image correction unit 220 image information acquisition unit 230 captured image correction unit

Claims (5)

A display unit for displaying moving images;
An imaging unit for imaging a subject;
A control unit that controls to correct a moving image displayed on the display unit based on correction information that represents the correction characteristics when the captured image captured by the imaging unit is corrected;
A moving image reproducing apparatus comprising: A storage unit that stores a plurality of correction information of corrections performed by the control unit;
The moving image reproduction according to claim 1, wherein the control unit performs control so as to correct a moving image displayed on the display unit based on a plurality of correction information stored in the storage unit. apparatus. A storage unit that stores a captured image captured by the imaging unit and correction information for correcting the captured image;
The control unit includes first image information representing image characteristics for each frame of a moving image displayed on the display unit, and second image information representing image characteristics of the captured image stored in the storage unit. If the first image information and the second image information substantially match, the image of the moving image frame corresponding to the first image information corresponds to the second image information. The moving image reproduction apparatus according to claim 1, wherein control is performed so as to perform correction based on correction information for correction of a captured image stored in the storage unit.   The control unit performs control so as to correct the moving image by changing a parameter set for correcting the moving image displayed on the display unit, and captures an image captured by the imaging unit. When the correction is performed on the image, the correction of the moving image based on the correction information of the correction of the captured image is performed rather than the correction of the moving image by changing the parameters set for correcting the moving image. The moving image reproducing apparatus according to claim 1, wherein control is performed so that the priority is given to the video. A display step for displaying a moving image;
An imaging step for imaging a subject;
A control step for controlling to correct the moving image displayed in the display step based on correction information representing the characteristics of the correction when the captured image captured in the imaging step is corrected;
A moving image reproduction method comprising:

Images