JP2021124546A - Display control device, control method, program, and storage medium - Google Patents

Abstract

To display a photographed image while preventing a decrease in visibility even when the frame rate of recording quickens during photographing of moving images using an anamorphic.SOLUTION: A display control device pertaining to the present invention comprises: acquisition means capable of acquiring first image data in which a subject image is formed with magnifications different in the horizontal and vertical directions; display control means capable of executing first resizing of generating, after performing a first process of cutting out a portion of first image data, a display image so that second image data subjected to the first process has a prescribed aspect ratio, and second resizing of performing resizing without performing the first process on the first image data; and control means for exercising control so as to perform the first resizing when outputting images to a first output destination that is of a first resolution, and perform the second resizing when outputting images to a second output destination that is of a second resolution smaller than the first resolution.SELECTED DRAWING: Figure 6

Description

The present invention relates to a technique for displaying an image taken by using an anamorphic lens.

The anamorphic lens is a lens for acquiring an image in a state of being compressed in the lateral direction at the time of shooting. An image taken with an anamorphic lens can be displayed in such a way that the image is stretched in the horizontal direction during post-processing, so that an image that looks close to the original subject can be obtained. Patent Document 1 discloses that a part of an image compressed in the horizontal direction at the time of photographing is cut out and scaling processing is performed at different magnifications in the vertical direction and the horizontal direction. Further, Patent Document 2 discloses that a user can set a frame rate to be recorded at the time of imaging and a frame rate to be reproduced as a method of changing the frame rate.

JP-A-2018-0375859 Japanese Unexamined Patent Publication No. 2016-116133

When performing scaling processing on a part of an image taken by using an anamorphic lens as in Patent Document 1, the time required for inputting the captured image and the time required for writing for display. May be different. For example, as in Patent Document 2, if the frame rate to be recorded at the time of imaging is set high by the user, writing for display may not be in time depending on the display size.

In view of the above problems, the present invention can display a captured image while preventing a decrease in visibility even if the frame rate of recording is increased during shooting of a moving image using an anamorphic. An object of the present invention is to provide a display control device.

One aspect of the present invention is an acquisition means capable of acquiring first image data in which a subject image is formed at different magnifications in the horizontal direction and the vertical direction, and a first method of cutting out a part of the first image data. After performing the above-mentioned processing, the first resizing to generate a display image so that the second image data subjected to the first processing has a predetermined aspect ratio, and the above-mentioned with respect to the first image data. When the image is output to the display control means capable of executing the second resizing that resizes without performing the first processing and the first output destination having the first resolution, the first When resizing and outputting an image to a second output destination having a second resolution smaller than the first resolution, the control means for controlling the second resizing is provided. It is a display control device characterized by.

According to the present invention, even if the frame rate of recording becomes high during shooting of a moving image using anamorphic, it is possible to display the shot image while preventing deterioration of visibility.

It is a block diagram of the image pickup apparatus which concerns on this embodiment. It is a figure which shows the flow of the image data which concerns on this embodiment. It is a figure which shows an example of the menu display which concerns on this embodiment. It is a figure which shows an example of the desk ease processing which concerns on this embodiment. It is a figure which shows the processing time of the resizing process which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on this Embodiment. It is a figure which shows an example of the resizing process from QHD to FHD which concerns on this embodiment.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

<Structure of Imaging Device 100>
FIG. 1A is a diagram showing an example of the configuration of an image pickup device 100 (video camera) as an example of the display control device of the present invention. The image pickup apparatus 100 has a lens unit 101 on the front surface, a panel 107 on the upper surface, a user operation unit 109 (hereinafter, operation unit 109) on the side surface, and an external output 115 on the back surface. The panel 107 is a display unit that is removable (detachable) with respect to the main body of the image pickup apparatus 100. The operation unit 109 is an operation member such as a seesaw-type grip zoom 109a (zoom key) and various switches (buttons) that receive various operations from the user. Various switches include a power switch for switching the power on / off, a menu button for displaying a menu screen, a mode switch for switching the operation mode, and the like. The mode changeover switch is a switch for switching the operation mode of the image pickup apparatus 100 to any one of "still image recording mode", "moving image shooting mode", and "reproduction mode". The external output 115 is a terminal for connecting a non-removable (non-detachable) display unit to the image pickup apparatus 100, and is, for example, an HDMI (registered trademark) terminal, an SDI terminal, or the like.

FIG. 1B is a diagram showing a block diagram of the image pickup apparatus 100 according to the present embodiment. The lens unit 101 includes a fixed lens group for condensing, a variable magnification lens group, an aperture and a correction lens group, and the like. The correction lens group has a function of correcting the image formation position moved by the movement of the variable magnification lens group and a function of performing focus adjustment. The lens unit 101 finally forms a subject image on the image plane of the image sensor 102 (sensor) described later. The lens unit 101 has an interchangeable lens that can be attached to the lens attachment portion. The interchangeable lens mounted on the lens unit 101 is not necessarily limited to the above-mentioned lens group. Further, the lens mounting portion is an anamorphic lens (compression rate is doubled) that optically reduces (compresses; squeezes) the subject image formed on the image sensor 102 in the horizontal direction (horizontal direction) by half. Can be installed. Further, an anamorphic lens (compression rate of 1.3 times) that optically reduces (squeezes) the subject image to 1 / 1.3 in the lateral direction can be attached to the lens mounting portion. That is, when an anamorphic lens is attached to the lens attachment portion, an image is formed in a state in which the subject image is optically compressed in the lateral direction.

When a perfect circular subject is photographed using an anamorphic lens, the subject is optically squeezed and compressed in the horizontal direction, so that the subject is imaged and imaged on the image sensor 102 as a vertically long elliptical subject. If this is displayed as it is, it will be displayed as a vertically long elliptical subject, but by performing deskease (stretching) processing in the horizontal direction (horizontal direction), the subject restored to a perfect circle can be displayed. In the present embodiment, the desk ease display when an anamorphic lens having a compression ratio of 2 is attached will be described, but when an anamorphic lens having another magnification is attached, the cutout range and the desk ease magnification will be described. Should be changed according to the compression ratio. In the image data that can be acquired by using the anamorphic lens, the distortion of the anamorphic lens differs greatly between the vicinity of the center of the image and the left and right edges, so that distortion may occur if the image is simply stretched horizontally. Therefore, it is possible to display an image that does not give a sense of discomfort to the user by cutting out the central area with less distortion or by applying a black band to the left and right edge areas (band display; displaying by superimposing the black band). can.

The image sensor 102 converts light into electric charges to generate an imaging signal. The generated imaging signal is output to the image processing unit 103. The image sensor 102 is an image sensor such as a CCD image sensor or a CMOS image sensor. As an image pickup element, all pixels on the image pickup surface are each composed of a pair of light receiving elements, and a pair of optical images formed by a microlens in each pixel can be photoelectrically converted by the pair of light receiving elements, so-called dual pixels. A type may be used.

The image processing unit 103 converts the image pickup signal input and acquired from the image sensor 102 into RAW data (RAW image). After that, the image processing unit 103 performs RAW development processing such as interpolation processing and image quality adjustment processing on the RAW data to generate YUV format image data corresponding to the RAW data, and stores the image data in the RAM 111. In the present embodiment, an example in which the size of the image data is QHD (number of pixels in the horizontal direction × number of pixels in the vertical direction = 3840 × 2160) will be described, but DCI 4K (4096 × 2160) and 8K (8192 × 4320) will be described. It may be.

The display resizing circuit 104 performs resizing processing and desk-easing processing described later on the YUV format image data stored in the RAM 111 to generate display image data and stores the image data in the RAM 111. Since the display resizing circuit 104 may execute a plurality of resizing processes in parallel, such as resizing the panel 107 to the resolution and resizing the display unit 115a to the resolution, a plurality of displays are displayed inside the image pickup apparatus 100. The resizing circuit 104 may be mounted.

The recording resizing circuit 105 performs resizing processing on the YUV format image data stored in the RAM 111, creates the recording image data, and stores the image data in the RAM 111. Normally, the recording resizing circuit 105 records the image data in the RAM 111 in the same size without resizing, but may also record an image having a small resolution for proxy editing.

The on-screen display (OSD) generation circuit 106 stores OSD data such as various setting menus, titles, and times in the RAM 111. The stored OSD data is combined with the display image data stored in the RAM 111, displayed on the panel 107 which is a display unit, or output to the outside by the external output 115.

The panel 107 is a display unit (display panel) such as a liquid crystal panel or an organic EL panel, and displays display image data and OSD. Normally, the resolution of the panel 107 is fixed, but it is conceivable that the panel 107 is removable and can be replaced with a higher resolution panel 107. In the present embodiment, an example in which the resolution of the panel 107 is HD (1280 × 720) will be described, but the resolution of the panel 107 is not particularly limited.

The system control unit 108 controls the entire image pickup apparatus 100.

The operation unit 109 is an operation unit that receives a user's operation input. Further, the operation unit 109 also includes a switch for selecting one of a camera mode for mainly performing camera shooting, a reproduction mode for reproducing and displaying, and a power-off mode for turning off the power. The grip zoom 109a is a zoom key for changing the display magnification of the captured image.

The ROM (Read only Memory) 110 is a non-volatile recording medium in which a program or the like executed by the system control unit 108 is stored, and is composed of, for example, a Flash ROM. The program data stored in the recording medium 113 may be loaded into the RAM 111 described later, and the RAM 111 may be used as the ROM 110. Further, a part of the area of the ROM 110 may be used for backing up and for holding the state of the system and the like.

The RAM (Random Access Memory) 111 is a volatile memory for use as a work area by the system control unit 108, the image processing unit 103, the compression / decompression circuit 114, and the like.

The media control unit 112 records the moving image data output to the RAM 111 on the recording medium 113 according to a format compatible with a computer such as a FAT file system. The recording medium 113 is a recording medium that is removable (detachable) from the image pickup device 100, and can be attached to a PC or the like in addition to the image pickup device 100.

The compression / decompression circuit 114 MPEG-compresses (encodes) the image data of the RAM 111 to generate moving image data, and outputs the moving image data to the RAM 111. The compression method is, for example, MPEG, H.M. 264, H. There are methods such as 265.

The external output 115 is an output unit such as HDMI or SDI, and outputs the display image data output to the RAM 111 by the display resizing circuit 104 to the external display unit 115a or the like. The external output 115 can output display image data as a signal such as 4K60P or 2K60P. The external output 115 normally outputs a video signal at the same resolution and frame rate as at the time of imaging (recording setting), but the output resolution and frame rate may be changed (for example, may be reduced) depending on the menu setting. ). Further, the external output 115 may acquire the display capability of the monitor connected to the image pickup apparatus 100 and output at the highest resolution and frame rate among the resolutions corresponding to the display capability.

The bus 116 is a data bus for exchanging data between each block of the image pickup apparatus 100, and each block of the image pickup apparatus 100 exchanges data via this bus.

<Flow of image data>
FIG. 2 is a diagram showing a flow of image data from shooting to display or recording in the image pickup apparatus 100. The lens unit 101 will be described as being an anamorphic lens having a vertical and horizontal compression ratio of 1: 2. Therefore, the subject is restored to the original aspect ratio by doubling the magnification (desk easing; stretching) in the horizontal direction (horizontal direction). By communicating with the lens unit 101, the image pickup apparatus 100 can acquire the compression ratio and automatically determine the magnification ratio in the lateral direction at the time of restoration. Further, even when the lens unit 101 does not support communication, the user can manually change the enlargement ratio by using the setting menu screen shown in FIG.

FIG. 3 is a diagram showing an example of a setting menu screen. The screen 301 is a screen for setting the type of the anamorphic lens. Here, since an anamorphic lens having a compression ratio of 1: 2 is used, the user selects "x2.0" corresponding to item 302 on the setting menu screen. As a result, the display resizing circuit 104 performs a resizing (desk-easing) process of changing the aspect ratio so as to be magnified twice in the horizontal direction. When "x1.4" is selected, the display resizing circuit 104 performs a resizing (desk-easing) process for changing the assist ratio so as to be enlarged 1.4 times in the horizontal direction. Further, when "OFF" is selected, the display resizing circuit 104 does not change the aspect ratio and only resizes according to the resolution of the display unit of the display destination (output destination).

Returning to the description of FIG. The subject 200 is a subject imaged by the image pickup apparatus 100. The imaged subject is imaged as an image 201 on the image sensor 102 in a state where the aspect ratio is compressed to 1: 2 by the lens unit 101. The imaged and photoelectrically converted imaging signal is input to the image processing unit 103 and stored in the RAM 111 as QHD size YUV format image data 202.

The YUV format image data 202 stored in the RAM 111 is subjected to the first desk-easing process described later by the display resizing circuit 104, and is stored in the RAM 111 as the display image data 203. The stored display image data 203 is output to the panel 107, and the image 204 is displayed.

Further, the YUV format image data 202 stored in the RAM 111 is subjected to a second desk-use process described later by the display resizing circuit 104, and is stored in the RAM 111 as the display image data 205. The stored display image data 203 is output to the outside via the external output 115, and the image 206 is displayed. Further, the display image data 205 stored in the RAM 111 is compressed by the compression / decompression circuit 114 and transmitted to the external device via the communication circuit 116. In the example of FIG. 2, two display resizing circuits 104 are used, but the above processing may be performed using one display resizing circuit 104.

Further, the YUV format image data 202 stored in the RAM 111 is stored in the RAM 111 as the recording image data 207 resized to the recording size by the recording resizing circuit 105. The stored image data 207 for recording is input to the compression / decompression circuit 114. After that, the recording image data 207 is compressed by the compression / decompression circuit 114 and stored in the RAM 111 as moving image data 208. The moving image data 208 is recorded on the recording medium 113 via the media control unit 112.

≪First desk ease processing≫
FIG. 4A is a diagram for explaining a first desk-easing process performed by the display resizing circuit 104. The upper side of FIG. 4A shows an example of YUV format image data 202 input to the display resizing circuit 104 (acquired by the display resizing circuit).

The width of the YUV format image data 202 will be W, and the height will be H. The display resizing circuit 104 cuts out a cutting range 401 of the width of the YUV format image data 202, which is a range of 1 / 4W to 3/4W and a height H from the left side of the image data. The scaling process is performed at different magnifications in the vertical and horizontal directions. By performing scaling processing in the horizontal direction (width scaling) at a magnification of twice the vertical direction (height scaling), the displayed image is in the horizontal direction rather than displaying the original image 201 as it is. Seems to have doubled. As a result, in the lens unit 101, the subject image compressed in the lateral direction with respect to the actual appearance of the subject is stretched twice in the lateral direction and approaches the original appearance of the subject. The desk-easing process that makes the horizontal scaling factor twice the vertical scaling factor is called the first desk-easing process (first resizing), and is generated by the first desk-easing process. The image data is as shown in the display image data 203 at the lower side of FIG. 4A.

The first desk-easing process generates the display image data 203 by using 1/2 of the width of the input image among the YUV format image data 202 input to the display resizing circuit 104. This is because the compression magnification in the lateral direction of the lens unit 101 in the present embodiment is 2 times, and in the case of other magnifications, the size of the cutout range 302 is not limited to this. The YUV format image data 202 is scanned in the horizontal direction (horizontal direction) and input to the display resizing circuit 104 line by line. Assuming that the input time Tin for this one line to enter the display resizing circuit 104 is Tin, the input time for the cutout range 302 is Tin / 2. Therefore, the time To out for writing out one line of the display image data 203 also needs to be within Tin / 2.

Here, if the time required to generate the image data for display is longer than the input time, the time from imaging to display becomes longer, or the captured image of each frame cannot be displayed. Alternatively, the image data for display is output in a state where it is cut off in the middle. Therefore, the display resizing circuit 104 needs to generate one line of the display image data 203 in the time of Tin / 2. However, since the display resizing circuit 104 has a performance limit, if the time available for processing for generating display data (that is, the Tin / 2 time) is shortened, the display image data 203 can be generated in time. It disappears. Tin depends on the frame rate of the image pickup apparatus 100. For example, when the frame rate is 60 [fps], Tin is 1/60 [s], and when the frame rate is 120 [fps], Tin is 1/120 [s]. That is, when the frame rate becomes high, the time available for the resizing process becomes short, and the process for display may not be in time.

≪Second desk ease processing≫
FIG. 4B is a diagram for explaining a second desk-easing process performed by the display resizing circuit 104. The left side of FIG. 4B shows an example of YUV format image data 202 input to the display resizing circuit 104 (acquired by the display resizing circuit).

As in FIG. 4A, the width of the YUV format image data 202 will be W, and the height will be H. The display resizing circuit 104 applies deskease processing to the width W and height H ranges of the YUV format image data 202. At this time, the scaling process is performed at different magnifications in the vertical direction (vertical direction) and the horizontal direction (horizontal direction). Reduce the magnification to half of the horizontal variable magnification. By performing scaling processing in the vertical direction (height side magnification) at half the magnification in the horizontal direction (width scaling), the displayed image will be in the vertical direction rather than displaying the original image 201 as it is. Looks like it's compressed. As a result, in the lens unit 101, the subject image compressed in the horizontal direction from the actual appearance of the subject is compressed in half in the vertical direction, and approaches the original appearance of the subject. In this way, by compressing the vertical direction without cutting out the horizontal direction of the input image (without cutting out the horizontal direction of the YUV format image data), the subject image compressed in the horizontal direction is originally obtained. The process of bringing the subject closer to the appearance is called the second desk-eas process.

At this time, the image data generated by the display resizing circuit 104 becomes like the display image data 205 on the right side of FIG. 4B. The YUV format image data 202 and the display image data 205 are not cut out in the horizontal direction. Therefore, the display resizing circuit 104 can write out one line of the display image data 205 by using the input time Tin in which one line is input to the display resizing circuit 104. For example, when a line with a number of pixels in the horizontal direction of 1920 pix is input in Tin, when the first desk easing process is performed, half is cut out, so that display data for 1920 pix is displayed in Tin / 2 time. Need to write. In the second desk-easing process, when one line having 1920 pixels in the horizontal direction is input with an input time of Tin, display data can be written using Tin.

However, since the image displayed by performing the second desk easing process is smaller than that when the first desk easing process is performed, the user who wants to display the image larger can use the first desk easing process. The visibility of the image is better than that of the second desk easing process. Further, the display image data 205 is not displayed as it is, and as shown in the image 206 of FIG. 2, the display image data 205 is displayed in a state where black bands are added at the top and bottom (side black processing). .. The area corresponding to the display image data 203 is an area of the display image data 205 surrounded by the range 402.

FIG. 5A is an example of the time Tin / α (Equation (1)) applied to the one-line output for the input frame when the image data of QHD size (3840 × 2160) is input. Here, F indicates the frame rate, and α indicates the compression magnification.
Time Tin / α = (1 / F) × (1/2160) × (1 / α) ・ ・ ・ (1)

Normally, the image data includes surplus pixels (optical black, etc.) outside the effective pixels, but the surplus pixels are excluded here for the sake of simplification. Further, as the number of pixels in one line of the output image data increases, the time actually required for the resizing process also increases. Since the performance of the display resizing circuit 104 is limited, it may not be possible to complete the generation of the display image data within the time Tin / α in the lateral enlargement processing if the size or frame rate of the image data 202 is high. be. Assuming that the time required to output one line is Tout, it is necessary to satisfy the following equation (2).
Tin / α> Toout ・ ・ ・ (2)

FIG. 5B is an example of the processing time Tout with respect to the size of the output image data. Here, the processing capacity of the display resizing circuit 104 is set to 800 MHz, and the calculation is performed using a model that outputs one pixel in one clock. Assuming that the width of the image data to be output is W pixels, the processing time Tout can be obtained by using the following equation (3).
Time Toout = W / 800MHz ・ ・ ・ (3)

Comparing Tin / α in FIG. 5A and Tout in FIG. 5B, in the case of the HD size panel 107, it is possible to perform double lateral enlargement processing up to 120 [fps]. On the other hand, in the case of external output in QHD size, the above equation (2) is no longer satisfied, and 1.4 times and 2 times horizontal enlargement processing cannot be performed. If the processing overflows in this way, the image processing unit 103 cannot output the image data of the next frame to the RAM 111, and the frames are dropped. Alternatively, it is possible that the image data of the next frame is overwritten and the image data is damaged.

In such a case, in the present embodiment, the display resizing circuit 104 performs the second desk-easing process. Specifically, the display resizing circuit 104 performs a reduction process of 1/2 in the vertical direction, and displays image data of 3840 × 1080 which is half of the QHD size which is the resolution of the display unit 115a in the vertical direction. It is stored in the RAM 111 as 205. The stored external output display image data is output as a QHD size video signal with black bands added at the top and bottom at the external output 115, and the image 206 is displayed on the display unit 115a. At this time, as a characteristic of the anamorphic lens, the left and right edges of the image are distorted, so the left and right edges are often masked with black bands. In the second desk-eas processing process, one line of the display image data 205 may be generated in one line of time Tin. That is, the following equation (4) may be satisfied.
α × Tin ＞ Toout ・ ・ ・ (4)
Should be satisfied.

FIG. 5C shows QHD size image data, which is an example of the time α × Tin that can be applied to 1-line output at the time of vertical reduction. Image data for display can be generated regardless of the output size and frame rate. As described above, the external output 115 masks the outside (end) of the display image data 205 in the range 402 from 1/4 to 3/4 in the horizontal direction with a black band, and then displays the display unit 115a (external display device). ). Further, when the anamorphic lens setting is "OFF" and the display size is QHD, the image data 202 can be input to the external output 115 as it is as display image data. Further, the range 402 of the stored display image data 205 may be transmitted to the outside through the communication means 116.

<Processing content>
FIG. 6 is a diagram showing an example of display processing in the present embodiment. This process is realized by expanding the program recorded in the ROM 110 into the RAM 111 and executing it by the system control unit 108. This process starts when the power is turned on to the image pickup apparatus 100 and the image pickup process is performed. In this process, the first desk ease process is performed when the record setting satisfies a predetermined condition, and the second desk ease process is performed when the record setting does not satisfy the predetermined condition.

In S601, the system control unit 108 acquires information on the display device (panel 107 or display unit 115a). Here, the information of the display device includes a resolution that can be displayed by the display device, a frame rate that can be displayed by the display device, and a physical screen size (inch). In the case of the display unit 115a, the system control unit 108 can acquire information from the display device by bidirectional communication via the external output 115. In the case of the panel 107 (display device fixed to the image pickup device 100), the system control unit 108 can acquire the design value held in advance without acquiring it by communication. When the display device can display images having a plurality of resolutions, the system control unit 108 may acquire the current (currently displayed; currently set) resolution from the display device. ..

In S602, the system control unit 108 sets the output size and the frame rate based on the acquired information of the display device. When the display device can display an image of the input size (the size of the image data 202 in the YUV format (for example, QHD)), the system control unit 108 sets the output size to be the same as the input size. If the display device cannot display the input size image, the system control unit 108 sets the output size to the maximum resolution that the display device can display. Similarly, regarding the frame rate, the system control unit 108 sets the maximum frame rate that can be displayed when the display cannot be performed at the set frame rate. In recent years, some display devices support high frame rates exceeding 60 [fps], but many display devices support only up to 60 [fps].

In S603, the system control unit 108 determines whether or not the type (setting) of the anamorphic lens is "x1.4". If it is "x1.4", the process proceeds to S604, and if not, the process proceeds to S613.

In S604, the system control unit 108 determines whether or not the output size is QHD. In the case of QHD, the process proceeds to S605, and if not, the process proceeds to S610.

In S605, the system control unit 108 determines whether or not the frame rate is 60 [fps] (threshold value) or less. If it is 60 [fps] or less, the process proceeds to S606, and if not (if it exceeds 60 [fps]), the process proceeds to S607.

In S606, the system control unit 108 controls the display resizing circuit 104 to perform x1.4 cutout / horizontal enlargement processing (first desk ease processing) to obtain QHD size display image data 203. Generate.

In S607, the system control unit 108 determines whether or not the physical screen size acquired in S601 is equal to or larger than a predetermined value. If it is equal to or more than the predetermined value, the process proceeds to S608, and if not (smaller than the predetermined value), the process proceeds to S609. That is, it is determined whether or not the resolution of the output destination acquired in S601 is equal to or higher than a predetermined value.
Proceed to.

In S608, the system control unit 108 controls the display resizing circuit 104 to perform a vertical reduction process (second desk ease process) of × 1.4 to generate display image data 205. Here, the system control unit 108 generates display image data 205 in which the area shown as a whole in the image data 202 is reduced to an area having a size of 3840 × 1542. As described above, the top, bottom, left, and right are masked with black bands and displayed in QHD size, but the effective image range is 2742 × 1542. The resolution of the effective image range is FHD (FullHD; 1920 x 1080) or higher, but if the physical screen size is small, the visibility may decrease because the black bands occupy the top, bottom, left, and right. ..

In S609, the system control unit 108 changes the output size to FHD. When the resolution is changed to FHD, the resolution is lowered, but in S611 described later, × 1.4 cut-out enlargement (first desk ease processing) is performed. As a result, the black bands on the top, bottom, left, and right are eliminated, and the entire screen is used for display. Therefore, it is possible to prevent a decrease in visibility when the screen size is small (S607-No). This is because when the resolution of the display destination is low (when the physical screen size acquired in S601 is small; when the formula (1) is 2160 or less), the frame rate is high and the time available for resizing processing is short. However, this is because it is possible to secure time for performing the above-mentioned display processing.

In S610, the system control unit 108 determines whether or not the output size is FHD. If it is FHD, the process proceeds to S611, and if not (in the case of HD), the process proceeds to S612.

In S611, the system control unit 108 controls the display resizing circuit 104 to perform x1.4 cutout / horizontal enlargement processing (first desk ease processing) to obtain FHD size display image data 203. Generate.

In S612, the system control unit 108 controls the display resizing circuit 104 to perform x1.4 cutout / horizontal enlargement processing (first desk ease processing) to obtain HD size display image data 203. Generate.

In S613, the system control unit 108 determines whether or not the type (setting) of the anamorphic lens is "x2.0". If it is "x2.0", the process proceeds to S614, and if it is not likely to proceed (in the case of "OFF"), the process proceeds to S621.

In S614, the system control unit 108 determines whether or not the output size is QHD. In the case of QHD, the process proceeds to S615, and if not, the process proceeds to S618.

In S615, the system control unit 108 determines whether or not the frame rate is 30 [fps] (threshold value) or less. If it is 30 [fps] or less, the process proceeds to S616, and if not (if it exceeds 30 [fps]), the process proceeds to S617. Here, the threshold value of S605 is 60 [fps], whereas that of S615 is 30 [fps]. This is because the higher the compression ratio (“× 1.4” or “× 2.0”, etc.) of the image data, the smaller the threshold value, so that the above equation (4) is satisfied.

In S616, the system control unit 108 controls the display resizing circuit 104 to perform x2.0 cutout / horizontal enlargement processing (first desk ease processing) to obtain QHD size display image data 203. Generate.

In S617, the system control unit 108 changes the output size to FHD. × 1 mentioned above
.. As in the case of No. 4 (S609), the black bands on the top, bottom, left, and right are eliminated and the entire screen is used for display. Therefore, even if the screen size is small, deterioration of visibility can be prevented. In the case of × 2.0, it becomes 3840 × 1080 when the vertical reduction processing is performed, and the effective range in which the left and right are cut is the same as that of FHD. Therefore, it is not necessary to mask the top, bottom, left, and right with black bands and display them in QHD size.

In S618, the system control unit 108 determines whether or not the output size is FHD. If it is FHD, the process proceeds to S619, and if not (in the case of HD), the process proceeds to S620.

In S619, the system control unit 108 controls the display resizing circuit 104 to perform a vertical reduction process (second desk ease process) of × 2.0 to generate FHD size display image data 205. .. Here, the vertical reduction process of × 2.0 is performed, but in the case of resizing from QHD to FHD, the same result as the cutout enlargement of × 2.0 can be obtained. This is because the 2x enlargement in the horizontal direction and the 1/2x enlargement from QHD to FHD cancel each other out, and the horizontal enlargement process is not performed.

FIG. 7 is a diagram showing an example of resizing processing from QHD to FHD. The system control unit 108 (display resizing circuit 104) compresses the image data 202 in the vertical direction by 1/2 times to generate the image data 701 (restores the aspect ratio). After that, the system control unit 108 (display resizing circuit 104) cuts out the central portion of the image data 701 to generate the FHD size display image data 203.

In S620, the system control unit 108 controls the display resizing circuit 104 to perform x2.0 cutout / horizontal enlargement processing (first desk ease processing) to obtain HD size display image data 203. Generate.

Next, when the type (setting) of the anamorphic lens is "OFF" (S613-No), the system control unit 108 performs only the resizing process without restoring the aspect ratio.

In S621, when the type (setting) of the anamorphic lens is "OFF" (S613-No), the system control unit 108 determines whether or not the output size is QHD. In the case of QHD, since the image data 202 is used as it is as the display image data of the QHD size, the present processing flow is terminated without newly generating the display image data (doing nothing). If it is not QHD, proceed to S622.

In S622, the system control unit 108 controls the display resizing circuit 104 to generate display image data without performing desk-eas processing. That is, the system control unit 108 performs resizing processing to the output size (FHD or HD) while maintaining the aspect ratio, and generates FHD or HD size display image data.

As in the above-described embodiment, even if the time that can be used for generating the display image data is shortened due to a high frame rate or the like, if the output resolution is low, the first desk ease process is performed. This makes it possible to generate display image data with good visibility.

Although the present invention has been described in detail based on the preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various embodiments within the scope of the gist of the present invention are also included in the present invention. included. Some of the above-described embodiments may be combined as appropriate.

In the above-described embodiment, the image pickup apparatus may perform the first desk-easing process or the second desk-easing process according to the resolution of the image to be output regardless of the frame rate of the image data. That is, when the image pickup apparatus outputs an image at the first resolution, the first desk ease process is performed, and when the image is output at a second resolution higher than the first resolution, the second display is performed. Squeeze processing may be performed.

Further, in the above-described embodiment, the image pickup apparatus may perform the first desk-easing process or the second desk-easing process depending on the resolution of the output image and the frame rate of the image data. That is, the image pickup apparatus performs the first desk-easing process when the frame rate of the image data is lower than the threshold value. Then, the image pickup apparatus performs the first desk-eas processing when the frame rate of the image data is higher than the threshold value and the image is output at a resolution lower than a predetermined value. Further, the image pickup apparatus performs a second desk-easing process when the frame rate of the image data is higher than the threshold value and the image is output at a resolution higher than a predetermined value.

Further, when a software program that realizes the functions of the above-described embodiment is supplied to a system or device having a computer capable of executing the program directly from a recording medium or by using wired / wireless communication, and the program is executed. Is also included in the present invention.

Therefore, in order to realize the functional processing of the present invention on a computer, the program code itself supplied and installed on the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

In that case, the form of the program does not matter, such as the object code, the program executed by the interpreter, the script data supplied to the OS, etc., as long as it has the function of the program. The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical / optical magnetic storage medium, or a non-volatile semiconductor memory.

Further, as a method of supplying the program, a method in which the computer program forming the present invention is stored in a server on the computer network and the connected client computer downloads and programs the computer program can be considered.

The above-mentioned various controls described as being performed by the system control unit 108 may be performed by one hardware, or the entire device may be controlled by sharing the processing among a plurality of hardware. ..

Further, although the present invention has been described in detail based on the preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various embodiments within the scope of the gist of the present invention are also included in the present invention. included. Furthermore, each of the above-described embodiments is merely an embodiment of the present invention, and each embodiment can be combined as appropriate.

Further, in the above-described embodiment, the case where the present invention is applied to an imaging device has been described as an example, but this is not limited to this example, and any display control device that controls the display of an image can be applied. .. That is, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, and the like. Further, the present invention can be applied to a printer device including a display, a digital photo frame, a music player, a game machine, an electronic book reader, a tablet terminal, a smartphone and the like. Further, the present invention can be applied to a home appliance device including a projection device and a display, an in-vehicle device, and the like.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

108: System control unit

Claims (14)

An acquisition means capable of acquiring first image data in which a subject image is formed at different magnifications in the horizontal direction and the vertical direction, and
After performing the first process of cutting out a part of the first image data, the first process of generating a display image so that the second image data subjected to the first process has a predetermined aspect ratio. A display control means capable of performing resizing and a second resizing of resizing the first image data without performing the first processing.
When the image is output to the first output destination having the first resolution, the first resizing is performed to the second output destination having the second resolution smaller than the first resolution. A display control device comprising: a control means for controlling to perform the second resizing when outputting an image. An acquisition means capable of acquiring first image data in which a subject image is formed at different magnifications in the horizontal direction and the vertical direction, and
When an image is output to a first output destination having a first resolution, there is a region surrounding the horizontal direction and the vertical direction of the captured image with a width equal to or larger than a predetermined width, which is smaller than the first resolution. When an image is output to a second output destination having a resolution of 2, a control means for controlling the horizontal direction and the vertical direction of the captured image so as not to be surrounded by the predetermined width or more is displayed. A display control device characterized by having. When the control means outputs an image to the first output destination, the control means has a region surrounding the horizontal direction and the vertical direction of the captured image with a width equal to or greater than a predetermined width, and the image is sent to the second output destination. The display control device according to claim 1, wherein when the image is output, the horizontal direction and the vertical direction of the captured image are controlled so as to be displayed without being surrounded by the predetermined width or more. The control means performs the first resizing when the moving image recording setting satisfies a predetermined condition, and performs the second resizing when the moving image recording setting does not satisfy the predetermined condition. The display control device according to claim 1 or 3, wherein the display control device is controlled in such a manner. The control means
When the frame rate of the first image data is lower than the threshold value, the first resizing is performed.
When the frame rate of the first image data is higher than the threshold value and the image is output to an output destination having a resolution lower than a predetermined value, the first resizing is performed.
When the frame rate of the first image data is higher than the threshold value and the image is output to an output destination having a resolution higher than the predetermined value, the second resizing is performed.
To control,
The display control device according to any one of claims 1, 3 and 4. The higher the compression ratio of the first image data, the smaller the threshold value.
The display control device according to claim 5. The control means acquires the current resolution from the display unit when the display unit to which the image is output can display an image having a plurality of resolutions, and the first resizing according to the acquired resolution. Whether to perform the second resizing,
The display control device according to any one of claims 1, 3 to 6, characterized in that. The control means performs the first resizing when the physical screen size of the display unit to which the image is output is smaller than the predetermined size, and when the screen size is larger than the predetermined size, the control means said. Do the second resizing,
The display control device according to any one of claims 1, 3 to 7, wherein the display control device is characterized by the above. Further having a transmission means for transmitting an image to an external display unit,
The display control device according to any one of claims 1 to 8, wherein the display control device is characterized by the above. The image acquired by the acquisition means is an image captured by using an anamorphic lens and is not desk-eased.
The display control device according to any one of claims 1 to 9, wherein the display control device is characterized by the above. The acquisition step of acquiring the first image data in which the subject image is imaged at different magnifications in the horizontal direction and the vertical direction, and
After performing the first process of cutting out a part of the first image data, the first process of generating a display image so that the second image data subjected to the first process has a predetermined aspect ratio. A display control step in which resizing and a second resizing that resizes the first image data without performing the first processing can be performed.
When the image is output to the first output destination having the first resolution, the first resizing is performed to the second output destination having the second resolution smaller than the first resolution. A control method comprising: a control step for controlling to perform the second resizing when outputting an image. The acquisition step of acquiring the first image data in which the subject image is imaged at different magnifications in the horizontal direction and the vertical direction, and
When an image is output to a first output destination having a first resolution, there is a region surrounding the horizontal direction and the vertical direction of the captured image with a width equal to or larger than a predetermined width, which is smaller than the first resolution. When the image is output to the second output destination having the resolution of 2, the control step for controlling the horizontal direction and the vertical direction of the captured image so as not to be surrounded by the predetermined width or more is displayed. A control method characterized by having. A program for causing a computer to function as each means of the display control device according to any one of claims 1 to 10. A computer-readable storage medium containing a program for causing the computer to function as each means of the display control device according to any one of claims 1 to 10.

Images