JP4948637B2 - Image processing method and apparatus - Google Patents

Description

  The present invention relates to an image processing method and apparatus, and more particularly, to an image processing method and apparatus for reproducing an input image stored in association with an attitude of an image detected at the time of image shooting, for example, a vertical direction.

  Conventionally, image processing apparatuses such as electronic cameras for recording and reproducing still images and moving images using a memory card having a solid-state memory element as a recording medium are already on the market, and electronic cameras equipped with an electronic viewfinder such as a color liquid crystal panel are also sold. Has been. According to these electronic cameras, it is possible to continuously display images before photographing and to determine the composition by the user of the electronic camera, and to reproduce and display the photographed images. In particular, the function of reproducing a photographed image immediately after photographing is highly convenient and is a useful function for the user of the electronic camera.

  By the way, in so-called image recording apparatuses such as digital cameras and digital video cameras, when a user shoots an image recording apparatus in a normal posture with respect to the vertical direction (hereinafter referred to as horizontal shooting), the image recording apparatus is aligned with respect to the vertical direction. When shooting in a posture rotated 90 degrees or 270 degrees (hereinafter referred to as vertical shooting). Further, there is a case where photographing (horizontal shooting) is performed with a posture rotated 180 degrees with respect to the top and bottom direction opposite to the normal posture.

  Therefore, an image recording apparatus that detects the attitude (top and bottom direction) of the image recording apparatus at the time of shooting by adding posture detection means such as a gyro, and records information by adding information indicating the top and bottom direction of the shot image. Is commercially available. In this image recording apparatus, for example, the top-and-bottom direction can be indicated by rotation angles such as 0 degrees, 90 degrees, 180 degrees, and 270 degrees according to the result of detecting the posture of the apparatus at the time of shooting. In addition, information indicating the vertical direction can be added as EXIF information in an image recording apparatus that encodes and records a captured image in the JPEG format.

  When the captured image recorded in this way is reproduced by a so-called image reproducing device such as a digital camera or a digital video camera, the captured image is converted to 90 degrees, 180 degrees, or based on information indicating the vertical direction in the EXIF information. It is played after rotating 270 degrees. For example, Patent Document 1 discloses a technique for rotating a photographed image so that the posture of the device at the time of photographing and the posture of the device at the time of reproduction coincide with each other, and Patent Document 2 discloses a case of rotating a still image but a moving image. A technology that does not rotate is proposed.

JP 2000-31329 A JP 2003-274366 A

  In the above rotation processing, for example, as shown in FIG. 3, when a vertically shot image is reproduced without rotation (FIG. 3A), the user views the image in the correct top-and-bottom direction. However, in the same manner as the image recording device at the time of shooting, it is necessary to rotate the image reproducing device. However, when the image reproducing device is rotated and reproduced (FIG. 3B), the user does not need to rotate the image reproducing device. You can see the image in the correct vertical direction.

  By the way, a conventional image recording apparatus such as a digital camera is generally provided with an image sensor such as a CCD or CMOS sensor having an aspect ratio of 4: 3 (may be 3: 2), and the aspect ratio of an image to be taken is 4 as well. : 3 (there may be 3: 2), and an image reproducing apparatus such as a digital camera generally includes display means such as an LCD or EVF having an aspect ratio of 4: 3.

  In such a case, for example, as shown in FIG. 3, the aspect ratio of the captured image and the aspect ratio of the display means are both 4: 3, and the vertically shot image is reproduced without being rotated (FIG. 3). 3 (a)) when rotating and reproducing (FIG. 3 (b)), the reproduced image must be displayed smaller than when reproduced without rotating, and the entire image is displayed. In order to do so, it was necessary to provide a margin (black area in the figure) on the left and right for display. At this time, the display area of the image in the entire display means is about 56% as shown in FIG.

  However, in recent digital video cameras and the like, there are some cameras equipped with an image sensor with an aspect ratio of 16: 9 because of wide compatibility and HD compatibility, and it is also possible to capture an image with an aspect ratio of 16: 9 that is wider than before. It became so. In such digital video cameras, not only those having display means with an aspect ratio of 4: 3 but also those having display means with an aspect ratio of 16: 9 have appeared.

  However, as shown in FIG. 4, for example, in the case of a digital video camera having a 16: 9 aspect ratio that is longer than the conventional aspect ratio of the captured image and the aspect ratio of the display means, the vertically shot image is rotated. When the image is reproduced without rotation (FIG. 4A), when the image is reproduced with rotation (FIG. 4B), the image display area occupying the entire display means is as shown in FIG. 4B. About 32%. If the aspect ratio of the captured image or the display means increases as in such a case, the display size (display area reproduced by the display means) when the vertically shot image is rotated becomes small. There is a problem that it is difficult for the user to view the reproduced image.

The present invention has been made starting from solving the above-described problems of the prior art, and an object of the present invention is to control an image processing apparatus that reproduces a captured image so that the user can easily see it, and its control. A method is provided.

In order to achieve the above object, an image processing apparatus of the present invention is an image processing apparatus for displaying an image in a horizontally long display area, and displays the image horizontally or vertically in the display area. A determination means for determining whether to display the image, and if the determination means determines that the image is to be displayed horizontally in the display area, the entire image is enlarged and reduced so that the image is displayed horizontally in the display area; When it is determined by the determination means that the image is displayed in the display area in the portrait orientation, the entire image is rotated and enlarged / reduced so that the image is displayed in the portrait orientation in the display area, and a preset portion of the image is displayed. Display control means for controlling the display so as to be displayed in an area where the entire image is not displayed in the display area .

According to another aspect of the present invention, there is provided a control method for an image processing apparatus for displaying an image in a horizontally long display area, wherein the discrimination means displays the image horizontally in the display area. Or when the display control means determines that the image is displayed horizontally in the display area in the determination process, the entire image is displayed horizontally in the display area. If the image is determined to be displayed vertically in the display area in the determination step, the entire image is rotated and enlarged so that the image is displayed vertically in the display area. And a display control step of controlling to display a preset portion of the image in an area of the display area where the entire image is not displayed .

  Furthermore, a program for causing a computer to execute the steps of the image processing method and a computer-readable storage medium storing the program are also provided.

According to the present invention, it is possible to provide an image processing method and apparatus for reproducing captured images so that the user can easily see them.

  In other words, according to the first embodiment of the present invention, it is possible to reproduce an image reproduced by a user with a simple configuration so that the user can easily see the image by processing based on the aspect ratio of the captured image.

  According to the second embodiment of the present invention, it is possible to perform more accurate reproduction so that the user can easily view the reproduced image by the process based on the occupation ratio of the reproduction screen of the reproduced image.

  According to the third embodiment of the present invention, the reproduction based on the size of the reproduced image can be performed with consideration given to making it easier for the user to view the reproduced image.

  According to the fourth embodiment of the present invention, it is possible to perform reproduction so that the user can recognize the reproduced image by the process of reproducing a plurality of captured images with different rotations.

  Furthermore, the present invention relates to a reproduction-only image reproduction apparatus including a personal computer or a television, and a digital camera or digital video camera with an electronic viewfinder for reproduction which is an image recording / reproduction apparatus in which the image recording apparatus and the image reproduction apparatus are integrated. In addition, the present invention can be applied to any system in which an image recording apparatus and an image reproduction apparatus, which are independent devices, are connected via a network or the like.

It is a block diagram which shows the structure of the image recording / reproducing apparatus in this embodiment. It is a figure which shows the structural example of the image data etc. which are memorize | stored in the memory in this embodiment. Example of a result obtained by reproducing a 4: 3 photographed image taken in the vertical direction on the 4: 3 display means and a result obtained by rotating a photographed 4: 3 photograph taken in the vertical direction and reproduced on the 4: 3 display means. FIG. An example of a result of replaying a 16: 9 photographed image taken in the vertical direction with the display means of 16: 9 and a result of replaying a photographed image taken in the vertical direction of 16: 9 and reproduced by the display means of 16: 9 FIG. 4 is a flowchart illustrating an example of a reproduction processing procedure according to the first embodiment. It is a figure which shows the example of the result reproduced | regenerated by the display means of 16: 9, without rotating the picked-up image of 16: 9 vertically taken in 1st Embodiment. It is a figure which shows the example of the result of having rotated the 4: 3 picked-up image taken vertically and reproducing | regenerating by the display means of 16: 9 in 1st Embodiment. It is a figure which shows the example of the result of having reproduced | regenerated by the display means of 4: 3, without rotating the 16: 9 picked-up image image | photographed vertically in 1st Embodiment. It is a figure which shows the example of the result of having rotated the picked-up image of 4: 3 taken vertically and reproducing | regenerating by the display means of 4: 3 in 1st Embodiment. It is a flowchart which shows the example of a procedure of the reproduction | regeneration processing in 2nd Embodiment. It is a figure explaining the ratio of the display area of the image which occupies the whole display means at the time of rotating and reproducing | regenerating a picked-up image with respect to the aspect ratio of the picked-up image and a display means in 2nd Embodiment. It is a figure which shows the example of the result of having rotated the 16: 9 picked-up image taken vertically and reproducing | regenerating by the display means of 4: 3 in 2nd Embodiment. 15 is a flowchart illustrating an example of a playback process procedure according to the third embodiment. 14 is a flowchart illustrating an example of a reproduction processing procedure according to the fourth embodiment. It is a figure which shows the example of the result of having rotated the 16: 9 picked-up image which carried out the vertical shooting in 4th Embodiment, and having arranged and reproduced | regenerated on the display means of 16: 9 with two types of magnification. It is a figure which shows the example of the result of having rotated the 4: 3 picked-up image vertically taken in 4th Embodiment, and reproducing | regenerating by arranging on the display means of 16: 9 by two types of magnifications . It is a figure which shows the example of the result of having rotated the 16: 9 picked-up image vertically taken in 4th Embodiment, and reproducing | regenerating by arranging on the display means of 4: 3 by two types of magnifications . It is a figure which shows the example of the result of having rotated the 4: 3 picked-up image image | photographed vertically in 4th Embodiment, and having arranged and reproduced | regenerated on the display means of 4: 3 by two types of magnification .

  Hereinafter, several examples of preferred embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to such an embodiment, and includes all technologies expected from the present embodiment for reproducing the reproduced image so that the user can easily see the image. In the following embodiments, the configuration range of the apparatus is divided by an image recording apparatus, an image reproducing apparatus, an image recording / reproducing apparatus, and the like, which are included in an image processing apparatus that processes an image. In the book, these are recognized as an image processing apparatus. Accordingly, in the following embodiments, the present invention will be described by taking a digital camera with a digital viewfinder for reproduction and a digital video camera as an example of an image recording / playback device, but a playback-only image playback device including a personal computer or a TV, The present invention can also be applied to a system in which an image recording apparatus and an image reproduction apparatus, which are independent devices, are connected via a network or the like, and these are also included in the present invention.

[First Embodiment]
<Characteristics of First Embodiment>
The image reproducing apparatus according to the first embodiment is an image recording apparatus at the time of photographing when acquiring and reproducing a recorded image added with information indicating the vertical direction obtained by posture detection at the time of photographing. The rotation angle of the image to be displayed on the display unit is determined according to the angle of the vertical direction of the image and the aspect ratio of the captured image, the acquired image is rotated at the determined rotation angle of the image, and the rotated image is It can process so that it may display according to the screen size of a display means. That is, when displaying a vertically shot image on the display unit, if it is determined that the displayed image is easy to view even if the vertically shot image is rotated and displayed on the display unit (the aspect ratio of the vertically shot image) Is larger than a threshold value (example: 14/9), the image is rotated 90 degrees (or 270 degrees), and the vertically captured image is rotated and displayed on the display unit. (When the aspect ratio of the vertically shot image is smaller than the threshold value), the image can be displayed without being rotated. Therefore, the user can always see the reproduced image adjusted for easy viewing.

<Configuration Example of Image Recording / Reproducing Device of the Present Embodiment: FIG. 1>
FIG. 1 is a block diagram showing a configuration of an image recording / reproducing apparatus according to the first embodiment.

In FIG. 1, a CPU 102 of an image recording / reproducing apparatus 100 according to the first embodiment is connected to a RAM 103, a ROM 104, a key operation processing unit 105, a gyro input processing unit 106, and an image processing engine 111 via a bus 101. The CPU 102 operates based on the control program 104a stored in the ROM 104, and uses the RAM 103 as a temporary information storage location during operation. The state of key operation by the user is held in the key operation processing unit 105, and the control program of the CPU 102 periodically acquires the state of key operation from the key operation processing unit 105. Posture state of the apparatus detected by the gyro during image capture is held by the gyro input processing unit 106, the control program of the CPU102 is periodically acquires the posture state from the gyro input processing unit 106. The control program of the CPU 102 controls the image processing engine 111 based on the acquired key operation state and the posture state, and performs an image recording operation and a reproduction operation.

<Operation Example of Image Recording / Reproducing Device of this Embodiment: FIGS. 1, 2, and 5>
(Recording operation: Figs. 1 and 2)
First, a recording operation of a captured image will be described with reference to FIG.

The image recording and reproducing apparatus 10 0 is provided with a mode dial (not shown), when the user adjust the mode dial to the recording mode, CPU 102 control program retrieves the state of the mode dial from the key operation unit 105, the recording Transition to the mode in which operation is performed.

The image recording and reproducing apparatus 10 0 and the shutter button (not shown) is provided, in a mode for performing the recording operation, since the user presses the shutter button, CPU 102 controls the program from the key operation unit 105 shutter button pressed in Get state. Thereby, the control program controls the image processing engine 111 as described below to perform a recording operation of the captured image. The memory card slot (not shown) in the image recording and reproducing apparatus 10 0 is provided with, captured images are recorded to the memory card in the said slot.

  First, the image processing engine 111 acquires raw data of a captured image from the CCD 107 by controlling a TG (timing generator) and an AFE (analog front end) (not shown). Next, the image processing engine 111 stores the raw data in the RAM 114. Next, the image processing engine 111 reads the raw data from the RAM 114, performs image processing such as color conversion, and converts each pixel into YCC data having luminance (Y) and color difference (Cb, Cr) information. Then, it is stored in the RAM 114 again. Next, the image processing engine 111 reads the YCC data from the RAM 114, encodes it using the JPEG codec 113, and stores it again in the RAM 114 as JPEG main image data including image size information. At the same time, the image processing engine 111 reads the YCC data from the RAM 114, reduces the thumbnail image size, and stores the thumbnail image in the RAM 114. Next, the image processing engine 111 reads out the YCC data reduced to the thumbnail image size from the RAM 114, encodes it using the JPEG codec 113, and stores it in the RAM 114 as JPEG thumbnail image data.

FIG. 2 shows data 150 (stored in the same manner as image 2, image 3,...) Including EXIF information 151, JPEG main image data 152, and JPEG thumbnail image data 153. An example of the configuration of the captured image storage unit 114a in the RAM 114 is shown. Additional information related to the image can be stored as EXIF information 151.

In parallel with the operation of the image processing engine 111 from the time the shutter button is pressed, the control program of the CPU 102 is detected by the gyro together with parameters such as the shutter speed and the aperture amount at the time of shooting and is detected by the gyro input processing unit 106. the posture state of the device during image capture which is held periodically acquired from the gyro input processing unit 106, an image from the CPU102 parameters 151a that indicates the vertical direction of the captured image obtained from the posture state at the time of photographing, the EXIF information It is stored in the RAM 114 via the processing engine 111. Here, the parameter indicating the top-and-bottom direction is indicated by a rotation angle such as 0 degrees, 90 degrees, 180 degrees, and 270 degrees according to the posture state at the time of shooting. At this time, for example, if the posture at the time of photographing is in the range of −80 degrees (280 degrees) to 80 degrees, the parameter is 0 degrees. Similarly, if the posture is in the range of more than 80 degrees to less than 100 degrees, 90 degrees, 100 degrees to 260 degrees. If this is the case, it may be more convenient to assign 180 degrees or more than 260 degrees to less than 280 degrees or 270 degrees.

  Finally, the image processing engine 111 reads and combines the EXIF information stored in the RAM 114, the thumbnail image data, and the main image data, and records them as a JPEG file on the memory card using the card control unit 115. To do.

(Playback operation: Fig. 1)
Next, an operation for reproducing a captured image will be described.

  When the user sets the mode dial to the playback mode, the control program of the CPU 102 acquires the mode dial state from the key operation processing unit 105 and shifts to a mode for performing a playback operation. In the mode for performing the reproduction operation, the control program controls the image processing engine 111 as follows to perform the reproduction operation of the captured image recorded on the memory card.

  First, the image processing engine 111 reads a JPEG file from the memory card using the card control unit 115 and stores it in the RAM 114. Next, the control program causes the CPU 102 to analyze the EXIF information 151 (FIG. 2) located at the head of the JPEG file stored in the RAM 114 via the image processing engine 111. Here, the parameter 151a (for example, rotation angles: 0, 90, 180, and 270 degrees) indicating the vertical direction of the captured image is acquired together with the shutter speed and the aperture amount at the time of shooting. Also, the position of the main image data 152 of the JPEG file is acquired.

  Next, the image processing engine 111 reads the main image data 152 of the JPEG file stored in the RAM 114, and analyzes and holds the size information of the image included therein using a JPEG codec. Next, the control program acquires the size information held by the image processing engine 111, and the size information and a parameter 151a (for example, rotation angle: 0, 90, 180, 270 degrees) indicating the top / bottom direction of the captured image. ) To determine the rotation angle when the captured image is reproduced. Next, the control program sets the rotation angle of the captured image to be reproduced for the image processing engine 111. Next, the image processing engine 111 reads the main image data of the JPEG file stored in the RAM 114, decodes it according to the set rotation angle using the JPEG codec 113, and stores it in the RAM 114 as YCC data.

  Next, the image processing engine 111 applies the YCC data stored in the RAM 114 so that the entire captured image fits in the output data according to the image format (SD or HD) of the output unit of the image processing engine 111. Thus, the enlargement / reduction processing is performed and stored in the RAM 114 as output data. When the aspect ratio of the YCC data and the aspect ratio of the output data do not match, the enlargement / reduction processing is performed so that the entire captured image fits in the center of the output data, and black is displayed in the left and right or upper and lower margin areas. Fill in the data to be displayed.

  Finally, the image processing engine 111 reads out the output data stored in the RAM 114 and continuously outputs the output data to the video output unit 122 and the LCD output unit 123, so that the captured image reproduced on the external monitor, the LCD, or the like can be obtained. Is displayed.

  The image recording / reproducing apparatus 100 is provided with a cross key (not shown). When the user presses the cross key in a playback operation mode, the control program of the CPU 102 is pressed by the key operation processing unit 105. Get the status of. Thereby, the control program performs the reproduction operation for the next (or previous) photographed image in the memory card.

(Example of processing procedure of reproduction processing: FIG. 5)
In the above reproduction operation, the reproduction process of the present embodiment including the determination process in which the control program 104a determines the rotation angle will be described with reference to the flowchart of FIG.

  First, in step S201, when the parameter indicating the vertical direction (rotation angle) of the photographed image is 0 degree or 180 degrees (horizontal photographing), the process proceeds to step S204, and the angle indicated by the parameter is used as it is for the photographed image. As the rotation angle at the time of reproduction, rotation (S204), assignment to a display screen (S205), and display processing (S206) are performed. That is, a display process is performed in which the angle indicated by the parameter is 0 degrees, and when the angle is 180 degrees, the display is rotated 180 degrees. On the other hand, when the parameter indicating the vertical direction (rotation angle) of the photographed image is 90 degrees or 270 degrees (vertical photographing), the process proceeds to step S202-1.

  In step S 202-1, the horizontal size w and vertical size h of the captured image (the horizontal direction and vertical direction here do not depend on the vertical direction at the time of shooting, but the horizontal size in the JPEG main image data). If the calculated aspect ratio w / h is smaller than the threshold value 14/9 (for example, 4/3), the step is calculated. Proceeding to S204, the angle indicated by the parameter is used as the rotation angle when the captured image is reproduced as it is. That is, display processing that rotates 90 degrees or 270 degrees is performed under these conditions. For example, when the image is 4: 3, the image is displayed rotated by 90 degrees or 270 degrees. On the other hand, when the calculated aspect ratio w / h is larger than 14/9 (for example, 16/9) in step S202-1, the process proceeds to step S203.

  In step S203, the parameter indicating the top-to-bottom direction at the time of shooting is replaced with 0 degree, that is, the rotation angle is set to 0 degree, and the process proceeds to step S204, where rotation (S204), allocation to the display screen (S205), and display processing (S206) are performed. ). That is, the display processing that rotates under this condition is invalidated. For example, in the case of a 16: 9 image, even a vertically shot image is not rotated and displayed.

  In FIG. 5, 14/9 is used as the threshold value, but this is set between a general aspect ratio of 4/3 and 16/9. It will be apparent to those skilled in the art that the aspect ratio of the screen changes as the aspect ratio changes and is not limited to 14/9, and the object of the present invention can be achieved even with different values.

<Examples of Images Displayed on Display Unit of this Embodiment: FIGS. 6 to 9>
Examples of images displayed on the display unit in the image recording / reproducing apparatus of the first embodiment are shown in FIGS. That is, when the aspect ratio of the display unit is 16: 9 and the aspect ratio of the vertically shot captured image is 16: 9, the image is reproduced without being rotated as shown in FIG. This is reproduced without rotation because the displayed image is small and difficult for the user to see when rotated. On the other hand, when the aspect ratio of the display unit is 16: 9 and the aspect ratio of the vertically shot captured image is 4: 3, the image is rotated and reproduced as shown in FIG. This is because the user can easily see the image displayed when the image is rotated and displayed (the display area of the display unit is 42%), and the image is rotated and reproduced.

  Similarly, when the aspect ratio of the display unit is 4: 3 and the aspect ratio of the vertically shot captured image is 16: 9, the image is reproduced without being rotated as shown in FIG. This is reproduced without rotation because the displayed image is small and difficult for the user to see when rotated. On the other hand, when the aspect ratio of the display unit is 4: 3 and the aspect ratio of the vertically shot captured image is 4: 3, the image is rotated and reproduced as shown in FIG. This is because the user can easily view the image displayed when the image is rotated and displayed (the display area of the display unit is 56%), and the image is rotated and reproduced. Therefore, the user can always see the reproduced image adjusted for easy viewing.

[Second Embodiment]
<Features of Second Embodiment>
In the image recording / playback apparatus according to the second embodiment, when a vertically shot shot image is rotated and played back, the shot image is played back depending on the ratio of the shot image display area to the displayable area of the display unit. The rotation angle of the image displayed on the display unit is determined. With such a configuration, the user can always see the reproduced image adjusted for easy viewing. In the following description, the description of points common to the image recording / playback apparatus of the second embodiment and the image recording / playback apparatus of the first embodiment is omitted because it overlaps, and only different points will be described below.

<Configuration Example of Image Recording / Reproducing Device of the Present Embodiment: FIG. 1>
The configuration of the image recording / reproducing apparatus of this embodiment is the same as that of the first embodiment.

<Operation Example of Image Recording / Reproducing Device of this Embodiment: FIG. 10>
Since the image recording of this embodiment is the same as that of the first embodiment, a procedure example of image reproduction will be described below.

  In an image recording / playback apparatus, when a vertically shot shot image is rotated and played back with respect to the aspect ratio (16: 9 or 4: 3) of the display unit and the aspect ratio (16: 9 or 4: 3) of the shot image The occupation ratio of the display area to the display screen is as shown in the table of FIG. In the second embodiment, 36.2% (81/224 = (9 × 9) / (16 × 14) where the display occupation ratio on the reproduction screen in the table of FIG. 11 is between 32% and 42%. ) Is set as a threshold value for determining whether to perform rotation / allocation. Such a threshold value is not limited to this example. It can be changed by changing the aspect ratio of the captured image or the display screen.

(Example of playback processing procedure: FIG. 10)
Discrimination processing in which the control program of the image recording / playback apparatus of the second embodiment discriminates the rotation angle in the playback operation similar to that described in the image recording / playback apparatus of the first embodiment with reference to FIGS. A reproduction process including the above will be described with reference to the flowchart of FIG.

  First, in step S201, when the parameter indicating the vertical direction (rotation angle) of the photographed image is 0 degree or 180 degrees (horizontal photographing), the process proceeds to step S204, and the angle indicated by the parameter is used as it is for the photographed image. As the rotation angle at the time of reproduction, rotation (S204), assignment to a display screen (S205), and display processing (S206) are performed. That is, a display process is performed in which the angle indicated by the parameter is 0 degrees, and when the angle is 180 degrees, the display is rotated 180 degrees. On the other hand, if the parameter indicating the vertical direction (rotation angle) of the captured image is 90 degrees or 270 degrees (vertical shooting), the process proceeds to step S202-2.

  Next, in step S202-2, the horizontal size w and the vertical size h of the captured image (the horizontal direction and the vertical direction here are not in the vertical direction at the time of reproduction, but in a normal posture). Indicates the horizontal and vertical directions, w> h), the horizontal size W of the display unit, and the vertical size H (the horizontal and vertical directions here are not close to the vertical direction at the time of playback) The ratio Hh / Ww of the display area with respect to the display screen when the captured image is rotated and reproduced is obtained, and the calculated display is obtained. When the area ratio Hh / Ww is larger than a threshold value, for example, 81/224 (= 36.2%), for example, 27/264 (= about 42%), the process proceeds to step S204, and the parameter Re-shoot the above image as it is. The rotation angle of when. That is, display processing that rotates 90 degrees or 270 degrees is performed under these conditions. On the other hand, if the calculated display area ratio Hh / Ww is smaller than the threshold value in step S202-2, for example, 81/256 (= about 32%), the process proceeds to step S203.

  In step S203, the parameter indicating the top-to-bottom direction at the time of shooting is replaced with 0 degree, that is, the rotation angle is set to 0 degree, and the process proceeds to step S204, where rotation (S204), allocation to the display screen (S205), and display processing (S206) are performed. ). That is, the display processing that rotates under this condition is invalidated.

<Example of Image Displayed on Display Unit of Present Embodiment: FIG. 12>
The image recording / reproducing apparatus according to the second embodiment, when compared with the image reproducing apparatus according to the first embodiment, is a captured image that is vertically shot at an aspect ratio of 16: 9 when the aspect ratio of the display unit is 4: 3. Is reproduced in a rotated manner as shown in FIG. 12, and an easy-to-view screen for the user can be displayed more accurately by considering the occupation ratio on the display screen.

[Third Embodiment]
<Characteristics of Third Embodiment>
The image recording / playback apparatus according to the third embodiment determines whether or not to rotate the vertically shot captured image by paying attention to the size of the playback image directly on the playback screen. With this configuration, for example, when the image reproduction apparatus is not a known fixed apparatus but can be connected in various ways, an image that is easy to view for the user can be provided more reliably. In the following description, the description of points common to the image recording / playback apparatuses of the third embodiment and the image recording / playback apparatuses of the first and second embodiments will be omitted, and will be omitted. To do.

<Configuration Example of Image Recording / Reproducing Device of the Present Embodiment: FIG. 1>
The configuration of the image recording / reproducing apparatus of this embodiment is the same as that of the first embodiment.

<Operation Example of Image Recording / Reproducing Device of this Embodiment: FIGS. 1 and 13>
(Example of playback processing procedure: FIG. 13)
The reproduction process of this embodiment including the determination process in which the control program 104a determines the rotation angle will be described with reference to the flowchart of FIG.

  First, in step S201, when the parameter indicating the vertical direction (rotation angle) of the photographed image is 0 degree or 180 degrees (horizontal photographing), the process proceeds to step S204, and the angle indicated by the parameter is used as it is for the photographed image. As the rotation angle at the time of reproduction, rotation (S204), assignment to a display screen (S205), and display processing (S206) are performed. That is, a display process is performed in which the angle indicated by the parameter is 0 degrees, and when the angle is 180 degrees, the display is rotated 180 degrees. On the other hand, when the parameter indicating the vertical direction (rotation angle) of the captured image is 90 degrees or 270 degrees (vertical shooting), the process proceeds to step S202-3.

  Next, in step S202-3, the horizontal size w and the vertical size h of the photographed image (the horizontal direction and the vertical direction here are not in the vertical direction at the time of reproduction, but in a normal posture). Indicates the horizontal and vertical directions, w> h), the horizontal size W of the display unit, and the vertical size H (the horizontal and vertical directions here are not close to the vertical direction at the time of playback) , Indicating the horizontal and vertical directions in a normal posture, W> H), and calculating the size of the image displayed on the display screen when the captured image is rotated and played back, by the area If the displayed area is larger than a predetermined size α which is a threshold value, for example, VGA (640 × 480) size, the process proceeds to step S204, where the angle indicated by the parameter is used as it is when the captured image is reproduced. The rotation angle. That is, display processing that rotates 90 degrees or 270 degrees is performed under these conditions. On the other hand, if the calculated display area is smaller than the predetermined size serving as the threshold value in step S202-3, the process proceeds to step S203.

  In step S203, the parameter indicating the top-to-bottom direction at the time of shooting is replaced with 0 degree, that is, the rotation angle is set to 0 degree, and the process proceeds to step S204, where rotation (S204), allocation to the display screen (S205), and display processing (S206) are performed. ). That is, the display processing that rotates under this condition is invalidated.

  Such a threshold value is not limited to this example. It can be changed depending on the reproduction environment such as the distance to the reproduction screen or the change of the user viewing the reproduction screen. In the above example, the display area is taken into consideration as the size of the display image. For example, various criteria for determining the size of the display image can be used as long as it is a criterion for easy viewing by the user, such as the length of a short side or the length of a diagonal line of the display image. Various standards can be considered and these are also included in the present invention. In the present embodiment, the reference example of whether or not to rotate the VGA (640 × 480) size is used. However, instead of using such a fixed value as a reference, for example, the screen size of the display device, the display screen, etc. The criteria for whether or not to rotate the display is taken into account, including the distance that the display device is viewed (that is, whether it is for personal use or a large screen for viewing by many people), and whether the display device can be rotated freely. You may make it provide. The present invention has been made by recognizing that it is not always easy for a user to see rotation by only the direction at the time of shooting, and the technical idea of controlling the rotation by predicting the display state is provided. These modifications are also included in the present invention.

[Fourth Embodiment]
<Features of Fourth Embodiment>
The image recording / playback apparatus according to the fourth embodiment is configured to display a shot image at different magnifications by using a margin part of the displayable area when rotating and playing back a shot image taken vertically. In the following description, the description of points common to the image recording / playback apparatuses of the fourth embodiment and the image recording / playback apparatuses of the first to third embodiments will be omitted, and will be omitted. To do.

<Configuration Example of Image Recording / Reproducing Device of the Present Embodiment: FIG. 1>
The configuration of the image recording / reproducing apparatus of this embodiment is the same as that of the first embodiment.

<Example of Operation of Image Recording / Reproducing Apparatus of this Embodiment: FIGS. 1 and 14>
(Playback operation: Fig. 1)
A reproduction operation of a captured image will be described. When the user sets the mode dial to the playback mode, the control program of the CPU 102 acquires the mode dial state from the key operation processing unit 105 and shifts to a mode for performing a playback operation. In the mode for performing the reproduction operation, the control program controls the image processing engine 111 as follows to perform the reproduction operation of the captured image recorded on the memory card.

  First, the image processing engine 111 reads a JPEG file from the memory card using the card control unit 115 and stores it in the RAM 114. Next, the control program causes the CPU 102 to analyze the EXIF information 151 (FIG. 2) located at the head of the JPEG file stored in the RAM 114 via the image processing engine 111. Here, the parameter 151a (for example, rotation angles: 0, 90, 180, and 270 degrees) indicating the vertical direction of the captured image is acquired together with the shutter speed and the aperture amount at the time of shooting. Also, the position of the main image data of the JPEG file is acquired.

  Next, the image processing engine 111 reads the main image data of the JPEG file stored in the RAM 114, and analyzes and holds the size information of the image included therein using the JPEG codec. Next, the control program discriminates the rotation angle at the time of reproducing the captured image, based on the parameter 151a (for example, the rotation angle: 0, 90, 180, 270 degrees) indicating the vertical direction of the captured image. Next, the control program sets the rotation angle of the captured image to be reproduced for the image processing engine 111. Next, the image processing engine 111 reads the main image data of the JPEG file stored in the RAM 114, decodes it according to the set rotation angle using the JPEG codec 113, and stores it in the RAM 114 as YCC data.

Next, the control program selects enlargement / reduction processing so that the entire image fits in the output data according to the image format (SD or HD) of the output unit of the image processing engine 111. Next, the image processing engine 111 follows the enlargement / reduction processing selected by the control program so that the entire photographed image fits in the output data according to the format (SD or HD) of the output unit of the image processing engine 111. The YCC data stored in the RAM 114 is enlarged / reduced and stored in the RAM 114 as output data. Finally, the image processing engine 111 reads out the output data stored in the RAM 114 and continuously outputs the output data to the video output unit 122 and the LCD output unit 123, so that the captured image reproduced on the external monitor, the LCD, or the like can be obtained. Is displayed.

(Example of playback processing procedure: FIG. 14)
In the above reproduction operation, a reproduction process including a process in which the control program selects the enlargement / reduction process will be described with reference to the flowchart of FIG.

  First, in step S301, when the parameter indicating the vertical direction (rotation angle) of the captured image is 0 degree or 180 degrees (horizontal shooting), a first enlargement / reduction process is performed in step S302. In the first enlargement / reduction process, when the aspect ratio of the YCC data and the aspect ratio of the output data match after rotation (0 degree or 180 degrees) corresponding to the top-to-bottom direction, the entire output data is captured. If the aspect ratio does not match, the image is enlarged or reduced so that the entire captured image fits in the center of the output data, and black is displayed in the left and right or upper and lower margin areas. This is the enlargement / reduction processing shown in the first to third embodiments.

  On the other hand, when the parameter indicating the rotation angle of the captured image is 90 degrees or 270 degrees (vertical shooting) in step S301, the process proceeds to step S303, and the second enlargement / reduction process is performed. In the second enlargement / reduction processing, after rotation (90 degrees or 270 degrees) corresponding to the vertical direction, enlargement / reduction is performed so that the whole of the photographed image fits in the left end of the output data, and the right margin area is displayed. This is enlargement / reduction processing for enlarging / reducing the trimming area provided in the photographed image so as to fit in the entire margin area.

<Example of Second Enlargement / Reduction Processing of Embodiment and Example of Image Displayed on Display Unit: FIGS. 15 to 18>
Next, an example of the second enlargement / reduction process will be described.

  For example, when the aspect ratio of the display unit is 16: 9 and the aspect ratio of the captured image is 4: 3, the captured image is displayed on the display unit as shown in FIG. 16 as a result of the second enlargement / reduction process. The Note that the horizontal size of the captured image is w, and the vertical size is h (the horizontal direction and the vertical direction here do not depend on the vertical direction at the time of shooting, but the horizontal direction and vertical direction in the JPEG main image data. The horizontal size of the display unit is W, and the vertical size is H (the horizontal and vertical directions here are not close to the vertical direction at the time of reproduction). , The horizontal and vertical directions in a normal posture are shown, and W> H). The size of the image 1100 rotated and displayed on the display unit is H in the vertical direction and h (H / w) in the horizontal direction.

  The size of the margin area 1102 of the display unit is H in the vertical direction and Wh (H / w) in the horizontal direction, and the trimming area 1101 positioned at the center of the image 1100 is doubled in the margin area 1102 (image 1100). The enlarged image is displayed with respect to the display magnification. Therefore, the size of the trimming area is H / 2 in the vertical direction and (Wh (H / w)) / 2 in the horizontal direction. The trimming area 1101 in the image 1100 informs the user which area is enlarged and displayed by blinking the frame line. Since the frame line is blinking, it is possible to confirm what is displayed below the frame line even when the trimming frame is fixed in one place.

  Similarly, when the aspect ratio of the display unit is 4: 3 and the aspect ratio of the captured image is 16: 9, the display is as shown in FIG. Similarly, when the aspect ratio of the display unit is 4: 3 and the aspect ratio of the captured image is 4: 3, the display is as shown in FIG.

  On the other hand, when the aspect ratio of the captured image is 16: 9 and the aspect ratio of the display unit is 16: 9, the horizontal size of the trimming area 1101 exceeds the horizontal size of the image 1100 in the above processing. Therefore, the horizontal size of the trimming region 1101 is limited to h (H / w) as in the image 1100, and the enlarged image of the trimming region 1101 is set to 2h (H / w) in the center of the margin region 1102. ) As shown in FIG.

[Other Embodiments]
Note that the object of the present invention is to provide an image reproducing apparatus configured not to include the imaging device such as the CCD 107 shown in FIG. 1 but to input a photographed image from the outside via a communication means or a storage medium and perform the reproducing operation. Needless to say, even if it is realized. Further, the object of the present invention is realized by software processing using a computer, regardless of which part is realized by a hardware circuit in the functional blocks and flowcharts shown in FIGS. 1, 5, 10, 13, and 14. Needless to say, this is achieved.

  Also, an object of the present invention is to supply a recording medium (or storage medium) that records a program code of software that realizes the functions of the above-described embodiments to a system or apparatus, and to computer (or CPU or CPU) of the system or apparatus. Needless to say, this can also be achieved when the MPU) reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, after the program code read from the recording medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  Also, program data for realizing the functions of the above-described embodiment is downloaded from the CD-ROM set in the own device or an external supply source such as the Internet to the memory of the own device, and A form in which the function is realized is also included in the present invention.

  When the present invention is applied to the above recording medium, it is preferable that the recording medium stores a program code corresponding to the processing procedure including the flowchart described above.

Claims (8)

An image processing apparatus for displaying an image in a horizontally long display area,
  A discriminating means for discriminating whether to display the image horizontally or vertically with respect to the display area;
  When it is determined by the determining means that the image is displayed in the display area in landscape orientation, the entire image is enlarged and reduced so that the image is displayed in landscape orientation in the display area. When it is determined that the image is displayed in the display area in the portrait orientation, the entire image is rotated and enlarged / reduced to be displayed in the portrait orientation in the display area, and the image is preset. An image processing apparatus comprising: a display control unit configured to control a portion to be displayed in an area where the entire image is not displayed in the display area. The image processing apparatus according to claim 1, wherein the determination unit performs the determination based on information on a posture state of the image capturing apparatus at the time of capturing the image, which is added to the image. The posture state information indicates the posture state of the imaging apparatus by information indicating a rotation of 0 degrees, 90 degrees, 180 degrees, or 270 degrees, and the determination unit includes the posture state information added to the image. If the image indicates a rotation of 0 degrees or 180 degrees, it is determined that the image is displayed in the display area in a landscape orientation, and the posture state information added to the image indicates a rotation of 90 degrees or 270 degrees. The image processing apparatus according to claim 1, wherein the image processing apparatus determines that the image is displayed in the display area in a portrait orientation. The display control means displays the preset part of the image in the display area in the portrait orientation when the discrimination means determines that the image is displayed in the display area in the portrait orientation. 4. The image processing apparatus according to claim 1, wherein the image processing apparatus is controlled to display in the area at a magnification larger than the magnification of 4. Imaging means;
  A recording unit that records an image captured using the imaging unit on a recording medium with information on a posture state of the imaging unit at the time of capturing the image;
The image processing apparatus according to claim 1, further comprising: A control method of an image processing apparatus for displaying an image in a horizontally long display area,
  A determination step of determining whether the determination means displays the image horizontally or vertically with respect to the display area;
  When it is determined that the image is displayed in the display area in the display area in the determination step, the display control means displays the entire image in an enlarged and reduced manner so that the image is displayed in the display area in a horizontal direction. When it is determined that the image is displayed in the display area in the display area in the determination step, the entire image is rotated and enlarged and displayed so as to be displayed in the display area in the portrait direction, and the image And a display control step for controlling the predetermined portion of the display area to be displayed in an area in which the entire image is not displayed in the display area. The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 thru | or 4. A computer-readable recording medium on which the program according to claim 7 is recorded.

Images