JP7150053B2 - IMAGING DEVICE, IMAGING METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to an image capturing apparatus, an image capturing method, and a program for capturing moving images, and more particularly to an image capturing apparatus, an image capturing method, and a program capable of extracting frames constituting a moving image as still images.

2. Description of the Related Art Techniques for capturing moving images for the purpose of extracting still images after capturing are known. With this technology, for example, a moving image is captured and recorded in order to capture an instantaneous event that cannot be predicted at any time, and the user selects a desired frame after capturing. is difficult to predict. For this reason, it is necessary to continue shooting moving images for a certain amount of time, and there are problems such as the remaining capacity of the recording medium becoming scarce and the user having to select desired frames from a long moving image requiring a great deal of effort. be.

Therefore, techniques for dealing with the above problems have been proposed. For example, in Patent Literature 1, a moving image is marked at the timing of obtaining a still image according to a user's instruction during recording of the moving image, and after the moving image recording is completed, the marked portion and the moving image frames before and after the marking are extracted and displayed, thereby enabling the user to view the moving image. It is described that the timing of still image extraction can be selected.

JP 2016-32303 A

However, since the technique described in Patent Document 1 described above only records marking information, image file search processing is required each time a marked frame is extracted and displayed, and the processing load on the imaging device is high. In addition, since the technique described in Patent Document 1 performs processing on the entire captured moving image, it takes time and effort to select frames to be extracted, and this problem becomes significant when the moving image capturing time is long. Furthermore, in Japanese Patent Application Laid-Open No. 2002-200010, no consideration is given to a change in composition while shooting a moving image for extracting still images. For this reason, the conventional technology does not allow the user to quickly and easily select a moving image frame with a desired composition when extracting a still image from a moving image.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and provides an imaging apparatus, an imaging method, and an imaging apparatus that enable a user to quickly and easily select a moving image frame with a desired composition when extracting a still image from a moving image. The purpose is to provide a program.

In order to achieve the above object, an imaging device according to a first aspect of the present invention is an imaging device comprising: a direction detection unit for detecting the direction of the imaging device with respect to the direction of gravity; an imaging unit for capturing moving images; a control unit, wherein the control unit generates a moving image file of a moving image captured by the imaging unit, and has a first moving image mode capable of extracting a still image from the moving image file; , when the direction detection unit determines that the direction of the imaging device has changed with respect to the direction of gravity, the moving image is divided to generate a first moving image and a second moving image. The second moving image files of the two moving images are stored in the same folder in the storage unit.

Normally, in still image shooting, the user sets the camera (imaging device) horizontally (the bottom surface of the camera is parallel to the ground) or vertically (the bottom surface of the camera is parallel to the direction of gravity) depending on the desired composition. Hold the camera like this and shoot. In video shooting, on the other hand, it is rare to switch between vertical and horizontal orientations during shooting. On the other hand, in moving image shooting for extracting still images, it is assumed that the way the camera is held changes between vertical and horizontal during moving image shooting due to the characteristic that frames are cut out after the fact. From this point of view, in the first aspect, when a change in direction with respect to the direction of gravity (a change in the attitude of the imaging device) occurs, the video is divided to generate the first video and the second video, and the first video and the second video are generated. The first moving image file and the second moving image file of the second moving image are saved in the same folder in the storage unit. Since the first moving image and the second moving image are separate files in this "same folder", it is possible to easily identify the posture of the imaging device when each moving image was shot, and the user can select frames with a desired composition. It is possible to quickly and easily select a moving image including the moving image by using the attitude of the imaging device as a clue.

Also, by saving the first moving image file and the second moving image file in the same folder, consecutive scenes can be saved in the same folder, and the scenes can be narrowed down only from the file information in the folder. . Therefore, it does not take a long time to search for moving image files. Also, if the orientation is switched in the middle of the moving image file, the display direction must be switched during file decoding, and the processing time becomes discontinuous, and the display does not switch smoothly. Since the video is split when it changes, the vertical and horizontal attributes can be known when the file is opened, and the display can be updated at regular intervals from the beginning of the file.

In addition, since the frame to be extracted as a still image can be found in the divided moving image whose recording time is shortened, the desired frame can be quickly found. Thus, according to the first aspect, when extracting a still image from a moving image, the user can quickly and easily select a moving image frame with a desired composition. Furthermore, since it is not necessary to use information such as tags and markings in the first mode, frame selection and still image extraction based on moving images recorded in the first mode can be performed by other imaging devices or image reproduction. It can also be performed with an apparatus or an image processing apparatus. Note that the storage unit that stores the moving image file may be provided in the imaging device, or may be an external device.

In the first aspect of the imaging device according to the second aspect, the control unit stores the first moving image file and the second moving image file based on information regarding the direction of the imaging device with respect to the direction of gravity. For example, it is possible to add information to a file name or a part thereof, a header or footer of a moving image file, and save the information.

In the imaging device according to the third aspect, in the first or second aspect, the control unit generates a third moving image that is between the first moving image and the second moving image in time, and the third moving image is the direction change Includes videos of the first period when the In the third aspect, the period during which the direction change occurs (first period) is set to a moving image (third moving image) different from the first and second moving images, so that the user can easily see the period during which the direction change occurs. making it identifiable.

In the third aspect of the imaging device according to the fourth aspect, the control unit does not store the third moving image in the storage unit. According to the fourth aspect, it is possible to save the capacity of the storage unit by not saving the moving image during the period in which the direction change occurs (the first period).

In the third aspect of the imaging device according to the fifth aspect, the control unit saves the third moving image in a folder different from the same folder. In the fifth aspect, by storing the moving image (third moving image) during the period in which the direction change occurs in a folder different from the folder for saving the first and second moving images, the first and second moving images and the third moving image The user can easily identify the moving image.

In any one of the third to fifth aspects, the imaging device according to the sixth aspect includes the third moving image including the moving image of the second period after the direction change is completed. Since there is a possibility that the shooting conditions may not be stable due to the change in composition even after the change in direction is completed, in the sixth aspect, even after the end of the change in direction is included in the third moving image in the second period. .

An imaging device according to a seventh aspect is the imaging device according to the sixth aspect, wherein the second period is a period during which the imaging device adjusts the exposure of the moving image, the white balance of the moving image, or the focus of the subject in the moving image. A seventh aspect specifically defines the content of the second period.

According to an eighth aspect, in any one of the first to seventh aspects, the first moving image file and the second moving image file share at least a part except for the extension of the file name. Thereby, the user can easily identify that the first moving image file and the second moving image file are files generated by dividing one moving image.

An imaging device according to a ninth aspect is any one of the first to eighth aspects, wherein the control unit has a second moving image mode different in imaging conditions from the first moving image mode, In moving image mode, when the direction detection unit detects a direction change, the moving image is not divided. For example, a normal moving image mode that does not presuppose extraction of still images from a moving image can be set as the second moving image mode.

In the imaging apparatus according to the tenth aspect, in the ninth aspect, the first moving image mode has at least one of a shutter speed, an autofocus speed, an automatic exposure follow-up speed, and a white balance follow-up speed with respect to the second moving image mode. One is set to fast and/or the frame rate is set to high for the second video mode. The tenth aspect defines one aspect of the difference in shooting conditions between the first moving image mode and the second moving image mode. High-quality still images can be extracted.

To achieve the above object, an imaging device according to an eleventh aspect of the present invention is an imaging device comprising a direction detection unit for detecting the direction of the imaging device with respect to the direction of gravity, an imaging unit for capturing moving images, and a control unit. a control unit having a first moving image mode capable of extracting a still image from a moving image file of a moving image captured by the imaging unit, and a second moving image mode having different shooting conditions from the first moving image mode, and controlling In the first moving image mode, the unit divides the moving image to generate a first moving image and a second moving image when the direction detecting unit determines that the direction of the imaging device has changed with respect to the gravity direction, and the control unit generates the second moving image. In moving image mode, when the direction detection unit determines that a direction change has occurred, the moving image is not divided. In the eleventh aspect, in the first moving image mode, the moving image is divided to generate the first moving image and the second moving image when a direction change with respect to the gravity direction (a change in the attitude of the imaging device) occurs. This makes it possible to easily identify the posture of the imaging device when each moving image is taken, and the user can easily select a moving image including a frame to be extracted as a still image by using the posture of the imaging device as a clue. Note that, for example, a normal moving image mode that does not presuppose extraction of still images from a moving image can be set as the second moving image mode.

An imaging device according to a twelfth aspect is the eleventh aspect, wherein the control unit converts the first moving image file of the first moving image and the second moving image file of the second moving image based on information about the direction of the imaging device with respect to the direction of gravity. save. For example, it is possible to add information to a file name or a part thereof, a header or footer of a moving image file, and save the information.

In the eleventh or twelfth aspect of the imaging device according to the thirteenth aspect, the control unit generates a third moving image that is between the first moving image and the second moving image in terms of time, and the third moving image changes direction. Includes a video of the first period in which is occurring. In the thirteenth aspect, the period during which the direction change occurs (first period) is a moving image (third moving image) different from the first and second moving images so that the user can easily identify it.

In the thirteenth aspect of the imaging device according to the fourteenth aspect, the control unit does not save the third moving image. According to the fourteenth aspect, it is possible to save the capacity of the storage unit by not saving the moving image during the period in which the direction change occurs (the first period).

In the imaging apparatus according to the fifteenth aspect, in the thirteenth aspect, the control unit stores the third moving image in a folder different from the folder storing the first moving image and the folder storing the second moving image. In the fifteenth aspect, by storing the moving image (third moving image) during the period in which the direction change occurs in a folder different from the folder for storing the first and second moving images, the first and second moving images and the third moving image The user can easily identify the moving image.

In the thirteenth or fourteenth aspect, the imaging device according to the sixteenth aspect is such that the third moving image also includes the moving image of the second period after the direction change is completed. In the sixteenth aspect, even after the change in direction is completed, the shooting conditions may not be stable due to the change in composition, so even after the change in direction is completed, it is included in the third moving image in the second period. .

An imaging device according to a seventeenth aspect is the imaging device according to the sixteenth aspect, wherein the second period is a period during which the imaging device adjusts the exposure of the moving image, the white balance of the moving image, or the focus of the subject in the moving image. The seventeenth aspect specifically defines the content of the second period.

An imaging device according to an eighteenth aspect is the imaging device according to any one of the eleventh to seventeenth aspects, wherein the file names of the first moving image file of the first moving image and the second moving image file of the second moving image are at least partly common. is doing. Thereby, the user can easily identify that the first moving image file and the second moving image file are files generated by dividing one moving image.

In the imaging device according to the nineteenth aspect, in the second or twelfth aspect, the control unit adds information about the direction of the imaging device with respect to the direction of gravity to the first moving image file and the second moving image file.

To achieve the above object, an imaging method according to a twentieth aspect of the present invention includes a direction detection unit for detecting the direction of an imaging device with respect to the direction of gravity, an imaging unit for capturing a moving image, and a moving image captured by the imaging unit. and a control unit having a first moving image mode capable of extracting a still image from a moving image file, wherein in the first moving image mode, the orientation detection unit changes the orientation of the imaging device with respect to the gravity direction. a generation step of dividing the moving image to generate a first moving image and a second moving image when the direction detecting unit determines that a direction change has occurred; a generating step of generating a first moving image and a second moving image; and a saving step of saving the moving image file and the second moving image file of the second moving image in the same folder of the storage unit. According to the twentieth aspect, as in the first aspect, when a change in direction with respect to the direction of gravity (a change in the posture of the imaging device) occurs in the first moving image mode, the moving image is divided into the first moving image and the second moving image. Generate videos. This makes it possible to easily identify the posture of the imaging device when each moving image is taken, and when the user extracts a still image from the moving image, the user can quickly and easily select a moving image frame with a desired composition. Note that the twentieth aspect may further include a configuration similar to that of the second to tenth aspects.

To achieve the above object, an imaging method according to a twenty-first aspect of the present invention includes a direction detection unit for detecting the direction of an imaging device with respect to the direction of gravity, an imaging unit for capturing a moving image, and a moving image captured by the imaging unit. a first moving image mode capable of extracting a still image from a moving image file and a control unit having a second moving image mode having different shooting conditions from the first moving image mode, wherein in the first moving image mode, a determination step in which the direction detection unit determines whether or not a direction change with respect to the direction of gravity of the imaging device has occurred; generating a first moving image and a second moving image, and not dividing the moving image in the second moving image mode. According to the twenty-first aspect, as in the eleventh aspect, when a change in direction with respect to the direction of gravity (a change in the posture of the imaging device) occurs in the first moving image mode, the moving image is divided into the first moving image and the second moving image. Generate videos. This makes it possible to easily identify the posture of the imaging device when each moving image is shot, and when the user extracts a still image from the moving image, the user can quickly and easily select a moving image frame with a desired composition. For example, a normal moving image mode in which still images are not extracted from a moving image can be set as the second moving image mode. The twenty-first aspect may further include the same configuration as the twelfth to eighteenth aspects.

To achieve the above object, a program according to a twenty-second aspect of the present invention includes: a direction detection unit for detecting the direction of an imaging device with respect to the direction of gravity; A program for use in an image capturing apparatus comprising a control unit having a first moving image mode capable of extracting a still image from a file, wherein in the first moving image mode, whether or not the orientation of the image capturing unit changes with respect to the direction of gravity of the image capturing unit. If the direction detection unit determines that a direction change has occurred in the first moving image mode, the control unit is caused to divide the moving image to generate a first moving image and a second moving image, and the control unit causes the first moving image to be generated. The first moving image file of the moving image and the second moving image file of the second moving image are stored in the same folder of the storage unit. The effects obtained by the twenty-first aspect are the same as those of the twentieth aspect. Note that the twenty-second aspect may further include the same configuration as those of the second to tenth aspects. In addition, a non-transitory recording medium in which computer-readable codes of programs of these aspects are recorded can also be cited as an aspect of the present invention.

To achieve the above object, a program according to a twenty-third aspect of the present invention includes: a direction detection unit for detecting the direction of an imaging device with respect to the direction of gravity; A program for use in an imaging device comprising a first moving image mode capable of extracting a still image from a file and a control unit having a second moving image mode in which shooting conditions are different from the first moving image mode, wherein in the first moving image mode, a direction causing the detection unit to determine whether or not a direction change has occurred with respect to the gravity direction of the imaging device, and if the direction detection unit determines that a direction change has occurred in the first moving image mode, causes the control unit to divide the moving image into the first moving image; and the second moving image is generated, and the moving image is not divided when the direction detection unit determines that the direction change has occurred in the second moving image mode. The effect obtained by the twenty-third aspect is the same as that of the twenty-first aspect. Note that the twenty-third aspect may further include a configuration similar to that of the twelfth to eighteenth aspects. In addition, a non-transitory recording medium in which computer-readable codes of programs of these aspects are recorded can also be cited as an aspect of the present invention.

As described above, according to the image capturing apparatus, image capturing method, and program of the present invention, the user can quickly and easily select moving image frames with a desired composition when extracting still images from moving images.

FIG. 1 is a diagram showing the configuration of a camera according to the first embodiment. FIG. 2 is a diagram showing the functional configuration of the image processing apparatus. FIG. 3 is a diagram showing the coordinate system of the camera. FIG. 4 is a diagram illustrating an example of posture determination and moving image division. FIG. 5 is a flow chart showing the processing of the imaging method. FIG. 6 is a diagram showing how a moving image is divided and recorded. FIG. 7 is a diagram showing changes in camera posture and how moving images are divided and saved. FIG. 8 is a diagram showing an example of a folder structure. FIG. 9 is another diagram showing an example of the folder structure. FIG. 10 is a diagram showing an example of icon display of a moving image file. FIG. 11 is a diagram showing a state in which thumbnail images are added to icons of moving image files. FIG. 12 is a diagram showing an example of scene narrowing-down display based on file information. FIG. 13 is another flow chart showing the processing of the imaging method. FIG. 14 is a diagram illustrating another example of posture determination and moving image division. FIG. 15 is a diagram showing still another example of posture determination and moving image division. FIG. 16 is a diagram showing still another example of posture determination and moving image division. FIG. 17 is a diagram showing still another example of posture determination and moving image division. FIG. 18 is still another flow chart showing the processing of the imaging method. FIG. 19 is still another flow chart showing the processing of the imaging method. FIG. 20 is a diagram showing how a moving image file is resized and played back. FIG. 21 is another diagram showing how a moving image file is resized and played back. FIG. 22 is a flow chart showing processing of moving image recording in the first mode and the second mode. FIG. 23 is a flowchart (a continuation of FIG. 22) showing the moving image recording process in the first mode and the second mode. FIG. 24 is a diagram showing an example of the folder structure regarding the first mode and the second mode. FIG. 25 is a diagram showing how a frame to be extracted as a still image is selected. FIG. 26 is an external view of a smartphone according to the second embodiment. FIG. 27 is a block diagram showing the configuration of the smartphone according to the second embodiment. FIG. 28 is a diagram illustrating the posture of the smartphone according to the second embodiment; FIG. 29 is a diagram illustrating how the orientation of the smartphone is determined based on the angle.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments for implementing an imaging apparatus, an imaging method, and a program according to the present invention will be described in detail below with reference to the accompanying drawings.

<First embodiment>

<Overall Configuration of Imaging Device>

FIG. 1 is a diagram showing the configuration of a camera 10 (imaging device) according to the first embodiment. The camera 10 is composed of an interchangeable lens 100 (imaging unit, imaging device) and an imaging device main body 200 (imaging unit, imaging device). form an image. The interchangeable lens 100 and the imaging device main body 200 can be attached and detached via a mount (not shown).

<Configuration of Interchangeable Lens>

The interchangeable lens 100 includes a zoom lens 110 , a focus lens 120 , an aperture 130 and a lens driver 140 . The lens driving unit 140 advances and retreats the zoom lens 110 and the focus lens 120 according to a command from the image processing device 240 (lens driving control unit 240F in FIG. 2) to perform zoom (optical zoom) adjustment and focus adjustment. Zoom adjustment and focus adjustment may be performed in response to a command from the image processing device 240, or may be performed in response to a zoom operation or focus operation (rotation of a zoom ring or focus ring (not shown), etc.) performed by the user. good. Also, the lens drive unit 140 controls the aperture 130 according to a command from the image processing device 240 to adjust the exposure. On the other hand, information such as the positions of the zoom lens 110 and the focus lens 120 and the opening degree of the diaphragm 130 is input to the image processing device 240 . Note that the interchangeable lens 100 has an optical axis L. As shown in FIG.

<Structure of imaging device body>

The imaging device main body 200 includes an imaging device 210 (imaging unit), an AFE 220 (AFE: Analog Front End, imaging unit), an A/D converter 230 (A/D: Analog to Digital, imaging unit), an image processing device 240, It includes an operation unit 250, a storage unit 260, a monitor 270, and an orientation sensor 280 (direction detection unit). The imaging device main body 200 may have a shutter (not shown) for blocking light to be transmitted through the imaging element 210 . The imaging element 210 has a light receiving surface in which a large number of light receiving elements are arranged in a matrix. Subject light transmitted through the zoom lens 110, the focus lens 120, and the diaphragm 130 forms an image on the light receiving surface of the imaging element 210, and is converted into an electric signal by each light receiving element. A color filter of R (red), G (green), or B (blue) is provided on the light receiving surface of the imaging device 210, and a color image of the subject can be acquired based on the signals of each color.

As the imaging element 210, various photoelectric conversion elements such as CMOS (Complementary Metal-Oxide Semiconductor) and CCD (Charge-Coupled Device) can be used. The AFE 220 removes noise and amplifies the analog image signal output from the imaging device 210, and the A/D converter 230 converts the captured analog image signal into a digital image signal having a gradation range.

<Configuration of image processing apparatus>

FIG. 2 is a diagram showing the functional configuration of the image processing device 240. As shown in FIG. The image processing device 240 includes an image acquisition section 240A, a direction detection section 240B (direction detection section), a control section 240C (control section), an event detection section 240D (event detection section), and a still image extraction section 240E (still image extraction section). , and a lens drive control section 240F (lens drive control section). Then, based on the digital image signal input from the A/D converter 230, the image processing device 240 shoots a moving image, generates a file, generates a still image file, processes a plurality of frames constituting a moving image, and extracts a still image. etc. are processed. Details of the processing by the image processing device 240 will be described later.

The functions of the image processing device 240 can be realized using various processors. Various processors include, for example, a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to realize various functions. The various processors described above also include programmable logic devices (PLDs) such as GPUs (Graphics Processing Units), which are processors specialized for image processing, and FPGAs (Field Programmable Gate Arrays). A programmable logic device is a processor whose circuitry can be changed after manufacture. Furthermore, a dedicated electric circuit, which is a processor having a circuit configuration specially designed for executing specific processing such as ASIC (Application Specific Integrated Circuit), is also included in the various processors described above.

When the processor or electric circuit described above executes software (program), the processor (computer) readable code of the software to be executed is stored in a non-temporary recording medium such as a ROM (Read Only Memory). The processor then references that software. Software stored in the non-temporary recording medium includes a program for executing the imaging method according to the present invention. Codes may be recorded in non-temporary recording media such as magneto-optical recording devices and semiconductor memories instead of ROMs. When processing using software, for example, RAM (Random Access Memory) is used as a temporary storage area, and data stored in EEPROM (not shown) can also be referred to. can.

The image processing device 240 includes a ROM 242 in addition to the above units. The ROM 242 stores computer-readable codes of programs necessary for photographing, recording, displaying images, etc. (including programs for executing the imaging method according to the present invention (the program according to the present invention)).

<Operation section>

The operation unit 250 includes a release button (not shown), operation buttons (for example, a cross button, a Quick button, an OK button, etc.), dials, switches, etc., and the user can set a shooting mode, instruct movie shooting, extract a still image, etc. Various operations can be performed. Note that the monitor 270 (touch panel type) may be used as the operation unit 250 .

<Memory unit>

A storage unit 260 (storage unit) is composed of non-temporary recording media such as various magneto-optical recording media and semiconductor memories, and their control circuits, and stores moving images, still images, and still images extracted from moving images. A recording medium that can be attached to and detached from the imaging apparatus main body 200 can be used. A captured image (moving image, still image) may be transmitted to and stored in an external recording medium (storage unit), for example, by wireless communication.

<Monitor and viewfinder>

The monitor 270 (display device) is configured by a touch panel type liquid crystal display panel, and can display moving images, still images, still image extraction frames, and the like. The monitor 270 can be arranged on the back side, the top side, or the like of the imaging device body 200 . Also, the camera 10 may have a viewfinder.

<Shooting mode>

The camera 10 can set any one of a still image shooting mode, a moving image shooting mode for still image extraction (first moving image mode), and a normal moving image shooting mode (second moving image mode) as a shooting mode. A still image shooting mode and a normal moving image shooting mode are modes similar to those of a normal digital camera. On the other hand, the movie shooting mode for extracting still images is a mode that allows you to extract still images from movie files. movies). Specifically, in the still image extraction video shooting mode, at least one of the shutter speed, autofocus speed, auto exposure tracking speed, and white balance tracking speed is set to a higher speed than in the normal video shooting mode, and/or the frame rate is set higher than the normal moving image shooting mode. Also, the resolution and frame rate are set to the maximum values that can be set by the camera 10 (eg, 4,000×2,000 pixels, 30 frames/second), and the color tone is also set on the premise of still image extraction. The upper limit of the ISO sensitivity is also set higher than in the normal movie shooting mode.

For example, the shutter speed is set to a value corresponding to the frame rate of the moving image to be recorded in the normal moving image shooting mode (1/30 second if the frame rate is 30 frames/second). On the other hand, in the moving image mode for still image extraction, the frame interval is set to be faster (for example, less than 1/30 second). In normal movie shooting mode, the shutter speed is set to a value corresponding to the frame rate of the movie so that the movie can be reproduced smoothly. For this reason, in the movie shooting mode for extracting still images, the shutter speed is set faster than in the normal movie shooting mode (faster than the frame interval). becomes possible.

Similarly, by raising the upper limit of ISO sensitivity (ISO: International Organization for Standardization), the shutter speed can be increased, thereby extracting a still image with less blurring. Further, the speed of autofocus, the follow-up speed of automatic exposure, the follow-up speed of auto white balance, etc. are set faster than in the normal moving image shooting mode. As a result, it is possible to obtain many frames in which the subject is in focus, frames with appropriate exposure, and the like. As for the frame rate, by setting a high rate, the frame interval of moving images becomes shorter, and the number of frames that can be extracted as still images increases.

According to the moving image shooting mode for still image extraction described above, the frames forming the moving image can be extracted as still images afterward. Therefore, the user can easily take a photograph of an event (natural phenomenon, accident, happening, etc.) whose occurrence is unknown at any time, a photograph of a subject whose state changes with the passage of time, or a photograph of an instantaneous state of a moving subject. be able to. At this time, as will be described in detail later, still images can be extracted not only at the timing when the still image recording is instructed, but also at other timings. Therefore, the user can obtain a still image at desired timing. Also, high-quality still images can be extracted by setting photographing conditions (shutter speed, resolution, frame rate, etc. described above) suitable for still image extraction. In addition to such post-extraction of still images, still images can be captured (saved) during moving image capturing in the moving image capturing mode for still image extraction and the normal moving image capturing mode.

<Position of camera>

FIG. 3 is a diagram showing an example of a coordinate system defined for the camera 10. As shown in FIG. In the example shown in FIG. 3, the X-axis is the direction of the optical axis (the direction toward the subject), the Z-axis is the direction toward the top of the imaging device body 200, and (X-axis, Y-axis, Z-axis) is a right-handed coordinate system. construct the system. Whether the camera 10 is in the vertical position (the top and bottom surfaces of the camera 10 are parallel to the vertical direction) or in the horizontal position (the top and bottom surfaces of the camera 10 are parallel to the ground) depends on, for example, the angle around the X-axis (imaging device main body 200 is set to 0 degrees when the top and bottom surfaces are parallel to the ground). FIG. 4 is a diagram showing an example of the relationship between the angle about the X-axis and the posture (vertical or horizontal). The direction detection unit 240B (direction detection unit) determines “horizontal” when the angle is in the range of (−45 deg to +45 deg) and (+135 deg to +225 deg), and otherwise determines “vertical”. can do. As described above, the camera 10 includes the orientation sensor 280 (for example, an acceleration sensor that measures acceleration in three axial directions; see FIG. 1). Ten poses (orientations relative to the direction of gravity) can be detected.

Note that the range of angles shown in FIG. 4 is an example, and the posture may be determined in a different range. For example, the direction detection unit 240B may determine "horizontal" when the angle is in the range of (-30deg or more +30deg or less) and (+150deg or more +210deg or less), and otherwise determine "vertical". .

<Processing in Movie Shooting Mode for Still Image Extraction>

An imaging method in the camera 10 having the above configuration will be described. FIG. 5 is a flow chart showing processing in the moving image shooting mode for still image extraction (first moving image mode). The flowchart of FIG. 5 shows one aspect of the imaging method according to the present invention, which can be executed using the program according to the present invention (the same applies to other flowcharts described later). For example, by operating a mode dial (not shown) of the operation unit 250, the camera 10 enters a still image extraction moving image shooting mode. Then, the control unit 240C sets the photographing unit (interchangeable lens 100, image sensor 210, AFE 220, A/D converter 230) to the first moving image mode. Then, the processing of FIG. 5 is started, and the control unit 240C initializes the state of the camera 10 (acquisition and display of a live view image, initialization of orientation, start of orientation determination, etc.) (step S100: initialization step). .

<Video recording start instruction>

The control unit 240C determines whether or not a moving image recording start instruction has been issued (step S110: photographing step, moving image generating step). For example, when a release button (not shown) of the operation unit 250 is pressed, the control unit 240C can determine that "an instruction to start recording a moving image has been issued." If the determination is affirmative, the process proceeds to step S120.

<Determination of Posture>

The orientation detection unit 240B detects the orientation (direction relative to the direction of gravity) of the camera 10 (imaging device) based on the output of the orientation sensor 280 (step S120: orientation detection step, judgment step). As described above with reference to FIGS. 3 and 4, the direction detection unit 240B determines whether the posture is “vertical”, “horizontal”, or “undefined” based on the angle about the X-axis. The "undefined" posture can include a state in which the angle around the Y-axis is large and "vertical" or "horizontal" cannot be specified, such as when the user holds the camera 10 facing the ground or the sky. . Note that the direction detection unit 240B does not have to detect the “undefined” state of the camera 10 .

<Recording to folder according to posture>

If it is determined in step S120 that the orientation of the camera 10 is "horizontal" or "undefined", the control unit 240C compresses the moving image captured by the imaging unit and generates a new moving image file. Then, the control unit 240C starts recording (saving) the horizontal moving image file in the vertical/horizontal moving image folder (first folder) of the storage unit 260 (step S130: moving image saving step). The control unit 240C can compress moving images in MPEG format (MPEG2, MPEG4, etc.), for example. On the other hand, when the orientation of the camera 10 is determined to be "vertical", the control unit 240C compresses the moving image captured by the imaging unit and generates a new moving image file. Then, the control unit 240C starts recording (saving) the vertical/horizontal moving image file in the same vertical/horizontal moving image folder (first folder) as the horizontal/horizontal moving image (step S140: moving image saving step). The control unit 240C may record and store voices together using a microphone (not shown) provided in the camera 10 . Note that “horizontal moving image” and “vertical moving image” are moving images when the orientation is horizontal or indefinite and vertical, respectively, and “vertical/horizontal moving image folder” is a folder for storing vertical and horizontal moving images.

<Treatment when posture is indeterminate>

Note that in the flowchart of FIG. 5, the same file continues to be recorded in the case of "when the attitude changes from horizontal to indefinite" and "when the attitude changes from indefinite to horizontal". Then, we will explain how to divide the video (record it in a separate file) by determining that "the posture has changed" when "the posture changes from vertical to undefined" and "when the posture changes from undefined to vertical". ing. However, when the posture is indefinite, it may be recorded in a folder different from the vertical/horizontal moving image folder, or may not be recorded (saved) (see other aspects described later).

The direction detection unit 240B continues to detect the orientation of the camera 10 in parallel with the moving image shooting, and determines "whether or not the direction of the camera 10 (imaging device) has changed with respect to the direction of gravity" (step S170: determination step). . “Whether or not a direction change has occurred” can be determined based on the output of the attitude sensor 280 (for example, acceleration in three axial directions). If the direction detection unit 240B determines that "a direction change has occurred", specifically, if it determines that "the orientation of the camera 10 has changed between 'horizontal or undefined' and 'vertical'" (step S170). Yes), the control unit 240C closes the moving image file being recorded and divides the moving image (step S180: moving image generating step). In this case, the controller 240C denies the determination in step S190 and returns to step S120.

In the returned step S120, the control unit 240C determines whether the orientation of the camera 10 is "vertical", "horizontal", or "undefined", in the same manner as at the start of recording the moving image file. Then, according to the result, the control unit 240C saves the new moving image file related to the division (moving image file different from the one before the division) in the vertical/horizontal moving image folder (the same folder as before the division) of the storage unit 260. . For example, when the orientation of the camera 10 changes from “horizontal” at the start of recording to “vertical”, the control unit 240C stores a horizontal moving image file (first moving image file of the first moving image) in the vertical/horizontal moving image folder at the start of recording. , and a new vertical moving image file (second moving image file of the second moving image) generated due to the posture change is also stored in the vertical/horizontal moving image folder. It should be noted that, in step S170, when the orientation of the camera changes again (Yes in step S170), the moving image does not have to be closed. This is because, if the moving image is repeatedly divided each time the posture of the camera changes, the number of moving image files may increase excessively.

The image acquisition unit 240A, the direction detection unit 240B, and the control unit 240C continue dividing the moving image and saving the moving image file according to the orientation of the camera 10 until the end of the moving image shooting (while the determination in step S190 is No). continue. The direction detection unit 240B and the control unit 240C add information about the attitude of the camera 10 (direction with respect to the direction of gravity) to the first moving image file and the second moving image file. It is preferable to be able to refer to The direction detection unit 240B and the control unit 240C can record information about the attitude of the camera 10 (information about the direction with respect to the direction of gravity) in the header and footer. For example, it can be "01" for vertical, "02" for horizontal, and "03" for others (oblique, indeterminate, etc.).

FIG. 6 is a diagram showing how moving images are divided and recorded. The upper part of the figure shows how frames 1001 to 1005 of the moving image are sequentially recorded. In this situation, it is assumed that the orientation of the camera 10 is switched from "horizontal or undefined" to "vertical" (Yes in step S170) at the time of recording up to frame 1002 (frame number: N-1). In this case, the control unit 240C closes the moving image file 1010 (first moving image file) being recorded and divides the moving image (step S180: moving image generating step). Then, the control unit 240C starts recording a moving image starting from frame 1003 (frame number: N) as a new moving image file 1020 (second moving image file) (step S140). Control unit 240C saves moving image files 1010 and 1020 in the same folder (vertical/horizontal moving image folder) of storage unit 260 .

FIG. 7 is a diagram showing an example of determining a direction change based on an angle about the X-axis. In the example of FIG. 7, the posture is determined to be "horizontal" during the elapsed time from 0 to t1, and the angle exceeds 45 degrees (threshold value) at the elapsed time t1 and is determined to be "vertical". The moving image is divided at time t1. The control unit 240C saves the moving image file of the first moving image (first moving image file) whose elapsed time is from 0 to t1 in the vertical/horizontal moving image folder (first folder), and saves the moving image file of the second moving image after the elapsed time t1 ( 2nd moving image file) is also saved in the vertical/horizontal moving image folder (first folder). Note that the attitude of the camera 10 may change minutely, so the control unit 240C may determine that "the attitude has changed" when the state in which the attitude has changed continues for a predetermined time or longer. Also, FIG. 7 shows an example in which the orientation of the camera 10 changes from horizontal to vertical. The moving image becomes the second moving image (the same applies to other aspects).

The control unit 240C not only changes the posture of the camera 10 (Yes in step S170), but also determines in step S150 that "an instruction to end moving image recording (instruction to end moving image shooting) has been issued", and in step S160 " Also when it is determined that the remaining capacity of the recording medium is not sufficient, the moving image file is closed (step S180). However, in these cases, the control unit 240C also terminates the moving image shooting (Yes in step S190). In step S150, the control unit 240C can determine that "an instruction to end moving image recording has been issued" when a release button (not shown) of the operation unit 250 is pressed, or when an end instruction is issued via the monitor 270. can.

Note that when the camera 10 shifts from the horizontal state to the vertical state and finishes shooting the moving image in the vertical state as in the example of FIG. corresponds to the instruction to end the second moving image or the instruction to end moving image shooting. Similarly, when moving from the vertical state to the horizontal state again and shooting the moving image in the horizontal state is finished, the end timing of the first moving image (horizontal moving image; moving image in the horizontal state) depends on the user's instruction to end the first moving image or the moving image. Respond to the shooting end instruction. These moving image end timings are the same in the aspects described later.

In the first embodiment, when it is determined that "the direction (orientation) of the camera 10 (imaging device) with respect to the direction of gravity has changed in the moving image shooting mode for still image extraction (first moving image mode)" as described above, the moving image is divided to generate a horizontal video (first video) and a vertical video (second video). The generated horizontal moving image file (first moving image file) and vertical moving image file (second moving image file) are stored in the vertical/horizontal moving image folder (first folder).

Thereby, the user can easily identify the posture of the camera 10 when each moving image is taken, and can easily select the moving image including the frame to be extracted as a still image by using the posture of the camera 10 as a clue. In addition, the user can search for a frame to be extracted as a still image in a divided moving image whose recording time is shortened, so that a desired frame can be quickly searched. Thus, according to the first embodiment, the user can quickly and easily select a moving image frame with a desired composition.

<Example of folder structure>

FIG. 8 is a diagram showing an example of a folder structure for storing moving images. In the example shown in the figure, the control unit 240C generates a folder for each shooting date, and within each folder for each shooting date, a folder is created for each shooting number. For example, in the flow chart of FIG. 5, one photographing number can be set from the instruction to start moving image recording in step S110 to the end of moving image recording in step S190. The number of vertical and horizontal moving images corresponding to the division of the moving image is saved in each shooting number folder. Such a folder structure can be displayed on the monitor 270 (display device) by the control unit 240C. FIG. 9 shows vertical and horizontal video folders (first folders) with different shooting numbers displayed on the monitor 270 by icons indicating each folder.

<Video file icon display and file name>

FIG. 10 is a diagram showing an example of icon display of a moving image file. The figure shows how the control unit 240C displays horizontal moving image files by adding the characters "horizontal moving image" to the horizontal icon and adding the characters "vertical moving image" to the vertical icon. A symbol in the icon indicates that the file can be reproduced by specifying the file. In the example of FIG. 10 , the control unit 240</b>C saves moving image files of horizontal moving images and vertical moving images (first moving image file, second moving image file) in the same folder of the storage unit 260 . In the example of FIG. 10, the file name is the file creation date (October 23, 2018), the shooting number (001), vertical and horizontal identification information (horizontal video is H (Horizontal), vertical video is V (Vertical)), It consists of a file number (001, 002, . Also, the moving image file of the horizontal moving image (first moving image file) and the moving image file of the vertical moving image (second moving image file) are partially common except for the extension of the file name. The vertical/horizontal identification information is an example of information regarding the direction of the camera 10 (imaging device) with respect to the direction of gravity.

The control unit 240C may include information indicating the file type, file creation date and time, and vertical/horizontal identification information in the file name. For example, the control unit 240C gives the file name "IMG201801011230_H.MOV" to the horizontal video file created at 12:30 on January 1, 2018, and creates the file name at 12:40 on January 1, 2018. A file name of "IMG201801011240_V.MOV" can be given to the vertical moving image file. "IMG" in the file name is an example of information indicating that the file is an image file. Also in this case, the file name may contain information indicating the file size.

FIG. 11 is a diagram showing how thumbnail images are displayed on icons of moving image files. Part (a) of the figure is an example in the case of horizontal moving images, and part (b) is an example in the case of vertical moving images. The image to be displayed may be the first frame of the horizontal moving image or the vertical moving image, or may be an intermediate frame (for example, a frame in which an event occurs).

By displaying icons, displaying thumbnails, and assigning file names to moving image files as illustrated above, the user can easily distinguish between vertical moving images and horizontal moving images, and can quickly and easily select a moving image containing a desired frame. Note that icon display, thumbnail display, and file name assignment of moving image files can be performed by the control unit 240C.

<Example of display of scenes narrowed down by file information>

FIG. 12 is a diagram showing an example of scene narrowing-down display based on file information. In the part (a) of the figure, the elapsed time from t6 to t7, the elapsed time from t7 to t8, the elapsed time from t8 to t9, and the elapsed time from t9 to t10 are represented by horizontal video file 1030, vertical video file 1032, and horizontal video file 1032, respectively. A file 1034 and a vertical moving image file 1036 are shown. The control unit 240C refers to the file information such as the file name and displays each moving image file on the monitor 270 (or an external display device) using a figure such as a rectangle having a width corresponding to the length of the recording time. The control unit 240C may perform identification display, such as adding different colors to the figure indicating the moving image file for the vertical moving image and the horizontal moving image.

In the example shown in part (a) of FIG. 12, it can be seen that the recording time of the horizontal moving image file 1030 and the vertical moving image file 1036 is long. Note that when the user selects a displayed moving image file via the monitor 270 and/or the operation unit 250, the control unit 240C displays the selected moving image file or a part of the frames that constitute the moving image file on the monitor 270 or externally. can be displayed on the display device. This allows the user to select a desired frame to be extracted as a still image (see "Extraction of Still Image", which will be described later, and FIG. 25).

At the time of scene narrowing display, the control unit 240C may also display the timing at which the event occurred. In the example shown in part (b) of FIG. 12, the timing at which the event detection unit 240D detects an event is indicated by a dotted line within the rectangle indicating the moving image file. By referring to the scene refinement display illustrated in FIG. 12, the user can quickly and easily select a moving image frame with a desired composition (horizontal moving image or vertical moving image) when extracting a still image from a moving image. .

<Other Aspects of Splitting and Recording Moving Images>

Another aspect of dividing and recording (saving) a moving image will be described. In the flowcharts described below, the same step numbers are assigned to the same processes as those in FIG. 5, and detailed description thereof will be omitted.

(Aspect 1: Aspect in which the period during posture change is set as an independent video file)

In the embodiment described with reference to FIG. 5, it is determined that "a direction change has occurred" when the posture of the camera 10 (angle around the X-axis) exceeds the threshold value. However, in mode 1 shown in FIG. 13, the moving image during posture change is stored in a separate folder as a moving image file separate from the horizontal moving image and the vertical moving image. FIG. 13 is a flowchart showing the processing in mode 1. The orientation detection unit 240B determines whether the orientation of the camera 10 (direction relative to the direction of gravity) is "horizontal", "vertical", or "changing". (step S122: posture detection step).

When the posture is "horizontal", the control unit 240C starts recording (saving) a moving image file (first moving image file) as a horizontal moving image (first moving image) in a vertical/horizontal moving image folder (first folder) (step S130). The control unit 240C starts recording a vertical moving image (second moving image) in the vertical/horizontal moving image folder (first folder) when the posture is "vertical" (step S140), and when the posture is "changing" Recording of the middle moving image (third moving image) in the changing moving image folder (third folder) is started (step S124). In aspect 1, the control unit 240C can create a changing moving image folder in addition to the vertical/horizontal moving image folder in the hierarchy below the folder of each shooting number.

In aspect 1, when the direction detection unit 240B determines that "the change in the orientation of the camera 10 has started or completed" (Yes in step S172 (determination step)), the moving image is divided (moving image file closed, new moving image file created). generate). At this time, since the posture of the camera 10 may change minutely due to camera shake or the like, the direction detection unit 240B determines that "posture change has started" when the angle around the X axis changes by a predetermined value or more within a certain period of time. . Similarly, direction detection section 240B preferably determines that "posture change is completed" when the time period in which the change in angle is within a predetermined range continues for a certain period of time or longer. For example, in the example shown in FIG. 14, the angle changes greatly between elapsed times t1 and t2, and is stable between elapsed times t3 and t4.

Therefore, the direction detection unit 240B determines that "the attitude is changing" from the elapsed time t2 to t4, and determines that "the attitude change is completed" at the elapsed time t4 (determination of the actual attitude and the attitude of the camera 10). Considering the time lag until the result changes). In response to this determination, the control unit 240C determines whether the horizontal video (first video; elapsed time 0 to t2) and vertical video (second video; after elapsed time t4) are displayed between the elapsed time t2 and t4. A transition moving image (third moving image) in between is generated. As shown in FIG. 14, the in-change moving image (third moving image) includes a first period moving image in which the orientation of the camera 10 is changing (a direction change occurs).

(Aspect 2: Aspect in which recording is continued to the video file of the previous period while the posture is changing)

In mode 1 described above, during a posture change (first period), an independent moving image (third moving image) is recorded. However, in mode 2, while the posture is changing, the moving image file for the period before the start of the change is continuously recorded. Specifically, as shown in FIG. 15, the direction detection unit 240B detects that the camera 10 (imaging device) is in the first state (the bottom surface of the camera 10 is in the horizontal direction; horizontal state), the second state (the camera It detects whether the bottom surface of 10 is in the direction of gravity; the vertical state) or the third state (the state in which it is changing between the first state and the second state). Then, the control unit 240C records the moving image of the period in which the camera 10 is in the third state (while the posture is changing) in the first moving image in the vertical/horizontal moving image folder (first folder).

(Aspect 3: Aspect in which a part of the period after the end of the posture change is continuously recorded in the moving image file during the period during the change)

In situations where the attitude of the camera 10 has changed, the composition on the screen of the monitor 270 may change significantly. In that case, the camera 10 (imaging device) has a period during which the exposure of the moving image (AE: AutoExposure), the white balance of the moving image (AWB: Auto White Balance), or the focus of the subject in the moving image (AF: Auto Focus) is adjusted. may be needed. Therefore, in mode 3, the predetermined period (second period) after the orientation change is completed is handled in the same manner as the case where the orientation of the camera 10 is changing even if the orientation is stable. Specifically, for example, as shown in FIG. 16, the direction detection unit 240B " The attitude of the camera 10 has changed."

Then, these first period and second period are recorded (saved) as a third moving image. That is, the third moving image also includes the moving image of the second period after the direction change ends. Note that the control unit 240C divides the moving image, sets the third moving image as a moving image file (third moving image file) separate from the horizontal moving image (first moving image) and the vertical moving image (second moving image), and creates a vertical/horizontal moving image folder (second moving image). 1 folder). However, the third moving image may be recorded in an independent folder for changing moving images (third folder). An image during a posture change or a period when the shooting conditions are unstable may be less important, but in mode 3, such a period is saved as a moving image separate from the horizontal moving image and the vertical moving image, so it is extracted as a still image. You can easily select the frame you want to use and organize video files and folders.

(Aspect 4: Aspect in which part of the period after the end of posture change is continuously recorded in the original video file)

In mode 3 described above, the period during which the orientation of the camera 10 is changing (the first period) and the period (the second period) determined after the orientation is changed are referred to as the "third moving image". However, in mode 4, as shown in FIG. 17, the first period (elapsed time from t2 to t4) and the second period (elapsed time from t4 to t5) are defined as the "first moving image" folder for vertical and horizontal moving images (first folder).

(Aspect 5: Aspect in which moving images are not recorded during a posture change)

In modes 1 to 4 described above, a moving image is recorded during the period during which the posture is changed (and the period until the posture is stabilized thereafter), but a mode in which the moving image is not recorded during the posture change can also be adopted. Specifically, as shown in the flowchart of FIG. 18, the orientation detection unit 240B detects whether the orientation of the camera 10 is horizontal (first state), vertical (second state), or changing (third state). is detected (step S122). Then, when the posture is changing, recording of the moving image file is not started (returns to step S122).

Also, if the orientation of the camera 10 is horizontal or vertical and the orientation of the camera 10 starts to change after recording of the moving image has started (steps S130 and S140) (Yes in step S174), the control unit 240C converts the moving image file to the moving image file in step S180. is closed and the process returns to step S122. Then, after the orientation of the camera 10 becomes horizontal or vertical, recording of moving images is resumed (steps S130 and S140). Images during posture changes may be of low importance, but in aspect 5, since moving images are not saved during such periods, the remaining capacity of the recording medium can be saved. In addition, the user can easily select a frame to be extracted as a still image and organize moving image files and folders.

(Aspect 6: Aspect in which the video is not divided immediately after the event occurs even if there is a posture change)

Mode 6 shown in the flowchart of FIG. 19 is a mode when the orientation change of the camera 10 is started or completed. In this aspect, if a specific event has occurred before a predetermined time (described as “T seconds before” in step S176) from the timing when the attitude change of the camera 10 is started or completed, the control unit 240C does not close the moving image file (Yes in step S176 and does not proceed to step S180). Therefore, in this case, the moving image is not divided. The same applies not only to the case where a specific event has occurred before a certain period of time, but also to the case where the specific event is occurring (continuing).

The control unit 240C can set the value of T (for example, 30 seconds, but other values are also possible) according to the user's operation via the operation unit 250 or without depending on the user's operation. can be done. The event detector 240D (see FIG. 2) determines whether a specific event has occurred by automatically detecting the amount of movement of the subject, changes in size and shape, changes in brightness, etc., through well-known image processing. may Further, whether or not a specific event has occurred may be determined by a user's instruction via the operation unit 250 (including the case where the monitor 270 is used as a touch panel). Alternatively, it may be determined that "an event has occurred" when the user instructs to shoot a still image while shooting a moving image. When an event is detected, the event detection section 240D and the control section 240C preferably add information indicating event detection timing to the moving image file (for example, record it in the header or footer). By referring to this information, the user can quickly and easily select a frame to be extracted as a still image (described later).

As described above, even if there is a change in the posture of the camera 10, if an event occurs before the change in posture, the video is not divided, so that the user can confirm the series of events related to the event in one video. A desired frame can be quickly and easily selected from the captured moving image. The flowchart in FIG. 19 is based on FIG. 13 (moving images during posture change are separate from horizontal and vertical moving images and stored in separate folders), but the flow charts in FIGS. good.

It should be noted that the control unit 240C may determine in which mode, including modes 1 to 6, the moving image is to be divided and recorded (saved) according to the user's operation via the operation unit 250. good.

<Example of changing the vertical and horizontal size of the image and displaying it>

The camera 10 can display the saved moving image (or a still image extracted from the moving image) on a display device such as the monitor 270 or the like. At this time, the moving image may be horizontally shot (shooting with the camera 10 in the “horizontal” orientation) or vertically (shooting with the camera 10 in the “vertical” orientation). If the captured image is displayed in an orientation different from that at the time of capturing while maintaining the size of the captured image, problems such as "a part of the image is not displayed" may occur. Therefore, when displaying an image (moving image, still image), the control unit 240C changes the size of the image in the vertical direction and/or the horizontal direction based on the relationship with the posture of the camera 10 at the time of shooting and display. can do. The orientation of the camera 10 at the time of photographing can be grasped by referring to file information such as information in the file name and header.

FIG. 20 is a diagram showing an example of resizing an image. Part (a) of the figure is an example of displaying an image 1040 shot horizontally on the monitor 270 (display device) with the camera 10 oriented horizontally (displayed in the same orientation as when shooting). In this case, there is no need to resize image 1040 . On the other hand, part (b) of FIG. 10 is an example of displaying an image with the camera 10 in the portrait orientation (displaying in a different orientation from when the image was taken). In this case, if the image 1040 is displayed as it is, the entire image cannot be displayed. Therefore, the control unit 240</b>C changes (reduces) the size of the image 1040 in the vertical and horizontal directions and displays it as an image 1042 . At this time, the control unit 240C maintains the aspect ratio of the image (for example, horizontal:vertical=16:9) before and after resizing.

FIG. 21 is a diagram showing another example of resizing an image. Part (a) of the figure is an example in which an image 1044 taken vertically is displayed in a state in which the camera 10 is oriented vertically. In this case, there is no need to resize image 1044 . On the other hand, part (b) of FIG. 10 is an example of displaying an image with the camera 10 in landscape orientation (displaying in a different orientation from when photographing). In this case, if displayed as it is, the entire image 1044 cannot be displayed. Therefore, the control unit 240</b>C changes (reduces) the size of the image 1044 in the vertical and horizontal directions and displays it as an image 1046 . At this time, the control unit 240C maintains the aspect ratio of the image (for example, horizontal:vertical=16:9) before and after resizing.

The size change illustrated in FIGS. 20 and 21 may be in one or both of the vertical and horizontal directions.

<Processing when Considering First Mode and Second Mode>

The flowcharts shown in FIGS. 5, 13, 18, and 19 describe the processing in the first moving image mode, but the processing in consideration of the first moving image mode and the second moving image mode will be described with reference to FIGS. . 5, 13, 18, and 19 are assigned the same step numbers, and detailed description thereof will be omitted.

The control unit 240C determines whether the camera 10 is in the first moving image mode (still image extraction moving image mode) or the second moving image mode (normal moving image mode) (step S112). In the case of the first moving image mode, the control unit 240C sets shooting conditions suitable for the first moving image mode (step S114). The shooting condition set in step S114 is a shooting condition that emphasizes the extraction of still images rather than the viewing of moving images, as described above in the section of "shooting mode". Since the processing after step S114 in the first moving image mode is the same as that of the flowchart of FIG. 5, detailed description thereof will be omitted.

On the other hand, if the result of determination in step S112 is the second moving image mode, the control unit 240C sets shooting conditions suitable for the second moving image mode (step S191 in FIG. 23). The direction detection unit 240B determines the orientation of the camera 10 (step S192), and the control unit 240C starts generating and recording (saving) a moving image file in the vertical/horizontal moving image folder (first folder) of the storage unit 260 (step S192). S193: moving picture saving step). The processing of steps S194 and S195 is the same as the processing of steps S150 and S160 for the first moving image mode. If there is an instruction to end moving image recording (Yes in step S194) and if the remaining capacity of the recording medium is insufficient (No in step S195), the control unit 240C closes the moving image file (step S197), and the step shown in FIG. The process returns to S190 (determining whether or not moving image shooting is finished).

If there is no moving image recording end instruction (No in step S194) and the remaining capacity of the recording medium is sufficient (Yes in step S195), the control unit 240C continues recording the moving image. However, in the second moving image mode, unlike the first moving image mode, even if the direction detection unit 240B does not determine that "a change in the posture (orientation) of the camera 10 has occurred" (No in step S196), When it is determined that "the posture has changed" (Yes in step S196), the moving image is not divided. In the second moving image mode, the moving image is not divided even if the orientation of the camera 10 changes. This is because it is not necessary to divide the .

<Folder structure when considering the first mode and second mode>

FIG. 24 is a diagram showing an example of the folder structure when considering the first mode and the second mode. In the example shown in the figure, the control unit 240C generates a folder for each shooting date, and within each folder for each shooting date, a folder for the first moving image mode and a folder for the second moving image mode are created. The control unit 240C creates a folder for each shooting number in the folder of the first moving image mode, and saves the number of horizontal and vertical moving images corresponding to the division of the moving image in the folder of each shooting number. On the other hand, in the second moving image mode, since the moving image is not divided according to the change in posture, the control unit 240C records the moving image file in the folder of the second moving image mode. In the example of FIG. 24, the folders for the first and second moving image modes are provided below the folder for the shooting date. may be provided. Such a folder structure can be displayed on the monitor 270 (display device) by the control unit 240C.

<Extraction of still image>

As described in the following example, the camera 10 can extract frames forming a moving image as still images. Still images may be extracted by another device such as a personal computer using the moving image file recorded by the camera 10 .

(Example 1)

The control unit 240C causes the monitor 270 to reproduce and display the file selected by the user from the moving image files of the horizontal moving image, the vertical moving image, or the changing moving image (the first moving image, the second moving image, and the third moving image). Instead of continuous reproduction, frame-by-frame reproduction (frame by frame display according to user's operation) may be performed. The control unit 240C stops the reproduction by the user's operation, and by confirming the selection of the frame displayed at the time of stopping according to the user's operation, the still image extracting unit 240E extracts the frame as a still image. .

(Example 2)

When information indicating event detection is added to the moving image file, the control unit 240C may refer to the information and display a plurality of frames including the frame in which the event was detected on the monitor 270 . FIG. 25 is an example of such a display, where five frames 1051-1055 are displayed, including frame 1052. FIG. Further, a camera-like icon 1060 is attached to a frame 1052 in which a still image is recorded (recording of a still image is an example of event occurrence) by the still image extraction unit 240E. The icon 1060 allows the user to understand that the frame is a still image recorded.

Note that the range of frames for still image extraction to be displayed is the frame (frame 1051) temporally before and the frame after (frame 1052 in the example of FIG. 25) with respect to the “frame corresponding to the recorded still image file”. 1053-1055) may be included. Further, the range of still image extraction frames to be displayed may be in a mode of "including only temporally previous frames" or a mode of "including only temporally subsequent frames". The range of still image extraction frames to be displayed may be set according to the user's designation, or may be set regardless of the user's designation. Also, the range of frames to be displayed may be specified by time (for example, one second before and after the frame in which the still image is recorded), or by the number of frames (for example, the number of frames before and after the frame in which the still image is recorded). 10 frames each). Alternatively, in the initial state, only the frames recorded as still images are displayed in a list, and a frame selected from the list may be displayed temporally before and/or after.

<Selection of frame to extract>

The user can select a frame from the displayed frames for still image extraction. In the example of FIG. 25, the user can select a frame by checking checkbox 1070 for the desired frame via operation unit 250 and/or monitor 270, and can select multiple frames. Note that the frame in which the still image specified by the user is recorded is not necessarily recorded at the best timing. By displaying other related frames together, the user may wish to select other frames. Therefore, it is preferable to be able to select from frames before and/or after in addition to the frame in which the still image was recorded. FIG. 25 shows a state in which the user has selected a frame 1053 different from the frame in which the still image was recorded (frame 1052 with icon 1060).

The still image extraction unit 240E extracts the selected frame as a still image (still image extraction step). Since the moving image file is stored in a moving image format such as MPEG format, the still image extraction unit 240E converts the selected frame data into a still image format (JPEG format, etc.). The user can extract a still image at a desired timing from the moving images (the first moving image, the second moving image, and the third moving image).

<Second embodiment>

Although the camera 10, which is a digital camera, has been described in the first embodiment, the configuration of the imaging device is not limited to this. Other imaging devices of the present invention can be, for example, a built-in or external PC camera (PC: Personal Computer), or a mobile terminal device having an imaging function as described below. .

Portable terminal devices, which are an embodiment of the imaging device of the present invention, include, for example, mobile phones, smart phones, PDAs (Personal Digital Assistants), and portable game machines. Hereinafter, a smart phone will be taken as an example, and a detailed description will be given with reference to the drawings.

FIG. 26 is a diagram showing the appearance of a smartphone 1 (imaging device) that is an embodiment of the imaging device of the present invention, in which part (a) is a front view and part (b) is a rear view. A smartphone 1 shown in FIG. 26 has a flat housing 2, and a display panel 21 (display device) as a display unit and an operation panel 22 (operation unit) as an input unit are provided on one surface of the housing 2. is provided with a display input unit 20 integrated with. The housing 2 includes a speaker 31, a microphone 32, an operation unit 40 (operation unit), camera units 41 and 42 (imaging device, imaging unit, event detection unit, control unit, still image extraction unit, lens drive control part) and a strobe 43 . The configuration of the housing 2 is not limited to this. For example, a configuration in which the display unit and the input unit are independent may be adopted, or a configuration having a folding structure or a slide mechanism may be adopted.

FIG. 27 is a block diagram showing the configuration of the smartphone 1 shown in FIG. 26. As shown in FIG. As shown in FIG. 27, the main components of the smartphone 1 are a wireless communication unit 11, a display input unit 20, a call unit 30, an operation unit 40, camera units 41 and 42, a strobe 43, and a storage unit. 50 (storage unit), an external input/output unit 60 , a GPS receiver unit 70 (GPS: Global Positioning System), a motion sensor unit 80 , a power supply unit 90 and a main control unit 101 . The main control unit 101 functions as an imaging unit, a moving image file generation unit, a still image file generation unit, an event detection unit, a display control unit, a still image extraction unit, and a lens drive control unit. In addition, as a main function of the smartphone 1, a radio communication function is provided to perform mobile radio communication via a base station device and a mobile communication network.

Radio communication section 11 performs radio communication with a base station apparatus accommodated in a mobile communication network according to instructions from main control section 101 . Such wireless communication is used to transmit and receive various file data such as audio data and image data, e-mail data, etc., and to receive Web data, streaming data, and the like.

Under the control of the main control unit 101, the display input unit 20 displays images (still images and/or moving images), character information, etc. to visually transmit information to the user, and also allows the user to operate the displayed information. It is a so-called touch panel for detecting, and includes a display panel 21 and an operation panel 22 .

As the display panel 21, an LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display), or the like is used as a display device. The operation panel 22 is a device that is placed so that an image displayed on the display surface of the display panel 21 can be visually recognized, and that detects one or more coordinates operated by a conductor such as a user's finger or pen. When such a device is operated by a user's finger or a conductor such as a pen, the operation panel 22 outputs a detection signal generated by the operation to the main control section 101 . Next, the main control unit 101 detects the operating position (coordinates) on the display panel 21 based on the received detection signal.

As shown in FIG. 26, the display panel 21 and the operation panel 22 of the smart phone 1 exemplified as an embodiment of the imaging device of the present invention are integrated to form the display input unit 20. However, the operation panel 22 is arranged to completely cover the display panel 21 . When such an arrangement is adopted, the operation panel 22 may have a function of detecting user operations in areas outside the display panel 21 as well. In other words, the operation panel 22 has a detection area for the overlapping portion overlapping the display panel 21 (hereinafter referred to as a display area) and a detection area for the outer edge portion not overlapping the display panel 21 (hereinafter referred to as a non-display area). ) may be provided.

The call unit 30 includes a speaker 31 and a microphone 32, converts the user's voice input through the microphone 32 into voice data that can be processed by the main control unit 101, and outputs the data to the main control unit 101; 11 or the external input/output unit 60 can decode the audio data and output it from the speaker 31 . Further, as shown in FIG. 26 , for example, the speaker 31 can be mounted on the same surface as the display input unit 20 is provided, and the microphone 32 can be mounted on the side surface of the housing 2 .

The operation unit 40 is a hardware key using a key switch or the like, and is a device that receives instructions from a user. For example, as shown in FIG. 26, the operation unit 40 is mounted on the side surface of the housing 2 of the smartphone 1, is turned on when pressed with a finger or the like, and is turned off by the restoring force of a spring or the like when the finger is released. It is a button type switch.

The storage unit 50 stores the control program and control data of the main control unit 101, application software, address data associated with names and telephone numbers of communication partners, data of sent and received e-mails, web data downloaded by web browsing, It stores downloaded content data and temporarily stores streaming data and the like. The storage unit 50 is configured by an internal storage unit 51 built into the smartphone and an external storage unit 52 having a detachable external memory slot. The internal storage unit 51 and the external storage unit 52 constituting the storage unit 50 are each realized using a known storage medium.

The external input/output unit 60 functions as an interface with all external devices connected to the smartphone 1 . The smartphone 1 is directly or indirectly connected to another external device through communication or the like via the external input/output unit 60 . Examples of communication means include Universal Serial Bus (USB), IEEE1394, and networks (eg, the Internet and wireless LAN). In addition, Bluetooth (registered trademark), RFID (Radio Frequency Identification), infrared communication (Infrared Data Association: IrDA) (registered trademark), UWB (Ultra Wide Band) (registered trademark), ZigBee (registered trademark) Trademark) can also be cited as a means of communication.

External devices connected to the smart phone 1 include, for example, a wired/wireless headset, a wired/wireless external charger, a wired/wireless data port, a memory card and a SIM (subscriber) connected through a card socket. Identity Module Card)/UIM (User Identity Module Card) cards. In addition, external audio and video devices connected via audio and video I/O (Input/Output) terminals, external audio and video devices wirelessly connected, smartphones connected wired/wirelessly, wired/wireless connected External devices such as a PDA, a wired/wireless personal computer, and an earphone can also be connected. The external input/output unit 60 can transmit data received from such an external device to each component inside the smartphone 1 and transmit data inside the smartphone 1 to an external device.

The motion sensor unit 80 includes, for example, a triaxial acceleration sensor and a tilt sensor, and detects physical movements of the smartphone 1 according to instructions from the main control unit 101 . By detecting the physical movement of the smartphone 1, the moving direction, acceleration, and posture of the smartphone 1 are detected. Such detection results are output to the main control section 101 . The power supply unit 90 supplies electric power stored in a battery (not shown) to each unit of the smartphone 1 according to instructions from the main control unit 101 .

The main control unit 101 includes a microprocessor, operates according to control programs and control data stored in the storage unit 50 , and controls each unit of the smartphone 1 including the camera unit 41 in an integrated manner. Further, the main control unit 101 has a mobile communication control function for controlling each unit of the communication system and an application processing function in order to perform voice communication and data communication through the wireless communication unit 11 .

The main control unit 101 also has an image processing function such as displaying an image on the display/input unit 20 based on image data (still image or moving image data) such as received data or downloaded streaming data. The image processing function is a function in which the main control unit 101 decodes image data, applies image processing to the decoded result, and displays the image on the display input unit 20 .

The camera units 41 and 42 are digital cameras (imaging devices) that perform electronic photography using imaging devices such as CMOS and CCD. Under the control of the main control unit 101, the camera units 41 and 42 convert image data (moving images and still images) obtained by imaging into compressed image data such as MPEG or JPEG, and record the data in the storage unit 50. Alternatively, it can be output through the external input/output unit 60 or the wireless communication unit 11 . In addition, the camera unit 41 is controlled by the main control unit 101 to divide and combine moving images, acquire high-quality still images (such as RAW images), replace and process frames, and extract still images from moving images. can also In the smart phone 1 shown in FIGS. 26 and 27, either one of the camera units 41 and 42 can be used for photographing, or the camera units 41 and 42 can be used simultaneously for photographing. A strobe 43 can be used when the camera unit 42 is used.

Also, the camera units 41 and 42 can be used for various functions of the smartphone 1 . For example, it is possible to display images acquired by the camera units 41 and 42 on the display panel 21, and to use the images of the camera units 41 and 42 as one of the operation inputs of the operation panel 22. FIG. Further, when the GPS receiving section 70 detects the position, it is also possible to detect the position by referring to the images from the camera sections 41 and 42 . Furthermore, by referring to the images from the camera units 41 and 42, the optical axis direction of the camera unit 41 of the smartphone 1 is determined without using the 3-axis acceleration sensor or in combination with the 3-axis acceleration sensor. It is also possible to determine the current usage environment. Of course, the images from the camera units 41 and 42 can also be used within the application software. In addition, positional information acquired by the GPS receiving unit 70 in image data of still images or moving images, audio information acquired by the microphone 32 (which may be converted into text information by performing audio-to-text conversion by the main control unit etc.), It is also possible to add posture information acquired by the motion sensor unit 80 and record it in the storage unit 50 or to output it through the external input/output unit 60 or the wireless communication unit 11 .

<Smartphone posture>

Part (a) of FIG. 28 is a diagram showing an example of a coordinate system defined for the smartphone 1. In this example, the attitude is determined by the angle around the X axis. Part (a) shows a vertical state at an angle of 0 degrees (second state in which the smartphone 1 (imaging device) is aligned with the direction of gravity), and part (b) of the figure shows a horizontal state at an angle of 90 degrees (smartphone 1 (imaging device) ) along the horizontal direction). Also, FIG. 29 is a diagram showing an example of the relationship between the angle around the X-axis and the orientation (vertical or horizontal) of the smartphone 1 . Note that, as in the case of the camera 10 according to the first embodiment (see FIG. 4), the horizontal state (the state shown in FIG. state, second state, third state) may be defined.

In the smartphone 1 having the configuration described above, as in the camera 10 according to the first embodiment, the processing of the imaging method according to the present invention (moving image shooting, division, and recording, still image recording, still image extraction, etc.) can be performed. can be executed. Specifically, in the smartphone 1, the processing executed by the image processing device 240 (each unit shown in FIG. 2) in the first embodiment (including the processing of aspects 1 to 6) is performed mainly by the camera units 41 and 42 and the main controller. By being executed by the unit 101, processing of the imaging method according to the present invention can be performed. In addition, the functions of the operation unit 250, the storage unit 260, and the monitor 270 in the first embodiment can be realized by the operation unit 40, the storage unit 50, the operation panel 22, the display panel 21, and the operation panel 22 in the smartphone 1, respectively. can.

As a result, the smartphone 1 according to the second embodiment has the same effect as the camera 10 according to the first embodiment (the user can quickly and easily select a video including frames with a desired composition). etc.) can be obtained.

Although the embodiments and other aspects of the present invention have been described above, the present invention is not limited to the above-described embodiments and aspects, and various modifications are possible without departing from the spirit of the present invention.

1 smartphone

2 housing

10 camera

11 wireless communication unit

20 display input unit

21 display panel

22 Operation panel

30 call part

31 speaker

32 Microphones

40 operation part

41 Camera section

42 Camera section

43 Strobe

50 storage unit

51 internal memory

52 external memory

60 external input/output unit

70 GPS receiver

80 motion sensor

90 power supply

100 interchangeable lenses

101 main control unit

110 zoom lens

120 focus lens

130 aperture

140 lens driver

200 imaging device body

210 image sensor

220 AFE

230 A/D converter

240 image processor

240A image acquisition unit

240B direction detector

240C control unit

240D event detector

240E still image extractor

240F lens drive controller

242 ROMs

250 Operation unit

260 storage unit

270 monitor

280 attitude sensor

1001 frames

1002 frames

1003 frames

1004 frames

1005 frames

1010 video files

1020 video files

1030 horizontal video file

1032 vertical video files

1034 horizontal video file

1036 vertical video files

1040 images

1042 images

1044 images

1046 images

1051 frames

1052 frames

1053 frames

1054 frames

1055 frames

1060 icon

1070 checkbox

L optical axis

S100 to S196 Each step of the imaging method

ST1 GPS satellite

STn GPS satellites

Claims (11)

An imaging device,
a direction detection unit that detects the direction of the imaging device with respect to the direction of gravity;
a shooting unit that shoots moving images;
a control unit;
with
The control unit has a first moving image mode capable of extracting a still image from a moving image file of a moving image captured by the imaging unit, and a second moving image mode having different shooting conditions from the first moving image mode,
In the first moving image mode, the control unit divides the moving image to generate a first moving image and a second moving image when the direction detection unit determines that a direction change with respect to the direction of gravity of the imaging device has occurred,
The imaging device, wherein the control unit does not divide the moving image when the direction detection unit determines that the direction change has occurred in the second moving image mode. The imaging device according to claim 1, wherein the control unit saves a first moving image file of the first moving image and a second moving image file of the second moving image based on information regarding a direction of the imaging device with respect to a direction of gravity. . The control unit generates a third moving image that is between the first moving image and the second moving image in time,
The imaging device according to claim 1 or 2, wherein the third moving image includes a moving image of the first period in which the direction change occurs. The imaging device according to claim 3, wherein said control unit does not save said third moving image. 4. The imaging apparatus according to claim 3, wherein said control unit stores said third moving image in a folder different from a folder storing said first moving image and a folder storing said second moving image. 5. The imaging device according to claim 3, wherein said third moving image also includes a moving image of a second period after said direction change is completed. 7. The imaging device according to claim 6, wherein said second period is a period during which said imaging device adjusts exposure of a moving image, white balance of a moving image, or focus of a subject in a moving image. 8. The imaging apparatus according to any one of claims 1 to 7, wherein file names of the first moving image file of the first moving image and the second moving image file of the second moving image are at least partially common. 3. The imaging apparatus according to claim 2, wherein said control unit adds information regarding a direction of said imaging apparatus with respect to said direction of gravity to said first moving image file and said second moving image file. A direction detection unit that detects the direction of the imaging device with respect to the direction of gravity, a shooting unit that shoots a moving image, a first moving image mode capable of extracting a still image from the file of the moving image shot by the shooting unit, and the first moving image mode An imaging method for an imaging device including a control unit having a second moving image mode with different imaging conditions,
a determination step of determining whether or not the direction detection unit has changed in the direction of gravity of the imaging device in the first moving image mode;
When the direction detection unit determines that the direction change has occurred, the moving image is divided in the first moving image mode to generate a first moving image and a second moving image, and the moving image is not divided in the second moving image mode. a video generation step;
An imaging method comprising: A direction detection unit that detects the direction of the imaging device with respect to the direction of gravity, a shooting unit that shoots a moving image, and a first moving image mode capable of extracting a still image from a moving image file of the moving image shot by the shooting unit, and the first moving image mode. and a control unit having a second moving image mode with different shooting conditions, wherein
In the first moving image mode, causing the direction detection unit to determine whether or not a direction change with respect to the direction of gravity of the imaging device has occurred;
when the direction detection unit determines that the direction change has occurred in the first moving image mode, causing the control unit to divide the moving image to generate a first moving image and a second moving image;
A program that does not divide a moving image when the direction detection unit determines that the direction change has occurred in the second moving image mode.

Images