JP2023024521A - Imaging device, imaging method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device, an imaging method, and a program that enable a user to easily select a frame of a moving image to be extracted as a still image.

SOLUTION: An imaging device according to the first aspect of the present invention includes a direction detection unit that detects the direction of an imaging device with respect to the direction of gravity, an imaging unit that shoots a moving image, and a control unit, and the control unit has a first moving image mode capable of generating a moving image file of the moving image captured by the imaging unit and extracting a still image from the moving image file, and in the first moving image mode, the control unit divides the moving image to generate a first moving image and a second moving image when the direction detection unit determines that the direction of the imaging device has changed with respect to the direction of gravity. A first moving image file of the first moving image and a second moving image file of a second moving image are stored in a storage unit, a third moving image is generated between the first moving image and the second moving image in time, and the third moving image includes a moving image of a first period of directional change.

SELECTED DRAWING: Figure 5

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to an image capturing apparatus, an image capturing method, and a program for capturing a moving image, and more particularly to an image capturing apparatus, an image capturing method, and a program capable of extracting frames constituting a moving image as still images.

2. Description of the Related Art Techniques for capturing moving images for the purpose of extracting still images after capturing are known. With this technology, for example, a moving image is shot and recorded in order to capture an instantaneous event (event) that may occur at any time, and the user selects a desired frame after shooting. is difficult to predict. For this reason, it is necessary to continue shooting movies for a certain amount of time, and the remaining capacity of the recording media is tight, or it takes a lot of effort for the user to select the frame where the event occurred from a long movie. I have a problem.

In order to deal with the above-mentioned problems, it is conceivable to tag or mark videos. For example, an event is detected, and a tag representing event information is added to the frame when the event occurs. Further, for example, in Patent Document 1, a moving image is marked at the timing of acquiring a still image according to a user instruction during recording of the moving image, and after the moving image recording is completed, the marked portion and the moving image frames before and after the marking are extracted and displayed so that the user can It is described that it is possible to select the timing of extracting a still image from a moving image.

JP 2016-32303 A

However, when adding tags as described above, tags representing event information are not standardized and can only be recognized by specific equipment including the imaging device used for shooting. tag information needs to be analyzed, and there is a problem that it takes time to reproduce the event frame. In addition, since the technique described in Patent Document 1 only records marking information, video decoding (still image processing) is required each time a marked frame is extracted and displayed, which imposes a high processing load on the imaging device. . As described above, the conventional technology does not allow the user to easily select frames of a moving image to be extracted as a still image.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging apparatus, an imaging method, and a program that enable a user to easily select a frame of a moving image to be extracted as a still image.

In order to achieve the above object, an imaging device according to a first aspect of the present invention is an imaging device comprising: a direction detection unit for detecting the direction of the imaging device with respect to the direction of gravity; an imaging unit for capturing moving images; a control unit, wherein the control unit has a first moving image mode capable of generating a moving image file of a moving image captured by the imaging unit and extracting a still image from the moving image file; , when the direction detection unit determines that the direction of the imaging device has changed with respect to the direction of gravity, the moving image is divided to generate a first moving image and a second moving image, and the control unit stores the first moving image file of the first moving image. The second moving image file of the second moving image is stored in the second folder of the storage unit.

Normally, in still image shooting, the user sets the camera (imaging device) horizontally (the bottom surface of the camera is parallel to the ground) or vertically (the bottom surface of the camera is parallel to the direction of gravity) depending on the desired composition. Hold the camera as shown. In video shooting, on the other hand, it is rare to switch between vertical and horizontal orientations during shooting. On the other hand, in moving image shooting for extracting still images, it is assumed that the way the camera is held changes between vertical and horizontal during moving image shooting due to the characteristic that frames are cut out after the fact. This vertical and horizontal change can be considered as a kind of event caused by the user's action. From such a point of view, in the first aspect, when the direction of the imaging device (posture of the imaging device) with respect to the direction of gravity changes, the moving image is divided to generate the first moving image and the second moving image, and the first moving image and the second moving image are generated. The first moving image file of the moving image and the second moving image file of the second moving image are stored in separate folders (first folder and second folder). This makes it possible to easily identify the posture of the imaging device when each moving image was shot, and the user can easily select a moving image including a frame to be extracted as a still image by using the posture of the imaging device as a clue. In addition, since the frame to be extracted as a still image can be found in the divided moving image whose recording time is shortened, the desired frame can be quickly found.

Thus, according to the first aspect, the user can easily select a frame of a moving image to be extracted as a still image. Furthermore, in the first mode, there is no need to use information such as tags and markings, so based on the moving image recorded in the first mode, selection of frames and extraction of still images can be performed by other imaging devices or image reproduction. It can also be performed with an apparatus or an image processing apparatus. Note that the storage unit may be provided in the imaging device, or may be an external device.

In the imaging device according to the second aspect, in the first aspect, the control unit generates a third moving image that is between the first moving image and the second moving image in time, and the third moving image is generated when the direction change occurs. Includes videos from the first period. In the second aspect, the period during which the direction change occurs (first period) is a moving image (third moving image) different from the first and second moving images so that the user can easily identify it.

In the imaging device according to the third aspect, in the first aspect, the first moving image precedes the second moving image in terms of time, and the direction detection unit captures the first state in which the imaging device is along the horizontal direction. detecting whether the device is in a second state in which the device is along the direction of gravity or in a third state in which the imaging device is changing between the first state and the second state; The moving image includes a moving image during the period when the imaging device is in the third state. In the third aspect, for example, an intermediate state between the first and second states, a state in which the direction of the imaging device is indefinite, a state in which the orientation of the imaging device is changing between the first state and the second state, etc. It can be in 3 states.

In the second aspect of the imaging device according to the fourth aspect, the control unit does not store the third moving image in the storage unit. According to the fourth aspect, it is possible to save the capacity of the storage unit by not saving the moving image during the period in which the direction change occurs (the first period).

In the second aspect of the imaging device according to the fifth aspect, the control unit saves the third moving image in the first folder of the storage unit. That is, the third moving image is saved in the same folder as the first moving image.

In the imaging device according to the sixth aspect, in the fourth or fifth aspect, the third moving image also includes the moving image of the second period after the direction change is completed. In the sixth aspect, even after the change in direction ends, the shooting conditions may not be stable due to the change in composition, so even after the change in direction ends, it is included in the third moving image in the second period. .

In the imaging device according to the seventh aspect, in the first or second aspect, the first moving image also includes the moving image of the second period after the direction change is completed. In the sixth aspect, since there is a possibility that the shooting conditions are not stable due to the change in composition even after the direction change is completed, the first moving image is included in the second period even after the direction change is completed. .

An imaging device according to an eighth aspect is the imaging device according to the sixth or seventh aspect, wherein the second period is a period during which the imaging device adjusts the exposure of the moving image, the white balance of the moving image, or the focus of the subject in the moving image. is. The eighth aspect specifically defines the content of the second period.

A ninth aspect of the imaging device according to any one of the first to eighth aspects, wherein the control unit adds information about the direction of the imaging device with respect to the direction of gravity to the first moving image file and the second moving image file. . For example, the information can be added to the file name or part thereof, the header or footer of the moving image file, and the like.

According to a tenth aspect, in any one of the first to ninth aspects, the control unit has a reception unit that receives a user's instruction, and the end timing of the second moving image is determined by the reception unit. , the reception of the second moving image end instruction or the moving image shooting end instruction. According to the tenth aspect, the user can end the second moving image at desired timing.

An imaging apparatus according to an eleventh aspect, in any one of the first to tenth aspects, is provided with a reception unit for receiving a user's instruction in the control unit or an event detection unit for detecting a specific event, the control unit does not divide the moving image if the orientation detection unit detects a direction change while the reception unit is accepting a user's instruction or the event detection unit is detecting a specific event. According to the eleventh aspect, if there is a direction change (orientation change) of the imaging device during the user's instruction or occurrence of a specific event, the moving image is not split, so that a sequence of scenes related to the user's instruction or event is generated. can be seen in one video.

An imaging apparatus according to a twelfth aspect is any one of the first to eleventh aspects, wherein the control unit has a second moving image mode different in imaging conditions from the first moving image mode, In moving image mode, if the direction detection unit detects a direction change, the moving image is not divided. For example, a normal moving image mode in which still images are not extracted from a moving image can be set as the second moving image mode.

In the imaging apparatus according to the thirteenth aspect, in the twelfth aspect, the first moving image mode has at least one of a shutter speed, an autofocus speed, an automatic exposure follow-up speed, and a white balance follow-up speed with respect to the second moving image mode. One is set to fast and/or the frame rate is set to high for the second video mode. The thirteenth aspect defines one aspect of the difference in shooting conditions between the first moving image mode and the second moving image mode. High-quality still images can be extracted.

According to a fourteenth aspect, in any one of the first to thirteenth aspects, the first moving image file and the second moving image file have at least part in common except for the extension of the file name. Thereby, the user can easily identify that the first moving image file and the second moving image file are files generated by dividing one moving image.

To achieve the above-described object, an imaging method according to a fifteenth aspect of the present invention includes a direction detection unit for detecting the direction of an imaging device with respect to the direction of gravity, an imaging unit for capturing a moving image, and a moving image captured by the imaging unit. and a control unit having a first moving image mode capable of extracting a still image from a file, wherein, in the first moving image mode, the direction detection unit changes the direction of the image capturing device with respect to the direction of gravity. a determining step of determining whether or not a direction change has occurred in the first moving image mode, a moving image generating step of dividing the moving image by the control unit to generate a first moving image and a second moving image; and a first moving image and a moving image saving step of saving the first moving image file of the above in a first folder of the storage unit, and saving the second moving image file of the second moving image in a second folder of the storage unit. According to the imaging method according to the fifteenth aspect, the user can easily select a desired frame from the captured moving image as in the first aspect. Note that the fifteenth aspect may further include the same configuration as those of the second to fourteenth aspects.

To achieve the above-described object, a program according to a sixteenth aspect of the present invention includes: a direction detection unit that detects the direction of an imaging device with respect to the direction of gravity; a shooting unit that shoots a moving image; and a control unit having a first moving image mode capable of extracting a still image from a file. If it is determined that a direction change has occurred in the first moving image mode, the control unit divides the moving image to generate a first moving image and a second moving image, and stores the first moving image file of the first moving image. a second moving image file of a second moving image in a second folder of a storage unit. According to the program according to the sixteenth aspect, the user can easily select a desired frame from the captured moving image, as in the first and fifteenth aspects. Note that the sixteenth aspect may further include the same configuration as those of the second to fourteenth aspects. Also, the programs of these aspects may be recorded in a non-temporary recording medium recording computer-readable codes.

As described above, according to the imaging apparatus, imaging method, and program of the present invention, a user can easily select a desired frame from a captured moving image.

FIG. 1 is a diagram showing the configuration of a camera according to the first embodiment. FIG. 2 is a diagram showing the functional configuration of the image processing apparatus. FIG. 3 is a diagram showing the coordinate system of the camera. FIG. 4 is a diagram illustrating an example of posture determination and moving image division. FIG. 5 is a flow chart showing the processing of the imaging method. 6A and 6B are diagrams showing changes in the posture of the camera and how the moving image is divided and saved. FIG. 7 is a diagram showing an example of a folder structure. FIG. 8 is another diagram showing an example of the folder structure. FIG. 9 is a diagram showing an example of icon display for a vertical video and a horizontal video. FIG. 10 is a diagram showing how thumbnail images are attached to a vertical moving image and a horizontal moving image. FIG. 11 is another flow chart showing the processing of the imaging method. FIG. 12 is a diagram showing another example of posture determination and moving image division. FIG. 13 is a diagram showing still another example of posture determination and moving image division. FIG. 14 is a diagram showing still another example of posture determination and moving image division. FIG. 15 is a diagram showing still another example of posture determination and moving image division. FIG. 16 is still another flow chart showing the processing of the imaging method. FIG. 17 is still another flow chart showing the processing of the imaging method. FIG. 18 is a flow chart showing processing of moving image recording in the first mode and the second mode. FIG. 19 is a flowchart (a continuation of FIG. 18) showing the moving image recording process in the first mode and the second mode. FIG. 20 is a diagram showing how a frame to be extracted as a still image is selected. FIG. 21 is an external view of a smartphone according to the second embodiment. FIG. 22 is a block diagram showing the configuration of the smartphone according to the second embodiment. FIG. 23 is a diagram illustrating the posture of the smartphone according to the second embodiment; FIG. 24 is a diagram illustrating how the orientation of the smartphone is determined based on the angle.

DETAILED DESCRIPTION OF THE INVENTION Modes for carrying out an imaging apparatus, an imaging method, and a program according to the present invention will be described in detail below with reference to the accompanying drawings.

<First embodiment>
<Overall Configuration of Imaging Device>
FIG. 1 is a diagram showing the configuration of a camera 10 (imaging device) according to the first embodiment. The camera 10 is composed of an interchangeable lens 100 (image pickup unit, image pickup device) and an image pickup device main body 200 (image pickup device). . The interchangeable lens 100 and the imaging device main body 200 can be attached and detached via a mount (not shown).

<Configuration of Interchangeable Lens>
The interchangeable lens 100 includes a zoom lens 110 , a focus lens 120 , an aperture 130 and a lens driver 140 . The lens driving unit 140 advances and retreats the zoom lens 110 and the focus lens 120 according to a command from the image processing device 240 (lens driving control unit 240F in FIG. 2) to perform zoom (optical zoom) adjustment and focus adjustment. Zoom adjustment and focus adjustment may be performed in response to a command from the image processing device 240, or may be performed in response to a zoom operation or focus operation (rotation of a zoom ring or focus ring (not shown), etc.) performed by the user. good. Also, the lens drive unit 140 controls the aperture 130 according to a command from the image processing device 240 to adjust the exposure. On the other hand, information such as the positions of the zoom lens 110 and the focus lens 120 and the opening degree of the diaphragm 130 is input to the image processing device 240 . Note that the interchangeable lens 100 has an optical axis L. As shown in FIG.

<Structure of imaging device body>
The imaging device main body 200 includes an imaging element 210 (imaging unit), an AFE 220 (AFE: Analog Front End, imaging unit), an A/D converter 230 (A/D: Analog to Digital, imaging unit), an image processing device 240, and an operation unit. 250, a storage unit 260, a monitor 270, and an orientation sensor 280 (direction detection unit). The imaging device main body 200 may have a shutter (not shown) for blocking light to be transmitted through the imaging element 210 . The imaging element 210 has a light receiving surface in which a large number of light receiving elements are arranged in a matrix. Subject light transmitted through the zoom lens 110, the focus lens 120, and the diaphragm 130 forms an image on the light receiving surface of the imaging element 210, and is converted into an electric signal by each light receiving element. A color filter of R (red), G (green), or B (blue) is provided on the light receiving surface of the imaging device 210, and a color image of the subject can be acquired based on the signals of each color. As the imaging element 210, various photoelectric conversion elements such as CMOS (Complementary Metal-Oxide Semiconductor) and CCD (Charge-Coupled Device) can be used. The AFE 220 removes noise and amplifies the analog image signal output from the imaging device 210, and the A/D converter 230 converts the captured analog image signal into a digital image signal having a gradation range.

<Configuration of image processing apparatus>
FIG. 2 is a diagram showing the functional configuration of the image processing device 240. As shown in FIG. The image processing device 240 includes an image acquisition section 240A, a direction detection section 240B (direction detection section), a control section 240C (control section), an event detection section 240D (event detection section), and a still image extraction section 240E (still image extraction section). , and a lens drive control section 240F (lens drive control section). The control unit 240</b>C has a reception unit that receives user instructions, and this reception unit can receive user instructions via the operation unit 250 . Then, based on the digital image signal input from the A/D converter 230, the image processing device 240 shoots a moving image, generates a file, generates a still image file, processes a plurality of frames constituting a moving image, and extracts a still image. etc. are processed. Details of the processing by the image processing device 240 will be described later.

The functions of the image processing device 240 can be implemented using various processors. Various processors include, for example, a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to realize various functions. In addition, the above-mentioned various processors include GPUs (Graphics Processing Units), which are processors specialized for image processing, and FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing. Programmable Logic Devices (PLDs) are also included. Furthermore, a dedicated electric circuit, which is a processor having a circuit configuration specially designed for executing specific processing such as ASIC (Application Specific Integrated Circuit), is also included in the various processors described above.

When the processor or electric circuit described above executes software (program), the processor (computer) readable code of the software to be executed is stored in a non-temporary recording medium such as a ROM (Read Only Memory). The processor then references that software. The software stored in the non-temporary recording medium includes a program for executing the imaging method according to the invention (a program for operating the imaging device according to the invention). Codes may be recorded in non-temporary recording media such as magneto-optical recording devices and semiconductor memories instead of ROMs. When processing using software, for example, RAM (Random Access Memory) is used as a temporary storage area, and data stored in EEPROM (not shown) can also be referred to. can.

The image processing device 240 includes a ROM 242 in addition to the above units. The ROM 242 stores computer-readable codes of programs necessary for photographing, recording, displaying images, etc. (including programs for executing the imaging method according to the present invention).

<Operation part>
The operation unit 250 has a release button (not shown), operation buttons (for example, cross button, Quick button, OK button, etc.), dials, switches, etc., and the user can set the shooting mode, instruct movie shooting, extract still images, etc. Various operations can be performed. Note that the monitor 270 (touch panel type) may be used as the operation unit 250 .

<Memory unit>
The storage unit 260 (storage unit) is composed of non-temporary recording media such as various magneto-optical recording media and semiconductor memories, and their control circuits, and stores moving images, still images, still images extracted from moving images, and the like. A recording medium that can be attached to and detached from the imaging apparatus main body 200 can be used. A captured image (moving image, still image) may be transmitted to and stored in an external recording medium (storage unit), for example, by wireless communication.

<Monitor and viewfinder>
The monitor 270 (display device) is configured by a touch panel type liquid crystal display panel, and can display moving images, still images, still image extraction frames, and the like. The monitor 270 can be arranged on the back side, the top side, or the like of the imaging device body 200 . Also, the camera 10 may have a viewfinder.

<Shooting mode>
The camera 10 can set any one of a still image shooting mode, a moving image shooting mode for still image extraction (first moving image mode), and a normal moving image shooting mode (second moving image mode) as a shooting mode. A still image shooting mode and a normal moving image shooting mode are modes similar to those of a normal digital camera. On the other hand, the movie shooting mode for extracting still images is a mode that allows you to extract still images from movie files. movies). Specifically, in the movie shooting mode for still image extraction, at least one of the shutter speed, autofocus speed, automatic exposure follow-up speed, and white balance follow-up speed is set to be faster than in the normal movie shooting mode, and/or the frame rate is set higher than the normal moving image shooting mode. Also, the resolution and frame rate are set to the maximum values that can be set by the camera 10 (eg, 4,000×2,000 pixels, 30 frames/second), and the color tone is also set on the premise of still image extraction. The upper limit of the ISO sensitivity is also set higher than in the normal movie shooting mode.

For example, the shutter speed is set to a value corresponding to the frame rate of the moving image to be recorded in the normal moving image shooting mode (1/30 second if the frame rate is 30 frames/second). On the other hand, in the moving image mode for still image extraction, the frame interval is set to be faster (for example, less than 1/30 second). In normal movie shooting mode, the shutter speed is set to a value corresponding to the frame rate of the movie so that the movie can be reproduced smoothly. For this reason, in the movie shooting mode for extracting still images, the shutter speed is set faster than in the normal movie shooting mode (faster than the frame interval). becomes possible. Similarly, by increasing the upper limit of ISO sensitivity (ISO: International Organization for Standardization), the shutter speed can be increased, thereby extracting a still image with less blurring. Further, the speed of autofocus, the follow-up speed of automatic exposure, the follow-up speed of auto white balance, etc. are set faster than in the normal moving image shooting mode. As a result, it is possible to obtain many frames in which the subject is in focus, frames with appropriate exposure, and the like. As for the frame rate, by setting a high rate, the frame interval of moving images becomes shorter, and the number of frames that can be extracted as still images increases.

According to the moving image shooting mode for still image extraction described above, the frames constituting the moving image can be extracted as still images afterward. Therefore, the user can easily take a photograph of an event (natural phenomenon, accident, happening, etc.) whose occurrence is unknown at any time, a photograph of a subject whose state changes with the passage of time, or a photograph of an instantaneous state of a moving subject. be able to. At this time, as will be described later in detail, still images can be extracted not only at the timing when the still image recording is instructed, but also at other timings. Therefore, the user can obtain a still image at desired timing. Also, high-quality still images can be extracted by setting shooting conditions suitable for still image extraction (shutter speed, resolution, frame rate, etc. described above). In addition to such post-extraction of still images, still images can be captured (saved) during moving image capturing in the moving image capturing mode for still image extraction and the normal moving image capturing mode.

<Position of camera>
FIG. 3 is a diagram showing an example of a coordinate system defined for the camera 10. As shown in FIG. In the example shown in FIG. 3, the X-axis is the direction of the optical axis (the direction toward the subject), the Z-axis is the direction toward the top of the imaging device body 200, and (X-axis, Y-axis, Z-axis) is a right-handed coordinate system. construct the system. Whether the camera 10 is in the vertical position (the top and bottom surfaces of the camera 10 are parallel to the vertical direction) or in the horizontal position (the top and bottom surfaces of the camera 10 are parallel to the ground) depends on, for example, the angle around the X-axis (imaging device main body 200 is set to 0 degrees when the top and bottom surfaces are parallel to the ground). FIG. 4 is a diagram showing an example of the relationship between the angle about the X-axis and the posture (vertical or horizontal). The direction detection unit 240B (direction detection unit) determines “horizontal” when the angle is in the range of (−45 deg to +45 deg) and (+135 deg to +225 deg), and otherwise determines “vertical”. can do. As described above, the camera 10 includes the orientation sensor 280 (for example, an acceleration sensor that measures acceleration in three axial directions; see FIG. 1). Ten poses (orientations relative to the direction of gravity) can be detected.

Note that the range of angles shown in FIG. 4 is an example, and the posture may be determined in a different range. For example, the direction detection unit 240B may determine "horizontal" when the angle is in the range of (-30deg or more +30deg or less) and (+150deg or more +210deg or less), and otherwise determine "vertical". .

<Processing in Movie Shooting Mode for Still Image Extraction>
An imaging method in the camera 10 having the above configuration will be described. FIG. 5 is a flow chart showing the processing in the moving image shooting mode for still image extraction. For example, by operating a mode dial (not shown) of the operation unit 250, the camera 10 enters a moving image shooting mode for still image extraction. Then, the control unit 240C sets the photographing unit (interchangeable lens 100, image sensor 210, AFE 220, A/D converter 230) to the first moving image mode. Then, the processing of FIG. 5 is started, and the control unit 240C initializes the state of the camera 10 (acquisition and display of the live view image, initialization of orientation, start of orientation determination, etc.) (step S100: initialization step ).

<Video recording start instruction>
The control unit 240C determines whether or not a moving image recording start instruction has been issued (step S110: photographing step, moving image generating step). For example, when a release button (not shown) of the operation unit 250 is pressed, it can be determined that "a moving image recording start instruction has been given." If the determination is affirmative, the process proceeds to step S120.

<Determination of Posture>
The orientation detection unit 240B detects the orientation (direction relative to the direction of gravity) of the camera 10 (imaging device) based on the output of the orientation sensor 280 (step S120: orientation detection step, judgment step). As described above with reference to FIGS. 3 and 4, the direction detection unit 240B determines whether the posture is “vertical”, “horizontal”, or “undefined” based on the angle about the X-axis. The "undefined" posture can include a state in which the angle around the Y-axis is large and "vertical" or "horizontal" cannot be specified, such as when the user holds the camera 10 facing the ground or the sky. . Note that the direction detection unit 240B does not have to detect the “undefined” state of the camera 10 .

<Recording to folder according to posture>
If it is determined in step S120 that the orientation of the camera 10 is "horizontal" or "undefined", the control unit 240C compresses the captured moving image and creates a new moving image file. Then, the control unit 240C starts recording (saving) the moving image file in the horizontal moving image folder (first folder) of the storage unit 260 (step S130: moving image saving step). Moving images can be compressed, for example, in MPEG format (MPEG2, MPEG4, etc.). On the other hand, when the orientation of the camera 10 is determined to be "vertical", the control unit 240C starts recording (saving) a moving image file in a vertical moving image folder (second folder) different from the horizontal moving image folder (step S140: moving picture saving step). A microphone (not shown) provided in the camera 10 may be used to record and store sounds together. In addition, "horizontal video" and "vertical video" are videos when the posture is horizontal or indefinite and vertical, respectively. is a folder to save .

<Treatment when posture is indeterminate>
In addition, in the flowchart of FIG. 5, recording in the same folder and the same file is continued in "when the attitude changes from horizontal to undefined" and "when the attitude changes from undefined to horizontal". Then, "when the posture changes from vertical to indefinite" and "when the posture changes from indefinite to vertical", it is determined that "the posture has changed", and the video is divided and recorded in a separate folder. ing. However, when the posture is indefinite, it may be recorded in the vertical video folder, or may be recorded in a folder different from the horizontal video folder and the vertical video folder (see other modes described later).

The direction detection unit 240B continues to detect the orientation of the camera 10 in parallel with the moving image shooting, and determines "whether or not the direction of the camera 10 (imaging device) has changed with respect to the direction of gravity" (step S170: determination step). . “Whether or not a direction change has occurred” can be determined based on the output of the attitude sensor 280 (for example, acceleration in three axial directions). If the direction detection unit 240B determines that "a direction change has occurred", specifically, if it determines that "the orientation of the camera 10 has changed between 'horizontal or undefined' and 'vertical'" (step S170). Yes), the control unit 240C closes the moving image file being recorded and divides the moving image (step S180: moving image generating step). Then, the determination in step S190 is denied and the process returns to step S120.

In the returned step S120, the control unit 240C determines whether the orientation of the camera 10 is "vertical", "horizontal", or "undefined", as at the start of recording the moving image file. Then, the control unit 240C saves the new moving image file related to the division in the horizontal moving image folder or the vertical moving image folder of the storage unit 260 according to the result. For example, when the orientation of the camera 10 changes from “horizontal” at the start of recording to “vertical”, the control unit 240C stores a moving image file (the first moving image file of the first moving image) in the horizontal moving image folder at the start of recording. A new moving image file (second moving image file of the second moving image) generated due to the posture change is stored in the vertical moving image folder. It should be noted that, in step S170, when the orientation of the camera changes again (Yes in step S170), the moving image does not have to be closed. This is because if the moving image is repeatedly divided each time the camera posture changes, the number of moving image files will increase too much.

The image acquisition unit 240A, the direction detection unit 240B, and the control unit 240C continue dividing the moving image and saving the moving image file according to the orientation of the camera 10 until the end of the moving image shooting (while the determination in step S190 is No). continue. The direction detection unit 240B and the control unit 240C add information about the attitude of the camera 10 (direction with respect to the direction of gravity) to the first moving image file and the second moving image file. It is preferable to be able to refer to The direction detection unit 240B and the control unit 240C can record information about the orientation of the camera 10 in the header and footer. For example, it can be "01" for vertical, "02" for horizontal, and "03" for others (oblique, indeterminate, etc.).

As in the example of FIG. 6, there is a case where the camera 10 shifts from the horizontal state to the vertical state and ends the shooting of the moving image in the vertical state. In this case, the end timing of the second moving image (vertical moving image; moving image in the vertical state) corresponds to reception of the user's second moving image end instruction or moving image shooting end instruction by the reception unit in the control unit 240C. This reception unit can receive a user's instruction via the operation unit 250 . Similarly, there is a case where the vertical state is changed to the horizontal state again, and the shooting of the moving image ends in the horizontal state. In this case, the end timing of the first moving image (horizontal moving image; moving image in horizontal state) corresponds to reception of the user's first moving image end instruction or moving image shooting end instruction by the reception unit. These moving image end timings are the same in the aspects described later.

FIG. 6 is a diagram showing an example of determining a direction change based on an angle about the X-axis. In the example of FIG. 6, the posture is determined to be "horizontal" during the elapsed time from 0 to t1, and the angle exceeds 45 degrees (threshold value) at the elapsed time t1 and is determined to be "vertical". The moving image is divided at time t1. The control unit 240C saves the moving image file of the first moving image (first moving image file) whose elapsed time is from 0 to t1 in the horizontal moving image folder (first folder), and saves the moving image file of the second moving image after the elapsed time t1 ( second moving image file) is stored in the vertical moving image folder (second folder). It should be noted that, since the attitude of the camera 10 may change minutely, it may be determined that "the attitude has changed" when the state in which the attitude has changed continues for a predetermined time or longer. Note that FIG. 6 shows an example in which the orientation of the camera 10 changes from horizontal to vertical. The moving image becomes the second moving image, the vertical moving image folder becomes the first folder, and the horizontal moving image folder becomes the second folder;

The control unit 240C not only changes the posture of the camera 10 (Yes in step S170), but also determines in step S150 that "an instruction to end moving image recording (instruction to end moving image shooting) has been issued", and in step S160 " Also when it is determined that the remaining capacity of the recording medium is not sufficient, the moving image file is closed (step S180). However, in these cases, the moving image shooting is also ended (Yes in step S190). In step S150, the control unit 240C can determine that "an instruction to end moving image recording has been issued" when a release button (not shown) of the operation unit 250 is pressed, or when an end instruction is issued via the monitor 270. can.

In the first embodiment, when it is determined that "the direction (orientation) of the camera 10 (imaging device) with respect to the direction of gravity has changed in the moving image capturing mode for still image extraction (first moving image mode)", A moving image is divided to generate a horizontal moving image (first moving image) and a vertical moving image (second moving image). Then, save the horizontal video file (first video file) in the horizontal video folder (first folder), and save the vertical video file (second video file) in the vertical video folder (second folder). do.

Thereby, the user can easily identify the posture of the camera 10 when each moving image is taken, and can easily select the moving image including the frame to be extracted as a still image by using the posture of the camera 10 as a clue. In addition, the user can search for a frame to be extracted as a still image in a divided moving image whose recording time is shortened, so that a desired frame can be quickly searched. Thus, according to the first embodiment, the user can easily select a desired frame from the captured moving image.

<Example of folder structure>
FIG. 7 is a diagram showing an example of a folder structure for storing moving images. In the example shown in the figure, a folder is generated for each shooting date, and a folder is generated for each shooting number within the folder for each shooting date. For example, in the flowchart of FIG. 5, one photographing number can be set from when the start of moving image recording is instructed in step S110 to when the moving image recording ends in step S190. A horizontal moving image folder and a vertical moving image folder are provided for each shooting number folder, and the horizontal moving image and the vertical moving image are stored, respectively. Such a folder structure can be displayed on the monitor 270 by the control unit 240C.

FIG. 8 shows how the folder for horizontal moving images (first folder) and the folder for vertical moving images (second folder) are displayed on the monitor 270 with horizontal and vertical icons, respectively. According to such a folder display mode, the user can easily visually grasp the vertical video folder and the vertical video folder, and can easily select a desired frame.

<Video file icon display and file name>
FIG. 9 is a diagram showing an example of icon display of a moving image file. Part (a) of the same figure shows a state in which a moving image file of a horizontal moving image is displayed with the characters "horizontal moving image" added to the horizontal icon. A symbol in the icon indicates that the file can be reproduced by specifying the file. Similarly, part (b) of FIG. 9 shows an example of icon display for a moving image file of a vertical moving image, and the vertical icon is displayed with the characters "vertical moving image" added. In the example of FIG. 9, the file name is file creation date (October 23, 2018), shooting number (001), vertical and horizontal identification information (horizontal video is H (Horizontal), vertical video is V (Vertical)), file number (001, 002, . . . ), file size (10 MB, etc.), and extension (here, “.mpg”). The moving image file of the horizontal moving image (first moving image file) and the moving image file of the vertical moving image (second moving image file) are partially common except for the extension of the file name. The vertical/horizontal identification information is an example of information regarding the direction of the camera 10 (imaging device) with respect to the direction of gravity.

The file name may include information indicating the file type, file creation date and time, and vertical/horizontal identification information. For example, give the file name "IMG201801011230_H.MOV" to the horizontal video file created at 2018/1/1 12:30, and the vertical video file created at 2018/1/1 12:40 A file name of "IMG201801011240_H.MOV" can be given to the file. "IMG" in the file name is an example of information indicating that the file is an image file. Also in this case, the file name may contain information indicating the file size.

FIG. 10 is a diagram showing how thumbnail images are displayed on icons. Part (a) of the figure is an example in the case of horizontal moving images, and part (b) is an example in the case of vertical moving images. The image to be displayed may be the first frame of the horizontal moving image or the vertical moving image, or may be an intermediate frame (for example, a frame in which an event occurs).

By displaying icons, displaying thumbnails, and assigning file names to moving image files as illustrated above, the user can easily distinguish between vertical moving images and horizontal moving images, and can easily select moving images containing desired frames. Note that icon display, thumbnail display, and file name assignment of moving image files can be performed by the control unit 240C.

<Other Aspects of Splitting and Recording Moving Images>
Another aspect of dividing and recording (saving) a moving image will be described. In the flowcharts described below, the same step numbers are assigned to the same processes as those in FIG. 5, and detailed description thereof will be omitted.

(Aspect 1: Aspect in which the period during posture change is set as an independent video file)
In the embodiment described with reference to FIG. 5, it is determined that "a direction change has occurred" when the posture of the camera 10 (angle around the X axis) exceeds the threshold value. However, in mode 1 shown in FIG. 11, the moving image during posture change is stored in a separate folder as a moving image file separate from the horizontal moving image and the vertical moving image. FIG. 11 is a flowchart showing the processing in mode 1. The orientation detection unit 240B determines whether the orientation of the camera 10 (direction relative to the direction of gravity) is "horizontal,""vertical," or "changing." (step S122: posture detection step).

When the posture is "horizontal", the control unit 240C starts recording (saving) a moving image file (first moving image file) in a horizontal moving image folder (first folder) as a horizontal moving image (first moving image) (step S130). If it is "vertical", the control unit 240C starts recording as a vertical moving image (second moving image) in the vertical moving image folder (second folder) (step S140). If it is "changing", recording is started as a changing moving image (third moving image) in a changing moving image folder (third folder) (step S124). In mode 1, in addition to the horizontal moving image folder and the vertical moving image folder, the changing moving image folder can be created in the hierarchy below the folder of each shooting number.

In aspect 1, when the direction detection unit 240B determines that "the change in the orientation of the camera 10 has started or completed" (Yes in step S172 (determination step)), the moving image is divided (moving image file closed, new moving image file created). generate). At this time, since the posture of the camera 10 may change minutely due to camera shake or the like, the direction detection unit 240B determines that "posture change has started" when the angle around the X axis changes by a predetermined value or more within a certain period of time. . Similarly, direction detection section 240B preferably determines that "posture change is completed" when the time period in which the change in angle is within a predetermined range continues for a certain period of time or longer. For example, in the example shown in FIG. 12, the angle changes greatly between elapsed times t1 and t2, and is stable between elapsed times t3 and t4.

Therefore, the direction detection unit 240B determines that "the attitude is changing" from the elapsed time t2 to t4, and determines that "the attitude change is completed" at the elapsed time t4 (determination of the actual attitude and attitude of the camera 10). Considering the time lag until the result changes). In response to this determination, the control unit 240C determines whether the horizontal video (first video; elapsed time from zero to t2) and vertical video (second video; after elapsed time t4) are displayed in time from elapsed time t2 to t4. A moving image (third moving image) in between is generated. As shown in FIG. 12, the moving image during change (third moving image) includes a moving image of the first period during which the orientation of the camera 10 is changing (direction change occurs).

(Aspect 2: Aspect in which recording is continued to the video file of the previous period while the posture is changing)
In mode 1 described above, during a posture change (first period), an independent moving image (third moving image) is recorded. However, in mode 2, while the posture is changing, the moving image file for the period before the start of the change is continuously recorded. Specifically, as shown in FIG. 13, in mode 2, the direction detection unit 240B detects that the camera 10 (imaging device) is in the first state (state in which the bottom surface of the camera 10 is in the horizontal direction; horizontal state), and in the second state. It detects whether it is in a state (a state in which the bottom surface of the camera 10 is aligned with the direction of gravity; vertical state) or a third state (a state in which the state changes between the first state and the second state). Then, the control unit 240C records the moving image of the period in which the camera 10 is in the third state (while the posture is changing) in the first moving image in the first folder (horizontal moving image folder).

(Aspect 3: Aspect in which a part of the period after the end of the posture change is continuously recorded in the moving image file during the period during the change)
In situations where the attitude of the camera 10 has changed, the composition on the screen of the monitor 270 may change significantly. In that case, the camera 10 (imaging device) has a period during which the exposure of the moving image (AE: AutoExposure), the white balance of the moving image (AWB: Auto White Balance), or the focus of the subject in the moving image (AF: Auto Focus) is adjusted. may be needed. Therefore, in mode 3, the predetermined period (second period) after the orientation change is completed is handled in the same manner as the case where the orientation of the camera 10 is changing even if the orientation is stable. Specifically, for example, as shown in FIG. 14, the direction detection unit 240B " The attitude of the camera 10 has changed."

Then, these first period and second period are recorded (saved) as a third moving image. That is, the third moving image also includes the moving image of the second period after the direction change ends. Note that the control unit 240C divides the moving image, sets the third moving image as a moving image file (third moving image file) separate from the horizontal moving image (first moving image) and the vertical moving image (second moving image), and creates a horizontal moving image folder (second moving image). 1 folder). However, the third moving image may be recorded in an independent folder for changing moving images (third folder). An image during a posture change or a period when the shooting conditions are unstable may be less important, but in mode 3, such a period is saved as a moving image separate from the horizontal moving image and the vertical moving image, so it is extracted as a still image. You can easily select the frame you want to use and organize video files and folders.

(Aspect 4: Aspect in which part of the period after the end of posture change is continuously recorded in the original video file)
In mode 3 described above, the period during which the orientation of the camera 10 is changed (first period) and the period (second period) determined after the orientation change is completed are referred to as the "third moving image". However, in aspect 4, as shown in FIG. 15, the first period (elapsed time t2 to t4) and the second period (elapsed time t4 to t5) are defined as the "first moving image" in the horizontal moving image folder (first folder).

(Aspect 5: Aspect in which moving images are not recorded during a posture change)
In modes 1 to 4 described above, a moving image is recorded during the period during which the posture is changed (and the period until the posture is stabilized thereafter), but a mode in which the moving image is not recorded during the posture change can also be adopted. Specifically, as shown in the flowchart of FIG. 16, the orientation detection unit 240B detects whether the orientation of the camera 10 is horizontal (first state), vertical (second state), or changing (third state). is detected (step S122). Then, when the posture is changing, recording of the moving image file is not started (returns to step S122).

Further, when the orientation of the camera 10 is horizontal or vertical and the orientation of the camera 10 starts to change after recording of the moving image has started (step S130, step S140) (Yes in step S174 (judgment step)), the control unit 240C moves to step At S180, the moving image file is closed and the process returns to step S122. Then, the camera 10 resumes recording the moving image after the orientation of the camera 10 becomes horizontal or vertical (steps S130 and S140). The closed video file is saved in the corresponding folder. Images during posture changes may be of low importance, but in aspect 5, since moving images are not saved during such periods, the remaining capacity of the recording medium can be saved. Also, the user can easily select a frame to be extracted as a still image and organize moving image files and folders.

(Aspect 6: Aspect in which the video is not divided immediately after the event occurs even if there is a posture change)
Aspect 6 shown in the flowchart of FIG. 17 is another aspect of the case where the attitude change of the camera 10 has started or completed (Yes in step S172 (determination step)). In this aspect, if a specific event has occurred before a predetermined time (described as "T seconds before" in step S176) from the timing when the attitude change of the camera 10 is started or completed, the control unit 240C Do not close the moving image file (Yes in step S176 and do not proceed to step S180). Therefore, in this case, the moving image is not divided. In addition, not only when a specific event has occurred before a certain period of time, but also when a specific event is being detected (an event is continuing), and when a user instruction is being received by the reception unit (user (including the case where the processing based on the instruction of the user or the processing based on the instruction of the user is being continued) is the same.

The control unit 240C can set the value of T (for example, 30 seconds, but other values are also possible) according to the user's operation via the operation unit 250 or without depending on the user's operation. can be done. Whether or not a specific event has occurred is determined by the event detection unit 240D (see FIG. 2) automatically detecting the amount of movement of the subject, changes in size and shape, changes in brightness, etc., through well-known image processing. You may Further, whether or not a specific event has occurred may be determined based on a user's instruction via the operation unit 250 or reception unit (including the case where the monitor 270 is used as a touch panel). For example, the control unit 240C may determine that "a (specific) event has occurred" when the reception unit receives a still image shooting instruction from the user during video shooting. When an event is detected, the event detection section 240D and the control section 240C preferably add information indicating event detection timing to the moving image file (for example, record it in the header or footer). By referring to this information, the user can easily select a frame to be extracted as a still image (described later).

As described above, even if there is a change in the posture of the camera 10, if an event occurs before the change in posture, the video is not divided, so that the user can confirm the series of events related to the event in one video. A desired frame can be easily selected from the captured moving image. The flowchart in FIG. 17 is based on FIG. 11 (a mode in which the moving image during posture change is stored separately from the horizontal and vertical moving images and stored in a separate folder), but the flow charts in FIGS. good.

In addition, in the camera 10, it may be determined in accordance with the user's operation via the operation unit 250 which of the modes 1 to 6 described above is used to divide and record (save) the moving image. .

<Processing when Considering First Mode and Second Mode>
The flowcharts shown in FIGS. 5, 11, 16, and 17 describe the processing in the first moving image mode, but the processing in consideration of the first moving image mode and the second moving image mode will be described with reference to FIGS. . 5, 11, 16, and 17 are assigned the same step numbers, and detailed description thereof will be omitted.

The control unit 240C determines whether the camera 10 is in the first moving image mode (still image extraction moving image mode) or the second moving image mode (normal moving image mode) (step S112). In the case of the first moving image mode, the control unit 240C sets shooting conditions suitable for the first moving image mode (step S114). The shooting condition set in step S114 is a shooting condition that emphasizes extraction of a still image rather than watching a moving image itself, as described above in the section of "shooting mode". Since the processing after step S114 in the first moving image mode is the same as the flowchart in FIG. 5, detailed description thereof will be omitted.

On the other hand, if the result of determination in step S112 is the second moving image mode, the control unit 240C sets shooting conditions suitable for the second moving image mode (step S191 in FIG. 19). The direction detection unit 240B determines the orientation of the camera 10 (step S192), and the control unit 240C starts recording (saving) the moving image file in the second moving image folder of the storage unit 260 (step S193: moving image saving step). . The processing of steps S194 and S195 is the same as the processing of steps S150 and S160 for the first moving image mode. If there is an instruction to end moving image recording (Yes in step S194) and if the remaining capacity of the recording medium is insufficient (No in step S195), the control unit 240C closes the moving image file (step S197), and the step in FIG. The process returns to S190 (determining whether or not moving image shooting is finished).

If there is no moving image recording end instruction (No in step S194) and the remaining capacity of the recording medium is sufficient (Yes in step S195), the control unit 240C continues recording the moving image. However, in the second moving image mode, unlike in the first moving image mode, the direction detection unit 240B does not detect a change in the posture (direction change) of the camera 10 (No in step S196), but also detects it (step Yes in S196) also does not divide the moving image. The reason why the moving image is not divided in the second moving image mode even if the posture of the camera 10 changes is that the second moving image mode is not a mode for shooting moving images for the purpose of extracting still images, but the moving image is divided like the first moving image mode. This is because it is not necessary to divide the .

<Example of folder structure>
When considering the first and second moving image modes, the control unit 240C can generate a folder for the first moving image mode and a folder for the second mode in folders for each shooting date in the storage unit 260 (storage unit). . In this case, the control unit 240C creates a folder for each shooting number in the folder for the first moving image mode, and creates a folder for horizontal movies (example of first folder) and a folder for vertical movies (example of first folder) in the folder for each shooting number. Example of two folders) are provided, and horizontal moving images (example of first moving image) and vertical moving images (example of second moving image) are stored respectively. On the other hand, for folders in the second moving image mode, the control unit 240C records files of moving images shot on the shooting date in folders for each shooting date. In addition, the control unit 240C does not provide the folders for the first and second moving image modes under the folder for the shooting date, but places the folders for the first and second moving image modes at the upper level, and then creates the folders for the shooting date at the lower level. may be provided. Such a folder structure can be displayed on the monitor 270 by the control section 240C, as in the case of FIG.

<Extraction of still image>
As described in the following example, the camera 10 can extract frames forming a moving image as still images. Still images may be extracted by another device such as a personal computer using the moving image file recorded by the camera 10 .

(Example 1)
The control unit 240C causes the monitor 270 to reproduce and display the file selected by the user from the moving image files of the horizontal moving image, the vertical moving image, or the changing moving image (the first moving image, the second moving image, and the third moving image). Instead of continuous reproduction, frame-by-frame reproduction (frame by frame display according to user's operation) may be performed. The control unit 240C stops the reproduction by the user's operation, and by confirming the selection of the frame displayed at the time of stopping according to the user's operation, the still image extracting unit 240E extracts the frame as a still image. .

(Example 2)
If information indicating the detection of an event is added to the moving image file, the control unit 240C refers to the information and displays a plurality of frames including the frame in which the event was detected on the monitor 270 (see FIG. 1). good too. FIG. 20 is an example of such a display, where five frames 1051-1055 are displayed, including frame 1052. FIG. Further, a camera-like icon 1060 is attached to a frame 1052 in which a still image is recorded (recording of a still image is an example of event occurrence) by the still image extraction unit 240E. This allows the user to recognize that the frame is a recorded still image.

Note that the range of frames for still image extraction to be displayed is temporally before (frame 1051) and after (frame 1052 in the example of FIG. 20) the frame corresponding to the recorded still image file. frames 1053-1055) may be included. Further, the range of still image extraction frames to be displayed may be in a mode of "including only temporally previous frames" or a mode of "including only temporally subsequent frames". The range of still image extraction frames to be displayed may be set according to the user's designation, or may be set regardless of the user's designation. Also, the range of frames to be displayed may be specified by time (for example, one second before and after the frame in which the still image is recorded), or by the number of frames (for example, the number of frames before and after the frame in which the still image is recorded). 10 frames each). Alternatively, in the initial state, only the frames recorded as still images are displayed in a list, and a frame selected from the list may be displayed temporally before and/or after.

<Selection of frame to extract>
The user can select a frame from the displayed frames for still image extraction. In the example of FIG. 20, the user can select a frame by checking checkbox 1070 for the desired frame via operation unit 250 and/or monitor 270, and can select multiple frames. Note that the frame in which the still image specified by the user is recorded is not necessarily recorded at the best timing. By displaying other related frames together, the user may wish to select other frames. Therefore, it is preferable to display on the monitor 270 such that the frame in which the still image was recorded, as well as the frames before and/or after it, can be selected. FIG. 20 shows a state in which the user has selected a frame 1053 different from the frame in which the still image was recorded (frame 1052 with icon 1060).

The still image extraction unit 240E extracts the selected frame as a still image (still image extraction step). Since the moving image file is stored in a moving image format such as MPEG format, the still image extraction unit 240E converts the selected frame data into a still image format (JPEG format, etc.). The user can extract a still image at a desired timing from the moving images (the first moving image, the second moving image, and the third moving image).

<Second embodiment>
Although the camera 10, which is a digital camera, has been described in the first embodiment, the configuration of the imaging device is not limited to this. Other imaging devices of the present invention can be, for example, a built-in or external PC camera (PC: Personal Computer), or a mobile terminal device having an imaging function as described below. .

Portable terminal devices, which are an embodiment of the imaging device of the present invention, include, for example, mobile phones, smart phones, PDAs (Personal Digital Assistants), and portable game machines. Hereinafter, a smart phone will be taken as an example, and a detailed description will be given with reference to the drawings.

FIG. 21 is a diagram showing the appearance of a smartphone 1 (imaging device) that is an embodiment of the imaging device of the present invention, in which part (a) is a front view and part (b) is a rear view. A smartphone 1 shown in FIG. 21 has a flat housing 2, and a display panel 21 (display device) as a display unit and an operation panel 22 (operation unit) as an input unit are provided on one surface of the housing 2. is provided with a display input unit 20 integrated with. The housing 2 includes a speaker 31, a microphone 32, an operation unit 40 (operation unit), camera units 41 and 42 (imaging device, imaging unit, event detection unit, control unit, still image extraction unit, lens drive control part) and a strobe 43 . The configuration of the housing 2 is not limited to this. For example, a configuration in which the display unit and the input unit are independent may be adopted, or a configuration having a folding structure or a slide mechanism may be adopted.

FIG. 22 is a block diagram showing the configuration of the smartphone 1 shown in FIG. 21. As shown in FIG. As shown in FIG. 22, the main components of the smartphone 1 include a wireless communication unit 11, a display input unit 20, a call unit 30, an operation unit 40, camera units 41 and 42, a strobe 43, and a storage unit. 50, an external input/output unit 60, a GPS receiving unit 70 (GPS: Global Positioning System), a motion sensor unit 80, a power supply unit 90, a main control unit 101 (shooting unit, moving image file generating unit, still image file generation unit, event detection unit, display control unit, still image extraction unit, lens drive control unit). In addition, as a main function of the smartphone 1, it has a wireless communication function for performing mobile wireless communication via a base station device and a mobile communication network.

Radio communication section 11 performs radio communication with a base station apparatus accommodated in a mobile communication network according to instructions from main control section 101 . Such wireless communication is used to transmit and receive various file data such as audio data and image data, e-mail data, etc., and to receive Web data, streaming data, and the like.

Under the control of the main control unit 101, the display input unit 20 displays images (still images and/or moving images), character information, etc. to visually transmit information to the user, and also allows the user to operate the displayed information. It is a so-called touch panel that detects, and includes a display panel 21 and an operation panel 22 .

In the display panel 21, an LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display), or the like is used as a display device. The operation panel 22 is a device that is placed so that an image displayed on the display surface of the display panel 21 can be visually recognized, and that detects one or more coordinates operated by a conductor such as a user's finger or pen. When such a device is operated by a user's finger or a conductor such as a pen, the operation panel 22 outputs a detection signal generated by the operation to the main control section 101 . Next, the main control unit 101 detects the operating position (coordinates) on the display panel 21 based on the received detection signal.

As shown in FIG. 21, a display panel 21 and an operation panel 22 of a smart phone 1 exemplified as an embodiment of the imaging device of the present invention are integrated to form a display input unit 20. 22 is arranged to completely cover the display panel 21 . When such an arrangement is adopted, the operation panel 22 may have a function of detecting user operations in areas outside the display panel 21 as well. In other words, the operation panel 22 has a detection area (hereinafter referred to as a display area) for the overlapping portion overlapping the display panel 21 and a detection area (hereinafter referred to as a non-display area) for the outer edge portion not overlapping the display panel 21. ) may be provided.

The call unit 30 includes a speaker 31 and a microphone 32, converts the user's voice input through the microphone 32 into voice data that can be processed by the main control unit 101, and outputs the data to the main control unit 101; 11 or the external input/output unit 60 can decode the audio data and output it from the speaker 31 . Further, as shown in FIG. 21 , for example, the speaker 31 can be mounted on the same surface as the display input unit 20 is provided, and the microphone 32 can be mounted on the side surface of the housing 2 .

The operation unit 40 is a hardware key using a key switch or the like, and is a device that receives instructions from a user. For example, as shown in FIG. 21, the operation unit 40 is mounted on the side surface of the housing 2 of the smartphone 1, is turned on when pressed with a finger or the like, and is turned off by the restoring force of a spring or the like when the finger is released. It is a button type switch.

The storage unit 50 stores control programs and control data of the main control unit 101, application software, address data associated with names and telephone numbers of communication partners, data of sent and received e-mails, web data downloaded by web browsing, It stores downloaded content data and temporarily stores streaming data and the like. The storage unit 50 is configured by an internal storage unit 51 built into the smartphone and an external storage unit 52 having a detachable external memory slot. The internal storage unit 51 and the external storage unit 52 that configure the storage unit 50 are each realized using a known storage medium.

The external input/output unit 60 functions as an interface with all external devices connected to the smartphone 1 . The smartphone 1 is directly or indirectly connected to another external device through communication or the like via the external input/output unit 60 . Examples of communication means include Universal Serial Bus (USB), IEEE1394, and networks (eg, the Internet and wireless LAN). In addition, Bluetooth (registered trademark), RFID (Radio Frequency Identification), infrared communication (Infrared Data Association: IrDA) (registered trademark), UWB (Ultra WideBand) (registered trademark), ZigBee (registered trademark) ) can also be cited as a means of communication.

External devices connected to the smartphone 1 include, for example, a wired/wireless headset, a wired/wireless external charger, a wired/wireless data port, a memory card or a SIM (subscriber) connected through a card socket. Identity Module Card)/UIM (User Identity Module Card) cards. In addition, external audio and video devices connected via audio and video I/O (Input/Output) terminals, external audio and video devices wirelessly connected, smart phones connected wired/wirelessly, wired/wireless connected External devices such as a PDA, a wired/wireless personal computer, and an earphone can also be connected. The external input/output unit 60 can transmit data received from such an external device to each component inside the smartphone 1 and transmit data inside the smartphone 1 to an external device.

The motion sensor unit 80 includes, for example, a triaxial acceleration sensor and a tilt sensor, and detects physical movements of the smartphone 1 according to instructions from the main control unit 101 . By detecting the physical movement of the smartphone 1, the moving direction, acceleration, and posture of the smartphone 1 are detected. Such detection results are output to the main control section 101 . The power supply unit 90 supplies power stored in a battery (not shown) to each unit of the smartphone 1 according to instructions from the main control unit 101 .

The main control unit 101 includes a microprocessor, operates according to control programs and control data stored in the storage unit 50 , and controls each unit of the smartphone 1 including the camera unit 41 in an integrated manner. Further, the main control unit 101 has a mobile communication control function for controlling each unit of the communication system and an application processing function in order to perform voice communication and data communication through the wireless communication unit 11 .

The main control unit 101 also has an image processing function such as displaying an image on the display/input unit 20 based on image data (still image or moving image data) such as received data or downloaded streaming data. The image processing function is a function in which the main control unit 101 decodes image data, applies image processing to the decoded result, and displays the image on the display input unit 20 .

The camera units 41 and 42 are digital cameras (imaging devices) that perform electronic photography using imaging devices such as CMOS and CCD. Under the control of the main control unit 101, the camera units 41 and 42 convert image data (moving images and still images) obtained by imaging into compressed image data such as MPEG or JPEG, and record the data in the storage unit 50. Alternatively, it can be output through the external input/output unit 60 or the wireless communication unit 11 . In addition, the camera unit 41 is controlled by the main control unit 101 to divide and combine moving images, acquire high-quality still images (such as RAW images), replace and process frames, and extract still images from moving images. can also In the smartphone 1 shown in FIGS. 21 and 22, either one of the camera units 41 and 42 can be used for photographing, or the camera units 41 and 42 can be used simultaneously for photographing. A strobe 43 can be used when the camera unit 42 is used.

Also, the camera units 41 and 42 can be used for various functions of the smartphone 1 . For example, the smartphone 1 can display images acquired by the camera units 41 and 42 on the display panel 21 . Further, the smartphone 1 can use the images of the camera units 41 and 42 as one of the operation inputs of the operation panel 22 . The smartphone 1 can also detect the position by referring to the images from the camera units 41 and 42 when the GPS receiver 70 detects the position. Furthermore, the smartphone 1 refers to the images from the camera units 41 and 42, and uses the light from the camera unit 41 of the smartphone 1 without using the three-axis acceleration sensor or in combination with the three-axis acceleration sensor. It is also possible to determine the axial direction and the current usage environment. Of course, the smartphone 1 can also use the images from the camera units 41 and 42 within the application software. In addition, the smartphone 1 uses position information acquired by the GPS receiving unit 70 for image data of still images or moving images, and voice information acquired by the microphone 32 (which is converted into text information by performing voice-to-text conversion by the main control unit or the like). Alternatively, posture information and the like acquired by the motion sensor unit 80 may be added and recorded in the storage unit 50 . The smartphone 1 can also output the image data of these still images or moving images through the external input/output unit 60 and the wireless communication unit 11 .

<Smartphone posture>
Part (a) of FIG. 23 is a diagram showing an example of a coordinate system defined for the smartphone 1. In this example, the attitude is determined by the angle around the X axis. Part (a) shows a vertical state at an angle of 0 degrees (second state in which the smartphone 1 (imaging device) is aligned with the direction of gravity), and part (b) of the figure shows a horizontal state at an angle of 90 degrees (smartphone 1 (imaging device) ) along the horizontal direction). Also, FIG. 24 is a diagram showing an example of the relationship between the angle around the X-axis and the orientation (vertical or horizontal) of the smartphone 1 . Note that, as in the case of the camera 10 according to the first embodiment (see FIG. 4), the horizontal state (the state shown in (b) of FIG. 23) is used as a reference (angle of 0 degrees) and the posture and state (first state, second state, third state) may be defined.

In the smartphone 1 configured as described above, as in the camera 10 according to the first embodiment, the processing of the imaging method according to the present invention (moving image shooting, division, and recording, still image recording, still image extraction, etc.) can be performed. can be executed. Specifically, the processing executed by the image processing device 240 (each unit shown in FIG. 2) in the first embodiment (including the processing of aspects 1 to 6) is performed by the camera units 41 and 42 and the main control unit in the smartphone 1. 101 can be executed. In addition, the functions of the operation unit 250, the storage unit 260, and the monitor 270 in the first embodiment can be realized by the operation unit 40, the storage unit 50, the operation panel 22, the display panel 21, and the operation panel 22 in the smartphone 1, respectively. can.

As a result, the smartphone 1 according to the second embodiment has the same effects as the camera 10 according to the first embodiment (selection of frames to be extracted as still images, easy organization of video files and folders). etc.) can be obtained.

Although the embodiments and other aspects of the present invention have been described above, the present invention is not limited to the above-described embodiments and aspects, and various modifications are possible without departing from the spirit of the present invention.

1 smartphone 2 housing 10 camera 11 wireless communication unit 20 display input unit 21 display panel 22 operation panel 30 call unit 31 speaker 32 microphone 40 operation unit 41 camera unit 42 camera unit 43 strobe 50 storage unit 51 internal storage unit 52 external storage unit 60 External Input/Output Unit 70 GPS Receiving Unit 80 Motion Sensor Unit 90 Power Supply Unit 100 Interchangeable Lens 101 Main Control Unit 110 Zoom Lens 120 Focus Lens 130 Aperture 140 Lens Drive Unit 200 Imaging Apparatus Body 210 Imaging Device 220 AFE
230 A/D converter 240 Image processing device 240A Image acquisition unit 240B Direction detection unit 240C Control unit 240D Event detection unit 240E Still image extraction unit 240F Lens drive control unit 242 ROM
250 Operation unit 260 Storage unit 270 Monitor 280 Attitude sensor 1051 Frame 1052 Frame 1053 Frame 1054 Frame 1055 Frame 1060 Icon 1070 Check box L Optical axis S100 to S197 Each step of imaging method ST1 GPS satellite STn GPS satellite

Claims (16)

An imaging device,
a direction detection unit that detects the direction of the imaging device with respect to the direction of gravity;
a shooting unit that shoots moving images;
a control unit;
with
The control unit has a first moving image mode capable of generating a moving image file of the moving image captured by the imaging unit and extracting a still image from the moving image file,
In the first moving image mode, the control unit divides the moving image to generate a first moving image and a second moving image when the direction detection unit determines that a direction change with respect to the direction of gravity of the imaging device has occurred,
The control unit
storing a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit;
generating a third animation that is between the first animation and the second animation in time;
The imaging device, wherein the third moving image includes a moving image of the first period in which the direction change occurs. the first moving image is earlier than the second moving image in time;
The direction detection unit detects a first state in which the imaging device is aligned in a horizontal direction, a second state in which the imaging device is aligned in a direction of gravity, or a state in which the imaging device is in the first state and the second state. Detecting whether it is in any of the third states that are changing between
The imaging device according to claim 1, wherein said first moving image includes a moving image of a period in which said imaging device is in said third state. The imaging device according to claim 1, wherein said control section does not store said third moving image in said storage section. The imaging apparatus according to claim 1, wherein said control unit stores said third moving image in the same folder as a folder storing said first moving image file. The imaging apparatus according to any one of claims 1, 3, and 4, wherein said third moving image also includes a moving image of a second period after said direction change is finished. The imaging device according to claim 1, wherein said first moving image also includes a moving image of a second period after said direction change is finished. 7. The imaging device according to claim 5, wherein said second period is a period during which said imaging device adjusts exposure of a moving image, white balance of a moving image, or focus of a subject in a moving image. 8. The imaging apparatus according to any one of claims 1 to 7, wherein said control unit adds information regarding a direction of said imaging apparatus with respect to a direction of gravity to said first moving image file and said second moving image file. The control unit has a reception unit that receives a user's instruction,
The imaging apparatus according to any one of claims 1 to 8, wherein the end timing of the second moving image corresponds to reception of a user's second moving image end instruction or moving image shooting end instruction by the reception unit. A reception unit that receives user instructions in the control unit, or an event detection unit that detects a specific event,
The control unit does not divide the moving image when the direction detection unit detects the direction change while the reception unit is accepting a user's instruction or the event detection unit is detecting a specific event. 10. The imaging device according to 9. The control unit has a second moving image mode with different shooting conditions from the first moving image mode,
The imaging apparatus according to any one of claims 1 to 10, wherein in the second moving image mode, the control unit does not divide the moving image when the direction detection unit detects the direction change. In the first moving image mode, at least one of a shutter speed, an autofocus speed, an automatic exposure follow-up speed, and a white balance follow-up speed is set higher than that in the second moving image mode, and/or 12. The imaging apparatus according to claim 11, wherein a higher frame rate is set for moving image mode. 13. The imaging apparatus according to any one of claims 1 to 12, wherein said first moving image file and said second moving image file have at least part in common except for an extension of a file name. 14. The control unit according to any one of claims 1 to 13, wherein the control unit saves the first moving image file in a first folder of the storage unit and saves the second moving image file in a second folder of the storage unit. imaging device. A direction detection unit that detects the direction of the imaging device with respect to the direction of gravity, a shooting unit that shoots a moving image, and a control unit that has a first moving image mode capable of extracting a still image from the file of the moving image shot by the shooting unit. An imaging method for an imaging device comprising
a determination step of determining whether or not the direction detection unit changes the direction of the imaging device with respect to the direction of gravity in the first moving image mode;
a moving image generation step of dividing the moving image by the control unit to generate a first moving image and a second moving image when it is determined that the direction change has occurred in the first moving image mode;
a moving image saving step in which the control unit saves a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit;
a third moving image generation step in which the control unit generates a third moving image that is between the first moving image and the second moving image in time;
has
The imaging method, wherein the third moving image includes a moving image of the first period in which the direction change occurs. A direction detection unit that detects the direction of the imaging device with respect to the direction of gravity, a shooting unit that shoots a moving image, and a control unit that has a first moving image mode capable of extracting a still image from the file of the moving image shot by the shooting unit. A program for operating an imaging device comprising
causing the direction detection unit to determine whether or not a direction change has occurred with respect to the gravity direction of the imaging device in the first moving image mode;
when it is determined that the direction change has occurred in the first moving image mode, the control unit divides the moving image to generate a first moving image and a second moving image;
storing a first moving image file of the first moving image and a second moving image file of the second moving image in a storage unit;
The control unit generates a third animation that is between the first animation and the second animation in time;
The program, wherein the third moving image includes a moving image of the first period in which the direction change occurs.

Images