JP2012253451A - Imaging device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device and a program that switches and uses either an output result from a triaxial gyro sensor or a calculation result of a motion vector according to a focal length, and display a guide image for allowing a user to reset a modified imaging posture to an original imaging posture.SOLUTION: An imaging posture information calculation unit 102 calculates the amount of displacement of an imaging posture of an imaging device 1 as imaging posture information on the basis of angular velocities detected by sensor units 401, 402, and 403 that detect angular velocities of rotations in triaxial orientations if a focal length is less than a predetermined value, or on the basis of a motion vector calculated by a motion vector calculation unit 101 if the focal length is the predetermined value or more. A guide information calculation unit 103 calculates guide information that allows a user to reset the imaging posture of the imaging device 1, and a display unit 305 displays a guide image generated on the basis of the guide information.

Description

  The present invention relates to an imaging device and a program, and more specifically, when a user changes the imaging posture of the imaging device and images a subject different from that before the change, the output of the triaxial sensor unit according to the focal length of the imaging The present invention relates to an imaging apparatus and a program for generating a guide image for switching back to the original imaging posture before change by using either a result or a motion vector calculation result.

  In imaging by an imaging device such as a digital video camera, after the user changes the orientation of the imaging device and images a subject different from the one before the change, only relying on the image displayed on the monitor included in the imaging device, In many cases, it is generally difficult to restore the orientation of the imaging device to the original state and resume imaging of the subject that was imaged before the orientation was changed.

  This is because the field of view of the captured image acquired through various lenses and displayed on the monitor is limited to a range of an angle of view that is much narrower than that of the user's naked eye. In this limited field of view, in order for the user to return the orientation of the imaging device, the subject or landscape existing around the original subject, or the subject imaged in the process of changing the orientation of the imaging device and their positions The user himself / herself remembers the relationship, and while carefully observing various subject images sequentially displayed on the monitor, panning and tilting must be performed until the original subject image is displayed.

  As a specific example, when a user is taking an image of a child at an athletic meet, the imaging device is pointed toward another subject, for example, a child of an acquaintance of the user, and the acquaintance of the acquaintance A case will be described in which after imaging a child, the imaging apparatus is directed toward the child again and imaging of the child is resumed. In this case, in order for the user to return the orientation of the imaging device, the positional relationship between another child or a landscape such as a flag that has been imaged around my child and their child is stored on the monitor. Then, the image pickup apparatus pans or tilts while carefully checking the images displayed sequentially.

  Here, when it is difficult to distinguish between the original subject and the subject imaged around the subject, such as when my child and the surrounding child are wearing the same gym clothes, the work to return the orientation of the imaging device is not Even more difficult. Also, if you are taking pictures of your child from a distance and taking images at a high zoom magnification, the field of view of the acquired image will be more limited, and the task of returning the orientation of the imaging device will be even more difficult It will be a thing.

  Therefore, the pan and tilt movements for returning the orientation of the image pickup device become awkward, the direction in which the image pickup device is directed goes too far over my child, and the troublesome pan and tilt are not in time for checking the monitor. Sometimes I missed some important scenes.

  In order to solve the above problems, for example, in Patent Document 1, the angular velocity signals in the horizontal direction and the vertical direction output from the vibration gyro element are integrated by an integration circuit, and the movement angle information therebetween is the amount of angular displacement. It is managed as movement instruction vector information for which the sign is inverted by the management circuit and returned to the initial camera angle, and the movement instruction vector information is displayed as guide information on the LCD screen so that the user can restore the original camera angle. A technique relating to an imaging apparatus that can be easily returned is disclosed.

JP 2006-5417 A

  However, in the photographing apparatus described in Patent Document 1, guide information is generated based on the two-axis angular velocity signals in the horizontal direction and the vertical direction, and therefore accurate guide information is accurate only with respect to two orthogonal axes of movement. If the movement cannot be generated and movement occurs around the three axes, it is assumed that accurate guide information cannot be generated.

  That is, for example, when the user captures an image with the imaging device fixed on a tripod head, accurate guide information can be generated. However, when the user performs imaging while holding the imaging device. For example, accurate guide information cannot be generated, and the user may be confused. Further, when imaging is performed at a long focal length with a large zoom magnification, there is a possibility that accurate guide information cannot be generated due to insufficient resolution of the vibrating gyro element.

  Furthermore, in general, a vibration gyro element having a small cumulative error in moving angle information based on angular velocity and having a high resolution is expensive and has a large size. Therefore, accurate guide information based on the technique described in Patent Document 1 is used. It is also speculated that there is a problem that the photographing apparatus is expensive and large in size.

  The present invention has been made in view of such problems. When a user generates a guide image to be displayed in order to return the imaging apparatus to the original imaging posture before the change, 3 is selected according to the focal length of the imaging. By switching between the output result of the axis gyro sensor or the calculation result of the motion vector, the user can easily and accurately move even in the case of movement in the three-axis direction, such as when performing hand-held imaging. An object of the present invention is to provide an imaging apparatus and program capable of returning the imaging posture.

  Further, according to the present invention, when generating a guide image to be displayed for the user to return the imaging apparatus to the original imaging posture before the change, the output result or motion vector of the three-axis gyro sensor according to the focal length of the imaging It is an object of the present invention to provide an imaging device and a program that allow a user to easily and accurately return an imaging posture even if a small gyro sensor with low resolution and low cost is used by switching any of the calculation results of To do.

  In order to solve the above-described problems, the present invention provides an imaging unit (110) that images a subject and acquires a captured image, and an imaging magnification adjustment unit that adjusts a focal length to adjust an imaging magnification of the imaging unit (110). An imaging device (1) comprising (119), the sensor unit (401, 402, 403) for detecting angular velocities of rotation in the horizontal direction, the vertical direction, and the tilt direction of the imaging device (1); A motion vector calculation unit (101) that calculates a motion vector of a captured image acquired by the imaging unit (110), and the angular velocity and motion vector calculation unit (101) detected by the sensor unit (401, 402, 403). From each of the calculated motion vectors, an imaging posture information calculation unit (10) that calculates a displacement amount between the first imaging posture and the second imaging posture of the imaging device (1) as imaging posture information. ) And the imaging posture information calculated by the imaging posture information calculation unit (102), guide information for the user to return the imaging device (1) from the second imaging posture to the first imaging posture. A guide information calculation unit (103) for calculating, and a display unit (305) for displaying a guide image generated from the guide information calculated by the guide information calculation unit (103). ) Is the sensor unit (401) when the focal length adjusted by the imaging magnification adjustment unit (119) when moving from the first imaging posture to the second imaging posture is less than a predetermined value. , 402, 403) calculates the imaging posture information from the angular velocities detected. When the imaging posture information is equal to or greater than the predetermined value, the imaging posture is calculated from the motion vector calculated by the motion vector calculation unit (101). Provided is an imaging device (1), characterized in that to calculate the distribution.

  In the imaging apparatus (1), the guide information calculated by the guide information calculation unit (103) includes an azimuth angle and an angular distance between the first imaging attitude and the second imaging attitude. Good.

  In order to solve the above problems, the present invention causes the sensor unit (401, 402, 403) included in the imaging device (1) to detect the angular velocities of rotation in the horizontal direction, vertical direction, and tilt direction of the imaging device, The motion vector of the captured image captured by the imaging device (1) is calculated, and the first velocity of the imaging device (1) is calculated from each of the angular velocity and the motion vector detected by the sensor unit (401, 402, 403). The amount of displacement between one imaging posture and the second imaging posture is calculated as imaging posture information, and the user takes the imaging device (1) from the second imaging posture based on the imaging posture information. Guide information for returning to a posture is calculated, a guide image is generated on the basis of the guide information, and a process for causing the computer to display on the display unit (305) of the imaging device (1) is executed. When the focal length of the imaging of the imaging apparatus (1) when the program is displaced from the first imaging posture to the second imaging posture is less than a predetermined value, the program is detected by the sensor unit. There is provided a program characterized in that the imaging posture information is calculated from an angular velocity and the imaging posture information is calculated from the motion vector when the value is equal to or greater than the predetermined value.

  According to the imaging device and the program according to the present invention, when the user generates a guide image to be displayed in order to return the imaging device to the original imaging posture before the change, the three-axis sensor unit according to the focal length of the imaging By switching and using either the output result or the motion vector calculation result, the user can easily and accurately return the imaging posture even when the movement in the three-axis direction occurs such as when performing hand-held imaging. be able to.

  Further, according to the imaging device and the program according to the present invention, when the user generates a guide image to be displayed in order to return the imaging device to the original imaging posture before the change, the three-axis is selected according to the focal length of the imaging. By switching and using either the output result of the sensor unit or the motion vector calculation result, the user can easily and accurately return the imaging posture even when using a small sensor unit with low resolution and low cost. .

1 is a block diagram illustrating an example of an internal configuration of an imaging apparatus (digital video camera) 1 according to an embodiment of the present invention. It is a schematic diagram of the imaging device 1 for demonstrating the angular velocity of the triaxial direction which a gyro sensor detects. It is a conceptual diagram for demonstrating the process which divides the imaging data temporarily stored in the main memory 201 into a some area | region, and specifies a reference | standard pixel block from each area | region. It is a conceptual diagram for demonstrating calculation of the motion vector between an inter-frame reference | standard pixel block and a corresponding | compatible pixel block. It is a conceptual diagram for demonstrating conversion to the angular velocity of the motion vector of imaging data itself. 5 is a flowchart of processing in which the imaging apparatus 1 generates and displays guide information for the user to return the imaging posture to the original state. 2 is a diagram schematically illustrating how an image is picked up by the image pickup apparatus 1. FIG. 3 is a schematic diagram of a display on a liquid crystal monitor 305 at the time of imaging by the imaging apparatus 1. FIG. 6 is a schematic diagram of a display on the liquid crystal monitor 305 when the imaging posture of the imaging device 1 is changed. FIG. It is a conceptual diagram for demonstrating a guide image. It is a figure for demonstrating the aspect which superimposes and displays a guide image on a captured image.

  DESCRIPTION OF EMBODIMENTS Preferred embodiments of an imaging device and a program according to the present invention will be described below with reference to the drawings. Specific numerical values and the like shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  FIG. 1 is a block diagram illustrating an example of an internal configuration of an imaging apparatus (digital video camera) 1 according to an embodiment of the present invention. The imaging device 1 can capture a moving image and a still image.

  The present invention is not limited to digital still cameras such as so-called compact digital cameras, mobile phones, PHS (Personal Handyphone System), PDA (Personal Digital Assistant), etc., as long as they can capture moving images or still images. It can also be used in other electronic devices.

  The central control unit 100 is configured by a semiconductor integrated circuit including a CPU (Central Processing Unit), a ROM (Read Only Memory) in which various programs are stored, a RAM (Random Access Memory) as a work area, and the like. The overall processing of the imaging apparatus 1 such as display, recording, and generation of guide information described later is comprehensively controlled.

  The imaging apparatus 1 includes an imaging unit 110 that includes a zoom lens 111, a diaphragm 112, a focus lens 113, and an imaging element 114. The zoom lens 111 is moved along the optical axis LA by the zoom actuator 119. The focus lens 113 is moved along the optical axis LA by a focus actuator (not shown). The diaphragm 112 operates by being driven by a diaphragm actuator (not shown). The imaging element 104 is configured by a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like.

  Imaging using the imaging apparatus 1 is performed according to the following procedure. First, the image sensor 114 photoelectrically converts light that has passed through the zoom lens 111, the diaphragm 112, and the focus lens 113 to generate an analog image signal of the subject.

  After the analog image signal processing unit 115 amplifies the analog image signal, the A / D conversion unit 116 converts the amplified signal into digital image data. The image input controller 117 takes in the digital image data output from the A / D conversion unit 116 as imaging data and stores it in the main memory 201 via the bus 200.

  The digital signal processing unit 118 takes in the imaging data stored in the main memory 201 based on a command from the central control unit 100 via the bus 200, performs predetermined signal processing, and data including a luminance signal and a color difference signal. Is generated. The digital signal processing unit 118 also performs various digital corrections such as offset processing, white balance adjustment processing, gamma correction processing, RGB interpolation processing, noise reduction processing, contour correction processing, color tone correction processing, and light source type determination processing.

  The main memory 201 temporarily stores various data such as imaging data and data including luminance signals and color difference signals. The main memory 201 is configured by a storage medium such as an EEPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable PROM), a nonvolatile RAM, a flash memory, or an HDD (Hard Disk Drive).

  Further, in the guide information generation process described later, when the user presses an operation button (not shown) of the operation unit 307, the main memory 201 records an imaging posture to be returned by the imaging device 1 as an initial imaging posture. When the user changes the imaging posture by panning or tilting the imaging device 1, the amount of displacement of the imaging posture is calculated based on this initial imaging posture.

  The flash ROM 202 stores various setting information related to the operation of the imaging apparatus 1, such as user setting information.

  The image processing unit 203 performs predetermined image processing on various data read from the main memory 201 in accordance with instructions from the central control unit 100 via the bus 200. For example, image data for various processing such as a menu image and an OSD image is generated, and the image data is output to the liquid crystal monitor 305 by being superimposed on the original image data read from the main memory 201. With this output, the image displayed on the liquid crystal monitor 305 is a combination of various image data.

  The image processing unit 203 also generates a guide image based on guide information generated by a guide information generation process described later, and outputs the guide image to the liquid crystal monitor 305 by superimposing it on the original imaging data.

  The VRAM 204 is used as a temporary storage area for image data for display on the liquid crystal monitor 305.

  The compression / decompression processing unit 205 performs predetermined compression processing on various data stored in the main memory 201 in accordance with an instruction from the central control unit 100 via the bus 200 to generate compressed data. Further, in accordance with a command from the central control unit 100, the compressed data stored in the card-type recording medium 304 or the like is subjected to decompression processing in a predetermined format to generate uncompressed data.

  Note that in the imaging apparatus 1 according to the present embodiment, a compression method conforming to the JPEG standard is used for still images, and MPEG2 standard or AVC / H. A compression method conforming to the H.264 standard is adopted.

  The media control unit 301 controls the writing of data to the card type recording medium 303 and the reading of data from the card type recording medium 303 through the card I / F 302 in accordance with instructions from the central control unit 100.

  As the card-type recording medium 303, an SD (Secure Digital) memory card that can be attached to and detached from the imaging apparatus 1 is used. However, this is merely an example, and instead of the card-type recording medium 303, another recording medium such as a DVD, a BD (Blu-ray Disc), or a flash memory is used, or an HDD built in the imaging apparatus 1 It goes without saying that it can be configured with (Hard Disc Drive) or the like.

  The liquid crystal monitor 305, the speaker 306, the operation unit 307, and the input / output terminal 308 are connected to the input / output I / F 304. The liquid crystal monitor 305 displays images generated from various image data such as imaging data and various menu image data temporarily recorded in the VRAM 204 or the main memory 201, for example.

  The speaker 305 can output sound for guiding the user based on guide information generated by, for example, guide information generation processing described later.

  The operation unit 307 includes an operation button including a relay switch and a power switch (not shown), a cross key, a joystick, a touch panel superimposed on the liquid crystal monitor 305, and the like. Accept.

  In a guide information generation process described later, when the user presses an operation button (not shown) of the operation unit 307, the imaging posture to be returned by the imaging device 1 is recorded as the initial imaging posture. The user also presses an operation button (not shown) to make a request for displaying a guide image on the liquid crystal monitor 305.

  The input / output terminal 307 is connected to a television monitor, a PC (Personal Computer) or the like (not shown), and inputs / outputs various data. The input / output terminal 307 includes, for example, an HDMI (High-Definition Multimedia Interface) terminal, a USB (Universal Serial Bus) terminal, various input terminals conforming to the IEEE 1394 standard, and the like.

  The imaging apparatus 1 includes an X-axis gyro sensor 401 that detects an angular speed of rotation in the horizontal direction of the imaging apparatus 1, a Y-axis gyro sensor 402 that detects an angular speed of rotation in the vertical direction, and a Z that detects angular speed of rotation in the tilt direction. An axis gyro sensor 403 is provided.

  FIG. 2 is a schematic diagram of the imaging apparatus 1 for explaining each angular velocity detected by each gyro sensor. The X-axis gyro sensor 401 detects an angular velocity ωx in the horizontal direction (X-axis direction in FIG. 2) of the imaging device 1. The Y-axis gyro sensor 402 detects an angular velocity ωy in the vertical direction (Y-axis direction in FIG. 2) of the imaging device 1. Then, the Z-axis gyro sensor 403 detects an angular velocity ωz in the tilt direction (Z-axis direction in FIG. 2) of the imaging device 1.

  Each of the detected angular velocities is transferred to the central control unit 100 and used in a guide information generation process for returning the imaging posture of the imaging device 1 to the original position by the user described below.

  Returning to the description of FIG. 1, the central control unit also performs a motion vector calculation unit 101, an imaging posture information calculation unit 102, a guide in the guide information generation process for the user to return the imaging posture of the imaging device 1 to the original position. It functions as the information calculation unit 103 and the determination unit 104.

  The motion vector calculation unit 101 calculates a motion vector of the subject image from the amount of displacement of the subject image data between frames of the imaging data temporarily stored in the main memory 201. Various known methods can be employed for calculating the motion vector.

  Here, the motion vector calculation unit 101 determines whether the motion vector calculated from the motion vector of each subject image data in the frame is caused by the movement of the subject itself to be imaged. It is also determined whether or not this is caused by panning or tilting the imaging device 1 to change the imaging posture.

  Hereinafter, this determination will be described with reference to FIGS. As shown in FIG. 3, the frame MH01 of the imaging data temporarily stored in the main memory 201 is divided into a plurality of areas. In FIG. 3, the frame MH01 is divided into a total of 40 areas divided into 8 in the horizontal direction and 5 in the vertical direction.

  A reference pixel block is specified for each of the divided areas. For example, in the area DA01 (area shown by hatching) in FIG. 3, the reference pixel block B01 is specified. A reference pixel block is also specified for the other 39 regions.

  Next, an area DA02 (area indicated by hatching) corresponding to the area DA01 of the frame MH01 is specified from the frame MH02 after a predetermined time from the frame MH01 shown in FIG. 4, and the corresponding pixels corresponding to the reference pixel block B01 are identified. The block B02 is specified. Then, a vector V01 between the reference pixel block B01 and the corresponding pixel block B02 is calculated as a motion vector.

  When the above motion vectors are calculated for all regions, and the motion vectors calculated for a predetermined number of regions or more are similar within a predetermined range, the user can pan the imaging device 1 with the calculated motion vectors. It is determined that it is caused by tilting or changing the imaging posture. On the other hand, when only the motion vectors calculated for a region less than the predetermined number are similar, it is determined that the calculated motion vectors are caused by the movement of the subject that is the subject of imaging. Is done.

  Then, the motion vector calculation unit 101 determines the motion of the imaging data itself from the average of the motion vectors of a plurality of regions used for determining that the imaging posture is changed by panning or tilting the imaging device 1 described above. Calculate the vector.

  Returning to the description of the central control unit 100 in FIG. 1, the imaging posture information calculation unit 102 detects the angular velocity or motion detected by the three gyro sensors of the X-axis gyro sensor 401, the Y-axis gyro sensor 402, and the Z-axis gyro sensor 403. From the motion vector of the imaging data itself calculated by the vector calculation unit 101, the amount of displacement from the initial imaging orientation that is the imaging orientation to be returned by the imaging device 1 due to panning or tilting performed by the user is calculated as imaging orientation information.

  When the angular velocities detected by the three gyro sensors are used, the output results are used as they are. When the motion vector of the imaging data itself calculated by the motion vector calculation unit 101 is used, the motion vector is used. Are converted into three rotational angular velocities: a horizontal direction, a vertical direction, and a tilt direction.

  With reference to FIG. 5, the conversion into the angular velocity will be described. FIG. 5 conceptually shows the motion vector AV of the imaging data MG01 itself calculated from the initial imaging posture before the imaging posture is changed to the imaging posture after the imaging posture is changed. FIG.

  The imaging posture information calculation unit 102 calculates a horizontal component vector Vh and a vertical component vector Vv of the motion vector AV. Further, the imaging posture information calculation unit 102 specifies the focal length f due to zooming of the zoom lens 111, and from the following equations (1) and (2), the horizontal angular velocity ωx and the vertical angular velocity ωy of the imaging device 1 are obtained. Is calculated.

ωx = 60 atan (Vh / f) (1)

ωy = 60 atan (Vv / f) (2)

  Then, the angular velocity ωz in the tilt direction is calculated from the change in angle ΔθZ between both ends of the motion vector around the point CP in the imaging data MG01 that coincides with the optical axis AL of the imaging unit 110.

  Then, the imaging posture information calculation unit 102 outputs the angular velocity ωx in the horizontal direction, the angular velocity ωy in the vertical direction, and the angular velocity ωy in the vertical direction calculated as described above from the motion vector of the imaging data itself output from each gyro sensor. From the angular velocity ωz, the horizontal rotation angle θx, the vertical rotation angle θy, and the rotation angle θz in the tilt direction are used as imaging posture information, respectively, from the following equations (3), (4), and (5). calculate.

θx = ∫ωx · sin θzdt (3)

θy = ∫ωy · cos θzdt (4)

θz = ∫ωzdt (5)

  Returning to the description of the central control unit 100 in FIG. 1, the guide information calculation unit 103 is based on the imaging posture information calculated by the imaging posture information calculation unit 102 and the orientation for the user to return the imaging device 1 to the imaging posture before the change. The angle θ and the angular distance d are calculated as guide information.

The azimuth angle θ for returning the imaging device 1 to the imaging posture before change is set to θ _{1 as} the azimuth angle of the initial imaging posture, θ _{2 as} the azimuth angle of the imaging posture after change, and the horizontal direction of each azimuth angle. Assuming that the components are θx ₁ and θx ₂ , the vertical components are θy ₁ and θy ₂ , and the components in the tilt direction are θz ₁ and θz ₂ , the following equations (6), (7), (8), Or it calculates by Formula (9).

When θy ₂ −θy ₁ = 0 ° and θx ₂ −θx ₁ = 0 °, it is calculated by the following equation (6).

θ = θz (6)

When θy ₂ −θy ₁ = 0 and 0 ° <θx ₂ −θx ₁ <180 °, it is calculated by the following equation (7).

θ = θz + 90 ° (7)

When θy ₂ −θy ₁ = 0 and 0 °> θx ₂ −θx ₁ > −180 °, it is calculated by the following equation (8).

θ = θz−90 ° (8)

  In other cases, it is calculated by the following equation (9).

θ = atan (sin (θx ₂ −θx ₁ ) / tan (θy ₂ −θy ₁ )) + θz
... (9)

  When the absolute value of the azimuth angle θ exceeds 180, the production of 360 is reduced until the absolute value becomes 180 or less.

  Then, when λ is cos θy cos θx, μ is sin θy cos θx, and ν is sin θy, the angular distance d is calculated by the following equation (10).

d = cos ⁻¹ (λ1λ2 + μ1μ2 + ν1ν2) (10)

  Returning to the description of the central control unit 100 in FIG. 1, when the guide information calculation unit 103 calculates guide information as described above, the image processing unit 203 displays on the liquid crystal monitor 305, and the user changes the imaging posture of the imaging device 1. A guide image for returning to the original imaging posture is generated and displayed superimposed on the imaging data.

  Since the guide information including the azimuth angle θ and the angular distance d is displayed on the liquid crystal monitor 305, the user can easily view the image displayed on the liquid crystal monitor 305 with a narrow visual field by referring to the guide image. In addition, it is possible to grasp the direction in which the imaging device 1 is returned to the original imaging posture and the size of the pan and tilt for that.

  The determination unit 104 calculates the guide information calculated by the guide information calculation unit 103 based on the focal length by zooming of the zoom lens 111 from the imaging posture information based on the angular velocity output from each gyro sensor, or the motion vector calculation unit 101 calculates the guide information. It is determined whether to calculate from the imaging posture information based on the angular velocity calculated from the calculated motion vector of the imaging data itself.

  When the focal length at the time of imaging is long, the correct guide information may not be generated due to insufficient resolution of the vibrating gyro element. Therefore, according to the determination of the focal length when the imaging posture is changed, the guide information calculated from the imaging posture information based on the angular velocity output by each gyro sensor and the motion vector of the imaging data itself calculated by the motion vector calculation unit 101 are used. By switching between the guide information calculated from the imaging posture information based on the calculated angular velocity, the user can easily and accurately return the imaging posture.

  As described above, according to the imaging apparatus 1, when the user generates a guide image to be displayed in order to return the imaging apparatus to the original imaging posture before the change, the three-axis gyro sensor according to the focal length of imaging. By switching and using either the output result or the motion vector calculation result, the user can easily and accurately return the imaging posture even when the movement in the three-axis direction occurs such as when performing hand-held imaging. be able to.

  Further, according to the imaging device 1, when the user generates a guide image to be displayed in order to return the imaging device to the original imaging posture before the change, the output result of the triaxial gyro sensor according to the focal length of the imaging Alternatively, by switching and using either of the motion vector calculation results, the user can easily and accurately return the imaging posture even with a low-cost and small-sized gyro sensor with low resolution.

  Next, a process (method) in which the imaging apparatus 1 generates and displays guide information for the user to return the imaging posture to the original will be described with reference to FIG. FIG. 6 is a flowchart of processing in which the imaging apparatus 1 generates and displays guide information for the user to return the imaging posture to the original state.

  When the user presses a recording button (not shown) on the operation unit 307, the imaging apparatus 1 starts imaging a subject (step S101). FIG. 7 is a diagram schematically showing the state of imaging by the imaging apparatus 1. In FIG. 7, the user points the imaging device 1 toward the subject 70 and images the subject 70.

  FIG. 8 shows a schematic diagram of the liquid crystal monitor 305 at this time. Since the imaging apparatus 1 is directed toward the subject 70, the subject image OG70 of the subject 70 is displayed on the liquid crystal monitor 305 that displays the captured image.

  Then, when the user presses an operation button (not shown) of the operation unit 307, the imaging posture of the imaging device 1 at that time is stored in the main memory 201 as the initial imaging posture to be returned (step S102).

  When the user changes the imaging posture of the imaging device 1 by panning or tilting (YES in step S103), a subject different from step S101 is imaged via the imaging unit 110 (step S104). FIG. 9 shows a schematic diagram of the liquid crystal monitor 305 at this time. The imaging posture of the imaging apparatus 1 is changed, and a subject image of a subject different from the subject 70 is displayed on the liquid crystal monitor 305.

  In step S104, another subject is imaged, and the user presses an operation button (not shown) of the operation unit 307 and requests a guide image for returning the imaging posture of the imaging device 1 to the initial imaging posture stored in step S102 (step S102). In S105, the determination unit 104 determines whether or not the focal length due to zooming of the zoom lens 111 when the imaging posture is changed in Step S103 is greater than or equal to a predetermined threshold (Step S106).

  When the determination unit 104 determines that the focal length due to zooming of the zoom lens 111 is equal to or greater than a predetermined threshold (YES in step S106), the imaging posture information calculation unit 102 calculates the imaging data itself calculated by the motion vector calculation unit 101. The displacement amount of the imaging posture of the imaging device 1 due to panning and tilting performed by the user is calculated as imaging posture information from the motion vector (step S107).

  The calculation of the imaging posture information based on the motion vector of the imaging data itself is performed as described above. In other words, the motion vector calculation unit 101 is obtained from the equations (1) and (2) and the change in angle between both ends of the motion vector centered on a point in the imaging data that coincides with the optical axis AL of the imaging unit 110. Is converted into an angular velocity of three rotations of an angular velocity ωx in the horizontal direction, an angular velocity ωy in the vertical direction, and an angular velocity ωz in the tilt direction.

  Then, the horizontal rotation angle θx and the vertical direction are calculated from the horizontal angular velocity ωx, the vertical angular velocity ωy, and the angular velocity ωz in the tilt direction by the above formulas (3), (4), and (5). Rotation angle θy and tilt rotation angle θz are respectively calculated as imaging posture information.

  When the determination unit 104 determines that the focal length due to zooming of the zoom lens 111 is less than a predetermined threshold (NO in step S106), the imaging posture information calculation unit 102 includes an X-axis gyro sensor 401 and a Y-axis gyro sensor 402. Then, from the angular velocities detected by the three gyro sensors of the Z-axis gyro sensor 403, the displacement of the imaging posture of the imaging device 1 due to the pan and tilt performed by the user is calculated as imaging posture information (step S108).

  Here, from the angular velocity ωx in the horizontal direction, the angular velocity ωy in the vertical direction, and the angular velocity ωz in the tilt direction output from each gyro sensor, the rotation angle θx in the horizontal direction, the rotation angle θy in the vertical direction, and the rotation angle in the tilt direction. θz is calculated as the imaging posture information based on the above-described equations (3), (4), and (5).

  When the imaging posture information is calculated from the motion vector of the imaging data itself in step S107 or the angular velocity detected by each gyro sensor in step S108, the guide information calculation unit 103 generates a guide image to be displayed on the liquid crystal monitor 305. Therefore, the guide information including the azimuth angle θ and the angular distance d for the user to return the imaging apparatus 1 to the imaging posture before the change is calculated.

  This guide information is calculated by the above formula (6), formula (7), formula (8), or formula (9).

  When the guide information is calculated in step S109, the image processing unit 203 generates a guide image based on the guide information, and superimposes the original captured image on the liquid crystal monitor 305. The display of this guide image will be described with reference to FIGS.

  FIG. 11 is a diagram for explaining an example of display of a guide image displayed on the liquid crystal monitor 305. A guide image having a length proportional to each distance d and having an azimuth angle θ from the horizontal direction of the imaging device (indicated by a line segment L01 in FIG. 11) and a direction to be returned is indicated by an image processing unit. 203 and displayed on the liquid crystal monitor 305.

  Then, as shown in FIG. 12, this guide image GG01 is superimposed and displayed on the original imaging data before returning the imaging posture of the imaging device 1 requested by the user in step S105.

  Then, when the user does not change the imaging posture of the imaging device 1 at step S103 (NO at step S103), when the user does not request guide information at step S105 (NO at step S105), or at step S110. When the guide image is generated and displayed, this process ends.

  As described above, according to the process of generating and displaying guide information for the user of the imaging apparatus 1 to return the imaging posture to the original, the guide that the user displays to return the imaging device to the original imaging posture before the change. When generating an image, by switching and using either the output result of the three-axis gyro sensor or the calculation result of the motion vector according to the focal length of the image pickup, the image in the three-axis direction for hand-held image pickup is used. Even when movement occurs, the user can easily and accurately return the imaging posture.

  Further, according to the process of generating and displaying guide information for the user of the imaging device 1 to return the imaging posture to the original, a guide image to be displayed for the user to return the imaging device to the original imaging posture before the change is generated. When switching between the output result of the three-axis gyro sensor or the calculation result of the motion vector according to the focal length of the imaging, the user can use the low-resolution, low-cost and small-sized gyro sensor. The imaging posture can be easily and accurately returned.

DESCRIPTION OF SYMBOLS 100 Central control part 101 Motion vector calculation part 102 Imaging orientation information calculation part 103 Guide information calculation part 104 Judgment part 110 Imaging part 111 Zoom lens 112 Aperture 113 Focus lens 114 Imaging element 201 Main memory 203 Image processing part 204 VRAM
304 I / O I / F
305 LCD monitor 306 Speaker 307 Operation unit 308 Input / output terminal

Claims (3)

An imaging apparatus including an imaging unit that images a subject and acquires a captured image, and an imaging magnification adjustment unit that adjusts a focal length and adjusts a magnification of imaging by the imaging unit,
A sensor unit for detecting an angular velocity of rotation in a horizontal direction, a vertical direction, and a tilt direction of the imaging device;
A motion vector calculation unit that calculates a motion vector of a captured image acquired by the imaging unit;
Imaging that calculates a displacement amount between the first imaging posture and the second imaging posture of the imaging device as imaging posture information from the angular velocity detected by the sensor unit or the motion vector calculated by the motion vector calculation unit An attitude information calculation unit;
A guide information calculation unit that calculates guide information for the user to return the imaging device from the second imaging posture to the first imaging posture from the imaging posture information calculated by the imaging posture information calculation unit;
A display unit for displaying a guide image generated from the guide information calculated by the guide information calculation unit,
When the focal length adjusted by the imaging magnification adjustment unit when the imaging posture information calculation unit is displaced from the first imaging posture to the second imaging posture is less than a predetermined value, the sensor unit The imaging posture information is calculated from the angular velocity detected by the camera, and the imaging posture information is calculated from a motion vector calculated by a vector calculation unit when the angular velocity is equal to or greater than the predetermined value.   The imaging apparatus according to claim 1, wherein the guide information calculated by the guide information calculation unit includes an azimuth angle and an angular distance between the first imaging attitude and the second imaging attitude. . Let the sensor unit of the imaging device detect the angular velocity of rotation in the horizontal direction, vertical direction, and tilt direction of the imaging device,
Calculating a motion vector of a captured image captured by the imaging device;
From each of the angular velocity and the motion vector detected by the sensor unit, a displacement amount between the first imaging posture and the second imaging posture of the imaging device is calculated as imaging posture information,
From the imaging posture information, the user calculates guide information for returning the imaging device from the second imaging posture to the first imaging posture,
A program for causing a computer to execute a process of generating a guide image based on the guide information and displaying the guide image on a display unit included in the imaging device,
The imaging posture information is calculated from the angular velocity detected by the sensor unit when the focal length of imaging of the imaging device when the first imaging posture is displaced to the second imaging posture is less than a predetermined value. And causing the imaging posture information to be calculated from the motion vector when the value is equal to or greater than the predetermined value.

Images