JP2007074077A - Imaging unit, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging unit that can accurately calculate the vertical direction of an object to be imaged that is placed horizontally, a program, and a recording medium. <P>SOLUTION: A process of measuring the gradient of the imaging unit and a process of storing at least one of the gradient and top-and-bottom direction, when the measured gradient is larger than a specified value are performed repeatedly, until a direction in which the imaging unit faces the object to be imaged is determined. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image capturing device, a program, and a recording medium, and more particularly to an image capturing device, a program, and a recording medium that process captured images as digital data.

  2. Description of the Related Art In recent years, image capturing apparatuses that electronically capture images, such as digital cameras, have become widespread. In the image pickup device, taking advantage of electronically storing images, new functions not provided in conventional film cameras and various applications are performed.

As one application example, there is a function of automatically determining the top / bottom direction of a captured image (hereinafter also referred to as a top / bottom direction automatic determination function). As a specific use scene of the top and bottom direction automatic determination function,
-When displaying a list, the top and bottom direction is unified and applied to display and image recognition.

  Hereinafter, these usage scenes and the conventional vertical direction determination technology will be described. First, when the list is displayed, the top and bottom directions are unified and displayed as follows.

  In the same way as conventional film cameras, electronic image pickup devices such as digital cameras or camera-equipped mobile phones often have a picked-up image in a vertically long or horizontally long rectangle instead of a 1: 1 aspect ratio. is there. For this reason, the user picks up the image pickup device directly or rotates it 90 ° to the left or right depending on whether he / she wants to pick up an image pickup target as a vertically long image or a horizontally long image.

  When a large number of electronically captured image data is reduced and displayed in a list, as described above, if the state of the image pickup device at the time of imaging (facing, rotated by 90 °) is not the same, the images displayed as a list are also displayed. The top-to-bottom direction is not unified and the list is not easy to see for users.

  If the top / bottom direction is automatically determined at the time of image capture, rotate the image if necessary so that the image is displayed along the top / bottom direction when saving the image, or the top / bottom direction information is stored in the image. For example, it is possible to attach the data to the data and display the data along the vertical direction. As a result, it is possible to solve the problem that the top-to-bottom direction of the images is not unified in the above-described list display.

  In order to solve this problem, as a method of automatically determining the top-and-bottom direction at the time of imaging, for example, in Japanese Patent Application Laid-Open No. 2004-229260 (Patent Document 1), according to the inclination at the time of imaging of a digital camera (image imaging apparatus). A technique (hereinafter also referred to as Conventional Technology A) is disclosed that determines the top / bottom direction of an image to be recorded using a built-in inclinometer and generates and records an image file corresponding to the determined top / bottom direction.

  Next, the application to image recognition is as follows. In the case of character recognition, after capturing a business card, it is possible to easily create an address book using the read recognition result with an item by reading the character written on the business card with a character recognition device with a business card item classification function Is possible.

  In the case of face recognition, it can be considered to search only a large number of image data showing a certain (person) face from a large amount of image data showing many (person) faces.

  Even in this case, as described above, the top-to-bottom direction is generally unknown in many cases depending on the state of the image pickup apparatus at the time of image pickup (facing, 90 ° rotation). It is inconvenient for the user to specify the top-and-bottom direction of the captured image each time, and the burden on the user is very large in the case of a large amount of image data. Therefore, before the image recognition process is performed, the top-and-bottom direction determination process for the captured image is required.

  In order to solve this problem, instead of the above-described method for automatically determining the top / bottom direction at the time of imaging, as a method for automatically determining the top / bottom direction based on the captured image, for example, JP-A-8-274980 (Patent Document 2) ) Discloses a technique (hereinafter also referred to as Conventional Technology B) for determining the top-and-bottom direction based on the recognition rate of characters.

Japanese Patent Laying-Open No. 2005-202477 (Patent Document 3) detects the eyes and mouth, and determines the top / bottom direction of the face image based on the position of the mouth relative to the position of the eyes (hereinafter also referred to as Conventional Technology C) ) Is disclosed.
Japanese Patent Laid-Open No. 2004-229260 JP-A-8-274980 JP 2005-202477 A

  However, in the technique disclosed in Japanese Patent Application Laid-Open No. 2004-229260 (Patent Document 1), as illustrated in FIG. 17, when imaging an imaging object 701 installed horizontally, the imaging unit of the image imaging device 10000 However, when it is directly in front of the imaging object 701, the magnitude of the gravitational acceleration applied to the axial directions DI401 and DI402 is almost zero, and the built-in inclinometer cannot determine the top-to-bottom direction of the imaging object 701. . Here, the imaging object 701 is paper on which characters “XYZ Corporation” are printed in the vertical direction.

  It is assumed that the inclinometer built in the image capturing apparatus 10000 is so high in performance that it can detect a slight gravitational acceleration (tilt). In this case, the left end of the image capturing device 10000 is raised in the direction D701 to move away from the imaging target 701, and the right end of the image capturing device 10000 is lowered in the direction D702 to be closer to the imaging target 701. Accordingly, the image capturing apparatus 10000 can be slightly inclined with respect to the imaging target 701 to the extent that the imaging result of the imaging target 701 is not affected. As a result, the inclinometer can determine the vertical direction.

However, there are the following problems in performing top-and-bottom determination with such a slight inclination.
(1) When the user picks up an image with the hand without fixing the image pickup device 10000, the user's hand shake is transmitted to the image pickup device 10000 when trying to hold a state close to horizontal as shown in FIG. there is a possibility. In this case, the inclination direction of the image pickup device 10000 fluctuates, and the inclinometer cannot accurately determine the top and bottom.
(2) It is assumed that the user tilts the image capturing device 10000 with his / her hand so that the top / bottom direction of the image capturing target 701 is correctly determined to the extent that the imaging result of the image capturing target 701 is not affected. In this case, there is a possibility that camera shake occurs due to the user pressing the shutter button of the image capturing apparatus 10000, and the tilt direction of the image capturing apparatus 10000 is changed. In this case, the inclinometer has a problem that it cannot accurately determine the top and bottom.

  Moreover, since it is necessary to incline the imaging device 10000 slightly with respect to the imaging object 701 placed horizontally, it is conceivable to give a psychological burden to the user.

  Furthermore, in terms of cost and the like, it is not always possible to attach a high-performance inclinometer that can detect a slight gravitational acceleration to the image capturing apparatus 10000. In this case, with a slight inclination, the inclinometer cannot determine the vertical direction. Therefore, as shown in FIG. 17, it is impossible to pick up an image in a substantially horizontal state and determine the vertical direction of the object to be picked up.

  Note that, when the related art B and the prior art C are applied, it is possible to determine the top / bottom direction of an imaged object including characters or a face imaged in a substantially horizontal state.

  However, if the image includes characters, it is necessary to apply the top / bottom determination processing for characters (conventional technology B or the like), and if the image includes a face, it is necessary to apply the top / bottom determination processing for faces (conventional technology C or the like). There is a drawback that it is troublesome for the user to specify which judgment processing is applied in accordance with the image. A further major drawback is that the prior art B or the prior art C cannot be applied to images that do not include characters or faces.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an image capturing apparatus capable of accurately calculating the vertical direction of an imaging target placed horizontally.

  Another object of the present invention is to provide a program capable of accurately calculating the vertical direction of an imaging target placed horizontally.

  Still another object of the present invention is to provide a recording medium on which a program capable of accurately calculating the vertical direction of an imaging target placed horizontally is recorded.

  In order to solve the above-described problem, an image imaging apparatus according to an aspect of the present invention includes an image imaging unit that images an imaging target, an inclination measurement unit that measures an inclination indicating an inclination angle of the image imaging apparatus, and Based on the inclination measured by the inclination measuring means, the vertical direction calculating means for calculating the vertical direction of the image pickup device, the inclination measured by the inclination measuring means, and the vertical direction calculated by the vertical direction calculating means Inclination information storage means for storing at least one of the information and a facing direction determination means for determining a facing direction of the image capturing apparatus with respect to the imaging target, and a process of measuring the tilt of the image capturing apparatus by the tilt measuring means When the measured inclination is equal to or greater than a predetermined value, information on at least one of the inclination and the vertical direction calculated by the vertical direction calculation means is stored in the inclination information storage means. Process and is time to directly facing direction is determined by the facing direction determining means is performed repeatedly.

  Preferably, when the measured inclination is smaller than a predetermined value, the top-and-bottom direction calculation unit calculates the top-and-bottom direction of the image pickup device based on the latest inclination or top-and-bottom information stored in the inclination information storage unit. calculate.

  Preferably, the information processing apparatus further includes an informing unit that informs at least one of the inclination of the image pickup apparatus and the up-and-down direction calculated by the up-and-down direction calculating unit.

  Preferably, the informing means informs at least one of the inclination degree of the image pickup apparatus and the top and bottom direction calculated by the top and bottom direction calculating means when the inclination measured by the inclination measuring means is equal to or greater than a predetermined value.

  Preferably, the notifying means notifies at least one of the inclination and the top and bottom information stored in the inclination information storage means when the inclination measured by the inclination measuring means is smaller than a predetermined value.

Preferably, the notification unit is a display device that displays an imaging target of the image imaging device,
The informing means displays at least one information of the inclination of the image pickup apparatus and the top-and-bottom direction calculated by the top-and-bottom direction calculating means together with the imaging target.

  Preferably, input means for receiving an instruction input from the user is further provided, and the facing direction determination means determines the facing direction of the image capturing apparatus with respect to the imaging target in accordance with the instruction input received by the input means.

  Preferably, the image storage device further includes an image storage device that stores the captured image captured by the image capturing device, and the image storage device stores the captured image in response to an instruction input received by the input device.

  Preferably, the captured image is stored in the image storage unit based on the facing direction of the image capturing apparatus determined by the facing direction determining unit.

Preferably, the inclination measuring means has a sensor for measuring gravitational acceleration.
Preferably, a recognizing unit for recognizing an image is further provided, and the recognizing unit recognizes the captured image captured by the image capturing unit based on the facing direction of the image capturing apparatus determined by the facing direction determining unit.

  According to another aspect of the present invention, there is provided a program for causing a computer in an image capturing apparatus to execute an image capturing process, the program capturing a target to be imaged and an inclination indicating an inclination angle of the image capturing apparatus. And calculating the top and bottom direction of the image pickup device based on the slope measured by the step of measuring the slope and the slope and top and bottom directions measured by the step of measuring the slope. Causing the computer to execute a step of storing at least one piece of information of the top-and-bottom direction calculated in the calculating step in a storage unit in the image capturing apparatus and a step of determining a facing direction of the image capturing apparatus with respect to the imaging target.

  Preferably, in the step of calculating the top / bottom direction, when the measured slope is smaller than a predetermined value, the top / bottom direction of the image pickup device is calculated based on the latest slope or the top / bottom direction information stored in the storage unit. calculate.

  According to still another aspect of the present invention, the recording medium is a medium on which a program is recorded.

  The image capturing apparatus according to the present invention includes a process for measuring the inclination of the image capturing apparatus, and a process for storing at least one information of the inclination and the vertical direction when the measured inclination is equal to or greater than a predetermined value. The process is repeated until the facing direction of the image capturing apparatus with respect to the imaging target is determined.

  Therefore, there is an effect that it is possible to accurately calculate the vertical direction of the imaging target placed horizontally.

  The program according to the present invention stores information on at least one of the measured inclination of the image pickup device and the calculated top-and-bottom direction of the image pickup device in a storage unit in the image pickup device.

  Therefore, there is an effect that it is possible to accurately calculate the vertical direction of the imaging target placed horizontally.

  The recording medium according to the present invention records a program. The program stores information on at least one of the measured inclination of the image pickup device and the calculated top-and-bottom direction of the image pickup device in a storage unit in the image pickup device.

  Therefore, there is an effect that it is possible to accurately calculate the vertical direction of the imaging target placed horizontally.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<First Embodiment>
FIG. 1 is an external view of an image capturing apparatus 1000 according to the present embodiment. FIG. 1A is a diagram illustrating the front of the image capturing apparatus 1000. With reference to FIG. 1A, the image capturing apparatus 1000 includes a housing 100, a lens group 110, and a shutter button 120. The lens group 110 is provided on the front surface of the housing 100. The shutter button 120 is provided on the top of the housing 100. The lens group 110 includes a plurality of optical lenses.

  FIG. 1B is a diagram illustrating the back surface of the image pickup apparatus 1000. With reference to FIG. 1B, the image capturing apparatus 1000 includes a display unit 150. The display unit 150 has a function of displaying various types of information to the user as characters, images, and the like, and operates as a notification unit. The display unit 150 may be any other display device such as a liquid crystal display (LCD (Liquid Crystal Display)), an FED (Field Emission Display), an organic EL display (Organic Electroluminescence Display), or a dot matrix. Good. On the display unit 150, images to be imaged are sequentially displayed during imaging.

  FIG. 2 is a block diagram illustrating an internal configuration of the image capturing apparatus 1000 according to the present embodiment. Referring to FIG. 2, the image capturing apparatus 1000 includes an image capturing unit 210 and a data bus 280.

  An image capturing unit 210 as an image capturing unit performs an image capturing process for converting a subject image into digital image data. The image capturing unit 210 is connected to the data bus 280.

  FIG. 3 is a block diagram illustrating a configuration of the image capturing unit 210 in the present embodiment. Referring to FIG. 3, the image capturing unit 210 includes a lens group 110, an image sensor 212, an A / D conversion unit 214, an imaging control unit 216, a temporary storage unit 217, and an input / output interface unit 218. Including.

  The image sensor 212 has a function of converting an input image to be imaged into an analog signal via the lens group 110. The imaging element 212 is an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The image sensor 212 transmits the converted analog signal to the A / D converter 214.

  The A / D conversion unit 214 converts the received analog signal into a digital signal, and transmits the digital signal to the imaging control unit 216.

  The temporary storage unit 217 has a function of temporarily storing data. The temporary storage unit 217 includes RAM (Random Access Memory), SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), SDRAM (Synchronous DRAM), DDR-SDRAM (Double Data Rate SDRAM), and RDRAM (Rambus Dynamic Random). (Access Memory), Direct-RDRAM (Direct Rambus Dynamic Random Access Memory), or any other circuit having a configuration capable of storing and storing data in a volatile manner.

  The imaging control unit 216 has a function of controlling each unit in the image imaging unit 210. The imaging control unit 216 causes the temporary storage unit 217 to store data based on the input digital signal.

  The imaging control unit 216 includes a microprocessor, an FPGA (Field Programmable Gate Array) which is an LSI (Large Scale Integration) that can be programmed, and an ASIC (Integrated Circuit) designed and manufactured for a specific application. Application Specific Integrated Circuit) or any other circuit having an arithmetic function may be used.

  The input / output interface unit 218 transmits and receives data between the imaging control unit 216 and the data bus 280.

  The image capturing unit 210 having the above configuration overwrites and stores the image data of the image to be captured input via the lens group 110 in the temporary storage unit 217 every predetermined time (for example, 0.1 second). .

  Referring to FIG. 2 again, the image capturing apparatus 1000 further includes a control unit 220, an input unit 230, a temporary storage unit 240, a storage unit 245, and a tilt measurement unit 250.

  A control unit 220, an input unit 230, a temporary storage unit 240, a storage unit 245, a display unit 150, a tilt measurement unit 250, a tilt information storage unit 260 and a recording medium access unit 270 are connected to the data bus 280.

  The storage unit 245 stores a program 246 for causing the control unit 220 to perform processing described later, various other programs, data, and the like. Data is accessed in the storage unit 245 by the control unit 220. The storage unit 245 is a flash memory that can hold data in a nonvolatile manner.

  The control unit 220 as a control unit has a function of performing various types of processing, arithmetic processing, and the like for each unit in the image capturing apparatus 1000 in accordance with a program 246 stored in the storage unit 245. The control unit 220 is the same as the imaging control unit 216 described above.

  The input unit 230 as an input unit detects a pressing operation of the shutter button 120 by the user, and transmits a signal indicating that to the control unit 220 (hereinafter also referred to as a pressing detection signal). That is, the input unit receives an instruction input from the user to the image capturing apparatus 1000. Note that the input unit 230 may detect a pressing operation of a button (not shown) by the user and transmit a signal indicating that to the control unit 220.

  The control unit 220 operates as a top / bottom direction calculation unit 222, a facing direction determination unit 224, and a recognition unit 226. The top / bottom direction calculation unit 222 calculates the top / bottom direction of the image capturing apparatus 1000 based on the direction and magnitude of gravity acceleration calculated by the inclination measurement unit 250 described later.

  The facing direction determination unit 224 determines the facing direction of the image capturing apparatus 1000 with respect to the imaging target in response to a press detection signal (instruction input from the user) from the input unit 230. The recognition unit 226 performs character recognition, face recognition, and the like, which will be described later.

  Further, the image capturing unit 210 performs an image capturing process in response to a press detection signal from the input unit 230.

  The temporary storage unit 240 has a function of temporarily storing data. The temporary storage unit 240 is the same as the temporary storage unit 217. The temporary storage unit 240 is accessed by the control unit 220 and operates as a work memory.

  The inclination measuring unit 250 as an inclination measuring unit has a function of measuring the inclination of the image capturing apparatus 1000 by detecting gravitational acceleration in two directions. The inclination measuring unit 250 is a gravitational acceleration sensor.

  The gravitational acceleration sensor has one or more directions for detecting gravitational acceleration. This detection direction is called an axis. A gravity acceleration sensor having one axis is referred to as a one-axis gravity acceleration sensor. The gravity acceleration sensors having two and three axes are referred to as a two-axis gravity acceleration sensor and a three-axis gravity acceleration sensor, respectively. Two or more axes of the gravitational acceleration sensor are orthogonal to each other. Inclination measuring unit 250 in the present embodiment is a biaxial gravitational acceleration sensor.

Next, an inclination measurement method by the inclination measurement unit 250 will be described.
FIG. 4 is a diagram for explaining a method of measuring the inclination of the plane P301. The inclination degree indicates an inclination angle. Referring to FIG. 4, directions D301 and D302 represent the axial directions of the gravitational acceleration sensor used for measuring the inclination of plane P301. A direction D303 represents a direction opposite to the direction D301. A direction D304 represents a direction opposite to the direction D302. As the axis of the gravitational acceleration sensor, the direction D301 and the direction D303 are on the same axis. Further, the direction D302 and the direction D304 are on the same axis.

A vector V301 is a gravity acceleration vector having a direction and a magnitude. The magnitude (standard value) of the gravitational acceleration is 9.8 m / s ² . In the following, the gravitational acceleration is represented by a general symbol g (g = 9.8 m / s ² ).

  P302 represents a horizontal plane. An angle formed by the direction D303 and the horizontal plane P302 is θ (theta).

  An angle formed by the direction D303 and the gravitational acceleration vector V301 is λ (lambda). A perpendicular line from the tip of the gravitational acceleration vector V301 to the direction D303 on the plane P301 is L301. A projection vector of the gravitational acceleration vector V301 in the direction D303 on the plane P301 is set to V302. In addition, an angle formed by the gravity acceleration vector V301 and the perpendicular L301 is A303.

Projection vector V302 represents a gravitational acceleration vector applied in the axial directions of directions D301 and D303. Therefore, the following formula (1) is established.
sin (size of angle A303) = | V302 | / | V301 | (1)
In the expression (1), | x | represents the size of the vector x.

The angle A303 is θ from 180-90-λ. Since | V301 | is a constant g, the expression (1) becomes the following expression (2).
sin θ = | V302 | / g (2)
When Expression (2) is transformed, the following Expression (3) is obtained.
θ = sin−1 (| V302 | / g) (3)
The gravitational acceleration sensor (tilt measuring unit 250) can measure the magnitude of gravitational acceleration in the axial direction. That is, in FIG. 4, the gravitational acceleration sensor can measure the magnitude of the gravitational acceleration vector V302.

  Therefore, the value of θ can be calculated from Equation (3). That is, the gravitational acceleration sensor can measure the inclination of the plane P301. Therefore, when the value of θ is less than 90, it can be determined that the direction D301 is the top in the top direction and the direction D303 is the top in the top direction. In addition, the inclination between the horizontal plane P302 and the plane P301 in this case can also be calculated.

  Also, contrary to the example of FIG. 4, even when the direction D301 is the top-and-bottom direction and the direction D303 is the top-and-bottom direction, it is possible to determine the top and bottom and calculate the inclination by the above method. .

  Next, a method for determining the vertical direction of the image capturing apparatus 1000 using the inclination measuring unit 250 built in the image capturing apparatus 1000 will be described.

  FIG. 5 is a diagram illustrating the biaxial directions of the inclination measuring unit 250 that is a biaxial gravitational acceleration sensor built in the image capturing apparatus 1000. Referring to FIG. 5, the directions of the two axes of inclination measuring unit 250 are assumed to be direction D401 and direction D402.

  When determining the top-to-bottom direction, if the gravitational acceleration in the direction D401 is close to the gravitational acceleration constant g and the gravitational acceleration in the direction D402 is almost zero, the top-and-bottom direction calculation means 222 has the top of the image pickup device 1000 in the top-to-bottom direction. It is determined that the lower part is the earth in the top and bottom direction. Hereinafter, the axis in the direction D401 in FIG. 5 is referred to as a first axis. The axis in the direction D402 is referred to as the second axis.

  If the gravitational acceleration in the direction D401 is almost 0 and the gravitational acceleration in the direction D402 is a value close to the gravitational acceleration constant g, it can be determined that the state of the image pickup apparatus 1000 is as shown in FIG. .

  That is, the top / bottom direction calculation unit 222 determines the top / bottom direction of the image capturing apparatus 1000 based on the direction and magnitude of the gravitational acceleration calculated by the inclination measurement unit 250.

  Referring to FIG. 2 again, the image capturing apparatus 1000 further includes a tilt information storage unit 260.

  The inclination information storage unit 260 as inclination information storage means has a function of temporarily storing data. The inclination information storage unit 260 is the same as the temporary storage unit 217. The inclination information storage unit 260 temporarily stores the direction and magnitude of the gravitational acceleration calculated by the inclination measurement unit 250 and the information on the vertical direction determined by the vertical direction calculation unit 222.

  FIG. 7 is a diagram illustrating data stored in the inclination information storage unit 260. Referring to FIG. 7, the inclination information storage unit 260 stores gravity acceleration direction data DT10, gravity acceleration magnitude data DT20, inclination degree data DT30, and top and bottom direction data DT40.

  Gravity acceleration direction data DT10 is data of the direction of gravity acceleration on the first axis and the direction of gravity acceleration on the second axis in FIG. For example, in FIG. 5, if gravitational acceleration is applied in the direction D401, the direction of gravitational acceleration is positive. Further, in FIG. 5, if gravitational acceleration is applied in the direction opposite to the direction D401, the direction of gravitational acceleration is negative.

The gravitational acceleration magnitude data DT20 is data of the magnitude of gravity acceleration on the first axis and the magnitude of gravity acceleration on the second axis. As described above, the magnitude of the gravitational acceleration on the first axis and the magnitude of the gravitational acceleration on the second axis are the magnitudes applied to the first and second axes. The unit is g (g = 9.8 m / s ² ).

  The inclination data DT30 is data of an angle formed by the first axis and the horizontal plane and an angle formed by the second axis and the horizontal plane. In the following, the angle formed by the first axis and the horizontal plane is also referred to as the first inclination. In addition, the angle formed by the second axis and the horizontal plane is also referred to as a second inclination. Each of the angle formed by the first axis and the horizontal plane and the angle formed by the second axis and the horizontal plane can be calculated from the direction of gravity acceleration and the magnitude of gravity acceleration, as described above.

  Therefore, the data of the angle formed by the first axis and the horizontal plane and the angle formed by the second axis and the horizontal plane are stored in the inclination information storage unit 260 if calculated from the direction and magnitude of the gravitational acceleration as necessary. It does not have to be.

  The vertical direction data DT40 is data indicating the vertical direction of the image capturing apparatus 1000. The top / bottom direction calculation means 222 determines that the direction of the first axis is the top / bottom direction if the first slope formed by the first axis and the horizontal plane is equal to or greater than a predetermined threshold (for example, 10 degrees). In addition, when the second slope formed by the second axis and the horizontal plane is equal to or greater than a predetermined threshold, the top / bottom direction calculation unit 222 determines that the direction of the second axis is the top / bottom direction.

  Moreover, the top / bottom direction calculation means 222 determines which direction of the axis is the top or bottom of the top / bottom direction based on the direction of gravity acceleration along the axis.

  Therefore, the top / bottom direction data DT40 indicates one of three states in which the first axis direction is the top / bottom direction, the second axis direction is the top / bottom direction, and the first and second axis directions are not the top / bottom direction.

  The top-and-bottom direction data DT40 may not be stored in the inclination information storage unit 260 if it is calculated from the direction and magnitude of the gravitational acceleration as necessary. When displaying only the top-and-bottom direction of the image capturing apparatus 1000 on the display unit 150, only the top-and-bottom direction data DT40 may be stored in the tilt information storage unit 260.

  Referring to FIG. 2 again, the image capturing apparatus 1000 further includes a recording medium access unit 270 and a recording medium 272.

The above-described program 246 is recorded on the recording medium 272.
The recording medium access unit 270 has a function of reading the program 246 from the recording medium 272 on which the program 246 is recorded. The program 246 stored in the recording medium 272 is read from the recording medium access unit 270 and stored in the storage unit 245 by the operation (installation process) of the control unit 220.

  The installation processing program is stored in advance in the storage unit 245, and the installation processing is performed by the control unit 220 based on the installation processing program.

  Note that the program 246 may not be stored in the storage unit 245. In this case, the control unit 220 reads the program 246 stored in the recording medium 272 via the recording medium access unit 270 and performs predetermined processing based on the program 246.

  The recording medium 272 is a medium that can be detached from the image capturing apparatus 1000. That is, the program 246 recorded on the recording medium 272 is recorded on a medium or the like and distributed as a program product. The recording medium 272 is also distributed as a program product.

  The recording medium 272 includes a floppy (registered trademark) disk, CF (Compact Flash) card, SM (Smart Media (registered trademark)), MMC (Multi Media Card), SD (Secure Digital) memory card, memory stick (registered trademark), Either an xD picture card or other nonvolatile memory may be used.

  The recording medium 272 stores image data captured by the image capturing apparatus 1000.

  Although details will be described later, the display unit 150 displays information on the first inclination, the second inclination, or the top-and-bottom direction calculated by the inclination measurement unit 250.

  FIG. 8 shows an example of an image 500 displayed on the display unit 150 when the image capturing apparatus 1000 is rotated 90 degrees to the right. Referring to FIG. 8, in the image 500, the calculation result of the vertical direction of the image capturing apparatus 1000 by the vertical direction calculation unit 222 is displayed in characters “up, down, left, right”. In addition, the characters “up, down, left, right” of the image 500 are rotated and displayed according to the calculation result of the top and bottom direction of the image capturing apparatus 1000. “Up” in the image 500 corresponds to the heaven in the top and bottom direction, and “bottom” in the image 500 corresponds to the ground in the top and bottom direction. Therefore, the user can easily know the vertical direction of the image capturing apparatus 1000 by referring to the image 500. In the image 500, an image to be imaged and characters “up, down, left, right” are displayed simultaneously.

  When the image pickup apparatus 1000 is tilted with respect to the horizontal plane, the user can intuitively determine the top-and-bottom direction if the determination result of the top-and-bottom direction is represented by a symbol or the like instead of characters.

  FIG. 9 shows an example of an image 510 displayed on the display unit 150 when the image capturing apparatus 1000 is tilted with respect to the horizontal plane P302. Referring to FIG. 9, an image 512 displays an arrow 512 as a symbol. The arrow 512 indicates “top” in the vertical direction of the image capturing apparatus 1000. In the image 510, an image to be imaged and an arrow 512 are displayed simultaneously.

  In the present embodiment, the notification by the notification means (display unit 150) is an image display. However, the notification by the notification means may be a notification by voice instead of displaying an image.

  Next, processing performed in the image capturing apparatus 1000 will be described. Note that the processing described below is started when the image capturing apparatus 1000 is powered on.

  FIG. 10 is a flowchart of processing performed in the image capturing apparatus 1000. With reference to FIG. 10, in step S110, as described above, the inclination measurement unit 250 measures the first inclination and the second inclination indicating the inclination of the image capturing apparatus 1000. Thereafter, the process proceeds to step S120.

  In step S120, the top / bottom direction calculation means 222 determines whether one of the first inclination and the second inclination is equal to or greater than a predetermined value A. Here, the predetermined value A is a value determined in consideration of the performance of the inclination measurement unit 250 (gravity acceleration sensor), an appropriate inclination for the user, and the like. The predetermined value A is a value in the range of 30 to 60, for example.

  If YES in step S120, the process proceeds to step S122. On the other hand, if NO at step S120, the process proceeds to step S132.

  In step S <b> 122, the top / bottom direction calculation unit 222 calculates the top / bottom direction of the image capturing apparatus 1000 based on the first slope and the second slope measured by the slope measuring unit 250. That is, the vertical direction of the imaging target is calculated. Thereafter, the process proceeds to step S124.

  In step S <b> 124, the control unit 220 generates an image (hereinafter also referred to as a notification image) displaying information based on the calculation result of the top-and-bottom direction of the image capturing apparatus 1000, and causes the display unit 150 to display the notification image. The notification image is, for example, the image 500 in FIG. In the notification image, an image to be imaged and at least one of information on the first or second inclination as the inclination and information on the top-and-bottom direction are displayed.

  FIG. 11 is a diagram showing a notification image 520 as an example. Referring to FIG. 11, the notification image 520 displays the information on the top and bottom directions with the characters “up, down, left, right”. In the notification image 520, an image to be imaged and characters “up, down, left, right” are displayed simultaneously. FIG. 11 shows the state of the image pickup apparatus 1000 when the first inclination or the second inclination is a predetermined value A or more.

  FIG. 12 is a diagram showing a notification image 600 when the image capturing apparatus 1000 is tilted 30 degrees to the right. Referring to FIG. 12, in notification image 600, a character image 610 that displays a character string is arranged. In the character image 610, a character string indicating that the image capturing apparatus 1000 is inclined 30 degrees to the right with respect to the horizontal plane P302 is displayed. In the notification image 600, an image to be imaged and a character image 610 are displayed at the same time.

  By displaying the notification image 600 on the display unit 150 as shown in FIG. 12, the user can be alerted so that the image capturing apparatus 1000 is not tilted. For this purpose, an instruction to the user such as “tilt 30 degrees to the left so that there is no tilt” may be displayed on the notification image.

  FIG. 13 is a diagram illustrating a notification image 600A when the image capturing apparatus 1000 is tilted 30 degrees to the right. Referring to FIG. 13, notification image 600 </ b> A is an image in which arrow 512 indicating information on the top and bottom direction is further displayed on notification image 600.

  Referring to FIG. 10 again, when the process of step S124 ends, the process proceeds to step S126.

  In step S126, the top / bottom direction calculation means 222 causes the inclination information storage unit 260 to store the inclination data DT30 described above based on the first inclination and the second inclination measured in step S110. When the inclination data storage unit 260 has already stored the inclination data DT30, the top / bottom direction calculation unit 222 overwrites and stores the latest inclination data DT30 in the inclination information storage unit 260.

  Further, the top / bottom direction calculation means 222 causes the tilt information storage unit 260 to store the top / bottom direction data DT40 described above based on the top / bottom direction information calculated in step S122. When the top / bottom direction data DT40 is already stored in the tilt information storage unit 260, the top / bottom direction calculation unit 222 overwrites and stores the latest top / bottom direction data DT40 in the tilt information storage unit 260. Thereafter, the process proceeds to step S150.

  In step S150, it is determined whether there is an instruction to determine the facing direction. Specifically, the facing direction determination unit 224 determines whether or not a pressing detection signal is received from the input unit 230. If YES in step S150, the process proceeds to step S152 described later. In step S150, YES is determined in the following case. That is, the user presses the shutter button 120 in order to determine the front direction of the image capturing apparatus 1000 corresponding to the top and bottom direction with reference to the top and bottom direction information of the notification image displayed on the display unit 150. This is when the operation is performed.

  On the other hand, if NO at step S150, the process at step S110 is performed again.

If NO at step S120 described above, the process proceeds to step S132.
In step S132, similarly to step S122, the vertical direction calculation unit 222 calculates the vertical direction of the image capturing apparatus 1000 based on the first and second inclinations measured by the inclination measurement unit 250. That is, the vertical direction of the imaging target is calculated.

  When the image capturing apparatus 1000 is parallel to the horizontal plane, the top / bottom direction calculating unit 222 determines the top / bottom direction of the image capturing apparatus 1000 based on the first and second tilt degrees measured by the tilt measuring unit 250. Cannot be judged.

  FIG. 14 is a diagram illustrating a state in which the image capturing apparatus 1000 is parallel to the horizontal plane P302. Referring to FIG. 14, an imaging object 701 is placed on the horizontal plane P302.

  Note that, before the image capturing apparatus 1000 is in the state of FIG. 14, it is always determined as YES in step S120. That is, when the image capturing apparatus 1000 is powered on, the image capturing apparatus 1000 is in the state shown in FIG. 14 unless the user intentionally makes it. This is because the user generally performs operations such as adjustment (focus, etc.) of the image capturing apparatus 1000 before the image capturing process. At this time, the image capturing apparatus 1000 is in a state where the top-to-bottom direction can be determined, and it is considered that the user does not often start an operation with the image capturing apparatus 1000 completely horizontal. Thereafter, the user takes the image capturing apparatus 1000 in a horizontal state in order to capture an image of the imaging object 701 placed horizontally.

  For this reason, the process of step S126 is always performed once before the image capturing apparatus 1000 enters the state of FIG. Therefore, the inclination information storage unit 260 stores the inclination data DT30 and the top / bottom direction data DT40.

  In this case, the top / bottom direction calculation unit 222 determines the top / bottom direction of the image capturing apparatus 1000 based on at least one of the slope data DT30 and the top / bottom direction data DT40 stored in the tilt information storage unit 260. Thereafter, the process proceeds to step S134.

  In step S134, the control unit 220 generates a notification image that displays information based on the calculation result of the top-and-bottom direction of the image capturing apparatus 1000, and causes the display unit 150 to display the notification image.

  FIG. 15 is a diagram illustrating a notification image 700 as an example when the image capturing apparatus 1000 is parallel to the horizontal plane P302. Referring to FIG. 15, the notification image 700 displays information on the top and bottom directions with characters “up, down, left, right”. In the notification image 700, the image of the imaging object 701 and the characters “up, down, left, right” are displayed at the same time.

  FIG. 16 is a diagram illustrating a notification image 700A as an example when the image capturing apparatus 1000 is parallel to the horizontal plane P302. Referring to FIG. 16, in the notification image 700 </ b> A, the arrow 710 indicating the information on the top and bottom direction of the image capturing apparatus 1000 and the fact that the image capturing apparatus 1000 is parallel to the image capturing object 701 (horizontal plane P <b> 302) are indicated by characters. It is displayed. Note that an image of the imaging object 701 is also displayed in the notification image 700A.

  Referring to FIG. 10 again, when the process of step S134 ends, the process proceeds to step S150 described above. If YES in step S150, the process proceeds to step S152.

  In step S152, the facing direction determination unit 224 uses the inclination data DT30 and the top and bottom direction data DT40 currently stored in the inclination information storage unit 260 as the reference for the vertical direction of the imaging object 701. It is determined as information on 1000 facing directions. Specifically, the facing direction determination unit 224 disables the update of the inclination degree data DT30 and the top and bottom direction data DT40 stored in the inclination information storage unit 260. Thereafter, the process proceeds to step S154.

  In step S154, an image correction process is performed. In the image correction process, the control unit 220 transmits a read control instruction for reading the image data stored in the temporary storage unit 217 to the image capturing unit 210 at the present time. When receiving the readout control instruction, the imaging control unit 216 in the image imaging unit 210 transmits the image data currently stored in the temporary storage unit 217 to the control unit 220.

  The control unit 220 receives the image data transmitted from the image capturing unit 210. Hereinafter, the image data received by the control unit 220 from the image capturing unit 210 is also referred to as captured image data.

  When it is necessary to correct the top / bottom direction of the received image data (captured image data) based on the top / bottom direction data DT40, which is information on the facing direction of the image capturing apparatus 1000, the control unit 220 performs processing such as rotation. Then, an image in which the vertical direction is corrected (hereinafter also referred to as a corrected image) is generated. For example, when the image capturing apparatus 1000 is in a state as shown in FIG. 8, if the user presses the shutter button 120, the part of the corrected image displayed as “up” is the upper part of the image. The image is long in the vertical direction.

  Control unit 220 does not correct the received image when it is not necessary to correct the vertical direction of the received image data based on vertical direction data DT40.

  Next, the control unit 220 stores the received image data (captured image data) in the temporary storage unit 240 when it is not necessary to correct the generated corrected image data or the vertical direction of the image data. Thereafter, the process proceeds to step S156.

  In step S156, recognition processing is performed. In the recognition process, the recognition unit 226 performs character recognition or face recognition based on the image stored in the temporary storage unit 240. That is, the recognition unit 226 performs processing such as character recognition or face recognition based on the corrected image when the top-to-bottom direction needs to be corrected. In other words, when the recognition unit 226 needs to correct the top / bottom direction, the recognition unit 226 recognizes the character of the captured image based on the information on the front (top) direction of the image pickup apparatus 1000 determined by the front direction determination unit 224. Alternatively, processing such as face recognition is performed.

  Note that data necessary for character recognition or face recognition is stored in the storage unit 245 in advance. Thereafter, the recognition unit 226 stores the recognition result in the storage unit 245. Thereafter, the process proceeds to step S160.

  In step S160, the control unit 220 causes the recording medium 272 to store the image data (corrected image data or captured image data) stored in the temporary storage unit 240. That is, when it is necessary to correct the vertical direction, the control unit 220 stores the captured image data in the recording medium 272 based on the information on the facing direction of the image capturing apparatus 1000 determined by the facing direction determination unit 224. Remember. That is, when it is necessary to correct the vertical direction, the control unit 220 stores the captured image data in the recording medium 272 based on the calculated information about the vertical direction of the imaging target. Thereafter, this process ends.

  In the present embodiment, when it is determined in step S150 that there is an instruction to determine the facing direction, the facing direction determination unit 224 determines the facing direction of the image capturing apparatus 1000 in step S152.

  However, the present embodiment can be implemented without an instruction for determining the facing direction. In this case, for example, if the image capturing apparatus 1000 is in a horizontal state for a certain period of time, the facing direction determination unit 224 uses the inclination data DT30 stored in the inclination information storage unit 260 for the imaging object 701. The information is determined as information on the directly facing direction of the image capturing apparatus 1000 based on the vertical direction.

  As described above, in the present embodiment, even when the imaging object 701 placed horizontally is imaged, the top-and-bottom information in an inclined state is retained, and the image imaging apparatus 1000 becomes nearly horizontal. By using the top-and-bottom information in the inclined state, the top-and-bottom direction can be accurately calculated even for the imaging object 701 placed horizontally.

  Further, a user who has the image capturing apparatus 1000 generally performs operations such as power-on of the image capturing apparatus 1000 and camera adjustment (focus, etc.) before the image capturing process. At this time, the image capturing apparatus 1000 is in a state where the top-to-bottom direction can be determined, and it is considered that the user does not often start an operation with the image capturing apparatus 1000 completely horizontal. Thereafter, the user brings the image capturing apparatus 1000 to a horizontal state when capturing an image of a horizontally placed subject.

  The degree of inclination of the image capturing apparatus 1000 to a certain degree necessary for the vertical direction determination in the present invention can be naturally acquired in the flow of imaging at this time, and therefore the burden on the user is small.

  In the present embodiment, information on the top and bottom directions is sequentially displayed on the display unit 150 together with the subject. Therefore, the user can take an image while simultaneously confirming the subject and the vertical direction. Therefore, the imaging target and the display in the top-and-bottom direction can be confirmed without greatly moving the line of sight, and imaging is easy.

  In the present embodiment, an image based on the determined top-and-bottom direction is generated and recognition processing is performed. Therefore, it becomes possible to recognize characters or faces more accurately.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a figure which shows the external view of the image imaging device in this Embodiment. It is a block diagram which shows the internal structure of the image imaging device in this Embodiment. It is a block diagram which shows the structure of the image imaging part in this Embodiment. It is a figure for demonstrating the measuring method of the inclination of the plane P301. It is a figure which shows the direction of 2 axis | shafts of the inclination measurement part which is a 2 axis | shaft gravity acceleration sensor built in the image pick-up device. It is a figure which shows the direction of 2 axis | shafts of the inclination measurement part which is a 2 axis | shaft gravity acceleration sensor built in the image pick-up device. It is a figure which shows the data memorize | stored in an inclination information storage part. An example of an image displayed on the display unit when the image pickup apparatus is rotated 90 degrees to the right is shown. An example of the image displayed on a display part at the time of inclining an imaging device with respect to a horizontal surface is shown. It is a flowchart of the process performed with an imaging device. It is a figure which shows the alerting | reporting image as an example. It is a figure which shows a alerting | reporting image in case the image imaging device is in the state inclined 30 degrees to the right. It is a figure which shows a alerting | reporting image in case the image imaging device is in the state inclined 30 degrees to the right. It is the figure which showed the state in which an imaging device is parallel with respect to a horizontal surface. It is a figure which shows the alerting | reporting image as an example when an image imaging device is parallel with respect to a horizontal surface. It is a figure which shows the alerting | reporting image as an example when an image imaging device is parallel with respect to a horizontal surface. It is the figure which showed the state in which an image imaging device is parallel with respect to an imaging target object.

Explanation of symbols

  110 lens group, 120 shutter button, 150 display unit, 210 image capturing unit, 220 control unit, 230 input unit, 246 program, 272 recording medium, 1000 image capturing device.

Claims (14)

An imaging device,
Image imaging means for imaging an imaging target;
An inclination measuring means for measuring an inclination indicating an angle of inclination of the image pickup device;
A top and bottom direction calculating unit that calculates a top and bottom direction of the image pickup device based on the tilt degree measured by the tilt degree measuring unit;
Inclination information storage means for storing information on at least one of the inclination measured by the inclination measurement means and the vertical direction calculated by the vertical direction calculation means;
A facing direction determining means for determining a facing direction of the image capturing apparatus with respect to the imaging target;
The inclination measurement process of the image pickup apparatus by the inclination measurement unit, and the inclination and the vertical direction calculated by the vertical direction calculation unit when the measured inclination is a predetermined value or more. The image capturing apparatus, wherein the process of storing at least one piece of information in the inclination information storage unit is repeatedly performed until the facing direction is determined by the facing direction determining unit.   When the measured inclination is smaller than the predetermined value, the top and bottom direction calculating means is configured to capture the image based on the latest inclination or the information on the top and bottom direction stored in the inclination information storage means. The image capturing apparatus according to claim 1, wherein the top-and-bottom direction of the apparatus is calculated.   The image capturing apparatus according to claim 1, further comprising a notifying unit that notifies at least one of the inclination of the image capturing apparatus and the top and bottom direction calculated by the top and bottom direction calculating unit.   The informing means, when the inclination measured by the inclination measuring means is equal to or greater than the predetermined value, at least one of the inclination of the image pickup device and the top-and-bottom direction calculated by the top-and-bottom direction calculating means. The image capturing apparatus according to claim 3, wherein information is reported.   When the inclination measured by the inclination measuring means is smaller than the predetermined value, the notifying means notifies at least one of the inclination and the top and bottom information stored in the inclination information storage means. The image pickup device according to claim 3. The notification means is a display device that displays the imaging target of the image imaging device,
The image capturing apparatus according to claim 3, wherein the notifying unit displays, together with the imaging target, information on at least one of the inclination of the image capturing apparatus and the top and bottom direction calculated by the top and bottom direction calculating unit. . It further comprises an input means for receiving an instruction input from the user,
The image capturing apparatus according to claim 1, wherein the facing direction determining unit determines a facing direction of the image capturing apparatus with respect to the imaging target in accordance with the instruction input received by the input unit. . Image storage means for storing a captured image captured by the image capturing means;
The image capturing apparatus according to claim 7, wherein the image storage unit stores the captured image in response to the instruction input received by the input unit.   The image capturing apparatus according to claim 8, wherein the captured image is stored in the image storing unit based on a facing direction of the image capturing apparatus determined by the facing direction determining unit.   The image capturing apparatus according to claim 1, wherein the inclination measuring unit includes a sensor that measures gravitational acceleration. It further comprises a recognition means for recognizing an image,
The recognition unit according to claim 1, wherein the recognition unit recognizes a captured image captured by the image capturing unit based on a facing direction of the image capturing apparatus determined by the facing direction determining unit. Imaging device. A program for causing a computer in an image capturing apparatus to execute an image capturing process,
Imaging an imaging target;
Measuring an inclination indicating an inclination angle of the image pickup device;
Calculating the top-to-bottom direction of the image pickup device based on the inclination measured by the step of measuring the inclination;
Storing at least one information of the top and bottom direction calculated by the step of calculating the slope and the top and bottom direction measured by the step of measuring the slope and the storage unit in the image capturing device;
A program for causing a computer to execute a step of determining a facing direction of the image capturing apparatus with respect to the imaging target.   The step of calculating the top-to-bottom direction is performed when the measured inclination is smaller than the predetermined value, based on the latest inclination or the information on the top-and-bottom direction stored in the storage unit. The program according to claim 12, wherein the top-and-bottom direction of a device is calculated.   A recording medium on which the program according to claim 12 or 13 is recorded.

Images