JP6142060B1 - smartphone - Google Patents

Abstract

The present invention provides a smartphone that operates an imaging device by moving a main body to realize positioning of an imaging frame and prevention of a so-called “camera shake” state. A smartphone includes an image pickup device that picks up an object to be picked up, a sensor that measures a position related to the three-axis direction of the image pickup device or an amount related to the position, and a three-axis direction output from the sensor. A control device 6 is provided that controls the operation of the imaging device 2 using at least one of the three sensor values corresponding to each. The sensor 5 measures the position in the three-axis direction of the imaging device 2 that images the object to be imaged or the amount related to the position, and the control device 6 3 corresponding to each of the three-axis directions output from the sensor 5. By controlling the operation of the imaging device 2 using at least one of the two sensor values, the user can control the operation of the imaging device 2 mounted on the smartphone by moving the smartphone body. [Selection] Figure 1

Description

  The present invention relates to a smartphone that controls the operation of an imaging apparatus.

  An imaging device is mounted on a smartphone, and an object to be imaged is regularly imaged using the imaging device. At the time of imaging, the user supports and fixes the smartphone with one or both hands and presses the imaging button. Here, the imaging button is provided on the surface on the opposite side of the smartphone with respect to the imaging direction of the imaging device. The user can take an image by operating the user's own side of the smartphone while looking at the object to be imaged in front.

  In so-called “self-portrait” in which a user operating a smartphone takes an image of himself / herself using the smartphone, the imaging button is on a surface not facing the user of the smartphone. For this reason, it is not easy to press the imaging button, and there is a high possibility that other problems will occur, such as a so-called “camera shake” state in which the smartphone moves and the imaging frame is misaligned. Note that such a problem caused by the operation of the imaging button may also occur when the self-taking is not performed.

  Thus, for example, Patent Document 1 discloses a system for carrying a controller independent of a smartphone and operating the smartphone wirelessly using the controller. However, it is not widely accepted to carry a controller with a smartphone. Rather, it is effective for the smartphone to have means other than the operation of the image capture button as means for causing the smartphone to perform imaging.

  As an operation of a smartphone by means other than the button operation, for example, Non-Patent Document 1 discloses a means for starting an application by an operation of shaking a smartphone. However, for imaging, it is necessary to satisfy requirements such as positioning of an imaging frame, prevention of a so-called “camera shake” state, and it is not sufficient to simply perform imaging by an operation of shaking a smartphone. Moreover, although some recent smartphones have a function of taking an image with a volume button or the like, many users do not have it.

  As described above, means for using the smartphone image pickup device by means other than the operation of the image pickup button has not been widely known.

JP 2006-072673 A

Shake-Android app on Google Play https://play.google.com/store/apps/details?id=com.adl.appshaker&hl=en

  It is an object of the present invention to provide a smartphone that operates an imaging device by moving a main body and realizes positioning of an imaging frame and prevention of a so-called “camera shake” state.

The smartphone of the present invention
It is a smartphone that controls the operation of the imaging device mounted on it by moving the smartphone body,
An imaging device for imaging an object to be imaged;
A sensor for measuring speed, acceleration, angular velocity or angular acceleration in the three-axis directions of the imaging device;
A control device that controls the operation of the imaging device using at least one of three sensor values corresponding to each of the three axial directions output from the sensor;
With
The controller is
The three sensor values are periodically received from the sensor , and when the change amount of at least one of the three sensor values exceeds a threshold value, the time point at which the threshold value is exceeded is determined as a time change pattern. As a start time, it is determined whether the time change patterns of the three sensor values correspond to a predetermined time change pattern,
After the predetermined time change pattern, it is determined whether or not the three sensor values are substantially constant with respect to time, and when the three sensor values are substantially constant, the image of the imaging device at the start time and the start time from comparing the image of the imaging apparatus at the point in time that has elapsed T ', to operate the imaging apparatus when the difference between the images has the elapsed only if less than the threshold value,
The value of T ′ is 1.5 to 2.5 seconds,
The predetermined time change pattern is a time change pattern generated by rotating the smartphone in one direction about any one of the three axes and then rotating the smartphone in the opposite direction to return to the position before the rotation. It is characterized by being.

  According to this feature, it becomes possible for the user to control the operation of the imaging device mounted on the smartphone by moving the smartphone main body so that the time change patterns of the three sensor values become predetermined. . The predetermined time change pattern is a time change pattern generated by rotating the smartphone in one direction about any one of the three axes and then rotating the smartphone in the opposite direction to return to the position before the rotation. And The user positions the smartphone in a posture to image the object to be imaged, rotates it in one direction about any one of the three axes from that state, and then rotates in the opposite direction to return to the position before the rotation ( The imaging frame can be positioned), stopped (so-called “hand-shake” is eliminated), and the operation of the imaging apparatus can be controlled after taking the posture for imaging the imaged object again.

The smartphone of the present invention
The control device causes the user to rotate the smartphone in one direction about any one of the three axes, and then rotate the smartphone in the opposite direction to return to the position before the rotation. The predetermined time change pattern is determined by receiving the three sensor values from.

  According to this feature, a predetermined time change pattern used for operating the imaging apparatus can be set uniquely for each user.

The smartphone of the present invention
The control device calculates a difference between the at least one sensor value and the predetermined time change pattern, and compares the sum of the difference with a threshold value.

  According to this feature, when the difference sum is smaller than the threshold value, it can be determined that the time change pattern of at least one sensor value corresponds to a predetermined time change pattern, and a reliable determination can be made.

The smartphone of the present invention
The sensor is an acceleration sensor that measures acceleration in the three-axis directions of the imaging device,
When the two sensor values of the three sensor values change with respect to time and the remaining one sensor value is substantially constant with respect to time, the control device determines the time of the two sensor values. It is characterized by determining a change pattern.

  According to this feature, the operation of the imaging device can be controlled by the control device by an acceleration sensor mounted on many smartphones. As for the acceleration sensor, as described later, two of the three sensor values change with time, and the remaining one sensor value is substantially constant with respect to time.

The smartphone of the present invention
In the predetermined time change pattern, the acceleration value, which is the sensor value, increases by (X) from (1) −9.8 m / s ² and returns to −9.8 m / s ² within time T. (2) Increase or decrease from 0 m / s ² by X2 and change back to 0 m / s ² within time T, (3) Increase from 9.8 m / s ² by X1 and within 9.8 T / 9.8 m / s ² is any one of the change pattern back to the s ^2,
The value of X1 is 4.9 to 14.7 m / s ² ,
The value of X2 is 8.5 to 9.8 m / s ² ,
The value of T is 0.8 to 1.8 seconds.

According to this feature, the operation of the imaging device mounted on the smartphone can be controlled by rotating the smartphone 60 to 120 degrees around 0.8 to 1.8 seconds around one of the three axes. It becomes possible to control. When the smartphone is moved over a period of 0.8 to 1.8 seconds, the acceleration of the movement is not so large, and the time changes as the time changes in the direction of gravitational acceleration (9.8 m / s ² ) with respect to the three axes. Patterns can be defined. Note that 4.9 m / s ² ≈9.8 m / s ² × (1-cos 60 °), 14.7 m / s ² ≈9.8 m / s ² × (1-cos 120 °), 8.5 m / s ² ≈9.8 m / s ² × sin 60 °, and a time change pattern is realized by rotation of 60 to 120 degrees.

The smartphone of the present invention
The sensor is an angular velocity sensor that measures a rotational angular velocity related to three axes of the imaging device,
When the sensor value of one of the three sensor values changes with respect to time and the remaining two sensor values are substantially constant with respect to time, the control device determines the time of the one sensor value. It is characterized by determining a change pattern.

  According to this feature, the operation of the imaging device can be controlled by the control device using the angular velocity sensor mounted on the smartphone. As for the angular velocity sensor, as described later, one of the three sensor values changes with time, and the remaining two sensor values are substantially constant with respect to time.

The smartphone of the present invention
In the predetermined time change pattern, the value of the angular velocity that is the sensor value increases or decreases from 0 (1) and returns to 0, and the absolute value of the integral of the sensor value is Y radians, (2) A change pattern in which a decrease or increase from 0 returns to 0, and the change in which the absolute value of the integral of the sensor value is Y radians continues during time T;
The value of Y is π / 3 to 2π / 3 radians,
The value of T is 0.8 to 1.8 seconds.

  According to this feature, the operation of the imaging device mounted on the smartphone can be controlled by rotating the smartphone 60 to 120 degrees around 0.8 to 1.8 seconds around one of the three axes. It becomes possible to control.

The smartphone of the present invention
When the change amount of at least one of the three sensor values exceeds a threshold value, the control device sets the time change pattern of the three sensor values as a start time point of the time change pattern when the threshold value is exceeded. It is characterized by judging.

  According to this feature, the user can grasp the start time of the rotation operation.

The smartphone of the present invention
The control device operates the imaging device when T ′ has elapsed from the start time,
The value of T ′ is 1.5 to 2.5 seconds.

  According to this feature, when the user rotates in one direction and then rotates in the opposite direction to return to the pre-rotation position in approximately 2 seconds (1.5 to 2.5 seconds) The imaging apparatus can be operated only in The user can operate for approximately 2 seconds and prevent malfunction of the imaging apparatus when the user does not perform the operation.

The smartphone of the present invention
The control device compares the image of the imaging device at the start time with the image of the imaging device at the time when T ′ has elapsed from the start time, and only when the difference between the two images is smaller than a threshold value It is characterized by operating.

  According to this feature, when the user positions the imaging frame at the start time, the imaging device can be operated when the smartphone is returned to that position.

The smartphone of the present invention
The control device compares the difference between the two images based on the difference in the histogram of the RGB luminance values of the two images.

  According to this feature, the image of the imaging device at the start time point and the image pickup time point can be compared with easy calculation.

  According to the smartphone of the present invention, it is possible to operate the imaging device by moving the main body, and to achieve positioning of the imaging frame and prevention of a so-called “camera shake” state.

FIG. 1 is a block diagram illustrating a configuration of a smartphone. FIG. 2 is a diagram illustrating an example of an operation for operating the imaging apparatus. FIG. 3 is a diagram illustrating the movement of the smartphone according to the operation of FIG. FIG. 4 is a diagram illustrating another example of an operation for operating the imaging apparatus. FIG. 5 is a diagram illustrating the movement of the smartphone according to the operation of FIG. FIG. 6 is a diagram showing a temporal change pattern of sensor values output by the acceleration sensor in the operation of FIG. FIG. 7 is a diagram showing a temporal change pattern of sensor values output by the acceleration sensor in the operation of FIG. FIG. 8 is a diagram showing a temporal change pattern of sensor values output by the angular velocity sensor in the operation of FIG.

  Hereinafter, an embodiment of the present invention will be described.

  In FIG. 1, the structure of the smart phone 1 which concerns on this embodiment is shown. The smartphone 1 is intended to control the operation of the imaging device mounted on the smartphone main body (simply referred to as the main body 1), and includes the imaging device 2, the display device 3, and the storage device 4. , Sensor 5 and control device 6. Note that these devices are mounted in a casing-like main body 1 (see, for example, FIG. 2) having a rectangular surface that extends in the direction of one axis. In addition, let the direction orthogonal to each other on one surface of the main body 1 provided with the display device 3 be the X-axis and Y-axis directions, and let the direction orthogonal to the one surface be the Z-axis direction. In imaging, the main body 1 is often supported with the display surface of the display device 3 facing the horizontal direction. In this state, the Z axis is the horizontal direction, one of the X axis and the Y axis is the vertical direction, and the other is the horizontal direction. Will turn to.

  The imaging device 2 is a device that images an object to be imaged through a lens (not shown) provided in the main body 1. As an example, a CCD image sensor or a CMOS image sensor can be employed. The imaging device 2 is incorporated in the main body 1. The operation of the imaging apparatus controlled in the present embodiment is to activate the electronic shutter of the imaging apparatus 2 and take a still image, that is, a photograph (also simply referred to as an image). It is possible to take a picture. It is also possible to record an image when the electronic shutter is not activated. An image picked up by the image pickup device 2 is expressed in a digital format such as a JPEG format, and transmitted to the display device 3 and the storage device 4 described later.

  The display device 3 is a device that displays an image captured by the imaging device 2 via a lens (not shown) or a captured image, and is provided on the surface of the main body 1.

  The storage device 4 is a device that stores an image picked up by the image pickup device 2, and can employ a memory such as a DRAM, for example. The storage device 4 is incorporated in the main body 1.

  The sensor 5 is a device that is mounted in the main body 1 and measures speed, acceleration, angular velocity, or angular acceleration in the three-axis directions of the imaging device 2. As the sensor 5, for example, an acceleration sensor that measures acceleration in the three-axis directions of the imaging device 2 can be employed. The acceleration in the three-axis direction of the imaging device 2 is measured by the acceleration sensor, and two of the three sensor values corresponding to each of the three-axis directions output from the acceleration sensor are used by the control device 6 described later. By controlling the operation of the imaging device 2, the user can control the operation of the imaging device 2 by moving the main body 1. As the sensor 5, an angular velocity sensor (also referred to as a gyro sensor) that measures a rotation angle with respect to the three axes of the imaging device 2 can be employed. The angular velocity sensor measures the rotation angle about the three axes of the imaging device 2, and the control device 6 described later uses one of the three sensor values corresponding to the rotation angle about the three axes output from the angular velocity sensor. By controlling the operation of the imaging device 2, the user can control the operation of the imaging device 2 by moving the main body 1.

  The sensor 5 is not limited to an acceleration sensor and an angular velocity sensor (gyro sensor), and a velocity sensor and an angular acceleration sensor may be employed.

  The control device 6 is a device that controls the operation of the imaging device 2. The control device 6 has a central processing unit (CPU), for example, and develops a function as a device that controls the operation of the imaging device 2 by executing an imaging application. The imaging application is stored in the storage device 4, for example, and can be read by the CPU. The control device 6 periodically receives three sensor values corresponding to each of the three axial directions from the sensor 5, and determines a time change pattern of at least one of the sensor values. Thereby, the operation of the imaging device 2 can be controlled based on the temporal change pattern of at least one of the three sensor values generated when the user moves the main body 1. The operation control of the imaging device 2 by the control device 6 will be described later.

  In addition, it is good also as providing a communication apparatus (not shown) and being able to transfer the image imaged by the imaging device 2 by this.

  FIG. 2 and FIG. 3 show an example of an operation for operating the imaging device 2 by the user and the movement of the main body 1 in this operation, respectively. As shown in FIG. 2, the user holds a short hand of the main body 1 with the right hand 8, directs the display device 3 toward himself (the back of the drawing), and supports the main body 1 sideways so that an object to be imaged, for example, a user Position itself in the imaging frame to take an image. From this state, the user (1) turns the wrist and swings the main body 1 about 90 degrees counterclockwise in FIG. Thereby, the main body 1 is supported vertically. From this state, the user (2) turns the wrist in the reverse direction and swings the main body 1 clockwise in FIG. As a result, the main body 1 is supported sideways and takes the posture for imaging the object to be imaged again. The user controls the operation of the imaging device 2 by (3) stopping the main body 1 in this state and (4) standing still. In this series of operations, as can be seen from FIG. 3, the main body 1 (1) rotates in one direction about the Z-axis passing near one short side, and then (2) rotates in the opposite direction to return to the position before rotation. Return to step (3). The same applies to the left hand instead of the right hand.

  Here, the main body 1 is rotated with respect to the Z-axis passing near the short side, but the Z-axis may pass through another position of the main body 1, for example, the Z-axis passing near the long side. It is good also as rotating about. That is, if the rotation is performed in a direction parallel to the Z axis shown in the figure, the X and Y coordinates of the rotation axis are arbitrary.

  FIG. 4 and FIG. 5 show another example of the operation for operating the imaging device 2 by the user and the movement of the main body 1 in this operation, respectively. As shown in FIG. 4, the user holds both the short sides of the main body 1 with both hands 9, directs the display device 3 toward himself (the back of the drawing), and supports the main body 1 sideways, whereby an object to be imaged, for example, a user Position itself in the imaging frame to take an image. From this state, the user (1) turns the wrist and tilts the main body 1 upward (or downward) by about 90 degrees. Thereby, the main body 1 is supported horizontally. From this state, the user (2) reverses the wrist and returns the main body 1. As a result, the main body 1 is supported sideways and takes the posture for imaging the object to be imaged again. The user (3) controls the operation of the imaging apparatus by holding the main body 1 stationary in this state. In this series of operations, as can be seen from FIG. 5, the main body 1 (1) tilts in one direction about the Y axis passing through the center of the main body (rotates with respect to the Y axis), and then (2) returns (Y axis To the original position) and (3) rest. Note that the same applies even if the operation is performed with one hand instead of both hands.

  Here, the main body 1 is tilted with respect to the Y axis passing through the center thereof, but may be tilted with respect to the X axis passing through the center of the main body 1. Further, the coordinates of the rotation axis are arbitrary.

  The operation control of the imaging device 2 by the control device 6 will be described. Operation control includes determination of a temporal change pattern of sensor values and determination of imaging conditions such as stillness.

(Judgment of temporal change pattern of sensor value)
The control device 6 detects three sensor values g _{d, n} from the sensor 5 (d = 1, 2, 3: d = 1 is for the X axis, d = 2 is for the Y axis, and d = 3 is for the Z axis. Value) periodically at intervals Δt. A numerical sequence g _{d, i} (i = 1 to N) with respect to time t _i (= iΔt) can be obtained. The control device 6 determines whether or not the numerical sequence g _{d, i} corresponds to a predetermined time change pattern. Here, variations can be considered in the “predetermined time change pattern” and the procedure for determination. This will be described below.

  Hereinafter, a case where the sensor 5 is an acceleration sensor will be described.

At each reception, the control device 6 changes at least one change amount Δg _{d, n} (= | g _{d, n + 1} −g _d ) among the absolute values of the three sensor values g _{d, n} (d = 1, 2, 3). _{, n |)} is determined whether more than a threshold value, the time t _n and the beginning of the time variation pattern when the threshold is exceeded. Hereinafter, for the sake of explanation, it is assumed that the change amount of d = 1 exceeds the threshold, but the same applies to d = 2 and d = 3.

Thereafter, the control device 6 determines whether one of the change amounts Δg _{d, n} (d = 2, 3) other than the change amount exceeding the threshold value exceeds the threshold value. Hereinafter, for the sake of explanation, it is assumed that the change amount of d = 2 exceeds the threshold, but the same applies to d = 3. That is, the change amount of d = 1, 2 exceeds the threshold value, and the change amount of d = 3 does not exceed the threshold value.

Thereafter, the control device 6 determines whether the values of g _{1, i} and g _{2, i} (i = n to N) correspond to a “predetermined time change pattern” or g _{3, i} (i = n to N). ) Is substantially constant. Regarding the value of g _{3, i} , if the difference from the value of g _{3, n} (| g _{3, n} −g _{3, i} |) is less than a certain value regardless of i, it shall be “almost constant”. Can do.

  The following procedure can be considered as to whether or not d = 1, 2 corresponds to the “predetermined time change pattern”.

(Procedure 1) A time t _i is obtained when the value of Δg _{d, i} reverses in polarity (Δg _{d, i} and Δg _{d, i + 1} are different in sign) _{, and} the value of g _{d, i} with respect to _i is expressed as “extreme value p) ". Thereafter, it is confirmed that the value of Δg _{d, i} approaches the value of Δg _{d, n} (| g _{d, n} −g _{d, i} |). (The value of i at this time is assumed to be U.) Here, if g _{d, n} ≈9.8 (m / s ² ) or g _{d, n} ≈−9.8 (m / s ² ), Since 4.9 ≦ | p−g _{d, n} | (X1) ≦ 14.7 (m / s ² ), it can correspond to a “predetermined time change pattern”. Further, _{g d,} if ^{_{n ≒ 0 (m / s 2}} ), 8.5 ≦ | "predetermined by (X2) that is ^{≦ 9.8 (m / s 2)} | p-g d, n It is possible to correspond to the “time change pattern”. Here, since the gravitational acceleration exists only in one direction, one of g1 _{, n} and g2 _{, n} is close to 0, and the other is close to 9.8 or -9.8.

(Procedure 2) At the same time when the user performs an operation for operating the imaging device 2, the control device 6 receives three sensor values from the sensor 5, and pre-stores the reference pattern f _i (i = 1 to M). I ask for it. An integrated value S = Σ _{d = 1 to 3} (Σ _{i = 1 to M} | g _{d, n + i−1−} f _d) of the absolute value of the difference between the sensor value g _{d, i} and the reference pattern f _{d, i} for each reception. _{, I} |). If the integrated value S is equal to or less than a predetermined threshold value, it can correspond to a “predetermined time change pattern”.

Having described Step 1 and Step 2, (1) increased by -9.8m / ^{s 2} from X1, the flow returns to -9.8m / ^{s 2} within time T changes, (2) 0 m / ^{s 2} Increase or decrease from time to time by X2, and return to 0 m / s ² within time T, (3) decrease from 9.8 m / s ² by X1 and return to 9.8 m / s ² within time T, It is only necessary to be able to determine whether or not the deviation from the reference pattern is small, and there are innumerable procedures for such determination. Either procedure may be used.

  FIG. 6 shows a time change pattern of sensor values output by the acceleration sensor in the operation of shaking the main body 1 shown in FIG. In the shaking operation, two sensor values of the three sensor values, that is, the sensor values for the X axis and the Y axis defined in FIG. 3 change with time, and the remaining one sensor value, that is, the definition in FIG. The sensor value regarding the Z axis to be performed is substantially constant with respect to time. That is, the Z-axis maintains a direction substantially perpendicular to the direction of gravity acceleration, and the XY plane rotates including the direction of gravity acceleration. In this case, the control device 6 can control the operation of the imaging device 2 by determining the temporal change pattern of the sensor values regarding the X axis and the Y axis.

  FIG. 7 shows a time change pattern of sensor values output from the acceleration sensor in the operation of tilting the main body 1 shown in FIG. In the tilting operation, two of the three sensor values, that is, the sensor values for the X axis and the Z axis defined in FIG. 5 change with time, and the remaining one sensor value, that is, the definition in FIG. The sensor value with respect to the Y axis is substantially constant with respect to time. The Y axis maintains a direction substantially perpendicular to the direction of gravity acceleration, and the XZ plane rotates including the direction of gravity acceleration. In this case, the control device 6 can control the operation of the imaging device 2 by determining the temporal change pattern of the sensor values regarding the X axis and the Z axis.

As described above, both the shaking operation and the tilting operation change two sensor values with respect to time, and the remaining one sensor value is substantially constant with respect to time. Since it rotates around the axis (Y axis, Z axis) orthogonal to the direction of gravity acceleration (-X direction before the start of operation), gravity acceleration does not change in the direction of the rotation axis, and the other two axes Gravity acceleration changes in the direction. In addition, the long diameter of the smartphone moves about 20 cm over at least about 1 second, and the acceleration required for the movement is 1 m / s ² or less (strictly speaking, the sensor position on the smartphone and the rotation center) Depending on the coordinates of and possibly smaller), a change in gravitational acceleration is reliably detected. Note that the rotation around the gravity acceleration direction axis (X-axis) does not change the direction of the smartphone relative to the gravitational acceleration, and all three sensor values do not change. I can't.

  Hereinafter, the case where the sensor 5 is an angular velocity sensor will be described.

The control device 6, at least one value is determined whether more than a threshold value, time variation patterns that point t _n when the threshold is exceeded for one of the absolute values of the three sensor values g _{d, n} for each reception The starting point of Hereinafter, for the sake of explanation, it is assumed that the value of d = 2 exceeds the threshold value, but the same applies to d = 1 and d = 3.

Thereafter, the control device 6 determines whether the value of g _{2, i} (i = n to N) corresponds to a “predetermined time change pattern” or g _{1, i} and g _{3, i} (i = n to N). ) To determine whether the values of g _{1, i} and g _{3, i} are substantially constant, the difference between the values of g _{1, n} and g _{3, n} (| g _{1, n} −g _{1, If i} |, | g3 _{, n−} g3 _{, i} |) are equal to or less than a certain value regardless of i, it can be “substantially constant”.

  The following procedure can be considered as to whether or not d = 2 corresponds to the “predetermined time change pattern”.

(Procedure 1) The time point t _{i at} which the value of g _{2, i} exceeding the threshold value returns close to 0 (| g _{2, i} | is smaller than the predetermined threshold value) is determined, and the time point for i is set to u. t = n 'seek _{= Σ i = n~u (g 2} , i), the integrated value S' t = _{g 2, i} integrated value S of up to u from the, π / 3 ≦ | S ' | ≦ 2π If it is / 3 radians, it corresponds to the first half of the “predetermined time change pattern”. Thereafter, the value of u is treated as the value of n, and the point in time when the value of g2 _{, i} returns to close to 0 after exceeding the threshold value is defined as u ′, and similarly the integrated value S ′ ′ _{= Σi} = u to u _'Find (g _{2, i} ). If the integrated value S ′ ′ is π / 3 ≦ | S ′ ′ | ≦ 2π / 3 radians and the positive and negative of S ′ and S ′ ′ are opposite, it corresponds to the “predetermined time change pattern”. Then.

(Procedure 2) At the same time when the user performs an operation for operating the imaging device 2, the control device 6 receives three sensor values from the sensor 5 and preliminarily receives the reference signal f _i (i = 1 to M). I ask for it. The sum of absolute values of the difference between the sensor value g _{d, i} and the reference signal f _i for each reception S = Σ _{i = 1 to M} | g _{n + i−1} −f _i | If the integrated value S is equal to or less than a predetermined threshold value, it can correspond to a “predetermined time change pattern”.

  The procedure 1 and the procedure 2 have been described above. Π / 3 ≦ | S ′ | ≦ 2π / 3 radians, π / 3 ≦ | S ′ ′ | ≦ 2π / 3 radians, and S ′ and S ′ ′ It is only necessary to be able to determine whether the sign is opposite or that the deviation from the reference pattern is small, and there are innumerable procedures for such determination. Either procedure may be used.

  FIG. 8 shows a time change pattern of the sensor value output from the angular velocity sensor in the operation of tilting the main body 1 shown in FIG. In the tilting operation, one of the three sensor values, that is, the sensor value related to the Y axis defined in FIG. 5 changes with time, and the remaining two sensor values, that is, the X axis defined in FIG. And the sensor value regarding the Z-axis becomes substantially constant with respect to time. The tilting operation is rotation about the Y axis, and an angular velocity about the Y axis is generated, and an angular velocity about the other axis is not generated. In this case, the control device 6 can control the operation of the imaging device 2 by determining the temporal change pattern of the sensor value with respect to the Y axis.

  In the case of a shaking operation, the rotation is about the Z axis, the sensor value related to the Z axis changes with time, and the sensor values related to the X axis and the Y axis become substantially constant with respect to time. Similarly, with respect to the rotation operation about the X axis, the sensor value related to the X axis changes with time, and the sensor value related to the Y axis and the Z axis becomes substantially constant with respect to time. The angular velocity sensor can detect a rotation operation around the X axis that cannot be detected by the acceleration sensor.

Regardless of whether the sensor 5 is an acceleration sensor or an angular velocity sensor, the operation starts from the start time (t _n ) to the end time (t _U when the sensor is an acceleration sensor, and t when the sensor is an angular velocity sensor. It is confirmed that the time until _{u ′} ) is 0.8 to 1.8 seconds. When the time from the start time to the end time of the operation is not 0.8 to 1.8 seconds, it does not correspond to the “predetermined time change pattern”.

  When comparing with the reference pattern (procedure 2), it is not necessary to confirm the time from the start time to the end time of the operation. If the time length of the reference pattern is 0.8 to 1.8 seconds, the integrated value S inevitably increases if the time from the start time to the end time of the operation is not 0.8 to 1.8 seconds. This is because it is considered.

(Judgment of imaging conditions such as still)
The control device 6 determines the stillness after the operation is finished. The determination of stillness may be made by continuing that the value of Δg _{d, i} is not more than a constant value for all of d = 1, 2, 3 over a predetermined time (for example, 0.2 seconds).

The control device 6 operates the imaging device 2 when the stationary state is determined only when the time from when the operation starts (t _n ) until the stationary state is determined is 1.5 to 2.5 seconds. And However, the imaging device 2 may be operated when the stillness is determined regardless of the time from the start of the operation until the stillness is determined.

  The control device 6 may compare the image at the start of the operation with the image when the stillness is determined, and may not operate the imaging device 2 when the difference between the two images is large. It is considered that the user positions the imaging frame (determines the position and direction of the imaging device) before starting the operation. If it is different from the positioned imaging frame, imaging is not performed and positioning displacement is prevented.

  For this reason, the control device 6 operates the imaging device 2 to acquire an image at the time when the operation is started and when stillness is determined.

  Various methods are known for comparing whether two images are similar, and any of them may be used. As an example, the following method enables high-speed processing.

Each pixel of the image data has a color value (for example, RGB value). Color values (often 24 bits) are clustered (considering close colors as the same color), eg 64 colors. For example, if only the upper 2 bits of each RGB value are viewed, there are 64 colors. Find a histogram of which of the 64 colors each pixel is throughout the image. That is, determine the total pixel appearance ratio p _c of the c-th color. Is _{Σ c = 1~64 p c = 1} .

The similarity R between two images is calculated as R _{= Σc = 1 to 64} (min (p _{1, c} , p _{2, c} )). min is the smaller of the two values. For the same image, R = 1. (Σ _{c = 1 to 64} (min (p _{1, c} , p _{2, c} )) = 1−Σ _{c = 1 to 64} (| p _{1, c} −p _{2, c} |), and the images are different. (It can be seen that the value of R decreases as the value increases.)

  When such high-speed calculation is performed, a plurality of images are processed by combining the time point when stillness is determined and the time points before and after that, and the image with the largest R is captured (stored in the smartphone). It becomes possible to do so.

  As described in detail above, the smartphone 1 according to the present embodiment includes the imaging device 2 that captures an object to be captured, the position of the imaging device 2 in the three axial directions, or the sensor 5 that measures the amount related to the position, and the sensor. 5 includes a control device 6 that controls the operation of the imaging device 2 by using at least one of the three sensor values corresponding to each of the three axial directions output from 5. The sensor 5 measures the position in the three-axis direction of the imaging device 2 that images the object to be imaged or the amount related to the position, and the control device 6 3 corresponding to each of the three-axis directions output from the sensor 5. By controlling the operation of the imaging device 2 using at least one of the two sensor values, the user can control the operation of the imaging device 2 mounted on the smartphone by moving the smartphone body.

  It is a smartphone that operates an imaging device by moving a main body and realizes positioning of an imaging frame and prevention of a so-called “camera shake” state. The use by many individual users is conceivable, and the implementation of the present invention by a smartphone manufacturer is expected.

1 Smartphone (main unit)
2 Imaging device 3 Display device 4 Storage device 5 Sensor 6 Control device

Claims (8)

It is a smartphone that controls the operation of the imaging device mounted on it by moving the smartphone body,
An imaging device for imaging an object to be imaged;
A sensor for measuring speed, acceleration, angular velocity or angular acceleration in the three-axis directions of the imaging device;
A control device that controls the operation of the imaging device using at least one of three sensor values corresponding to each of the three axial directions output from the sensor;
With
The controller is
The three sensor values are periodically received from the sensor , and when the change amount of at least one of the three sensor values exceeds a threshold value, the time point at which the threshold value is exceeded is determined as a time change pattern. As a start time, it is determined whether the time change patterns of the three sensor values correspond to a predetermined time change pattern,
After the predetermined time change pattern, it is determined whether or not the three sensor values are substantially constant with respect to time, and when the three sensor values are substantially constant, the image of the imaging device at the start time and the start time from comparing the image of the imaging apparatus at the time of the lapse of T ', it is operated the imaging device when the difference between the images has the elapsed only if less than the threshold value,
The value of T ′ is 1.5 to 2.5 seconds,
The predetermined time change pattern is a time change pattern generated by rotating the smartphone in one direction about any one of the three axes and then rotating the smartphone in the opposite direction to return to the position before the rotation. A smartphone characterized by being.   The control device causes the user to rotate the smartphone in one direction about any one of the three axes, and then rotate the smartphone in the opposite direction to return to the position before the rotation. The smartphone according to claim 1, wherein the predetermined time change pattern is determined by receiving the three sensor values from.   The smartphone according to claim 2, wherein the control device calculates a difference between the at least one sensor value and the predetermined time change pattern, and compares the sum of the difference with a threshold value. The sensor is an acceleration sensor that measures acceleration in the three-axis directions of the imaging device,
When the two sensor values of the three sensor values change with respect to time and the remaining one sensor value is substantially constant with respect to time, the control device determines the time of the two sensor values. The smart phone according to claim 1, wherein a change pattern is determined. The time variation to a predetermined pattern, the acceleration value is a sensor value, (1) increased by -9.8m / ^{s 2} from X1, the flow returns to -9.8m / ^{s 2} within the time T changes (2) Increase or decrease from 0 m / s ² by X2 and change back to 0 m / s ² within time T, (3) Decrease from 9.8 m / s ² by X1 and 9.8 m within time T Any of the change patterns back to / s ² ,
The value of X1 is 4.9 to 14.7 m / s ² ,
The value of X2 is 8.5 to 9.8 m / s ² ,
The smartphone according to claim 4 , wherein the value of T is 0.8 to 1.8 seconds. The sensor is an angular velocity sensor that measures a rotation angle with respect to three axes of the imaging device,
When the sensor value of one of the three sensor values changes with respect to time and the remaining two sensor values are substantially constant with respect to time, the control device determines the time of the one sensor value. The smart phone according to claim 1, wherein a change pattern is determined. In the predetermined time change pattern, the value of the angular velocity that is the sensor value increases or decreases from 0 (1) and returns to 0, and the absolute value of the integral of the sensor value is Y radians, (2) A change pattern in which a decrease or increase from 0 returns to 0, and the change in which the absolute value of the integral of the sensor value is Y radians continues during time T;
The value of Y is π / 3 to 2π / 3 radians,
The smartphone according to claim 6 , wherein the value of T is 0.8 to 1.8 seconds. The smartphone according to claim 1, wherein the control device compares the difference between the two images based on a difference in a histogram of RGB luminance values of the two images.