WO2021152711A1 - Interval imaging device - Google Patents

Interval imaging device Download PDF

Info

Publication number
WO2021152711A1
WO2021152711A1 PCT/JP2020/003053 JP2020003053W WO2021152711A1 WO 2021152711 A1 WO2021152711 A1 WO 2021152711A1 JP 2020003053 W JP2020003053 W JP 2020003053W WO 2021152711 A1 WO2021152711 A1 WO 2021152711A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion
interval
imaging
image
shooting
Prior art date
Application number
PCT/JP2020/003053
Other languages
French (fr)
Japanese (ja)
Inventor
奥山　宣隆
奥　万寿男
吉澤　和彦
Original Assignee
マクセル株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by マクセル株式会社 filed Critical マクセル株式会社
Priority to JP2021573677A priority Critical patent/JP7343621B2/en
Priority to PCT/JP2020/003053 priority patent/WO2021152711A1/en
Publication of WO2021152711A1 publication Critical patent/WO2021152711A1/en
Priority to JP2023140669A priority patent/JP2023164903A/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present invention relates to an interval imaging device used as a life log camera or the like.
  • Patent Document 1 discloses an automatic imaging device that determines whether a state in which shooting processing is possible or a state in which shooting processing is not possible, and controls not to execute shooting when it is determined that shooting processing is not possible. There is.
  • Patent Document 1 when the image pickup device exists in a dark place where a sufficient amount of light is not satisfied, or when the image pickup lens is covered with a jacket or the like, automatic photographing is not performed. It is stated that it will be possible to take only images that are significant to the user.
  • Patent Document 1 In conventional techniques such as Patent Document 1, it is considered not to shoot an image meaningless to the user, but the relationship between the user's motion state and the shooting conditions described above is not particularly considered.
  • the present invention has been made in view of the above points, and an object of the present invention is to provide an interval imaging device that performs interval imaging under suitable imaging conditions according to a state of movement of a user.
  • the interval imaging device of the present invention has an imaging unit capable of shooting in a plurality of shooting modes, a motion sensor that detects the motion of the interval imaging device, and the presence or absence of motion based on the detection results of the motion sensor.
  • a motion determination unit for determination a timing generation unit that generates a trigger signal for imaging by the imaging unit at predetermined intervals, and a shooting control unit that controls the imaging unit and the timing generation unit according to the determination result of the motion determination unit.
  • the shooting control unit controls the imaging unit to switch between a shooting mode when there is movement (moving state) and a shooting mode when there is no movement (stationary state).
  • the imaging unit includes a standard mode for shooting at a standard image angle and a wide-angle mode for shooting at a wide angle
  • the photographing control unit shoots at the imaging unit in the standard mode in a moving state and in a wide-angle mode in a stationary state. Control to shoot with.
  • the imaging unit can acquire captured images in a plurality of formats
  • the imaging control unit acquires one of the captured images in the plurality of formats in the moving state and a plurality of captured images in the stationary state with respect to the imaging unit. Controls to acquire all captured images of the format.
  • an interval imaging device that efficiently acquires an image effective as a history image according to a state of movement of a user.
  • FIG. 5 is a wearing state diagram of the interval imaging device (omnidirectional camera 3) according to the second embodiment. Block diagram of omnidirectional camera 3. The external view of the interval imaging apparatus (HMD4) which concerns on Example 3.
  • FIG. 5 is a mounting state diagram of the interval imaging device 6 according to the fourth embodiment. The block diagram of the interval image pickup apparatus 6. A time chart showing an example of interval shooting. A flowchart showing an interval shooting operation.
  • the interval imaging device of the present invention is attached to the user and intermittently performs external photographing, and includes an imaging unit (camera) capable of photographing in a plurality of photographing modes.
  • an imaging unit camera
  • the configuration of the imaging unit will be specifically described in each case.
  • FIG. 1 is a diagram showing a state in which the user wears the interval imaging device 2 according to the first embodiment.
  • the interval imaging device 2 is attached to the head of the user 1 by the attachment belt 7.
  • the interval imaging device 2 is provided with a standard camera 11 and a wide-angle camera 12 for photographing the front of the user 1 as an imaging unit for externally photographing.
  • FIG. 2 is a block diagram of the interval imaging device 2.
  • the sensors include an acceleration sensor 13, a gyro sensor 14, a geomagnetic sensor 15, a position sensor 16, and an illuminance sensor 17.
  • the user setting unit 18, the communication unit 19, the CPU (or microprocessor) 20, the RAM 21, and the flash ROM 22 are provided, and each unit is connected by an internal bus 26.
  • the standard camera 11 and the wide-angle camera 12 are cameras that capture a standard angle-of-view image and a wide-angle image, respectively, and are used by switching according to the movement state of the user 1 as described later.
  • the image to be captured is for obtaining the user's action history, and may be a still image or a short moving image.
  • the captured image data is stored in the captured data holding unit 25 or in the external storage via the communication unit 19.
  • the acceleration sensor 13 and the gyro sensor 14 detect the movement and shaking of the interval image pickup device 2 (that is, the user 1 who wears the interval image pickup device 2).
  • the acceleration sensor 13 detects acceleration in the uniaxial, biaxial or triaxial directions
  • the gyro sensor 14 detects angular velocities in the uniaxial, biaxial or triaxial directions.
  • the geomagnetic sensor 15 acquires the orientation information of the interval imaging device 2, and the position sensor (for example, the GPS receiving unit) 16 acquires the position information, and handles these information as metadata of the captured image by attaching it to the image data.
  • the illuminance sensor 17 is used to detect the brightness of the surroundings and adjust the shooting conditions of the camera.
  • the user setting unit 18 is an operation input unit for the user to set the time interval (shooting interval) of interval shooting to a desired value.
  • Communication unit 19 includes all or part of communication functions such as mobile communication such as 4G, wireless LAN communication, and Bluetooth (registered trademark) communication.
  • the communication unit 19 can also transmit the image data taken by the standard camera 11 or the wide-angle camera 12 to the external storage via the network.
  • the CPU 20 expands the program 23 stored and stored in the flash ROM 22 into the RAM 21, and by executing this, controls each component of the interval imaging device 2 and realizes various functions.
  • the flash ROM 22 includes a program 23 and a shooting data holding unit 25, and the program 23 further includes a motion determination process 24a, a timing generation process 24b, and a shooting control process 24c that constitute the interval shooting application 24.
  • the interval shooting application 24 is executed, corresponding functional blocks (motion determination unit 24a, timing generation unit 24b, shooting control unit 24c) are configured.
  • the motion determination unit 24a compares the detection values of the acceleration sensor 13 and the gyro sensor 14 with a threshold value set separately, and determines whether or not the interval imaging device 2 (that is, the user 1 wearing the interval imaging device 2) is moving. do. When the interval imaging device 2 (user 1) is moving at a constant speed, it is determined that there is movement (movement state). The current velocity can be calculated by integrating the acceleration over time.
  • the timing generation unit 24b generates a trigger signal indicating the shooting timing at the set shooting interval.
  • the shooting control unit 24c switches the shooting mode, that is, switches between the standard camera 11 and the wide-angle camera 12, and controls the timing generation unit 24b to generate a trigger signal based on the determination result of the motion determination unit 24a.
  • the CPU 20, RAM 21, and flash ROM 22 may be mounted on one integrated circuit.
  • FIG. 3 is a diagram showing an example of a standard angle-of-view image and a wide-angle image.
  • a standard mode and a wide-angle mode are provided as shooting modes, and these modes are appropriately switched.
  • the standard angle-of-view image 81 is photographed by the standard camera 11, and in the wide-angle mode, the wide-angle image 82 is photographed by the wide-angle camera 12.
  • the standard angle-of-view image 81 is, for example, an image including the periphery of the person 83 in front, but the wide-angle image 82 can capture a wide range of backgrounds, which is effective for grasping the background situation in more detail. ..
  • 4A and 4B are time charts showing an example of interval shooting. From the top of the drawing, the time transitions of the motion determination result, shooting timing, and shooting mode are shown.
  • the motion determination result is a determination of the motion state of the interval imaging device 2 by the motion determination unit 24a.
  • the state with movement and its period will be referred to as “exercise state” and “exercise period”
  • the state without movement and its period will be referred to as “stationary state” and “stationary period”.
  • the "exercise state” is described as “exercise”
  • the "stationary state” is described as “stationary”.
  • the shooting timing signals T1 to T11 are generated by the timing generation unit 24b, and based on this, a trigger signal to the camera is generated.
  • the shooting mode has a standard mode and a wide-angle mode, and is switched by the shooting control unit 24c.
  • the shooting timing interval ⁇ T is constant except for the intervals [T2, T3] and [T8, T9], and the user's action history is obtained in seconds, minutes, or hours via the user setting unit 18.
  • the shooting interval ⁇ T is set to about several minutes.
  • T3 and T9 where the motion state changes from the motion state to the stationary state, the interval of the shooting timing up to that point is reset and a new shooting timing is set. That is, T3 and T9 become new shooting timings, and thereafter, the intervals ⁇ T are set with reference to these timings.
  • the image is taken in the standard mode using only the standard camera 11.
  • the standard camera 11 may have a smaller number of pixels or the like than the wide-angle camera 12.
  • the image is taken in the wide-angle mode using the wide-angle camera 12.
  • shooting may be performed using the standard camera 11 at the same time.
  • FIG. 4B shows an increase in the shooting timing during the rest period as compared with FIG. 4A.
  • the shooting timings of T3a and T3b are added, and in the period of [T9, T10], the shooting timings of T9a and T9b are added, and shooting is performed in the wide-angle mode.
  • the shooting interval ⁇ T'in the stationary period is set to about several tens of seconds. This makes it possible to observe the surrounding situation in more detail when there are moving or changing objects in the surroundings.
  • FIG. 5 is a flowchart showing the interval shooting operation.
  • the shooting operation proceeds by the interval shooting application 24 (movement determination unit 24a, timing generation unit 24b, shooting control unit 24c).
  • the timer value t As parameters, the timer value t, the movement flag F, the shooting interval ⁇ T during the exercise period (first shooting interval), and the shooting interval ⁇ T'(second shooting interval) during the stationary period are used.
  • S103 Acquires the outputs of the acceleration sensor 13 and the gyro sensor 14.
  • S104 The motion determination unit 24a determines the motion state.
  • S105 Proceed to S106 in the case of a moving state, and proceed to S109 in the case of a stationary state.
  • S107 The timer value t and the first shooting interval ⁇ T are compared. If t ⁇ ⁇ T, the process proceeds to S108, otherwise the process proceeds to S115.
  • S108 Shoot in the standard mode, that is, with the standard camera 11. As a result, in the moving state, shooting in the standard mode is performed at the first shooting interval ⁇ T.
  • S113 The captured image is saved in the captured data holding unit 25, or transmitted from the communication unit 19 to the external storage and saved.
  • S114: The timer value t is reset (t 0).
  • an image effective for observing the background can be acquired by the wide-angle camera 12 when stationary, and more abundant image information can be acquired by narrowing the shooting interval.
  • shooting since shooting is performed immediately after the transition from the moving state to the stationary state, the shooting in the stationary state is not missed even if the stationary period is short.
  • the capacity of the shooting data holding unit 25 and the power consumption of the device can be saved, and an efficient interval imaging device can be realized.
  • the switching of the shooting mode is to switch between shooting with the standard camera 11 and shooting with the wide-angle camera 12 according to the state of movement, but once both cameras 11 and 12 After shooting with, only the shot image corresponding to the motion state may be selected and saved in the shooting data holding unit 25. This also applies to the following examples.
  • FIG. 6 is a diagram showing a state in which the user wears the interval imaging device (omnidirectional camera 3) according to the second embodiment.
  • the user 1 attaches the omnidirectional camera 3 to the head using a mounting jig 8 such as a helmet.
  • the omnidirectional camera 3 has a first half celestial sphere optical system 31 for photographing the front of the user 1 and a second half celestial sphere optical system 32 for photographing the rear of the user 1.
  • FIG. 7 shows a block diagram of the omnidirectional camera 3.
  • the same components as those of the interval imaging device 2 of the first embodiment (FIG. 2) are designated by the same reference numerals, and redundant description will be omitted.
  • the first half celestial sphere optical system 31 is composed of a fisheye lens or the like and captures the first half celestial sphere of the user 1.
  • the second half celestial sphere optical system 32 is also composed of a fisheye lens or the like and captures the second half celestial sphere of the user 1.
  • the captured images of the first half celestial sphere and the second half celestial sphere are projected onto the image sensor (imaging element) 33 and taken out as image data. At that time, by selecting the detection region of the image sensor, it is possible to separately acquire the captured image of the first half celestial sphere and the captured image of the second half celestial sphere.
  • Interval shooting using the omnidirectional camera 3 is performed in the same manner as the shooting timing and shooting mode shown in FIGS. 4A and 4B of the first embodiment. Further, the flowchart of the shooting operation is performed in the same manner as in FIG. However, regarding the shooting mode, in the standard mode, the first half celestial sphere image using the first half celestial sphere optical system 31 is acquired. Further, in the wide-angle mode, a spherical image using both the first half celestial sphere optical system 31 and the second half celestial sphere optical system 32 is acquired.
  • the second embodiment it is possible to acquire an image effective for background observation by a spherical image when stationary, and to obtain a historical image at a predetermined shooting interval even when exercising.
  • Example 3 describes a case where a head-mounted display (HMD) is used as an imaging unit for interval shooting.
  • HMD head-mounted display
  • FIG. 8 is an external view of the interval imaging device (head-mounted display (HMD) 4) according to the third embodiment.
  • the configuration includes a left visual line camera 41, a right visual line camera 42, left and right projection optical systems 44a and 44b, a display optical system 45 such as a lens and a screen, a speaker 46, a microphone 47, a frame housing 48a to 48c, and a nose pad 49.
  • It has a controller 50 and the like.
  • the controller 50, the left line-of-sight camera 41, the right line-of-sight camera 42, the speaker 46, and the microphone 47 are arranged in the frame housings 48a to 48c.
  • the arrangement location does not have to be as shown in FIG.
  • a group of sensors such as an acceleration sensor, a gyro sensor, and a position sensor are built in.
  • the left line-of-sight camera 41 and the right line-of-sight camera 42 are three-dimensional cameras capable of photographing the front of the user's line of sight and measuring the distance to an object in the real space captured by the photographed image by using the difference between the left eye and the right eye. It is configured. By using the measured distance as depth information of the captured image, it is useful for understanding the captured image.
  • the controller 50 takes in the image of the real space taken by the left visual line camera 41 and the right visual line camera 42, the distance data to the object in the real space, and the like into the internal memory and the CPU. Further, the controller 50 creates images projected by the projection optical systems 44a and 44b by computer graphics or the like, and also creates audio output from the speaker 46.
  • the projection optical systems 44a and 44b and the display optical system 45 serve as a display unit of the HMD4.
  • the projection optical systems 44a and 44b divide the image of the virtual object created by the controller 50 into a left-eye image and a right-eye image, and project and display the image on the display optical system 45.
  • the user 1 sees a landscape or a real object in front of the display optical system 45 through the transmissive display optical system 45, and superimposes and visually recognizes an image of a virtual object projected from the projection optical systems 44a and 44b on the display optical system 45. be able to.
  • FIG. 9 shows a block diagram of the head-mounted display (HMD) 4.
  • HMD head-mounted display
  • the same components as those in FIG. 8 are designated by the same reference numerals, and the other elements include a sensor group 43, a distance calculation unit 51, a communication unit 52, a CPU 53, a RAM 54, an image RAM 55, and a flash ROM 56. Each part is connected by an internal bus 60.
  • the projection optical system 44 corresponds to the projection optical systems 44a and 44b of FIG. 8, and the left-eye image and the right-eye image are independently projected onto the display optical system 45.
  • a method may be used in which the left-eye image and the right-eye image interleaved by one projector are projected, and the left-eye image and the right-eye image are projected on the respective eyes by the shutter optical system.
  • an optical system using a holographic lens may be used.
  • the communication unit 52 has a plurality of communication functions such as mobile communication, wireless LAN and Bluetooth (registered trademark), and connects the HMD4 to an external storage or the like via a network.
  • the CPU 53 expands the program stored in the flash ROM 56 into the RAM 54 and executes the program to control the operation of each component of the HMD 4.
  • the image RAM 55 stores video data to be transmitted to the projection optical system 44.
  • the flash ROM 56 includes a basic operation program 57, a processing program of the interval shooting application 58, and a shooting data holding unit 59.
  • the basic operation program 57 performs the projection operation process as the HMD 4, and the interval shooting application 58 performs the motion determination process 24a, the timing generation process 24b, and the shooting control process 24c described in the first embodiment (FIG. 2). Further, the shooting data holding unit 59 stores the image data shot by the camera.
  • the interval shooting using the HMD4 in this embodiment is performed in the same manner as the shooting timing and shooting mode shown in FIGS. 4A and 4B of the first embodiment. Further, the flowchart of the shooting operation is performed in the same manner as in FIG. However, regarding the shooting mode, in the standard mode, the left visual line camera 41 or the right visual line camera 42 is used for shooting. Further, in the wide-angle mode, three-dimensional photography is performed using both the left visual line camera 41 and the right visual line camera 42.
  • the third embodiment it is possible to acquire an image effective for background observation by a three-dimensional image when stationary, and to obtain a historical image at a predetermined shooting interval even when exercising. In addition, it is possible to reduce the amount of captured data and power consumption of the historical image during exercise.
  • the optical axes of the lenses of the left visual line camera 41 and the right visual line camera 42 can be substantially aligned with the user's line of sight, so that the landscape actually seen by the user is a three-dimensional photograph. Can be saved as. Further, by operating the projection optical system 44 and the display optical system 45, there is an advantage that the captured image can be viewed retroactively in time without interrupting the interval shooting.
  • Example 4 has a configuration in which the motion determination process in Example 1 is determined at a plurality of levels. Specifically, the movement of the user's head and the movement of the body are detected, and the movements are compared to control shooting.
  • FIG. 10 is a diagram showing a state in which the user wears the interval imaging device 6 according to the fourth embodiment.
  • the main body portion 6a of the interval imaging device 6 (hereinafter referred to as the device main body portion) is attached to the head of the user 1 by the attachment belt 7.
  • the apparatus main body 6a is provided with a standard camera 11 and a wide-angle camera 12.
  • the sensor terminal 6b is attached to the body (for example, the chest) of the user 1.
  • the sensor terminal 6b detects the movement of the body of the user 1.
  • motion determination is performed using the motion data of the user's head and the motion data of the user's torso.
  • first movement data the movement of the user's head
  • second movement data the movement of the user's torso
  • FIG. 11 shows a block diagram of the interval imaging device 6.
  • the configuration of the device main body 6a is the same as that of the first embodiment (FIG. 2), and the acceleration sensor 13 and the gyro sensor 14 are “first motion sensors” that detect the movement of the user's head.
  • the interval shooting application 24 includes a motion determination process 24a, a timing generation process 24b, and a shooting control process 24c. In this figure, some components of the device main body 6a are omitted.
  • the configuration of the sensor terminal 6b includes an acceleration sensor 61, a gyro sensor 62, a communication unit 63, a CPU (or microcomputer) 64, a RAM 65, and a flash ROM 66.
  • the acceleration sensor 61 and the gyro sensor 62 are "second movement sensors" that detect the movement of the user's body.
  • the communication unit 63 supports a low power communication protocol such as Bluetooth (registered trademark), and performs one-to-one communication with the communication unit 19 of the device main body 6a.
  • the operation program 67 stored in the flash ROM 66 is for performing a process of acquiring the motion data of the user's body by the second motion sensors 61 and 62 and transmitting the motion data to the apparatus main body 6a.
  • the sensor terminal 6b acquires the movement data of the user's body in conjunction with the shooting operation of the device main body 6a. Specifically, when a motion data request is received from the device body 6a, the detected values of the acceleration sensor 61 and the gyro sensor 62 are transmitted to the device body 6a as second motion data.
  • the motion determination unit 24a determines the motion state at a plurality of levels. In the judgment, the correlation between the movement data of both is examined together with the state of each movement.
  • both the first motion data and the second motion data are "with motion” and have the same direction and the same size (correlation), that is, the head and the body move in synchronization. If so, it is determined to be "exercise state 1", and shooting is performed in the standard mode (standard camera 11 only).
  • the first motion data and the second motion data are in different directions and different sizes (no correlation), that is, when the head and the torso are moving randomly out of synchronization, it is determined to be "exercise state 2". Then, the camera shooting is restricted (stopped).
  • both the first motion data and the second motion data are "no motion”
  • shooting is performed in the wide-angle mode (wide-angle camera 12).
  • FIG. 12 is a time chart showing an example of interval shooting.
  • the notation in the figure is the same as in the first embodiment (FIG. 4A).
  • the motion determination result determines the state of motion in three stages of "exercise 1", “exercise 2", and "stationary".
  • the shooting mode in addition to the standard mode and wide-angle mode, a mode for limiting shooting has been added.
  • the shooting timing interval ⁇ T is set in the same manner as in FIG. 4A, and in T23 and T30 where the motion determination transitions from the moving state to the stationary state, the shooting timing interval up to that point is reset and a new shooting timing is set. Will be done.
  • the motion judgment result is "exercise 1"
  • the image is taken in the standard mode, for example, T21 and T22.
  • the motion determination result is "exercise 2”
  • camera shooting is restricted, for example, T27 and T28.
  • shooting is performed in the wide-angle mode, for example, T23 and T24.
  • the shooting timing interval ⁇ T is set to be equal in the standard mode and the wide-angle mode, but as shown in FIG. 4B, it may be set to a different interval ⁇ T'in the wide-angle mode.
  • FIG. 13 is a flowchart showing the interval shooting operation.
  • the steps having the same contents as the flowchart shown in the first embodiment (FIG. 5) are given the same numbers.
  • the shooting timing interval is ⁇ T in the standard mode and ⁇ T'in the wide-angle mode.
  • the steps different from the flowchart of FIG. 5 are as follows.
  • S103a Acquires the output of the first motion sensor (accelerometer 13, gyro sensor 14) of the device main body 6a.
  • S103b Acquires the output of the second motion sensor (accelerometer 61, gyro sensor 62) of the sensor terminal 6b.
  • the motion determination unit 24a compares the first motion data with the second motion data, and performs three-step motion determination. If the first and second motion data are "with motion” and have a correlation, it is regarded as "exercise state 1". If the first and second motion data are "with motion” and there is no correlation, it is set as “exercise state 2". When both the first and second motion data are "no motion”, it is regarded as "stationary state".
  • the motion is determined by using the motion sensors attached to the user's head and the body, and if there is no correlation between the motion data even if both are in the motion state, the camera I tried to limit the shooting.
  • the camera I tried to limit the shooting.
  • the motion state may be determined at a plurality of levels according to the size of the motion data by using only the first motion sensors 13 and 14 of the apparatus main body 6a. That is, it is determined that a small movement or a large movement is performed, and in the case of a large movement exceeding a predetermined value, unnecessary shooting is reduced by limiting the camera shooting, and the same effect can be obtained.
  • the present invention is not limited to these, and a part of the configuration of one embodiment can be replaced with another embodiment. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. All of these belong to the category of the present invention, and the numerical values appearing in the text and figures are merely examples, and even if different ones are used, the effect of the present invention is not impaired.
  • the interval imaging device of the present invention is attached to the user to acquire the user's history image (life log)
  • the present invention is not limited to this, and the interval imaging device can be used as an animal, a vehicle, or the like. It can also be applied when it is attached to a moving body and an image of its action history is acquired.
  • the functions and the like of the invention may be implemented by hardware by designing a part or all of them by, for example, an integrated circuit. Further, it may be implemented by software by interpreting and executing an operation program by a microprocessor unit, a CPU, or the like. Further, the implementation range of the software is not limited, and the hardware and the software may be used together.
  • HMD Head mount display
  • 6a Device body
  • 6b Sensor terminal
  • 11 Standard camera
  • 12 Wide angle camera
  • 13 Acceleration sensor
  • 14,62 Gyro sensor
  • 19,63 Communication unit
  • 24,58 Interval shooting application
  • 24a Motion judgment processing
  • 24b Timing generation processing
  • 24c Shooting control processing
  • 25,59 Shooting Data holding unit
  • 31 first half celestial optical system
  • 32 second half celestial optical system
  • 33 imaging sensor
  • 42 right visual line camera.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Exposure Control For Cameras (AREA)
  • Cameras In General (AREA)

Abstract

The present invention provides an interval imaging device that captures images under suitable image-capture conditions in accordance with the state of a user's motion. An interval imaging device 2 comprises: imaging units 11, 12 capable of capturing an image in a plurality of image-capture modes; motion sensors 13, 14 for detecting the motion of the interval imaging device; a motion determination unit 24a for determining the presence of motion from the detection results of the motion sensors; a timing generation unit 24b for generating at prescribed intervals a trigger signal for the imaging units to capture an image; and an image-capture control unit 24c for controlling the imaging units and the timing generation unit in accordance with the determination result of the motion determination unit. The imaging units are provided, for example, with a standard mode for capturing an image at a standard angle of view and a wide-angle mode for capturing an image at a wide angle of view, and the image-capture control unit controls the imaging units so as to capture an image in the standard mode when there is a motion (moving state) and capture an image in the wide-angle mode when there is no motion (stationary state).

Description

インターバル撮像装置Interval imaging device
　本発明は、ライフログカメラなどとして用いられるインターバル撮像装置に関する。 The present invention relates to an interval imaging device used as a life log camera or the like.
　ユーザ等の移動体に装着して、カメラ撮影を間欠的に自動的に行い、移動体の行動履歴を得るインターバル撮像装置（ライフログカメラ）が知られている。この場合、撮影は自動的に行われるので、撮影に適さない状態であっても撮影が実行されるという問題がある。これに関し特許文献１では、撮影処理が可能な状態か不可能な状態かを判定し、撮影処理が不可能な状態と判定したときは、撮影を実行しないよう制御する自動撮像装置が開示されている。 There is known an interval imaging device (life log camera) that is attached to a moving body such as a user to automatically take pictures with a camera intermittently and obtain an action history of the moving body. In this case, since the shooting is performed automatically, there is a problem that the shooting is executed even in a state unsuitable for shooting. In this regard, Patent Document 1 discloses an automatic imaging device that determines whether a state in which shooting processing is possible or a state in which shooting processing is not possible, and controls not to execute shooting when it is determined that shooting processing is not possible. There is.
特開２００９－１４７６４７公報JP-A-2009-147647
　特許文献１によれば、例えば、撮像装置が十分な光量を満たさないような暗所に存在する場合や、撮像レンズが上着等で覆われた状態にある場合に、自動撮影が行われないようにし、利用者にとって有意な画像のみを撮影することが可能になると述べられている。 According to Patent Document 1, for example, when the image pickup device exists in a dark place where a sufficient amount of light is not satisfied, or when the image pickup lens is covered with a jacket or the like, automatic photographing is not performed. It is stated that it will be possible to take only images that are significant to the user.
　特許文献１を始め従来のインターバル撮像装置では、ユーザの動きの状態に関わらず撮影条件は一律に定められていた。しかしながら、ユーザの行動履歴を把握するという観点では、ユーザの動きの状態（静止状態であるか、運動状態であるか）に応じて、行動履歴に対する着目点が異なってくる。例えば、静止時にはユーザを取り巻く広い範囲の風景を収めた画像を取得するのが好ましく、運動時には目の前の狭い範囲の風景の変化の画像を取得するのが好ましい。また、限られた記憶部に撮影データを効率よく保存するために、静止時と運動時とで、行動履歴として撮影する画像の時間間隔（インターバル）の設定も重要になる。 In conventional interval imaging devices such as Patent Document 1, shooting conditions are uniformly defined regardless of the state of movement of the user. However, from the viewpoint of grasping the user's action history, the point of interest for the action history differs depending on the state of the user's movement (whether it is in a stationary state or a moving state). For example, it is preferable to acquire an image containing a wide range of landscape surrounding the user when stationary, and it is preferable to acquire an image of changes in the landscape in a narrow range in front of the user when exercising. Further, in order to efficiently store the photographed data in the limited storage unit, it is important to set the time interval (interval) of the image to be photographed as the action history between the stationary state and the exercised state.
　特許文献１を始め従来の技術では、ユーザにとって無意味な画像を撮影しないことは考慮されているが、上記したユーザの動きの状態と撮影条件との関係については、特に考慮されなかった。 In conventional techniques such as Patent Document 1, it is considered not to shoot an image meaningless to the user, but the relationship between the user's motion state and the shooting conditions described above is not particularly considered.
　本発明は上記の点を鑑みてなされたものであり、その目的は、ユーザの動きの状態に応じて好適な撮影条件でインターバル撮影を行うインターバル撮像装置を提供することにある。 The present invention has been made in view of the above points, and an object of the present invention is to provide an interval imaging device that performs interval imaging under suitable imaging conditions according to a state of movement of a user.
　上記課題を解決するために、本発明のインターバル撮像装置は、複数の撮影モードで撮影可能な撮像部と、インターバル撮像装置の動きを検出する動きセンサと、動きセンサの検出結果から動きの有無を判定する動き判定部と、撮像部が撮影するためのトリガー信号を所定の間隔で発生するタイミング生成部と、動き判定部の判定結果に応じて撮像部とタイミング生成部を制御する撮影制御部と、を備え、撮影制御部は撮像部に対し、動き有り（運動状態）のときの撮影モードと動き無し（静止状態）のときの撮影モードを切り替えるよう制御する。 In order to solve the above problems, the interval imaging device of the present invention has an imaging unit capable of shooting in a plurality of shooting modes, a motion sensor that detects the motion of the interval imaging device, and the presence or absence of motion based on the detection results of the motion sensor. A motion determination unit for determination, a timing generation unit that generates a trigger signal for imaging by the imaging unit at predetermined intervals, and a shooting control unit that controls the imaging unit and the timing generation unit according to the determination result of the motion determination unit. The shooting control unit controls the imaging unit to switch between a shooting mode when there is movement (moving state) and a shooting mode when there is no movement (stationary state).
　好ましくは、撮像部は、標準画角で撮影する標準モードと、広角で撮影する広角モードとを備え、撮影制御部は撮像部に対し、運動状態では標準モードで撮影し、静止状態では広角モードで撮影するよう制御する。 Preferably, the imaging unit includes a standard mode for shooting at a standard image angle and a wide-angle mode for shooting at a wide angle, and the photographing control unit shoots at the imaging unit in the standard mode in a moving state and in a wide-angle mode in a stationary state. Control to shoot with.
　あるいは、撮像部は、複数形式の撮影画像を取得することが可能であり、撮影制御部は撮像部に対し、運動状態では複数形式の撮影画像のうちの１つを取得し、静止状態では複数形式の撮影画像を全て取得するよう制御する。 Alternatively, the imaging unit can acquire captured images in a plurality of formats, and the imaging control unit acquires one of the captured images in the plurality of formats in the moving state and a plurality of captured images in the stationary state with respect to the imaging unit. Controls to acquire all captured images of the format.
　本発明によれば、ユーザの動きの状態に応じて履歴画像として有効な画像を効率良く取得するインターバル撮像装置を提供できる。 According to the present invention, it is possible to provide an interval imaging device that efficiently acquires an image effective as a history image according to a state of movement of a user.
実施例１に係るインターバル撮像装置２の装着状態図。The wearing state figure of the interval imaging apparatus 2 which concerns on Example 1. FIG. インターバル撮像装置２のブロック図。The block diagram of the interval image pickup apparatus 2. 標準画角画像と広角画像の例を示す図。The figure which shows the example of a standard angle-of-view image and a wide-angle image. インターバル撮影の例を示すタイムチャート。A time chart showing an example of interval shooting. インターバル撮影の例を示すタイムチャート。A time chart showing an example of interval shooting. インターバル撮影動作を示すフローチャート。A flowchart showing an interval shooting operation. 実施例２に係るインターバル撮像装置（全方位カメラ３）の装着状態図。FIG. 5 is a wearing state diagram of the interval imaging device (omnidirectional camera 3) according to the second embodiment. 全方位カメラ３のブロック図。Block diagram of omnidirectional camera 3. 実施例３に係るインターバル撮像装置（ＨＭＤ４）の外観図。The external view of the interval imaging apparatus (HMD4) which concerns on Example 3. FIG. ＨＭＤ４のブロック図。Block diagram of HMD4. 実施例４に係るインターバル撮像装置６の装着状態図。FIG. 5 is a mounting state diagram of the interval imaging device 6 according to the fourth embodiment. インターバル撮像装置６のブロック図。The block diagram of the interval image pickup apparatus 6. インターバル撮影の例を示すタイムチャート。A time chart showing an example of interval shooting. インターバル撮影動作を示すフローチャート。A flowchart showing an interval shooting operation.
　以下、図面を参照しながら、本発明の実施形態について説明する。本発明のインターバル撮像装置は、ユーザに装着され外部の撮影を間欠的に行うものであって、複数の撮影モードで撮影可能な撮像部（カメラ）を備えている。以下の実施例では、撮像部の構成を場合分けして具体的に説明する。 Hereinafter, embodiments of the present invention will be described with reference to the drawings. The interval imaging device of the present invention is attached to the user and intermittently performs external photographing, and includes an imaging unit (camera) capable of photographing in a plurality of photographing modes. In the following examples, the configuration of the imaging unit will be specifically described in each case.
　図１は、実施例１に係るインターバル撮像装置２をユーザが装着した状態を示す図である。ユーザ１の頭部には、インターバル撮像装置２が装着ベルト７にて装着されている。インターバル撮像装置２には、外部の撮影を行う撮像部として、ユーザ１の前方を撮影する標準カメラ１１と広角カメラ１２とが備えられている。 FIG. 1 is a diagram showing a state in which the user wears the interval imaging device 2 according to the first embodiment. The interval imaging device 2 is attached to the head of the user 1 by the attachment belt 7. The interval imaging device 2 is provided with a standard camera 11 and a wide-angle camera 12 for photographing the front of the user 1 as an imaging unit for externally photographing.
　図２は、インターバル撮像装置２のブロック図である。前記した標準カメラ１１と広角カメラ１２の他、センサ類として、加速度センサ１３、ジャイロセンサ１４、地磁気センサ１５、位置センサ１６、照度センサ１７、を有する。また、ユーザ設定部１８、通信部１９、ＣＰＵ（もしくはマイクロコンピュータ）２０、ＲＡＭ２１、フラッシュＲＯＭ２２を備え、各部は内部バス２６で接続されている。 FIG. 2 is a block diagram of the interval imaging device 2. In addition to the standard camera 11 and the wide-angle camera 12 described above, the sensors include an acceleration sensor 13, a gyro sensor 14, a geomagnetic sensor 15, a position sensor 16, and an illuminance sensor 17. Further, the user setting unit 18, the communication unit 19, the CPU (or microprocessor) 20, the RAM 21, and the flash ROM 22 are provided, and each unit is connected by an internal bus 26.
　標準カメラ１１と広角カメラ１２は、それぞれと標準画角画像と広角画像を撮影するカメラであり、後述するようにユーザ１の動きの状態に応じて切り替えて使用する。なお撮影する画像はユーザの行動履歴を得るためのものであり、静止画であっても、また短い動画であってもよい。撮影された画像データは、撮影データ保持部２５に、あるいは通信部１９を介して外部のストレージに保存される。 The standard camera 11 and the wide-angle camera 12 are cameras that capture a standard angle-of-view image and a wide-angle image, respectively, and are used by switching according to the movement state of the user 1 as described later. The image to be captured is for obtaining the user's action history, and may be a still image or a short moving image. The captured image data is stored in the captured data holding unit 25 or in the external storage via the communication unit 19.
　加速度センサ１３とジャイロセンサ１４では、インターバル撮像装置２（すなわちこれを装着したユーザ１）の動きや揺れ等を検出する。加速度センサ１３は、一軸、二軸あるいは三軸方向の加速度を検出し、ジャイロセンサ１４は、一軸、二軸あるいは三軸方向の角速度を検出する。 The acceleration sensor 13 and the gyro sensor 14 detect the movement and shaking of the interval image pickup device 2 (that is, the user 1 who wears the interval image pickup device 2). The acceleration sensor 13 detects acceleration in the uniaxial, biaxial or triaxial directions, and the gyro sensor 14 detects angular velocities in the uniaxial, biaxial or triaxial directions.
　地磁気センサ１５ではインターバル撮像装置２の方位情報を、位置センサ（例えばＧＰＳ受信部）１６では位置情報を取得し、これらの情報を撮影画像のメタデータとして、画像データに添付して取り扱う。照度センサ１７は周囲の明るさを検知して、カメラの撮影条件を調整するために用いられる。 The geomagnetic sensor 15 acquires the orientation information of the interval imaging device 2, and the position sensor (for example, the GPS receiving unit) 16 acquires the position information, and handles these information as metadata of the captured image by attaching it to the image data. The illuminance sensor 17 is used to detect the brightness of the surroundings and adjust the shooting conditions of the camera.
　ユーザ設定部１８は、インターバル撮影の時間間隔（撮影間隔）等をユーザが所望値に設定するための操作入力部である。 The user setting unit 18 is an operation input unit for the user to set the time interval (shooting interval) of interval shooting to a desired value.
　通信部１９は、４Ｇなどのモバイル通信、ワイアレスＬＡＮ通信、Ｂｌｕｅｔｏｏｔｈ（登録商標）通信などの全て、もしくは一部の通信機能を含む。通信部１９は、標準カメラ１１や広角カメラ１２で撮影した画像データを、ネットワークを介して外部のストレージに送信することも可能である。 Communication unit 19 includes all or part of communication functions such as mobile communication such as 4G, wireless LAN communication, and Bluetooth (registered trademark) communication. The communication unit 19 can also transmit the image data taken by the standard camera 11 or the wide-angle camera 12 to the external storage via the network.
　ＣＰＵ２０は、フラッシュＲＯＭ２２に記憶格納されているプログラム２３をＲＡＭ２１に展開し、これを実行することによって、インターバル撮像装置２の各構成部の制御を行い、各種の機能を実現する。 The CPU 20 expands the program 23 stored and stored in the flash ROM 22 into the RAM 21, and by executing this, controls each component of the interval imaging device 2 and realizes various functions.
　フラッシュＲＯＭ２２は、プログラム２３と撮影データ保持部２５を含み、さらにプログラム２３は、インターバル撮影アプリ２４を構成する動き判定処理２４ａ、タイミング生成処理２４ｂ、撮影制御処理２４ｃを含む。インターバル撮影アプリ２４が実行されることで、それぞれ対応する機能ブロック（動き判定部２４ａ、タイミング生成部２４ｂ、撮影制御部２４ｃ）が構成される。 The flash ROM 22 includes a program 23 and a shooting data holding unit 25, and the program 23 further includes a motion determination process 24a, a timing generation process 24b, and a shooting control process 24c that constitute the interval shooting application 24. When the interval shooting application 24 is executed, corresponding functional blocks (motion determination unit 24a, timing generation unit 24b, shooting control unit 24c) are configured.
　動き判定部２４ａは、加速度センサ１３とジャイロセンサ１４の検出値を、別途設定している閾値と比較して、インターバル撮像装置２（すなわちこれを装着しているユーザ１）の動きの有無を判定する。なお、インターバル撮像装置２（ユーザ１）が一定速度で移動している場合は、動き有（運動状態）と判定する。なお、現在の速度は、加速度を時間積分することで算出することができる。 The motion determination unit 24a compares the detection values of the acceleration sensor 13 and the gyro sensor 14 with a threshold value set separately, and determines whether or not the interval imaging device 2 (that is, the user 1 wearing the interval imaging device 2) is moving. do. When the interval imaging device 2 (user 1) is moving at a constant speed, it is determined that there is movement (movement state). The current velocity can be calculated by integrating the acceleration over time.
　タイミング生成部２４ｂは、設定された撮影間隔で撮影のタイミングを示すトリガー信号を生成する。撮影制御部２４ｃは、動き判定部２４ａの判定結果に基づき、撮影モードの切替、すなわち標準カメラ１１と広角カメラ１２との切替を行うとともに、タイミング生成部２４ｂに対しトリガー信号の生成について制御する。
  なお、ＣＰＵ２０、ＲＡＭ２１、フラッシュＲＯＭ２２は、１つの集積回路に実装されてもよい。 The timing generation unit 24b generates a trigger signal indicating the shooting timing at the set shooting interval. The shooting control unit 24c switches the shooting mode, that is, switches between the standard camera 11 and the wide-angle camera 12, and controls the timing generation unit 24b to generate a trigger signal based on the determination result of the motion determination unit 24a.
The CPU 20, RAM 21, and flash ROM 22 may be mounted on one integrated circuit.
　図３は、標準画角画像と広角画像の例を示す図である。本実施例では、撮影モードとして標準モードと広角モードを備え、これを適宜切り替える。標準モードでは、標準カメラ１１で標準画角画像８１を撮影し、広角モードでは、広角カメラ１２で広角画像８２を撮影する。標準画角画像８１は、例えば前方の人物８３の周囲を含む画像となるが、広角画像８２は、広い範囲の背景が撮影可能であり、背景の状況をより詳細に把握するために有効となる。 FIG. 3 is a diagram showing an example of a standard angle-of-view image and a wide-angle image. In this embodiment, a standard mode and a wide-angle mode are provided as shooting modes, and these modes are appropriately switched. In the standard mode, the standard angle-of-view image 81 is photographed by the standard camera 11, and in the wide-angle mode, the wide-angle image 82 is photographed by the wide-angle camera 12. The standard angle-of-view image 81 is, for example, an image including the periphery of the person 83 in front, but the wide-angle image 82 can capture a wide range of backgrounds, which is effective for grasping the background situation in more detail. ..
　図４Ａと図４Ｂは、インターバル撮影の例を示すタイムチャートである。図面の上から、動き判定結果、撮影タイミング、撮影モードについて、それぞれの時間推移を示している。 4A and 4B are time charts showing an example of interval shooting. From the top of the drawing, the time transitions of the motion determination result, shooting timing, and shooting mode are shown.
　動き判定結果は、動き判定部２４ａによりインターバル撮像装置２の動きの状態を判定したものである。以下では、動き有の状態とその期間を「運動状態」「運動期間」、動き無の状態とその期間を「静止状態」「静止期間」と呼ぶことにする。この図では、「運動状態」を「運動」、「静止状態」を「静止」と記述している。 The motion determination result is a determination of the motion state of the interval imaging device 2 by the motion determination unit 24a. In the following, the state with movement and its period will be referred to as "exercise state" and "exercise period", and the state without movement and its period will be referred to as "stationary state" and "stationary period". In this figure, the "exercise state" is described as "exercise", and the "stationary state" is described as "stationary".
　撮影タイミング信号Ｔ１～Ｔ１１は、タイミング生成部２４ｂにより生成され、これに基づきカメラへのトリガー信号を発生する。撮影モードは、標準モードと広角モードがあり、撮影制御部２４ｃにより切り替えられる。 The shooting timing signals T1 to T11 are generated by the timing generation unit 24b, and based on this, a trigger signal to the camera is generated. The shooting mode has a standard mode and a wide-angle mode, and is switched by the shooting control unit 24c.
　図４Ａから説明する。撮影タイミングの間隔ΔＴは、区間［Ｔ２，Ｔ３］、［Ｔ８，Ｔ９］を除いて一定であり、ユーザ設定部１８を介して、秒、分あるいは時間単位で、ユーザの行動履歴を得るのに適した値に設定される。例えば、撮影間隔ΔＴを数分程度に設定する。 The explanation will be given from FIG. 4A. The shooting timing interval ΔT is constant except for the intervals [T2, T3] and [T8, T9], and the user's action history is obtained in seconds, minutes, or hours via the user setting unit 18. Set to a suitable value. For example, the shooting interval ΔT is set to about several minutes.
　動きの状態が運動状態から静止状態に遷移するＴ３とＴ９においては、それまでの撮影タイミングの間隔がリセットされ、新たな撮影タイミングが設定される。すなわち、Ｔ３とＴ９が新たな撮影タイミングとなり、以降、これを基準に間隔ΔＴで設定される。 In T3 and T9 where the motion state changes from the motion state to the stationary state, the interval of the shooting timing up to that point is reset and a new shooting timing is set. That is, T3 and T9 become new shooting timings, and thereafter, the intervals ΔT are set with reference to these timings.
　これにより、ユーザが静止状態へ遷移後直ぐに撮影が行なわれるので、たとえ静止状態の期間が撮影タイミングの間隔ΔＴより短い場合でも、カメラ撮影を逃すことがない。なお、Ｔ３とＴ９の静止状態への遷移直後では、ユーザの身体が不安定な状態の場合がある。よって、加速度センサ１３、ジャイロセンサ１４の詳細な値を解析し、安定した静止状態であることを確認した後に撮影を実施することにより、一層安定した撮影画像を得ることができる。 As a result, shooting is performed immediately after the user transitions to the stationary state, so even if the stationary state period is shorter than the shooting timing interval ΔT, the camera shooting is not missed. Immediately after the transition of T3 and T9 to the stationary state, the user's body may be in an unstable state. Therefore, a more stable captured image can be obtained by analyzing the detailed values of the acceleration sensor 13 and the gyro sensor 14 and performing imaging after confirming that the gyro sensor 14 is in a stable stationary state.
　次に、動き判定結果に応じて撮影モードを切り替える。動きの状態が運動状態であるタイミング（Ｔ１、Ｔ２、Ｔ５など）では、標準カメラ１１のみを用いた標準モードで撮影する。標準カメラ１１は、広角カメラ１２に比較して画素数等を少なくしてもよい。 Next, switch the shooting mode according to the motion judgment result. At the timing when the motion state is the motion state (T1, T2, T5, etc.), the image is taken in the standard mode using only the standard camera 11. The standard camera 11 may have a smaller number of pixels or the like than the wide-angle camera 12.
　一方、動きの状態が静止状態であるタイミング（Ｔ３、Ｔ４、Ｔ９）では、広角カメラ１２を使用する広角モードで撮影する。なお、広角モードでは、同時に標準カメラ１１を用いた撮影を行ってもよい。 On the other hand, at the timing when the motion state is stationary (T3, T4, T9), the image is taken in the wide-angle mode using the wide-angle camera 12. In the wide-angle mode, shooting may be performed using the standard camera 11 at the same time.
　図４Ｂは、図４Ａに対して、静止期間での撮影タイミングを増加させたものである。静止状態の［Ｔ３，Ｔ４］の期間では、Ｔ３ａ、Ｔ３ｂの撮影タイミングを追加し、［Ｔ９，Ｔ１０］の期間では、Ｔ９ａ，Ｔ９ｂの撮影タイミングを追加し、広角モードで撮影を行っている。例えば、静止期間での撮影間隔ΔＴ’を数１０秒程度に設定する。これにより、周囲に動く物体や変化する物体がある場合に、周囲の状況をより詳細に観察することができる。 FIG. 4B shows an increase in the shooting timing during the rest period as compared with FIG. 4A. In the stationary state [T3, T4], the shooting timings of T3a and T3b are added, and in the period of [T9, T10], the shooting timings of T9a and T9b are added, and shooting is performed in the wide-angle mode. For example, the shooting interval ΔT'in the stationary period is set to about several tens of seconds. This makes it possible to observe the surrounding situation in more detail when there are moving or changing objects in the surroundings.
　図５は、インターバル撮影動作を示すフローチャートである。撮影動作は、インターバル撮影アプリ２４（動き判定部２４ａ、タイミング生成部２４ｂ、撮影制御部２４ｃ）によって進行する。パラメータとして、タイマー値ｔ、動きフラグＦ、運動期間の撮影間隔ΔＴ（第１撮影間隔）、静止期間の撮影間隔ΔＴ’（第２撮影間隔）を用いる。動きフラグＦは、運動状態ではＦ＝１、静止状態ではＦ＝０とする。 FIG. 5 is a flowchart showing the interval shooting operation. The shooting operation proceeds by the interval shooting application 24 (movement determination unit 24a, timing generation unit 24b, shooting control unit 24c). As parameters, the timer value t, the movement flag F, the shooting interval ΔT during the exercise period (first shooting interval), and the shooting interval ΔT'(second shooting interval) during the stationary period are used. The movement flag F is set to F = 1 in the moving state and F = 0 in the stationary state.
　Ｓ１０１：撮影を開始するとタイマー値ｔをリセットする（ｔ＝０）。
  Ｓ１０２：動きフラグＦの初期値をＦ＝１（運動状態）に設定する。
  Ｓ１０３：加速度センサ１３、ジャイロセンサ１４の出力を取得する。
  Ｓ１０４：動き判定部２４ａにより、動きの状態を判定する。 S101: When shooting is started, the timer value t is reset (t = 0).
S102: The initial value of the motion flag F is set to F = 1 (exercise state).
S103: Acquires the outputs of the acceleration sensor 13 and the gyro sensor 14.
S104: The motion determination unit 24a determines the motion state.
　Ｓ１０５：運動状態の場合はＳ１０６へ、静止状態の場合はＳ１０９へ進む。
  Ｓ１０６：現在の動きフラグＦが１以外（Ｆ＝０）の場合はＦ＝１に設定する。
  Ｓ１０７：タイマー値ｔと第１撮影間隔ΔＴを比較する。ｔ≧ΔＴならＳ１０８へ、そうでなければＳ１１５へ進む。
  Ｓ１０８：標準モードで、すなわち標準カメラ１１で撮影する。これにより運動状態においては、第１撮影間隔ΔＴで標準モードの撮影がなされる。 S105: Proceed to S106 in the case of a moving state, and proceed to S109 in the case of a stationary state.
S106: When the current movement flag F is other than 1 (F = 0), it is set to F = 1.
S107: The timer value t and the first shooting interval ΔT are compared. If t ≧ ΔT, the process proceeds to S108, otherwise the process proceeds to S115.
S108: Shoot in the standard mode, that is, with the standard camera 11. As a result, in the moving state, shooting in the standard mode is performed at the first shooting interval ΔT.
　Ｓ１０９：Ｓ１０５で静止状態の場合は、現在の動きフラグＦが１かどうかで分岐する。Ｆ＝１であればＳ１１０へ、それ以外（Ｆ＝０）の場合はＳ１１２へ進む。
  Ｓ１１０：動きフラグＦを１から０に書き換える。
  Ｓ１１１：広角モードで、すなわち広角カメラ１２で撮影する。これにより運動状態から静止状態に遷移したタイミングで、広角モードの撮影がなされる（図４ＢのタイミングＴ３，Ｔ９）。
  Ｓ１１２：静止状態（Ｆ＝０）が継続しているので、タイマー値ｔと第２撮影間隔ΔＴ’を比較する。ｔ≧ΔＴ’ならＳ１１１へ進み、静止状態において、第２撮影間隔ΔＴ’で広角モードの撮影がなされる。ｔ≧ΔＴ’でなければＳ１１５へ進む。 S109: In the case of the rest state in S105, branching is performed depending on whether or not the current movement flag F is 1. If F = 1, the process proceeds to S110, and if it is not (F = 0), the process proceeds to S112.
S110: The motion flag F is rewritten from 1 to 0.
S111: Shoot in the wide-angle mode, that is, with the wide-angle camera 12. As a result, shooting in the wide-angle mode is performed at the timing of transition from the moving state to the stationary state (timings T3 and T9 in FIG. 4B).
S112: Since the stationary state (F = 0) continues, the timer value t and the second shooting interval ΔT'are compared. If t ≧ ΔT', the process proceeds to S111, and in the stationary state, shooting in the wide-angle mode is performed at the second shooting interval ΔT'. If t ≧ ΔT', the process proceeds to S115.
　Ｓ１１３：撮影した画像を、撮影データ保持部２５に保存する、もしくは通信部１９から外部ストレージに送信して保存する。
  Ｓ１１４：タイマー値ｔをリセットする（ｔ＝０）。
  Ｓ１１５：タイマー値ｔをカウントアップする。
  Ｓ１１６：撮影を継続するかどうかを判定する。継続する場合はＳ１０３へ戻り、上記の処理を繰り返す。 S113: The captured image is saved in the captured data holding unit 25, or transmitted from the communication unit 19 to the external storage and saved.
S114: The timer value t is reset (t = 0).
S115: The timer value t is counted up.
S116: Determine whether to continue shooting. When continuing, the process returns to S103 and the above process is repeated.
　以上説明した実施例１の構成によれば、静止時には広角カメラ１２により背景の観察に有効な画像を取得できるとともに、撮影間隔を狭くすることで画像情報をより豊富に取得することができる。また、運動状態から静止状態へ遷移後直ちに撮影を行なうので、静止期間が短くても静止状態での撮影を撮り逃すことがない。一方、運動時には標準カメラ１１のみを用いて撮影するので、撮影データ保持部２５の容量と装置の消費電力を節約し、効率の良いインターバル撮像装置を実現できる。 According to the configuration of the first embodiment described above, an image effective for observing the background can be acquired by the wide-angle camera 12 when stationary, and more abundant image information can be acquired by narrowing the shooting interval. In addition, since shooting is performed immediately after the transition from the moving state to the stationary state, the shooting in the stationary state is not missed even if the stationary period is short. On the other hand, since only the standard camera 11 is used for shooting during exercise, the capacity of the shooting data holding unit 25 and the power consumption of the device can be saved, and an efficient interval imaging device can be realized.
　なお、上記の説明では、撮影モードの切替とは、動きの状態に応じて標準カメラ１１で撮影するか、あるいは広角カメラ１２で撮影するかを切り替えるものとしたが、一旦両方のカメラ１１，１２で撮影した後で、動きの状態に対応する撮影画像のみを選択して撮影データ保持部２５に保存するようにしても良い。このことは、以下の実施例においても同様である。 In the above description, the switching of the shooting mode is to switch between shooting with the standard camera 11 and shooting with the wide-angle camera 12 according to the state of movement, but once both cameras 11 and 12 After shooting with, only the shot image corresponding to the motion state may be selected and saved in the shooting data holding unit 25. This also applies to the following examples.
　実施例２では、インターバル撮影のための撮像部として、全方位カメラを用いる場合について説明する。 In the second embodiment, a case where an omnidirectional camera is used as an imaging unit for interval shooting will be described.
　図６は、実施例２に係るインターバル撮像装置（全方位カメラ３）をユーザが装着した状態を示す図である。ユーザ１は、ヘルメットなどの装着治具８を用いて、全方位カメラ３を頭部に装着する。全方位カメラ３は、ユーザ１の前方を撮影する前半天球光学系３１と、ユーザ１の後方を撮影する後半天球光学系３２を有する。 FIG. 6 is a diagram showing a state in which the user wears the interval imaging device (omnidirectional camera 3) according to the second embodiment. The user 1 attaches the omnidirectional camera 3 to the head using a mounting jig 8 such as a helmet. The omnidirectional camera 3 has a first half celestial sphere optical system 31 for photographing the front of the user 1 and a second half celestial sphere optical system 32 for photographing the rear of the user 1.
　図７は、全方位カメラ３のブロック図を示す。実施例１（図２）のインターバル撮像装置２と同じ構成要素については、同一の符号を付しており、重複する説明を省略する。 FIG. 7 shows a block diagram of the omnidirectional camera 3. The same components as those of the interval imaging device 2 of the first embodiment (FIG. 2) are designated by the same reference numerals, and redundant description will be omitted.
　全方位カメラ３において、前半天球光学系３１は魚眼レンズ等で構成され、ユーザ１の前半天球を捉え、同様に後半天球光学系３２も魚眼レンズ等で構成され、ユーザ１の後半天球を捉える。捉えられた前半天球と後半天球の画像は、撮像センサ（撮像素子）３３上に投影され、画像データとして取り出される。その際、撮像素子の検出領域を選択することで、前半天球の撮影画像と後半天球の撮影画像に分けて取得することができる。 In the omnidirectional camera 3, the first half celestial sphere optical system 31 is composed of a fisheye lens or the like and captures the first half celestial sphere of the user 1. Similarly, the second half celestial sphere optical system 32 is also composed of a fisheye lens or the like and captures the second half celestial sphere of the user 1. The captured images of the first half celestial sphere and the second half celestial sphere are projected onto the image sensor (imaging element) 33 and taken out as image data. At that time, by selecting the detection region of the image sensor, it is possible to separately acquire the captured image of the first half celestial sphere and the captured image of the second half celestial sphere.
　全方位カメラ３を用いたインターバル撮影は、実施例１の図４Ａ，４Ｂに示した撮影タイミングと撮影モードと同様に行う。また、撮影動作のフローチャートは図５と同様に行う。ただし、撮影モードに関しては、標準モードでは前半天球光学系３１を用いた前半天球画像を取得する。また広角モードでは、前半天球光学系３１と後半天球光学系３２の両方を用いた全天球画像を取得する。 Interval shooting using the omnidirectional camera 3 is performed in the same manner as the shooting timing and shooting mode shown in FIGS. 4A and 4B of the first embodiment. Further, the flowchart of the shooting operation is performed in the same manner as in FIG. However, regarding the shooting mode, in the standard mode, the first half celestial sphere image using the first half celestial sphere optical system 31 is acquired. Further, in the wide-angle mode, a spherical image using both the first half celestial sphere optical system 31 and the second half celestial sphere optical system 32 is acquired.
　以上の構成により、実施例２によれば、静止時には全天球画像による背景観察に有効な画像を取得できるとともに、運動時にも所定の撮影間隔で履歴画像を得ることができる。 With the above configuration, according to the second embodiment, it is possible to acquire an image effective for background observation by a spherical image when stationary, and to obtain a historical image at a predetermined shooting interval even when exercising.
　実施例３は、インターバル撮影のための撮像部として、ヘッドマウントディスプレイ（ＨＭＤ）を用いる場合について説明する。 Example 3 describes a case where a head-mounted display (HMD) is used as an imaging unit for interval shooting.
　図８は、実施例３に係るインターバル撮像装置（ヘッドマウントディスプレイ（ＨＭＤ）４）の外観図である。その構成は、左目視線カメラ４１、右目視線カメラ４２、左右の投影光学系４４ａ，４４ｂ、レンズやスクリーン等の表示光学系４５、スピーカ４６、マイク４７、フレーム筐体４８ａ～４８ｃ、ノーズパッド４９、コントローラ５０などを有する。コントローラ５０、左目視線カメラ４１、右目視線カメラ４２、スピーカ４６、マイク４７は、フレーム筐体４８ａ～４８ｃに配置される。なお、配置場所は図８の通りでなくてもよい。さらに、加速度センサ、ジャイロセンサ、位置センサ等のセンサ群を内蔵する。 FIG. 8 is an external view of the interval imaging device (head-mounted display (HMD) 4) according to the third embodiment. The configuration includes a left visual line camera 41, a right visual line camera 42, left and right projection optical systems 44a and 44b, a display optical system 45 such as a lens and a screen, a speaker 46, a microphone 47, a frame housing 48a to 48c, and a nose pad 49. It has a controller 50 and the like. The controller 50, the left line-of-sight camera 41, the right line-of-sight camera 42, the speaker 46, and the microphone 47 are arranged in the frame housings 48a to 48c. The arrangement location does not have to be as shown in FIG. Furthermore, a group of sensors such as an acceleration sensor, a gyro sensor, and a position sensor are built in.
　ユーザ１は、フレーム筐体４８ａ，４８ｂとノーズパッド４９で、ＨＭＤ４を自身の頭部に装着する。左目視線カメラ４１と右目視線カメラ４２は、ユーザの視線前方を撮影するとともに、左目と右目の視差を利用して、撮影画像が捉える現実空間の物体までの距離の測定が可能な三次元カメラを構成している。測定された距離は撮影画像の奥行き情報として利用することで、撮影画像の理解に役立つ。 User 1 attaches the HMD4 to his or her head with the frame housings 48a and 48b and the nose pad 49. The left line-of-sight camera 41 and the right line-of-sight camera 42 are three-dimensional cameras capable of photographing the front of the user's line of sight and measuring the distance to an object in the real space captured by the photographed image by using the difference between the left eye and the right eye. It is configured. By using the measured distance as depth information of the captured image, it is useful for understanding the captured image.
　コントローラ５０は、左目視線カメラ４１、右目視線カメラ４２で撮影した現実空間の画像、さらに現実空間の物体までの距離データ等を、内部のメモリやＣＰＵに取り込む。さらにコントローラ５０は、投影光学系４４ａ，４４ｂで投影する映像をコンピュータグラフィックス等で作成し、またスピーカ４６から出力する音声を作成する。 The controller 50 takes in the image of the real space taken by the left visual line camera 41 and the right visual line camera 42, the distance data to the object in the real space, and the like into the internal memory and the CPU. Further, the controller 50 creates images projected by the projection optical systems 44a and 44b by computer graphics or the like, and also creates audio output from the speaker 46.
　投影光学系４４ａ，４４ｂと表示光学系４５は、ＨＭＤ４の表示部となる。投影光学系４４ａ，４４ｂは、コントローラ５０が作成した仮想物体の映像を、左目用映像と右目用映像に分けて、表示光学系４５に投影して表示する。ユーザ１は、透過型の表示光学系４５を介して前方の風景や現実物体を見るとともに、投影光学系４４ａ，４４ｂから投影された仮想物体の映像を、表示光学系４５上で重ねて視認することができる。 The projection optical systems 44a and 44b and the display optical system 45 serve as a display unit of the HMD4. The projection optical systems 44a and 44b divide the image of the virtual object created by the controller 50 into a left-eye image and a right-eye image, and project and display the image on the display optical system 45. The user 1 sees a landscape or a real object in front of the display optical system 45 through the transmissive display optical system 45, and superimposes and visually recognizes an image of a virtual object projected from the projection optical systems 44a and 44b on the display optical system 45. be able to.
　図９は、ヘッドマウントディスプレイ（ＨＭＤ）４のブロック図を示す。図８と同一の構成要素には同一の符号を付しており、それ以外の要素として、センサ群４３、距離算出部５１、通信部５２、ＣＰＵ５３、ＲＡＭ５４、画像ＲＡＭ５５、フラッシュＲＯＭ５６を有し、各部は内部バス６０で接続されている。 FIG. 9 shows a block diagram of the head-mounted display (HMD) 4. The same components as those in FIG. 8 are designated by the same reference numerals, and the other elements include a sensor group 43, a distance calculation unit 51, a communication unit 52, a CPU 53, a RAM 54, an image RAM 55, and a flash ROM 56. Each part is connected by an internal bus 60.
　投影光学系４４は、図８の投影光学系４４ａ，４４ｂに対応し、左目用映像と右目用映像を独立に表示光学系４５に投影させる。他に、１つのプロジェクタでインタリーブした左目用映像と右目用映像を投影し、シャッタ光学系で、左目用映像と右目用映像をそれぞれの目に投影させる方式でもよい。さらには、ホログラフィックレンズを用いた光学系でもよい。 The projection optical system 44 corresponds to the projection optical systems 44a and 44b of FIG. 8, and the left-eye image and the right-eye image are independently projected onto the display optical system 45. Alternatively, a method may be used in which the left-eye image and the right-eye image interleaved by one projector are projected, and the left-eye image and the right-eye image are projected on the respective eyes by the shutter optical system. Further, an optical system using a holographic lens may be used.
　通信部５２は、モバイル通信、ワイアレスＬＡＮやＢｌｕｅｔｏｏｔｈ（登録商標）等の複数の通信機能を有し、ＨＭＤ４を、ネットワークを介して外部ストレージ等に接続させる。 The communication unit 52 has a plurality of communication functions such as mobile communication, wireless LAN and Bluetooth (registered trademark), and connects the HMD4 to an external storage or the like via a network.
　ＣＰＵ５３は、フラッシュＲＯＭ５６に格納されているプログラムをＲＡＭ５４に展開し、これを実行することによって、ＨＭＤ４の各構成部の動作制御を行う。画像ＲＡＭ５５は、投影光学系４４に送出する映像データを格納している。 The CPU 53 expands the program stored in the flash ROM 56 into the RAM 54 and executes the program to control the operation of each component of the HMD 4. The image RAM 55 stores video data to be transmitted to the projection optical system 44.
　フラッシュＲＯＭ５６には、基本動作プログラム５７、インターバル撮影アプリ５８の処理プログラム、及び撮影データ保持部５９が含まれる。基本動作プログラム５７はＨＭＤ４としての投影動作処理を行い、インターバル撮影アプリ５８は、実施例１（図２）で述べた動き判定処理２４ａ、タイミング生成処理２４ｂ、撮影制御処理２４ｃを行うものである。また撮影データ保持部５９は、カメラで撮影された画像データを保存する。 The flash ROM 56 includes a basic operation program 57, a processing program of the interval shooting application 58, and a shooting data holding unit 59. The basic operation program 57 performs the projection operation process as the HMD 4, and the interval shooting application 58 performs the motion determination process 24a, the timing generation process 24b, and the shooting control process 24c described in the first embodiment (FIG. 2). Further, the shooting data holding unit 59 stores the image data shot by the camera.
　本実施例におけるＨＭＤ４を用いたインターバル撮影は、実施例１の図４Ａ，４Ｂに示した撮影タイミングと撮影モードと同様に行う。また、撮影動作のフローチャートは図５と同様に行う。ただし、撮影モードに関しては、標準モードでは左目視線カメラ４１または右目視線カメラ４２を使用して撮影する。また、広角モードでは、左目視線カメラ４１と右目視線カメラ４２の両方を用いて三次元撮影を行う。 The interval shooting using the HMD4 in this embodiment is performed in the same manner as the shooting timing and shooting mode shown in FIGS. 4A and 4B of the first embodiment. Further, the flowchart of the shooting operation is performed in the same manner as in FIG. However, regarding the shooting mode, in the standard mode, the left visual line camera 41 or the right visual line camera 42 is used for shooting. Further, in the wide-angle mode, three-dimensional photography is performed using both the left visual line camera 41 and the right visual line camera 42.
　以上の構成により、実施例３によれば、静止時には三次元画像による背景観察に有効な画像を取得できるとともに、運動時にも所定の撮影間隔で履歴画像を得ることができる。また、運動時の履歴画像の撮影データ量や消費電力を削減することができる。 With the above configuration, according to the third embodiment, it is possible to acquire an image effective for background observation by a three-dimensional image when stationary, and to obtain a historical image at a predetermined shooting interval even when exercising. In addition, it is possible to reduce the amount of captured data and power consumption of the historical image during exercise.
　さらに本実施例のようにＨＭＤ４を用いる場合には、左目視線カメラ４１と右目視線カメラ４２のレンズの光軸をユーザの視線とほぼ一致させることができるので、実際にユーザが見る風景を立体写真として保存することができる。また、投影光学系４４と表示光学系４５を動作させることで、インターバル撮影を中断することなく、撮影済みの画像を時間的に遡って見ることができるという利点がある。 Further, when the HMD4 is used as in the present embodiment, the optical axes of the lenses of the left visual line camera 41 and the right visual line camera 42 can be substantially aligned with the user's line of sight, so that the landscape actually seen by the user is a three-dimensional photograph. Can be saved as. Further, by operating the projection optical system 44 and the display optical system 45, there is an advantage that the captured image can be viewed retroactively in time without interrupting the interval shooting.
　実施例４は、実施例１における動き判定処理を複数レベルで判定する構成とした。具体的には、ユーザの頭部の動きと胴体の動きをそれぞれ検出し、それらの動きを比較して撮影制御を行う。 Example 4 has a configuration in which the motion determination process in Example 1 is determined at a plurality of levels. Specifically, the movement of the user's head and the movement of the body are detected, and the movements are compared to control shooting.
　図１０は、実施例４に係るインターバル撮像装置６をユーザが装着した状態を示す図である。実施例１（図１）と同様に、ユーザ１の頭部には、インターバル撮像装置６の本体部６ａ（以下、装置本体部）が装着ベルト７にて装着されている。装置本体部６ａには、標準カメラ１１と広角カメラ１２とが備えられている。さらに本実施例では、ユーザ１の胴体（例えば胸部）にセンサ端末６ｂが装着されている。センサ端末６ｂはユーザ１の胴体の動きを検出するものである。 FIG. 10 is a diagram showing a state in which the user wears the interval imaging device 6 according to the fourth embodiment. Similar to the first embodiment (FIG. 1), the main body portion 6a of the interval imaging device 6 (hereinafter referred to as the device main body portion) is attached to the head of the user 1 by the attachment belt 7. The apparatus main body 6a is provided with a standard camera 11 and a wide-angle camera 12. Further, in this embodiment, the sensor terminal 6b is attached to the body (for example, the chest) of the user 1. The sensor terminal 6b detects the movement of the body of the user 1.
　本実施例のインターバル撮像装置６では、ユーザの頭部の動きデータとユーザの胴体の動きデータとを用いて、動き判定を行なう。以下では、２つの動きデータを区別するために、ユーザの頭部の動きについては「第１の動きデータ」と呼び、ユーザの胴体の動きについては「第２の動きデータ」と呼ぶことにする。 In the interval imaging device 6 of this embodiment, motion determination is performed using the motion data of the user's head and the motion data of the user's torso. In the following, in order to distinguish between the two movement data, the movement of the user's head will be referred to as "first movement data", and the movement of the user's torso will be referred to as "second movement data". ..
　図１１は、インターバル撮像装置６のブロック図を示す。装置本体部６ａの構成は実施例１（図２）と同様であり、加速度センサ１３とジャイロセンサ１４は、ユーザの頭部の動きを検出する「第１の動きセンサ」となる。インターバル撮影アプリ２４には、図２で述べたように、動き判定処理２４ａ、タイミング生成処理２４ｂ、撮影制御処理２４ｃが含まれている。なお、この図では装置本体部６ａの一部の構成要素を省略している。 FIG. 11 shows a block diagram of the interval imaging device 6. The configuration of the device main body 6a is the same as that of the first embodiment (FIG. 2), and the acceleration sensor 13 and the gyro sensor 14 are “first motion sensors” that detect the movement of the user's head. As described in FIG. 2, the interval shooting application 24 includes a motion determination process 24a, a timing generation process 24b, and a shooting control process 24c. In this figure, some components of the device main body 6a are omitted.
　センサ端末６ｂの構成は、加速度センサ６１、ジャイロセンサ６２、通信部６３、ＣＰＵ（もしくはマイクロコンピュータ）６４、ＲＡＭ６５、フラッシュＲＯＭ６６を有する。このうち加速度センサ６１とジャイロセンサ６２は、ユーザの胴体の動きを検出する「第２の動きセンサ」となる。 The configuration of the sensor terminal 6b includes an acceleration sensor 61, a gyro sensor 62, a communication unit 63, a CPU (or microcomputer) 64, a RAM 65, and a flash ROM 66. Of these, the acceleration sensor 61 and the gyro sensor 62 are "second movement sensors" that detect the movement of the user's body.
　通信部６３は、例えばＢｌｕｅｔｏｏｔｈ（登録商標）などの低電力通信プロトコルに対応しており、装置本体部６ａの通信部１９との間で、一対一の通信を行う。フラッシュＲＯＭ６６に格納する動作プログラム６７は、第２の動きセンサ６１，６２にてユーザの胴体の動きデータを取得し、装置本体部６ａへ送信する処理を行うためのものである。 The communication unit 63 supports a low power communication protocol such as Bluetooth (registered trademark), and performs one-to-one communication with the communication unit 19 of the device main body 6a. The operation program 67 stored in the flash ROM 66 is for performing a process of acquiring the motion data of the user's body by the second motion sensors 61 and 62 and transmitting the motion data to the apparatus main body 6a.
　センサ端末６ｂは、装置本体部６ａの撮影動作に連動してユーザの胴体の動きデータを取得する。具体的には、装置本体部６ａから動きデータ要求を受信すると、加速度センサ６１、ジャイロセンサ６２の検出値を第２の動きデータとして装置本体部６ａに送信する。 The sensor terminal 6b acquires the movement data of the user's body in conjunction with the shooting operation of the device main body 6a. Specifically, when a motion data request is received from the device body 6a, the detected values of the acceleration sensor 61 and the gyro sensor 62 are transmitted to the device body 6a as second motion data.
　本実施例のインターバル撮像装置６は、ユーザの頭部（第１の動きセンサ１３，１４）の第１の動きデータと、ユーザの胴体（第２の動きセンサ６１，６２）の第２の動きデータとを用いて、動き判定部２４ａにより動きの状態を複数レベルで判定する。判定では、それぞれの動きの状態とともに両者の動きデータの相関を調べる。 In the interval imaging device 6 of this embodiment, the first motion data of the user's head (first motion sensors 13 and 14) and the second motion of the user's body (second motion sensors 61 and 62) Using the data, the motion determination unit 24a determines the motion state at a plurality of levels. In the judgment, the correlation between the movement data of both is examined together with the state of each movement.
　具体的には、第１の動きデータと第２の動きデータが共に「動き有」で同方向、同程度の大きさである場合（相関有）、すなわち頭部と胴体が同期して動いている場合は「運動状態１」と判定し、標準モード（標準カメラ１１のみ）による撮影を行う。第１の動きデータと第２の動きデータが異なる方向、異なる大きさである場合（相関無）、すなわち頭部と胴体が同期せずにランダムに動いている場合は「運動状態２」と判定し、カメラ撮影を制限（中止）する。第１の動きデータと第２の動きデータがいずれも「動き無」の場合は「静止状態」と判定し、広角モード（広角カメラ１２）による撮影を行う。 Specifically, when both the first motion data and the second motion data are "with motion" and have the same direction and the same size (correlation), that is, the head and the body move in synchronization. If so, it is determined to be "exercise state 1", and shooting is performed in the standard mode (standard camera 11 only). When the first motion data and the second motion data are in different directions and different sizes (no correlation), that is, when the head and the torso are moving randomly out of synchronization, it is determined to be "exercise state 2". Then, the camera shooting is restricted (stopped). When both the first motion data and the second motion data are "no motion", it is determined to be "stationary state", and shooting is performed in the wide-angle mode (wide-angle camera 12).
　図１２は、インターバル撮影の例を示すタイムチャートである。図の表記は実施例１（図４Ａ）と同様である。動き判定結果は、動きの状態を「運動１」「運動２」「静止」の３段階で判定している。撮影モードは、標準モードと広角モードの他に、撮影を制限するモードを追加している。撮影タイミングの間隔ΔＴは図４Ａと同様に設定しており、動き判定が運動状態から静止状態に遷移するＴ２３とＴ３０においては、それまでの撮影タイミングの間隔がリセットされ、新たな撮影タイミングが設定される。 FIG. 12 is a time chart showing an example of interval shooting. The notation in the figure is the same as in the first embodiment (FIG. 4A). The motion determination result determines the state of motion in three stages of "exercise 1", "exercise 2", and "stationary". As for the shooting mode, in addition to the standard mode and wide-angle mode, a mode for limiting shooting has been added. The shooting timing interval ΔT is set in the same manner as in FIG. 4A, and in T23 and T30 where the motion determination transitions from the moving state to the stationary state, the shooting timing interval up to that point is reset and a new shooting timing is set. Will be done.
　動き判定結果が「運動１」のときは、例えばＴ２１、Ｔ２２のように、標準モードで撮影する。動き判定結果が「運動２」のときは、例えばＴ２７、Ｔ２８のように、カメラ撮影を制限する。動き判定結果が「静止」のときは、例えばＴ２３、Ｔ２４のように、広角モードで撮影する。 When the motion judgment result is "exercise 1", the image is taken in the standard mode, for example, T21 and T22. When the motion determination result is "exercise 2", camera shooting is restricted, for example, T27 and T28. When the motion determination result is "stationary", shooting is performed in the wide-angle mode, for example, T23 and T24.
　これにより、ユーザが例えば激しい動きをしている時や、急に顔を振り横に向けた場合には「運動２」と判定され、カメラ撮影を制限（中止）するので、無駄な撮影を回避することができる。そしてユーザの頭部と胴体が同期した動きになると、「運動１」と判定されて標準モードでの撮影に戻る。 As a result, for example, when the user is making a violent movement or suddenly swings his face and turns to the side, it is judged as "exercise 2" and the camera shooting is restricted (stopped), so that unnecessary shooting is avoided. can do. Then, when the movement of the user's head and body is synchronized, it is determined as "exercise 1" and the shooting returns to the standard mode.
　なお、図１２では、撮影タイミングの間隔ΔＴは標準モードと広角モードとで等しく設定したが、図４Ｂのように、広角モードでは異なる間隔ΔＴ’に設定してもよい。 In FIG. 12, the shooting timing interval ΔT is set to be equal in the standard mode and the wide-angle mode, but as shown in FIG. 4B, it may be set to a different interval ΔT'in the wide-angle mode.
　図１３は、インターバル撮影動作を示すフローチャートである。実施例１（図５）で示したフローチャートと同一の内容のステップには、同一の番号を付している。本実施例では、動きの状態を３段階に分け、「運動状態２」と動きフラグＦ＝２を追加している。撮影タイミングの間隔は、標準モードではΔＴ、広角モードではΔＴ’とする。図５のフローチャートと異なるステップは以下の通りである。 FIG. 13 is a flowchart showing the interval shooting operation. The steps having the same contents as the flowchart shown in the first embodiment (FIG. 5) are given the same numbers. In this embodiment, the motion state is divided into three stages, and "exercise state 2" and the motion flag F = 2 are added. The shooting timing interval is ΔT in the standard mode and ΔT'in the wide-angle mode. The steps different from the flowchart of FIG. 5 are as follows.
　Ｓ１０３ａ：装置本体部６ａの第１の動きセンサ（加速度センサ１３、ジャイロセンサ１４）の出力を取得する。
  Ｓ１０３ｂ：センサ端末６ｂの第２の動きセンサ（加速度センサ６１、ジャイロセンサ６２）の出力を取得する。 S103a: Acquires the output of the first motion sensor (accelerometer 13, gyro sensor 14) of the device main body 6a.
S103b: Acquires the output of the second motion sensor (accelerometer 61, gyro sensor 62) of the sensor terminal 6b.
　Ｓ１０４ａ：動き判定部２４ａにより、第１の動きデータと第２の動きデータを比較し、３段階の動き判定を行う。第１、第２の動きデータが「動き有」で相関があれば、「運動状態１」とする。第１、第２の動きデータが「動き有」で相関がなければ、「運動状態２」とする。第１、第２の動きデータがいずれも「動き無」の場合は「静止状態」とする。 S104a: The motion determination unit 24a compares the first motion data with the second motion data, and performs three-step motion determination. If the first and second motion data are "with motion" and have a correlation, it is regarded as "exercise state 1". If the first and second motion data are "with motion" and there is no correlation, it is set as "exercise state 2". When both the first and second motion data are "no motion", it is regarded as "stationary state".
　Ｓ１０６～Ｓ１０８：判定の結果、運動状態１の場合は動きフラグＦ＝１とし、第１撮影間隔ΔＴで標準モードの撮影を行う。
  Ｓ１０９～Ｓ１１２：判定の結果、静止状態の場合は動きフラグＦ＝０とし、第２撮影間隔ΔＴ’で広角モードの撮影を行う。
  Ｓ１２１～Ｓ１２２：判定の結果、運動状態２の場合は動きフラグＦ＝２とし、撮影を行わずに次の撮影タイミングまで待機する。 S106 to S108: As a result of the determination, in the case of the motion state 1, the motion flag F = 1 is set, and the standard mode shooting is performed at the first shooting interval ΔT.
S109 to S112: As a result of the determination, in the stationary state, the motion flag F = 0 is set, and the wide-angle mode shooting is performed at the second shooting interval ΔT'.
S121 to S122: As a result of the determination, in the case of the motion state 2, the motion flag F = 2 is set, and the camera waits until the next shooting timing without shooting.
　以上説明したように、実施例４によれば、ユーザの頭部と胴体に装着した動きセンサを用いて動き判定を行い、両者が運動状態であっても動きデータに相関がない場合は、カメラ撮影を制限するようにした。これにより無駄な撮影を削減し、撮影データ保持部の容量や装置の消費電力を節約する、効率の良いインターバル撮像装置を実現できる。 As described above, according to the fourth embodiment, the motion is determined by using the motion sensors attached to the user's head and the body, and if there is no correlation between the motion data even if both are in the motion state, the camera I tried to limit the shooting. As a result, it is possible to realize an efficient interval imaging device that reduces unnecessary shooting and saves the capacity of the shooting data holding unit and the power consumption of the device.
　なお、実施例４の変形として、装置本体部６ａの第１の動きセンサ１３，１４のみを用いて、運動状態をその動きデータの大きさに応じて複数レベルで判定してもよい。すなわち、小さな動き、大きな動きといった判定を行い、所定値を超える大きな動きの場合はカメラ撮影を制限することで無駄な撮影を削減し、同様の効果が得られる。 As a modification of the fourth embodiment, the motion state may be determined at a plurality of levels according to the size of the motion data by using only the first motion sensors 13 and 14 of the apparatus main body 6a. That is, it is determined that a small movement or a large movement is performed, and in the case of a large movement exceeding a predetermined value, unnecessary shooting is reduced by limiting the camera shooting, and the same effect can be obtained.
　以上、本発明に係るいくつかの実施例を説明したが、本発明はこれらに限られるものではなく、ある実施例の構成の一部を他の実施例に置き換えることが可能である。また、ある実施例の構成に、他の実施例の構成を加えることも可能である。これらは全て本発明の範疇に属するものであり、さらに文中や図中に現れる数値はあくまで一例であり、異なるものを用いても本発明の効果を損なうものでない。 Although some embodiments of the present invention have been described above, the present invention is not limited to these, and a part of the configuration of one embodiment can be replaced with another embodiment. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. All of these belong to the category of the present invention, and the numerical values appearing in the text and figures are merely examples, and even if different ones are used, the effect of the present invention is not impaired.
　例えば、上記実施例では、本発明のインターバル撮像装置をユーザに装着して、ユーザの履歴画像（ライフログ）を取得する場合について説明したが、これに限らず、インターバル撮像装置を動物や車両などの移動体に装着して、その行動履歴の画像を取得する場合にも適用できる。 For example, in the above embodiment, the case where the interval imaging device of the present invention is attached to the user to acquire the user's history image (life log) has been described, but the present invention is not limited to this, and the interval imaging device can be used as an animal, a vehicle, or the like. It can also be applied when it is attached to a moving body and an image of its action history is acquired.
　また、発明の機能等は、それらの一部または全部を、例えば集積回路で設計する等によりハードウェアで実装しても良い。また、マイクロプロセッサユニット、ＣＰＵ等が動作プログラムを解釈して実行することによりソフトウェアで実装しても良い。また、ソフトウェアの実装範囲を限定するものでなく、ハードウェアとソフトウェアを併用しても良い。 Further, the functions and the like of the invention may be implemented by hardware by designing a part or all of them by, for example, an integrated circuit. Further, it may be implemented by software by interpreting and executing an operation program by a microprocessor unit, a CPU, or the like. Further, the implementation range of the software is not limited, and the hardware and the software may be used together.
　１：ユーザ、２，６：インターバル撮像装置、３：全方位カメラ、４：ヘッドマウントディスプレイ（ＨＭＤ）、６ａ：装置本体部、６ｂ：センサ端末、１１：標準カメラ、１２：広角カメラ、１３，６１：加速度センサ、１４，６２：ジャイロセンサ、１９，６３：通信部、２４，５８：インターバル撮影アプリ、２４ａ：動き判定処理、２４ｂ：タイミング生成処理、２４ｃ：撮影制御処理、２５，５９：撮影データ保持部、３１：前半天球光学系、３２：後半天球光学系、３３：撮像センサ、４１：左目視線カメラ、４２：右目視線カメラ。 1: User, 2, 6: Interval imager, 3: Omnidirectional camera, 4: Head mount display (HMD), 6a: Device body, 6b: Sensor terminal, 11: Standard camera, 12: Wide angle camera, 13, 61: Acceleration sensor, 14,62: Gyro sensor, 19,63: Communication unit, 24,58: Interval shooting application, 24a: Motion judgment processing, 24b: Timing generation processing, 24c: Shooting control processing, 25,59: Shooting Data holding unit, 31: first half celestial optical system, 32: second half celestial optical system, 33: imaging sensor, 41: left visual line camera, 42: right visual line camera.

Claims (9)

  1. 　ユーザに装着され外部の撮影を間欠的に行うインターバル撮像装置であって、
    　複数の撮影モードで撮影可能な撮像部と、
    　前記インターバル撮像装置の動きを検出する動きセンサと、
    　前記動きセンサの検出結果から動きの有無を判定する動き判定部と、
    　前記撮像部が撮影するためのトリガー信号を所定の間隔で発生するタイミング生成部と、
    　前記動き判定部の判定結果に応じて前記撮像部と前記タイミング生成部を制御する撮影制御部と、を備え、
    　前記撮影制御部は前記撮像部に対し、動き有り（以下、運動状態）のときの撮影モードと動き無し（以下、静止状態）のときの撮影モードを切り替えるよう制御することを特徴とするインターバル撮像装置。     It is an interval imaging device that is attached to the user and intermittently takes pictures of the outside.
    An image pickup unit that can shoot in multiple shooting modes,
    A motion sensor that detects the motion of the interval imaging device and
    A motion determination unit that determines the presence or absence of motion from the detection result of the motion sensor,
    A timing generation unit that generates a trigger signal for shooting by the imaging unit at predetermined intervals, and
    A shooting control unit that controls the imaging unit and the timing generation unit according to the determination result of the motion determination unit is provided.
    The shooting control unit controls the imaging unit to switch between a shooting mode when there is movement (hereinafter, moving state) and a shooting mode when there is no movement (hereinafter, stationary state). Device.
  2. 　請求項１に記載のインターバル撮像装置であって、
    　前記動き判定部の判定結果が運動状態から静止状態へ遷移したとき、前記撮影制御部は前記タイミング生成部に対し、撮影のためのトリガー信号を発生させるとともに、前記撮像部に対し静止状態に対応する撮影モードで撮影するよう制御することを特徴とするインターバル撮像装置。     The interval imaging device according to claim 1.
    When the determination result of the motion determination unit transitions from the motion state to the stationary state, the imaging control unit generates a trigger signal for imaging to the timing generation unit and corresponds to the stationary state with respect to the imaging unit. An interval imaging device characterized in that it is controlled to shoot in a shooting mode.
  3. 　請求項１に記載のインターバル撮像装置であって、
    　前記動き判定部の判定結果が運動状態のとき、前記撮影制御部は前記タイミング生成部に対し、第１の間隔でトリガー信号を発生させ、
    　前記動き判定部の判定結果が静止状態のとき、前記撮影制御部は前記タイミング生成部に対し、前記第１の間隔より短い第２の間隔でトリガー信号を発生させることを特徴とするインターバル撮像装置。     The interval imaging device according to claim 1.
    When the determination result of the motion determination unit is in the motion state, the imaging control unit generates a trigger signal at the first interval for the timing generation unit.
    When the determination result of the motion determination unit is in a stationary state, the imaging control unit generates a trigger signal for the timing generation unit at a second interval shorter than the first interval. ..
  4. 　請求項１に記載のインターバル撮像装置であって、
    　前記撮像部は、標準画角で撮影する標準モードと、広角で撮影する広角モードとを備え、
    　前記撮影制御部は前記撮像部に対し、運動状態では標準モードで撮影し、静止状態では広角モードで撮影するよう制御することを特徴とするインターバル撮像装置。     The interval imaging device according to claim 1.
    The imaging unit includes a standard mode for shooting at a standard angle of view and a wide-angle mode for shooting at a wide angle.
    The imaging control unit is an interval imaging device that controls the imaging unit to shoot in a standard mode in a moving state and in a wide-angle mode in a stationary state.
  5. 　請求項１に記載のインターバル撮像装置であって、
    　前記撮像部は、複数形式の撮影画像を取得することが可能であり、
    　前記撮影制御部は前記撮像部に対し、運動状態では前記複数形式の撮影画像のうちの１つを取得し、静止状態では前記複数形式の撮影画像を全て取得するよう制御することを特徴とするインターバル撮像装置。     The interval imaging device according to claim 1.
    The imaging unit can acquire captured images in a plurality of formats, and can acquire captured images in a plurality of formats.
    The photographing control unit is characterized in that it controls the imaging unit to acquire one of the captured images of the plurality of formats in a moving state and to acquire all the captured images of the plurality of formats in a stationary state. Interval imaging device.
  6. 　請求項５に記載のインターバル撮像装置であって、
    　前記撮像部は、前記複数形式の撮影画像として、前半天球の画像と後半天球の画像を取得することが可能であり、
    　前記撮影制御部は前記撮像部に対し、運動状態では前半天球の画像を取得し、静止状態では前半天球と後半天球を含む全天球の画像を取得するよう制御することを特徴とするインターバル撮像装置。     The interval imaging device according to claim 5.
    The imaging unit can acquire an image of the first half celestial sphere and an image of the second half celestial sphere as the captured images of the plurality of formats.
    The imaging control unit controls the imaging unit to acquire an image of the first half celestial sphere in a moving state and an image of a whole celestial sphere including the first half celestial sphere and the second half celestial sphere in a stationary state. Device.
  7. 　請求項５に記載のインターバル撮像装置であって、
    　前記撮像部は、前記複数形式の撮影画像として、左目視線の画像と右目視線の画像を取得することが可能であり、
    　前記撮影制御部は前記撮像部に対し、運動状態では左目視線の画像と右目視線の画像のいずれかを取得し、静止状態では左目視線の画像と右目視線の画像の両方を取得するよう制御することを特徴とするインターバル撮像装置。     The interval imaging device according to claim 5.
    The imaging unit can acquire an image of the left visual line and an image of the right visual line as the captured images of the plurality of formats.
    The imaging control unit controls the imaging unit to acquire either an image of the left visual line or an image of the right visual line in a moving state, and to acquire both an image of the left visual line and an image of the right visual line in a stationary state. An interval imaging device characterized by this.
  8. 　請求項１に記載のインターバル撮像装置であって、
    　前記インターバル撮像装置の動きを検出する前記動きセンサ（以下、第１の動きセンサ）の他に、前記インターバル撮像装置が装着される前記ユーザの第１の部位とは異なる第２の部位に装着され、前記ユーザの第２の部位の動きを検出する動きセンサ（以下、第２の動きセンサ）を備え、
    　前記動き判定部は、前記第１の動きセンサで検出した第１の動きデータと、前記第２の動きセンサで検出した第２の動きデータとを比較し、両者の動きデータの相関の有無により運動状態を複数のレベルで判定し、
    　前記動き判定部の判定結果が運動状態であっても前記両者の動きデータに相関がない場合には、前記撮影制御部は前記撮像部に対し、撮影を制限するよう制御することを特徴とするインターバル撮像装置。     The interval imaging device according to claim 1.
    In addition to the motion sensor (hereinafter, the first motion sensor) that detects the motion of the interval image pickup device, the interval image pickup device is attached to a second portion different from the first portion of the user to which the interval image pickup device is attached. A motion sensor (hereinafter referred to as a second motion sensor) for detecting the motion of the second portion of the user is provided.
    The motion determination unit compares the first motion data detected by the first motion sensor with the second motion data detected by the second motion sensor, and depending on the presence or absence of correlation between the motion data of both. Judging the state of exercise at multiple levels,
    Even if the determination result of the motion determination unit is in a motion state, if there is no correlation between the motion data of both, the imaging control unit controls the imaging unit to limit imaging. Interval imaging device.
  9. 　請求項１に記載のインターバル撮像装置であって、
    　前記動き判定部は、前記動きセンサで検出した動きデータの大きさによって運動状態を複数のレベルで判定し、
    　前記動き判定部の判定結果が運動状態であってもその動きデータが所定値を超える場合には、前記撮影制御部は前記撮像部に対し、撮影を制限するよう制御することを特徴とするインターバル撮像装置。     The interval imaging device according to claim 1.
    The motion determination unit determines the motion state at a plurality of levels based on the size of the motion data detected by the motion sensor.
    Even if the determination result of the motion determination unit is in a motion state, if the motion data exceeds a predetermined value, the imaging control unit controls the imaging unit to limit imaging. Imaging device.
PCT/JP2020/003053 2020-01-28 2020-01-28 Interval imaging device WO2021152711A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021573677A JP7343621B2 (en) 2020-01-28 2020-01-28 interval imaging device
PCT/JP2020/003053 WO2021152711A1 (en) 2020-01-28 2020-01-28 Interval imaging device
JP2023140669A JP2023164903A (en) 2020-01-28 2023-08-31 Interval imaging apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/003053 WO2021152711A1 (en) 2020-01-28 2020-01-28 Interval imaging device

Publications (1)

Publication Number Publication Date
WO2021152711A1 true WO2021152711A1 (en) 2021-08-05

Family

ID=77078046

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/003053 WO2021152711A1 (en) 2020-01-28 2020-01-28 Interval imaging device

Country Status (2)

Country Link
JP (2) JP7343621B2 (en)
WO (1) WO2021152711A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009118135A (en) * 2007-11-06 2009-05-28 Sony Corp Imaging apparatus and imaging method
JP2009267792A (en) * 2008-04-25 2009-11-12 Panasonic Corp Imaging apparatus
JP2010011343A (en) * 2008-06-30 2010-01-14 Toyota Motor Corp Image acquisition device, image acquisition method, and moving body
JP2013153329A (en) * 2012-01-25 2013-08-08 Nikon Corp Electronic apparatus
JP2015005809A (en) * 2013-06-19 2015-01-08 ソニー株式会社 Information processing device, information processing method, and program
JP2019121855A (en) * 2017-12-28 2019-07-22 キヤノン株式会社 Imaging device and control method thereof

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4725595B2 (en) 2008-04-24 2011-07-13 ソニー株式会社 Video processing apparatus, video processing method, program, and recording medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009118135A (en) * 2007-11-06 2009-05-28 Sony Corp Imaging apparatus and imaging method
JP2009267792A (en) * 2008-04-25 2009-11-12 Panasonic Corp Imaging apparatus
JP2010011343A (en) * 2008-06-30 2010-01-14 Toyota Motor Corp Image acquisition device, image acquisition method, and moving body
JP2013153329A (en) * 2012-01-25 2013-08-08 Nikon Corp Electronic apparatus
JP2015005809A (en) * 2013-06-19 2015-01-08 ソニー株式会社 Information processing device, information processing method, and program
JP2019121855A (en) * 2017-12-28 2019-07-22 キヤノン株式会社 Imaging device and control method thereof

Also Published As

Publication number Publication date
JP2023164903A (en) 2023-11-14
JP7343621B2 (en) 2023-09-12
JPWO2021152711A1 (en) 2021-08-05

Similar Documents

Publication Publication Date Title
US11860511B2 (en) Image pickup device and method of tracking subject thereof
JP7408678B2 (en) Image processing method and head mounted display device
JP6094190B2 (en) Information processing apparatus and recording medium
US20170085841A1 (en) Image Recording System, User Wearable Device, Imaging Device, Image Processing Device, Image Recording Method And Storage Medium
JP2011166756A (en) Photographing apparatus and photographing system
KR20220128585A (en) Wearable image pickup apparatus, portable device and calibrator that communicate with image pickup apparatus, control methods therefor, and storage media storing control programs therefor
JP6533761B2 (en) INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING SYSTEM, AND INFORMATION PROCESSING METHOD
KR20170119201A (en) Method and system for remote control of camera in smart phone
WO2021152711A1 (en) Interval imaging device
JP6499993B2 (en) Information processing apparatus, information processing system, and information processing method
CN107426522B (en) Video method and system based on virtual reality equipment
JP6921204B2 (en) Information processing device and image output method
RU2782312C1 (en) Image processing method and display device mounted on the head
US20240121506A1 (en) System consisting of electronic device and notification apparatus, control method thereof, and electronic device and notification apparatus
JP2018160809A (en) Image processing apparatus, imaging system, image processing method, and program
JP2022140420A (en) Imaging apparatus, display unit, video transfer system, control method, and program
JP2023154539A (en) Imaging device, control method for imaging device, display device, and imaging system
JP2023143437A (en) Control device, control method, and control program
JP2022140424A (en) Image pickup apparatus, control method therefor, and program
JP2024064186A (en) Imaging system, device and controlling method
JP2024097267A (en) Imaging apparatus, calibrator, and method for controlling the same, and program
JP2023055074A (en) Imaging apparatus and method for controlling the same, and program
JP2024017297A (en) Imaging apparatus, wearable device, control method, program, and system
JP2022140418A (en) Image pickup apparatus
JP2023146003A (en) Wearable terminal

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20916849

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021573677

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20916849

Country of ref document: EP

Kind code of ref document: A1