US20170272623A1 - Direction control device, direction control method and recording medium - Google Patents
Direction control device, direction control method and recording medium Download PDFInfo
- Publication number
- US20170272623A1 US20170272623A1 US15/532,157 US201515532157A US2017272623A1 US 20170272623 A1 US20170272623 A1 US 20170272623A1 US 201515532157 A US201515532157 A US 201515532157A US 2017272623 A1 US2017272623 A1 US 2017272623A1
- Authority
- US
- United States
- Prior art keywords
- image
- camera
- unit
- face
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B15/00—Special procedures for taking photographs; Apparatus therefor
- G03B15/02—Illuminating scene
- G03B15/03—Combinations of cameras with lighting apparatus; Flash units
- G03B15/04—Combinations of cameras with non-electronic flash apparatus; Non-electronic flash units
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B17/00—Details of cameras or camera bodies; Accessories therefor
- G03B17/02—Bodies
-
- G06K9/00228—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0014—Image feed-back for automatic industrial control, e.g. robot with camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/681—Motion detection
- H04N23/6812—Motion detection based on additional sensors, e.g. acceleration sensors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
- H04N23/685—Vibration or motion blur correction performed by mechanical compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
-
- H04N5/232—
-
- H04N5/23212—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Definitions
- the disclosed subject matter relates to a direction control device and the like.
- the disclosed subject matter relates to the control of the photographing direction of a camera.
- the wearable cameras include helmet-mounted wearable cameras, glasses-mounted wearable cameras, and additionally badge-mounted wearable cameras incorporated into recorders which are mounted on the chests of photographers.
- PTL 1 describes a method for managing the position of each camera by recognizing image data captured by each camera.
- PTL 2 describes a method for determining a self-position by using a plane mirror in order to determine the direction of a robot.
- a helmet-mounted or glasses-mounted wearable camera may eliminate adjusting the photographing direction of the camera in case of re-mounting the wearable camera after adjusting the photographing direction of the camera.
- the photographing direction of a badge-mounted wearable camera which is mounted on the chest, shoulder, or back may be changed from that in previous mounting due to variations in the body shapes of wearers, the deviations of mounting locations, or the like. Therefore, it is necessary to adjust the photographing direction of the camera while confirming an image captured by the camera with a monitor after the camera has been mounted.
- An object of the disclosed subject matter is to provide a technology for solving the problems described above.
- a direction control device includes: camera means for acquiring an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction; image-processing means for calculating a position of a first setting image that represents the subject in the captured image; posture detection means for detecting a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and camera control means for shifting the photographing direction of the camera based on the difference.
- a direction control method includes: acquiring an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction; calculating a position of a first setting image that represents the subject in the captured image; detecting a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and shifting the photographing direction of the camera based on the difference.
- a recording medium storing a direction control program that causes a computer to execute: acquiring an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction; calculating a position of a first setting image that represents the subject in the captured image; detecting a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and shifting the photographing direction of the camera based on the difference.
- the photographing direction of a camera can be set to a predetermined direction when a terminal with the camera is mounted on a wearer.
- FIG. 1 is a view illustrating an example of adjusting the photographing direction of a camera relating to a wearable terminal according to a first example embodiment.
- FIG. 2 is a block diagram illustrating a configuration of the wearable terminal according to the first example embodiment.
- FIG. 3 is a block diagram illustrating a configuration of an image-processing unit of the wearable terminal according to the first example embodiment.
- FIG. 4A is a view representing a position of a setting image 3 A in a captured image.
- FIG. 4B is a view representing a position of a setting image 3 B in a reference image.
- FIG. 5 is a flowchart illustrating operation of the wearable terminal according to the first example embodiment.
- FIG. 6 is a view illustrating an example of adjusting a photographing direction of a camera relating to a wearable terminal according to a second example embodiment.
- FIG. 7 is a block diagram illustrating a configuration of a wearable terminal according to a second example embodiment.
- FIG. 8 is a view illustrating a position of a face image in a captured image.
- FIG. 9 is a flowchart illustrating operation of the wearable terminal according to the second example embodiment.
- FIG. 10 is a view illustrating an example of adjusting a photographing direction of a camera relating to a wearable terminal according to a third example embodiment.
- FIG. 11 is a block diagram illustrating a configuration of a wearable terminal according to the third example embodiment.
- FIG. 12 is a flowchart illustrating operation of the wearable terminal according to the third example embodiment.
- FIG. 13 is a block diagram illustrating a configuration of a wearable terminal according to a fourth example embodiment.
- FIG. 14 is a flowchart illustrating operation of the wearable terminal according to the fourth example embodiment.
- FIG. 15A is a top schematic view illustrating an example of mounting a front camera unit and a back camera unit of a wearable terminal according to a fifth example embodiment.
- FIG. 15B is a front schematic view illustrating an example of mounting the front camera unit of the wearable terminal according to the fifth example embodiment.
- FIG. 15C is a back schematic view illustrating the example of mounting the front camera unit and the back camera unit of the wearable terminal according to the fifth example embodiment.
- FIG. 16 is a block diagram illustrating the configuration of the wearable terminal according to the fifth example embodiment.
- FIG. 17 is a block diagram illustrating a hardware configuration realized a processing unit or a control unit of the wearable terminal with a computer according to the first to fifth example embodiments.
- Example embodiments of a direction control device of the disclosed subject matter will be described in detail with reference to the drawings.
- an example in which the direction control device is applied to a wearable terminal is described below.
- the directions of arrows illustrated in the block diagrams in the drawings are examples, and the directions of the signals between the blocks are not limited thereto.
- FIG. 1 is a view illustrating an example of adjusting the photographing direction of the camera relating to a wearable terminal according to the first example embodiment.
- a predetermined subject 6 is first photographed by the camera (not illustrated) of the wearable terminal 10 . Then, the camera unit of the wearable terminal 10 acquires an image captured by photographing the subject 6 . In the captured image, the image portion of the subject 6 is referred to as a setting image.
- the setting image is used for setting the photographing direction of the camera to a predetermined direction.
- the image-processing unit of the wearable terminal 10 calculates the image position of a first setting image which represents the subject 6 in the acquired captured image.
- the posture detection unit of the wearable terminal 10 detects the difference between the image position of a second setting image which represents the subject 6 in a reference image registered in advance and the position of the first setting image.
- the reference image is an image captured when the photographing direction of the camera of the wearable terminal 10 mounted on a wearer 2 is a predetermined direction.
- Conditions for photographing the subject 6 are allowed to be similar between the captured image and the reference image.
- the position of the subject 6 photographed for acquiring the captured image and the reference image, the focal length of the lens of the camera, or the like is allowed to be the same.
- the photographing direction of the camera is controlled based on the detected difference.
- FIG. 2 is a block diagram illustrating the configuration of the wearable terminal 10 according to the first example embodiment.
- the wearable terminal 10 includes a camera unit 11 , an image-processing unit 12 , a posture detection unit 13 , and a camera control unit 14 .
- the camera unit 11 of the wearable terminal 10 acquires an image captured by photographing the predetermined subject 6 by the camera (not illustrated). Then, the camera unit 11 notifies the image-processing unit 12 of the captured image S 11 .
- a wearable terminal includes a camera (not illustrated) in each example embodiment.
- the camera unit 11 includes a shift unit (not illustrated) for changing the photographing direction of the camera.
- the image-processing unit 12 of the wearable terminal 10 generates verification image information S 12 based on the captured image S 11 and notifies the posture detection unit 13 of the verification image information S 12 . Details on the verification image information S 12 will be described later.
- FIG. 3 is a block diagram illustrating the configuration of the image-processing unit 12 of the wearable terminal 10 .
- the image-processing unit 12 includes a feature extraction unit 125 , a feature verification unit 126 , a database 123 , and a coordinate calculation unit 127 .
- the feature extraction unit 125 of the image-processing unit 12 extracts the feature amount of a setting image 3 A based on the input captured image S 11 .
- the feature extraction unit 125 notifies the coordinate calculation unit 127 of the input captured image S 11 .
- SIFT Scale-Invariant Feature Transform
- a setting image 3 B in the reference image and the feature amount of the setting image 3 B are registered in advance.
- the reference image is an image captured when the photographing direction of the camera of the wearable terminal 10 mounted on a wearer is a predetermined direction.
- position information that represents the image position of the setting image 3 B in the reference image (hereinafter expressed as the position information of the setting image 3 B) is also registered in advance.
- An example of the position information of the setting image 3 B is the coordinate information of the setting image 3 B in the reference image.
- the feature verification unit 126 of the image-processing unit 12 verifies the feature amount of the setting image 3 A in the captured image extracted by the feature extraction unit 125 against the feature amount of the setting image 3 B registered in the database 123 .
- the feature verification unit 126 notifies the coordinate calculation unit 127 of the position information on the setting image 3 B in the reference image registered in the database 123 when the setting image 3 A in the captured image and the setting image 3 B in the reference image match with each other as a result of the verification.
- the coordinate calculation unit 127 of the image-processing unit 12 calculates position information that represents the image position of the setting image 3 A in the captured image (hereinafter expressing the position information of the setting image 3 A) based on the captured image S 11 notified from the feature extraction unit 125 .
- An example of the position information of the setting image 3 A calculated by the coordinate calculation unit 127 is coordinate information.
- the coordinate calculation unit 127 notifies the posture detection unit 13 of the position information of the setting image 3 A in the captured image and the position information of the setting image 3 B in the reference image registered in the database 123 , as the verification image information S 12 .
- the posture detection unit 13 generates posture difference information S 13 that represents the difference between the position of the setting image 3 A in the captured image and the position of the setting image 3 B in the reference image, based on the notified verification image information S 12 , and notifies the camera control unit 14 of the posture difference information S 13 . Details on the posture difference information S 13 will be described later.
- FIG. 4A is a view representing the position of the setting image 3 A in the captured image
- FIG. 4B is a view representing the position of the setting image 3 B in the reference image.
- Each of the setting image 3 A in the captured image and the setting image 3 B in the reference image includes an image as a reference point.
- Such images as reference points are included in the setting images 3 A and 3 B, for example, by using an object with a predetermined color or shape as a subject. An image in which the color or shape of part of a subject is a reference point is also acceptable.
- Each of the position of the reference point of the setting image 3 A in the captured image illustrated in FIG. 4A and the position of the reference point of the setting image 3 B in the reference image illustrated in FIG. 4B is represented as a point of intersection of a horizontal line (continuous line) and a vertical line (continuous line).
- a point of intersection of a horizontal line (broken line) and a vertical line (broken line) in FIG. 4A indicates the position of the reference point of the setting image 3 B.
- the position of the reference point of the setting image 3 A in the captured image shifts horizontally left over a distance V 1 and vertically downward over a distance V 2 compared with the position of the reference point of the setting image 3 B in the reference image.
- the posture detection unit 13 generates the posture difference information S 13 based on the position information of the reference point of the setting image 3 B and the position information of the reference point of the setting image 3 A.
- the posture difference information S 13 is, for example, the horizontal shift distance V 1 and vertical shift distance V 2 of the reference point.
- the horizontal shift distance V 1 and the vertical shift distance V 2 can be determined from the coordinate information of the reference point of the setting image 3 A and the coordinate information of the reference point of the setting image 3 B.
- the camera control unit 14 Based on the posture difference information S 13 , the camera control unit 14 generates shift amount information such that the position of the reference point of the setting image 3 A in the captured image approaches the position of the reference point of the setting image 3 B in the reference image. Then, the camera control unit 14 notifies the shift unit (not illustrated) of the camera unit 11 of the shift amount information S 14 to control the photographing direction of the camera.
- FIG. 5 is a flowchart illustrating the operation of the wearable terminal 10 according to the first example embodiment.
- the camera unit 11 of the wearable terminal 10 acquires an image captured by photographing the predetermined subject 6 by the camera (step P 1 ) and notifies the image-processing unit 12 of the captured image S 11 . Then, the image-processing unit 12 of the wearable terminal 10 extracts the setting image 3 A from the captured image S 11 , of which the notification has been provided, and calculates the position of the setting image 3 A in the captured image (step P 2 ).
- the feature extraction unit 125 of the image-processing unit 12 extracts the feature amount of the setting image 3 A from the captured image, and the feature verification unit 126 verifies the feature amount of the setting image 3 B in the reference image registered in the database 123 in advance against the extracted feature amount of the setting image 3 A.
- the feature verification unit 126 notifies the coordinate calculation unit 127 of the verification results together with the position information of the setting image 3 B in the reference image registered in the database 123 in advance.
- the coordinate calculation unit 127 calculates the position of the setting image 3 A in the captured image and notifies the posture detection unit 13 of the position, together with the position information of the setting image 3 B in the reference image, as the verification image information S 12 .
- the posture detection unit 13 detects the difference between the position of the setting image 3 A in the captured image and the position of the setting image 3 B in the reference image (step P 3 ).
- the posture detection unit 13 notifies the camera control unit 14 of the difference information between the positions of the setting image 3 A and the setting image 3 B as the posture difference information S 13 .
- the camera control unit 14 Based on the posture difference information S 13 , the camera control unit 14 generates the shift amount information S 14 for a state in which the position of the reference point of the setting image 3 A in the captured image approaches the position of the reference point of the setting image 3 B in the reference image.
- the camera control unit 14 shifts the azimuth angle of the photographing direction of the camera to the left.
- the distance V 3 between the camera and the reference point of the reference image is the distance of a photographing position at the time of capturing the reference image.
- the distance V 3 is acquired together with the position information of the reference image from the database 123 .
- the camera control unit 14 shifts the elevation angle of the photographing direction of the camera downward.
- the camera control unit 14 notifies the shift unit of the camera unit 11 of the shift amount information S 14 to control the photographing direction of the camera (step P 5 ).
- the difference is absent (No in step P 4 )
- the operation is ended.
- the example in which the feature extraction unit 125 of the image-processing unit 12 notifies the coordinate calculation unit 127 of the captured image S 11 is described.
- the first example embodiment is not limited thereto.
- two of the feature extraction unit 125 and coordinate calculation unit 127 of the image-processing unit 12 may also be notified of the captured image S 11 from the camera unit 11 . In this case, the need for notifying the coordinate calculation unit 127 of the captured image S 11 from the feature extraction unit 125 of the image-processing unit 12 is eliminated.
- the wearable terminal 10 of the first example embodiment enables the photographing direction of the camera in the camera unit to be set to a predetermined direction when the wearable terminal 10 is worn.
- the wearable terminal 10 of the first example embodiment photographs the subject 6 by the camera, calculates the differences (horizontal direction: V 1 , vertical direction: V 2 ) between the position of the setting image 3 A in the captured image and the position of the setting image 3 B in the reference image, and further shifts the photographing direction of the camera based on the calculated difference information such that the position of the setting image 3 A in the captured image approaches the position of the setting image 3 B in the reference image.
- FIG. 6 is a view illustrating an example in which the photographing direction of the camera of a wearable terminal according to the second example embodiment is adjusted.
- a wearer 2 stands in front of a mirror 4 and photographs a mirror image 2 ′ of the wearer 2 in the mirror 4 , by using a wearable terminal 20 mounted on the wearer 2 in order to allow the wearer 2 with the wearable terminal 20 to be a subject.
- a face image of the mirror image 2 ′ of the wearer 2 is used as a setting image in a captured image.
- Conditions for photographing the mirror image 2 ′ are allowed to be similar between the captured image and a reference image.
- the position of a subject 6 photographed for acquiring the captured image and the reference image, the focal length of the lens of the camera, or the like is allowed to be the same.
- FIG. 7 is a block diagram illustrating the configuration of the wearable terminal 20 according to the second example embodiment.
- the wearable terminal 20 includes a camera unit 11 , an image-processing unit 22 , a posture detection unit 23 , and a camera control unit 14 .
- the same configurations as those of the wearable terminal 10 of the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted.
- the mirror image 2 ′ of the wearer 2 is photographed by the camera of the wearable terminal 20 targeted for adjustment of the photographing direction thereof.
- the camera unit 11 of the wearable terminal 20 acquires the captured image and notifies the image-processing unit 22 of the image of the mirror image 2 ′ as the captured image S 11 .
- a face detection unit 121 in the image-processing unit 22 detects the face image of the wearer 2 from the captured image S 11 .
- the face detection unit 121 notifies a face verification unit 122 of the detected face image.
- a known technology for detecting a face area can be applied to the detection of the face image.
- the face detection unit 121 notifies a face coordinate calculation unit 124 of the captured image S 11 .
- the face verification unit 122 of the image-processing unit 22 verifies a face image of the wearer 2 , registered in a database 123 A in advance, against the face image detected by the face detection unit 121 .
- the feature amount of the face image is extracted using SIFT to verify the face image, for example, as described in the first example embodiment.
- the extraction of the feature amount of the face image in the captured image is performed in the face detection unit 121 or the face verification unit 122 .
- the feature amount data of the face image is registered in the database 123 A in the case of the verification using the feature amount of the face image.
- the face image detected by the face detection is reversed horizontally with respect to an actual face image of the wearer 2 because of being an image captured by photographing the mirror image 2 ′, reflected by the mirror 4 , by the camera. Therefore, the face image of the wearer 2 , registered in the database 123 A of the image-processing unit 22 , is also regarded as the face image of the mirror image 2 ′ of the wearer 2 .
- Conditions for photographing the mirror image 2 ′ of the wearer 2 are allowed to be similar between photographing for adjusting the photographing direction of the camera and photographing for registration in the database.
- the photographing position of the mirror image 2 ′, the focal length of the lens of the camera, or the like is allowed to be the same.
- the position information of the face image of the mirror image 2 ′ of the wearer 2 is registered in the database 123 A of the image-processing unit 22 .
- the face image of the mirror image 2 ′ of the wearer 2 is a face image in an image (hereinafter expressed as a reference image) captured when the photographing direction of the camera is a predetermined direction.
- a reference image an image captured when the photographing direction of the camera is a predetermined direction.
- a rectangle is formed around the face of the wearer 2 in the face image, and the coordinates of the corners of the rectangular are regarded as the position information of the face image.
- Figures formed around the face of the wearer 2 in order to specify the position information of the face image may have shapes of a circle or a polygon in addition to the rectangle.
- identification information for identifying the wearer 2 and wearer information including the height data of the wearer 2 are registered in the database 123 A.
- the identification information is, for example, an arbitrary character string assigned to each wearer.
- the face verification unit 122 sends the position information of the face image in the reference image and the wearer information to the face coordinate calculation unit 124 .
- the face coordinate calculation unit 124 of the image-processing unit 22 calculates the position information of the face image in the captured image of which the notification has been provided from the face detection unit 121 .
- the position information of the face image in the captured image a rectangle is formed around the face of the wearer 2 , and the coordinates of the corners of the rectangle are regarded as the position information of the face image, like the position information of the face image in the above-described reference image.
- FIG. 8 is a view illustrating the position of the face image in the captured image of the mirror image 2 ′.
- the position of the face image V in the captured image is defined by the distance (horizontal direction: V 4 , vertical direction: V 5 ) between the left or top edge of the captured image and the rectangular image based on the coordinates of the rectangle formed around the face image.
- the face coordinate calculation unit 124 sends verification image information S 22 including the position information of the face image in the captured image, the position information of the face image in the reference image, and the wearer information to the posture detection unit 23 .
- the posture detection unit 23 generates posture difference information S 13 that represents the difference between the position of the face image in the captured image and the position of the face image in the reference image based on the position information of the face image in the captured image and the position information of the face image in the reference image.
- the posture detection unit 23 includes the function of generating correction information that is reflective of camera position information associated with the height of the wearer 2 , in addition to the function of generating the posture difference information S 13 that represents the difference between the positions of the face images in the captured image and the reference image.
- the photographing direction of the camera based on the posture difference information S 13 becomes a direction toward the face of the wearer 2 , reflected by the mirror 4 , when the photographing direction of the camera is adjusted using the face image of the wearer 2 , reflected by the mirror 4 , like the second example embodiment, as illustrated in FIG. 6 .
- the position of the camera mounted on the wearer 2 is at a height of 180 cm, and therefore, it is difficult to allow a subject closer to the feet to be included in the captured image.
- the height of the wearer 2 is 140 cm
- the position of the camera is at a height of 120 cm, and it is difficult to allow an upper part of the subject to be included in the captured image. Therefore, it is necessary to make a correction for the elevation angle of the photographing direction of the camera, reflective of the height of the wearer 2 on which the wearable terminal 20 is mounted.
- a camera posture calculation unit 131 in the posture detection unit 23 calculates correction information on the photographing direction of the camera, reflective of the height of the wearer 2 , based on the camera position information registered in a camera posture database 132 in the posture detection unit 23 . Then, the camera posture calculation unit 131 adds the correction information to the posture difference information S 13 and notifies the camera control unit 14 of thus obtained posture difference information.
- As the camera position information camera position information corresponding to the wearer is read from the camera posture database 132 by using the identification information included in the wearer information.
- the camera control unit 14 controls the photographing direction of the camera based on: the posture difference information S 13 that is calculated in the posture detection unit 23 and that represents the difference between the positions of the face images in the captured image and the reference image; and the correction information based on the camera position information.
- FIG. 9 is a flowchart illustrating the operation of the wearable terminal 20 according to the second example embodiment.
- the camera unit 11 of the wearable terminal 20 acquires the image captured by photographing the mirror image 2 ′ of the wearer 2 , reflected by the mirror 4 , by the camera (step P 11 ) and notifies the image-processing unit 22 of the captured image S 11 . Then, the image-processing unit 22 of the wearable terminal 20 detects the face image of the wearer 2 from the captured image S 11 of which the notification has been provided (step P 12 ) and calculates the position of the face image in the captured image.
- the face detection unit 121 of the image-processing unit 12 detects the face image from the captured image (step P 12 ), and the face verification unit 122 verifies the face image stored in the database 123 A in advance against the extracted face image (step P 13 ).
- the process of the face verification unit 122 returns to step P 11 when the face images do not match with each other (No in step P 14 ).
- the face verification unit 122 sends the position information of the face image in the reference image and the wearer information, registered in the database 123 A, to the face coordinate calculation unit 124 .
- the face coordinate calculation unit 124 calculates the position of the face image in the captured image (step P 15 ) and sends the wearer information, together with the position information of the face image in the reference image, to the posture detection unit 23 .
- the posture detection unit 23 confirms whether the difference between the positions of the face image in the captured image and the face image in the reference image is present (step P 16 ).
- the posture difference information is sent to the camera control unit 14 .
- the camera control unit 14 gives an instruction to a shift unit (not illustrated) of the camera unit 11 to control the photographing direction of the camera (step P 17 ).
- the wearable terminal 20 repeats step P 11 to step P 17 until the difference between the positions of the face image in the captured image and the face image in the reference image becomes absent.
- the photographing direction of the camera is controlled based on the correction information (step P 18 ).
- the second example embodiment the example in which the face detection unit 121 of the image-processing unit 22 notifies the face coordinate calculation unit 124 of the captured image S 11 is described.
- the second example embodiment is not limited thereto.
- two of the face detection unit 121 and face coordinate calculation unit 124 of the image-processing unit 22 may also be notified of the captured image S 11 from the camera unit 11 . In this case, the need for notifying the face coordinate calculation unit 124 of the captured image S 11 from the face detection unit 121 of the image-processing unit 22 is eliminated.
- the wearable terminal 20 of the second example embodiment enables the photographing direction of the camera in the camera unit to be set to a predetermined direction, like the first example embodiment.
- the reason for this is because the wearable terminal 20 of the second example embodiment photographs the mirror image 2 ′ of the wearer 2 by the camera, calculates the difference (horizontal direction: V 4 , vertical direction: V 5 ) between the position of the face image in the captured image and the position of the face image in the reference image, and further shifts the photographing direction of the camera based on the calculated difference information such that the position of the face image in the captured image approaches the position of the face image in the reference image.
- the wearable terminal of the second example embodiment enables the photographing direction of the camera to be controlled based on the camera position information associated with the height of the wearer 2 . Because the data of the wearer 2 is read from the database 123 A when the face images match with each other as a result of the verification of the face images, the protection of personal information is also facilitated.
- the third example embodiment is an example in which a two-dimensional bar code is used for a subject for adjustment of the photographing direction of a camera.
- the same configurations as those of the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted.
- FIG. 10 is a view illustrating an example in which the photographing direction of the camera of a wearable terminal according to the third example embodiment is adjusted.
- the camera (not illustrated) of the wearable terminal 30 mounted on a wearer 2 is a camera targeted for adjustment of the photographing direction thereof.
- the two-dimensional bar code 5 is photographed by the camera of the wearable terminal 30 .
- FIG. 11 is a block diagram illustrating the configuration of the wearable terminal 30 according to the third example embodiment.
- the wearable terminal 30 includes a camera unit 11 , an image-processing unit 32 , a posture detection unit 13 , and a camera control unit 14 .
- the camera unit 11 of the wearable terminal 30 acquires an image captured by photographing the two-dimensional bar code 5 and notifies the image-processing unit 32 of the captured image S 11 .
- the image-processing unit 32 of the wearable terminal 30 generates image information S 32 from the input captured image S 11 and notifies the posture detection unit 13 of the image information S 32 .
- the image-processing unit 32 includes information extraction unit 128 and coordinate calculation unit 129 .
- the information extraction unit 128 extracts bar code information from an image of the two-dimensional bar code 5 in the input captured image S 11 .
- the bar code information of the two-dimensional bar code 5 includes the information of the size of the two-dimensional bar code 5 and the installation position (height) of the two-dimensional bar code 5 .
- the coordinate calculation unit 129 calculates the position information of the two-dimensional bar code in the captured image and notifies the posture detection unit 13 of the position information, together with the bar code information, as the image information S 32 .
- the position information of the two-dimensional bar code can be defined by the coordinates of the image of the two-dimensional bar code in the captured image and by the respective distances (horizontal direction and vertical direction) between the left and top edges of the captured image and the left and top edges of the image of the two-dimensional bar code.
- the posture detection unit 13 of the wearable terminal 30 generates posture difference information S 13 from the image position of the two-dimensional bar code in the captured image and the bar code information based on the input image information S 32 and notifies the camera control unit 14 of the posture difference information S 13 .
- the camera control unit 14 of the wearable terminal 30 instructs the camera unit 11 such that the position of a setting image 3 A in the captured image approaches the position of a setting image 3 B in a reference image to control the photographing direction of the camera.
- FIG. 12 is a flowchart illustrating the operation of the wearable terminal 30 according to the third example embodiment.
- the same operations of the third example embodiment as those of the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted.
- the image-processing unit 32 of the wearable terminal 30 acquires the captured image of the two-dimensional bar code (step P 21 ).
- the information extraction unit 128 detects the two-dimensional bar code 5 from the input captured image S 11 (step P 22 ) and extracts the bar code information of the two-dimensional bar code 5 (step P 23 ).
- the bar code information includes the information of the size of the two-dimensional bar code and the installation height of the two-dimensional bar code.
- the coordinate calculation unit 129 calculates the position of the two-dimensional bar code image from the two-dimensional bar code image in the captured image S 11 (step P 24 ).
- the coordinate calculation unit 129 notifies the posture detection unit 13 of the calculated position information of the two-dimensional bar code image.
- a camera posture calculation unit 131 in the posture detection unit 13 calculates posture difference information from the position information of the two-dimensional bar code image as well as the size of the two-dimensional bar code and the installation height of the two-dimensional bar code, included in the bar code information.
- the camera control unit 14 controls a camera shift amount for instructing a shift unit (not illustrated) in the camera unit 11 to eliminate a direction deviation to control the photographing direction of the camera of the camera unit 11 (step P 26 ).
- the wearable terminal 30 enables the photographing direction of the camera in the camera unit to be set to a predetermined direction, like the third example embodiment.
- the information of the installation height or size of the two-dimensional bar code can be obtained from the photographed two-dimensional bar code, and therefore, the storage capacity of the database can be reduced.
- FIG. 13 is a block diagram illustrating the configuration of a wearable terminal 40 according to the fourth example embodiment.
- the wearable terminal 40 includes a camera unit 11 , an image-processing unit 12 , a posture detection unit 13 , a camera control unit 14 , and a sensor unit 15 .
- the sensor unit 15 of the wearable terminal 40 is mounted on the camera unit 11 and has the function of storing the photographing direction of a camera after the photographing direction of a camera has been controlled by the camera unit 11 .
- the sensor unit 15 includes a triaxial gyro sensor or a 6-axis sensor.
- a direction toward which the gyro sensor is directed can be registered as the adjusted photographing direction of the camera by determining the photographing direction of the camera by the camera control unit 14 and by then activating the gyro function of the sensor unit 15 .
- the wearable terminal 40 can correct the photographing direction of the camera without re-capturing a predetermined image by using the direction information of the gyro sensor of the sensor unit 15 .
- the sensor unit 15 of the fourth example embodiment can be applied to all of the first example embodiment to the third example embodiment.
- FIG. 14 is a flowchart illustrating the operation of the wearable terminal 40 according to the fourth example embodiment.
- the descriptions of the same operations as those of the first example embodiment are omitted.
- the posture detection unit 13 confirms whether a difference is present between the positions of a setting image 3 A in a captured image and a setting image 3 B in a reference image.
- the posture detection unit 13 calculates posture difference information S 13 and notifies the camera control unit 14 of the posture difference information S 13 .
- the camera control unit 14 gives an instruction (shift amount information S 14 ) to the camera unit 11 (step P 35 ) to control the photographing direction of the camera.
- step P 34 When the difference between the positions is absent (No in step P 34 ), the gyro function of the sensor unit 15 is activated, and the photographing direction of the camera in a state in which the difference between the positions is absent is stored as the initial direction of the gyro sensor (step P 36 ).
- the sensor unit 15 After the adjustment of the photographing direction of the camera by the posture detection unit 13 , the sensor unit 15 confirms whether the difference between the photographing direction of the camera and the initial direction of the gyro sensor is present (step P 37 ). When the difference between the directions is present (Yes in step P 37 ), the sensor unit 15 calculates direction difference information (angular difference) between the photographing direction and the initial direction and notifies the camera control unit 14 of the direction difference information. Based on the direction difference information, the camera control unit 14 generates the shift amount information S 14 to eliminate the direction difference and gives an instruction to the camera unit 11 to re-control the photographing direction of the camera (step P 38 ).
- direction difference information angular difference
- the wearable terminal 40 according to the fourth example embodiment enables the photographing direction of the camera in the camera unit to be set to a predetermined direction, like the first example embodiment.
- the wearable terminal 40 according to the fourth example embodiment enables the photographing direction of the camera to be corrected without re-capturing a predetermined image.
- FIG. 15A is a top schematic view illustrating an example in which the front camera unit and back camera unit of the wearable terminal are mounted.
- FIG. 15B is a front schematic view illustrating an example in which the front camera unit of the wearable terminal is mounted.
- FIG. 15C is a back schematic view illustrating an example in which the front camera unit and back camera unit of the wearable terminal are worn.
- the front camera unit 16 of the wearable terminal is mounted on the right shoulder of a wearer 2
- the back camera unit 17 is mounted on the upper portion of the back.
- the photographing direction of the camera of the front camera unit 16 is a direction in which the wearer 2 photographs from the front
- the photographing direction of the camera of the back camera unit 17 is a direction in which the wearer 2 photographs from the back.
- the photographing directions of the cameras of the front camera unit 16 and the back camera unit 17 are opposite in direction to each other.
- the elevation angles of the photographing direction of the camera of the front camera unit 16 and the photographing direction of the camera of the back camera unit 17 are respectively symmetric with respect to the horizontal plane.
- FIG. 16 is a block diagram illustrating the configuration of the wearable terminal according to the fifth example embodiment.
- the same configurations as those of the wearable terminal 10 according to the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted as appropriate.
- the wearable terminal 50 includes the front camera unit 16 , an image-processing unit 12 , a posture detection unit 13 , a camera control unit 18 , and the back camera unit 17 .
- the back camera unit 17 includes a camera (not illustrated) and a shift unit (not illustrated) for changing the photographing direction of the camera.
- the front camera unit 16 of the wearable terminal 50 of the fifth example embodiment acquires an image captured by photographing a predetermined subject by the camera (not illustrated). Then, the front camera unit 16 notifies the image-processing unit 12 of the captured image S 11 .
- the image-processing unit 12 of the wearable terminal 50 generates verification image information S 12 based on the captured image S 11 and notifies the posture detection unit 13 of the verification image information S 12 .
- the configuration and operation of the posture detection unit 13 are the same as those of the wearable terminal 10 according to the first example embodiment, and therefore, the detailed descriptions thereof are omitted.
- the posture detection unit 13 sends generated posture difference information S 13 to the camera control unit 18 .
- the camera control unit 18 of the wearable terminal 50 gives an instruction to the front camera unit 16 to control the photographing direction of the camera. Further, the camera control unit 18 instructs the back camera unit 17 to be set to be opposite in direction to the photographing direction of the front camera unit 16 to control the photographing direction of the camera. The back camera unit 17 may be controlled simultaneously with or after the control of the front camera unit 16 .
- the photographing direction of a camera in another camera unit can be easily adjusted according to the fifth example embodiment, as described above.
- the reason for this is because the adjustment of the photographing direction of the other camera is allowed to be reflective of a symmetry property between the photographing direction of the camera of the front camera unit 16 and the photographing direction of the camera of the back camera unit 17 .
- the wearable terminal 50 enables the photographing direction of the camera in the camera unit to be set to the predetermined direction, like the first example embodiment.
- the wearable terminal according to the fifth example embodiment enables the photographing direction of the camera of the back camera unit 17 to be adjusted by giving an instruction that the photographing direction of the camera of the back camera unit 17 is set to be opposite in direction to the photographing direction of the front camera unit 16 to control the back camera unit 17 .
- FIG. 17 is a view illustrating a hardware configuration in which each control unit or each processing unit of the wearable terminals 10 , 20 , 30 , 40 , and 50 according to the first to fifth example embodiments is implemented by a computer device.
- each control unit or each processing unit of the wearable terminals 10 , 20 , 30 , 40 , and 50 includes a CPU (Central Processing Unit) 901 and a communication I/F (communication interface) 902 for network connection.
- Each control unit or each processing unit of the wearable terminals 10 , 20 , 30 , 40 , and 50 further includes a memory 903 and a storage device 904 such as a hard disk in which a program is stored.
- the CPU 901 is connected to an input device 905 and an output device 906 via a system bus 907 .
- the CPU 901 runs an operating system to control the wearable terminals according to the first to fifth example embodiments.
- the CPU 901 read, for example, programs and data from a recording medium mounted in a drive device to the memory 903 .
- the CPU 901 has, for example, the function of processing an information signal input from each function unit in each example embodiment and executes processing of various functions based on the programs.
- the storage device 904 is, for example, an optical disk, a flexible disk, a magneto-optical disk, an external hard disk, a semiconductor memory, or the like.
- a storage medium in part of the storage device 904 is a non-volatile storage device, in which the programs are stored. The programs may also be downloaded from an external computer that is connected to a communication network and is not illustrated.
- the input device 905 is implemented by, for example, a mouse, a keyboard, a touch panel, or the like, and is used for input manipulation.
- the output device 906 is implemented by, for example, a display, and is used to output and confirm information or the like processed by the CPU 901 .
- each example embodiment is implemented by the hardware configuration illustrated in FIG. 17 .
- each implementation unit included in the wearable terminals 10 , 20 , 30 , 40 , and 50 is not particularly limited.
- the wearable terminals may be implemented by one device obtained by physical linking or may be implemented by plural devices which are two or more physically separated devices that are wired or wirelessly connected.
Abstract
A direction control device includes: a camera unit that acquires an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction; an image-processing unit that calculates a position of a first setting image that represents the subject in the captured image; a posture detection unit that detects a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and a camera control unit that shifts the photographing direction of the camera based on the difference.
Description
- The disclosed subject matter relates to a direction control device and the like. In particular, the disclosed subject matter relates to the control of the photographing direction of a camera.
- Providing reduction in size and weight of cameras, various wearable cameras capable of photographing situations in hands-free manners have come. The wearable cameras include helmet-mounted wearable cameras, glasses-mounted wearable cameras, and additionally badge-mounted wearable cameras incorporated into recorders which are mounted on the chests of photographers.
- With regard to a data-processing device that manages the positions of plural cameras,
PTL 1 describes a method for managing the position of each camera by recognizing image data captured by each camera. -
PTL 2 describes a method for determining a self-position by using a plane mirror in order to determine the direction of a robot. - [PTL 1] International Publication No. WO 2008/087974
- With regard to the photographing direction of the camera when mounting the wearable camera, a helmet-mounted or glasses-mounted wearable camera may eliminate adjusting the photographing direction of the camera in case of re-mounting the wearable camera after adjusting the photographing direction of the camera.
- In contrast, the photographing direction of a badge-mounted wearable camera which is mounted on the chest, shoulder, or back may be changed from that in previous mounting due to variations in the body shapes of wearers, the deviations of mounting locations, or the like. Therefore, it is necessary to adjust the photographing direction of the camera while confirming an image captured by the camera with a monitor after the camera has been mounted.
- Therefore, to provide a technology for capable of setting the photographing direction of a camera to a predetermined direction when a terminal with the camera is mounted on a wearer.
- An object of the disclosed subject matter is to provide a technology for solving the problems described above.
- A direction control device according to one aspect of the disclosed subject matter includes: camera means for acquiring an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction; image-processing means for calculating a position of a first setting image that represents the subject in the captured image; posture detection means for detecting a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and camera control means for shifting the photographing direction of the camera based on the difference.
- A direction control method according to one aspect of the disclosed subject matter includes: acquiring an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction; calculating a position of a first setting image that represents the subject in the captured image; detecting a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and shifting the photographing direction of the camera based on the difference.
- A recording medium according to one aspect of the disclosed subject matter storing a direction control program that causes a computer to execute: acquiring an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction; calculating a position of a first setting image that represents the subject in the captured image; detecting a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and shifting the photographing direction of the camera based on the difference.
- In the disclosed subject matter, the photographing direction of a camera can be set to a predetermined direction when a terminal with the camera is mounted on a wearer.
-
FIG. 1 is a view illustrating an example of adjusting the photographing direction of a camera relating to a wearable terminal according to a first example embodiment. -
FIG. 2 is a block diagram illustrating a configuration of the wearable terminal according to the first example embodiment. -
FIG. 3 is a block diagram illustrating a configuration of an image-processing unit of the wearable terminal according to the first example embodiment. -
FIG. 4A is a view representing a position of asetting image 3A in a captured image. -
FIG. 4B is a view representing a position of asetting image 3B in a reference image. -
FIG. 5 is a flowchart illustrating operation of the wearable terminal according to the first example embodiment. -
FIG. 6 is a view illustrating an example of adjusting a photographing direction of a camera relating to a wearable terminal according to a second example embodiment. -
FIG. 7 is a block diagram illustrating a configuration of a wearable terminal according to a second example embodiment. -
FIG. 8 is a view illustrating a position of a face image in a captured image. -
FIG. 9 is a flowchart illustrating operation of the wearable terminal according to the second example embodiment. -
FIG. 10 is a view illustrating an example of adjusting a photographing direction of a camera relating to a wearable terminal according to a third example embodiment. -
FIG. 11 is a block diagram illustrating a configuration of a wearable terminal according to the third example embodiment. -
FIG. 12 is a flowchart illustrating operation of the wearable terminal according to the third example embodiment. -
FIG. 13 is a block diagram illustrating a configuration of a wearable terminal according to a fourth example embodiment. -
FIG. 14 is a flowchart illustrating operation of the wearable terminal according to the fourth example embodiment. -
FIG. 15A is a top schematic view illustrating an example of mounting a front camera unit and a back camera unit of a wearable terminal according to a fifth example embodiment. -
FIG. 15B is a front schematic view illustrating an example of mounting the front camera unit of the wearable terminal according to the fifth example embodiment. -
FIG. 15C is a back schematic view illustrating the example of mounting the front camera unit and the back camera unit of the wearable terminal according to the fifth example embodiment. -
FIG. 16 is a block diagram illustrating the configuration of the wearable terminal according to the fifth example embodiment. -
FIG. 17 is a block diagram illustrating a hardware configuration realized a processing unit or a control unit of the wearable terminal with a computer according to the first to fifth example embodiments. - Example embodiments of a direction control device of the disclosed subject matter will be described in detail with reference to the drawings. In each example embodiment, an example in which the direction control device is applied to a wearable terminal is described below. The directions of arrows illustrated in the block diagrams in the drawings are examples, and the directions of the signals between the blocks are not limited thereto.
- Adjustment of the photographing direction of the camera of a wearable terminal will be described with reference to the drawings.
FIG. 1 is a view illustrating an example of adjusting the photographing direction of the camera relating to a wearable terminal according to the first example embodiment. - For adjusting the photographing direction, a
predetermined subject 6 is first photographed by the camera (not illustrated) of thewearable terminal 10. Then, the camera unit of thewearable terminal 10 acquires an image captured by photographing thesubject 6. In the captured image, the image portion of thesubject 6 is referred to as a setting image. The setting image is used for setting the photographing direction of the camera to a predetermined direction. - The image-processing unit of the
wearable terminal 10 calculates the image position of a first setting image which represents thesubject 6 in the acquired captured image. The posture detection unit of thewearable terminal 10 detects the difference between the image position of a second setting image which represents thesubject 6 in a reference image registered in advance and the position of the first setting image. - The reference image is an image captured when the photographing direction of the camera of the
wearable terminal 10 mounted on awearer 2 is a predetermined direction. - Conditions for photographing the
subject 6 are allowed to be similar between the captured image and the reference image. For example, the position of thesubject 6 photographed for acquiring the captured image and the reference image, the focal length of the lens of the camera, or the like is allowed to be the same. - In the control of the camera of the
wearable terminal 10, the photographing direction of the camera is controlled based on the detected difference. - The configuration of the
wearable terminal 10 will be described in detail below with reference to the drawings.FIG. 2 is a block diagram illustrating the configuration of thewearable terminal 10 according to the first example embodiment. As illustrated inFIG. 2 , thewearable terminal 10 includes acamera unit 11, an image-processingunit 12, aposture detection unit 13, and acamera control unit 14. - The
camera unit 11 of thewearable terminal 10 acquires an image captured by photographing thepredetermined subject 6 by the camera (not illustrated). Then, thecamera unit 11 notifies the image-processingunit 12 of the captured image S11. Unless otherwise noted, a wearable terminal includes a camera (not illustrated) in each example embodiment. In addition, thecamera unit 11 includes a shift unit (not illustrated) for changing the photographing direction of the camera. - The image-processing
unit 12 of thewearable terminal 10 generates verification image information S12 based on the captured image S11 and notifies theposture detection unit 13 of the verification image information S12. Details on the verification image information S12 will be described later. -
FIG. 3 is a block diagram illustrating the configuration of the image-processingunit 12 of thewearable terminal 10. As illustrated inFIG. 3 , the image-processingunit 12 includes afeature extraction unit 125, afeature verification unit 126, adatabase 123, and a coordinatecalculation unit 127. - The
feature extraction unit 125 of the image-processingunit 12 extracts the feature amount of asetting image 3A based on the input captured image S11. Thefeature extraction unit 125 notifies the coordinatecalculation unit 127 of the input captured image S11. For example, SIFT (Scale-Invariant Feature Transform) can be used in the extraction of the feature amount. Not only the SIFT described above but also another technique may be used in the extraction of the feature amount. - In the
database 123 of the image-processingunit 12, a settingimage 3B in the reference image and the feature amount of the settingimage 3B are registered in advance. The reference image is an image captured when the photographing direction of the camera of thewearable terminal 10 mounted on a wearer is a predetermined direction. In thedatabase 123, position information that represents the image position of the settingimage 3B in the reference image (hereinafter expressed as the position information of the settingimage 3B) is also registered in advance. An example of the position information of the settingimage 3B is the coordinate information of the settingimage 3B in the reference image. - The
feature verification unit 126 of the image-processingunit 12 verifies the feature amount of thesetting image 3A in the captured image extracted by thefeature extraction unit 125 against the feature amount of the settingimage 3B registered in thedatabase 123. Thefeature verification unit 126 notifies the coordinatecalculation unit 127 of the position information on thesetting image 3B in the reference image registered in thedatabase 123 when thesetting image 3A in the captured image and the settingimage 3B in the reference image match with each other as a result of the verification. - The coordinate
calculation unit 127 of the image-processingunit 12 calculates position information that represents the image position of thesetting image 3A in the captured image (hereinafter expressing the position information of thesetting image 3A) based on the captured image S11 notified from thefeature extraction unit 125. An example of the position information of thesetting image 3A calculated by the coordinatecalculation unit 127 is coordinate information. - Further, the coordinate
calculation unit 127 notifies theposture detection unit 13 of the position information of thesetting image 3A in the captured image and the position information of the settingimage 3B in the reference image registered in thedatabase 123, as the verification image information S12. - The
posture detection unit 13 generates posture difference information S13 that represents the difference between the position of thesetting image 3A in the captured image and the position of the settingimage 3B in the reference image, based on the notified verification image information S12, and notifies thecamera control unit 14 of the posture difference information S13. Details on the posture difference information S13 will be described later. -
FIG. 4A is a view representing the position of thesetting image 3A in the captured image, andFIG. 4B is a view representing the position of the settingimage 3B in the reference image. Each of thesetting image 3A in the captured image and the settingimage 3B in the reference image includes an image as a reference point. Such images as reference points are included in thesetting images - Each of the position of the reference point of the
setting image 3A in the captured image illustrated inFIG. 4A and the position of the reference point of the settingimage 3B in the reference image illustrated inFIG. 4B is represented as a point of intersection of a horizontal line (continuous line) and a vertical line (continuous line). In such a case, a point of intersection of a horizontal line (broken line) and a vertical line (broken line) inFIG. 4A indicates the position of the reference point of the settingimage 3B. - As illustrated in
FIG. 4A , the position of the reference point of thesetting image 3A in the captured image shifts horizontally left over a distance V1 and vertically downward over a distance V2 compared with the position of the reference point of the settingimage 3B in the reference image. - The
posture detection unit 13 generates the posture difference information S13 based on the position information of the reference point of the settingimage 3B and the position information of the reference point of thesetting image 3A. The posture difference information S13 is, for example, the horizontal shift distance V1 and vertical shift distance V2 of the reference point. The horizontal shift distance V1 and the vertical shift distance V2 can be determined from the coordinate information of the reference point of thesetting image 3A and the coordinate information of the reference point of the settingimage 3B. - Based on the posture difference information S13, the
camera control unit 14 generates shift amount information such that the position of the reference point of thesetting image 3A in the captured image approaches the position of the reference point of the settingimage 3B in the reference image. Then, thecamera control unit 14 notifies the shift unit (not illustrated) of thecamera unit 11 of the shift amount information S14 to control the photographing direction of the camera. - The operation of the
wearable terminal 10 according to the first example embodiment will now be described with reference to the drawings.FIG. 5 is a flowchart illustrating the operation of thewearable terminal 10 according to the first example embodiment. - First, the
camera unit 11 of thewearable terminal 10 acquires an image captured by photographing thepredetermined subject 6 by the camera (step P1) and notifies the image-processingunit 12 of the captured image S11. Then, the image-processingunit 12 of thewearable terminal 10 extracts thesetting image 3A from the captured image S11, of which the notification has been provided, and calculates the position of thesetting image 3A in the captured image (step P2). - Specifically, the
feature extraction unit 125 of the image-processingunit 12 extracts the feature amount of thesetting image 3A from the captured image, and thefeature verification unit 126 verifies the feature amount of the settingimage 3B in the reference image registered in thedatabase 123 in advance against the extracted feature amount of thesetting image 3A. When the features of the images match with each other, thefeature verification unit 126 notifies the coordinatecalculation unit 127 of the verification results together with the position information of the settingimage 3B in the reference image registered in thedatabase 123 in advance. The coordinatecalculation unit 127 calculates the position of thesetting image 3A in the captured image and notifies theposture detection unit 13 of the position, together with the position information of the settingimage 3B in the reference image, as the verification image information S12. - The
posture detection unit 13 detects the difference between the position of thesetting image 3A in the captured image and the position of the settingimage 3B in the reference image (step P3). When the difference is present (Yes in step P4), theposture detection unit 13 notifies thecamera control unit 14 of the difference information between the positions of thesetting image 3A and the settingimage 3B as the posture difference information S13. Based on the posture difference information S13, thecamera control unit 14 generates the shift amount information S14 for a state in which the position of the reference point of thesetting image 3A in the captured image approaches the position of the reference point of the settingimage 3B in the reference image. For example, when the position of the reference point of thesetting image 3A is on the left side of the position of the reference point of the settingimage 3B, thecamera control unit 14 shifts the azimuth angle of the photographing direction of the camera to the left. The shift amount of the azimuth angle in this case is calculated by, for example, tan θ1=V1/V3, from the horizontal shift distance V1 of the reference point included in the posture difference information S13 and a distance V3 (not illustrated) between the camera and the reference point of the reference image. The distance V3 between the camera and the reference point of the reference image is the distance of a photographing position at the time of capturing the reference image. The distance V3 is acquired together with the position information of the reference image from thedatabase 123. - When the position of the reference point of the
setting image 3A is on the downside of the position of the reference point of the settingimage 3B, thecamera control unit 14 shifts the elevation angle of the photographing direction of the camera downward. The shift amount of the elevation angle in this case is calculated by, for example, tan θ2=V2/V3, from the vertical shift distance V2 of the reference point included in the posture difference information S13 and the distance V3 (not illustrated) between the camera and the reference point of the reference image. Thecamera control unit 14 notifies the shift unit of thecamera unit 11 of the shift amount information S14 to control the photographing direction of the camera (step P5). When the difference is absent (No in step P4), the operation is ended. - In the first example embodiment, the example in which the
feature extraction unit 125 of the image-processingunit 12 notifies the coordinatecalculation unit 127 of the captured image S11 is described. However, the first example embodiment is not limited thereto. For example, two of thefeature extraction unit 125 and coordinatecalculation unit 127 of the image-processingunit 12 may also be notified of the captured image S11 from thecamera unit 11. In this case, the need for notifying the coordinatecalculation unit 127 of the captured image S11 from thefeature extraction unit 125 of the image-processingunit 12 is eliminated. - As above, the
wearable terminal 10 of the first example embodiment enables the photographing direction of the camera in the camera unit to be set to a predetermined direction when thewearable terminal 10 is worn. - The reason for this is because the
wearable terminal 10 of the first example embodiment photographs the subject 6 by the camera, calculates the differences (horizontal direction: V1, vertical direction: V2) between the position of thesetting image 3A in the captured image and the position of the settingimage 3B in the reference image, and further shifts the photographing direction of the camera based on the calculated difference information such that the position of thesetting image 3A in the captured image approaches the position of the settingimage 3B in the reference image. - A second example embodiment will now be described with reference to the drawings. The second example embodiment is an example in which a wearer is used as a subject for adjustment of the photographing direction of a camera.
FIG. 6 is a view illustrating an example in which the photographing direction of the camera of a wearable terminal according to the second example embodiment is adjusted. As illustrated inFIG. 6 , awearer 2 stands in front of amirror 4 and photographs amirror image 2′ of thewearer 2 in themirror 4, by using awearable terminal 20 mounted on thewearer 2 in order to allow thewearer 2 with thewearable terminal 20 to be a subject. In the second example embodiment, a face image of themirror image 2′ of thewearer 2 is used as a setting image in a captured image. Conditions for photographing themirror image 2′ are allowed to be similar between the captured image and a reference image. For example, the position of a subject 6 photographed for acquiring the captured image and the reference image, the focal length of the lens of the camera, or the like is allowed to be the same. -
FIG. 7 is a block diagram illustrating the configuration of thewearable terminal 20 according to the second example embodiment. As illustrated inFIG. 7 , thewearable terminal 20 includes acamera unit 11, an image-processingunit 22, aposture detection unit 23, and acamera control unit 14. In thewearable terminal 20 illustrated inFIG. 7 , the same configurations as those of thewearable terminal 10 of the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted. - First, the
mirror image 2′ of thewearer 2, reflected by themirror 4, is photographed by the camera of thewearable terminal 20 targeted for adjustment of the photographing direction thereof. Thecamera unit 11 of thewearable terminal 20 acquires the captured image and notifies the image-processingunit 22 of the image of themirror image 2′ as the captured image S11. - A
face detection unit 121 in the image-processingunit 22 detects the face image of thewearer 2 from the captured image S11. Theface detection unit 121 notifies aface verification unit 122 of the detected face image. A known technology for detecting a face area can be applied to the detection of the face image. In addition, theface detection unit 121 notifies a face coordinatecalculation unit 124 of the captured image S11. - The
face verification unit 122 of the image-processingunit 22 verifies a face image of thewearer 2, registered in adatabase 123A in advance, against the face image detected by theface detection unit 121. In the verification of the face image, the feature amount of the face image is extracted using SIFT to verify the face image, for example, as described in the first example embodiment. In this case, the extraction of the feature amount of the face image in the captured image is performed in theface detection unit 121 or theface verification unit 122. In addition to the face image of thewearer 2, registered in advance, the feature amount data of the face image is registered in thedatabase 123A in the case of the verification using the feature amount of the face image. - The face image detected by the face detection is reversed horizontally with respect to an actual face image of the
wearer 2 because of being an image captured by photographing themirror image 2′, reflected by themirror 4, by the camera. Therefore, the face image of thewearer 2, registered in thedatabase 123A of the image-processingunit 22, is also regarded as the face image of themirror image 2′ of thewearer 2. - Conditions for photographing the
mirror image 2′ of thewearer 2 are allowed to be similar between photographing for adjusting the photographing direction of the camera and photographing for registration in the database. For example, the photographing position of themirror image 2′, the focal length of the lens of the camera, or the like is allowed to be the same. - The position information of the face image of the
mirror image 2′ of thewearer 2 is registered in thedatabase 123A of the image-processingunit 22. The face image of themirror image 2′ of thewearer 2 is a face image in an image (hereinafter expressed as a reference image) captured when the photographing direction of the camera is a predetermined direction. With regard to the position information of the face image in the reference image, for example, a rectangle is formed around the face of thewearer 2 in the face image, and the coordinates of the corners of the rectangular are regarded as the position information of the face image. Figures formed around the face of thewearer 2 in order to specify the position information of the face image may have shapes of a circle or a polygon in addition to the rectangle. In addition, identification information for identifying thewearer 2 and wearer information including the height data of thewearer 2 are registered in thedatabase 123A. The identification information is, for example, an arbitrary character string assigned to each wearer. - When the detected face image and the face image registered in the
database 123A match with each other, theface verification unit 122 sends the position information of the face image in the reference image and the wearer information to the face coordinatecalculation unit 124. - Then, the face coordinate
calculation unit 124 of the image-processingunit 22 calculates the position information of the face image in the captured image of which the notification has been provided from theface detection unit 121. With regard to the position information of the face image in the captured image, a rectangle is formed around the face of thewearer 2, and the coordinates of the corners of the rectangle are regarded as the position information of the face image, like the position information of the face image in the above-described reference image. -
FIG. 8 is a view illustrating the position of the face image in the captured image of themirror image 2′. As illustrated inFIG. 8 , the position of the face image V in the captured image is defined by the distance (horizontal direction: V4, vertical direction: V5) between the left or top edge of the captured image and the rectangular image based on the coordinates of the rectangle formed around the face image. The face coordinatecalculation unit 124 sends verification image information S22 including the position information of the face image in the captured image, the position information of the face image in the reference image, and the wearer information to theposture detection unit 23. - The
posture detection unit 23 generates posture difference information S13 that represents the difference between the position of the face image in the captured image and the position of the face image in the reference image based on the position information of the face image in the captured image and the position information of the face image in the reference image. - Further, the
posture detection unit 23 includes the function of generating correction information that is reflective of camera position information associated with the height of thewearer 2, in addition to the function of generating the posture difference information S13 that represents the difference between the positions of the face images in the captured image and the reference image. - The photographing direction of the camera based on the posture difference information S13 becomes a direction toward the face of the
wearer 2, reflected by themirror 4, when the photographing direction of the camera is adjusted using the face image of thewearer 2, reflected by themirror 4, like the second example embodiment, as illustrated inFIG. 6 . For example, when the height of thewearer 2 is 200 cm, the position of the camera mounted on thewearer 2 is at a height of 180 cm, and therefore, it is difficult to allow a subject closer to the feet to be included in the captured image. In contrast, when the height of thewearer 2 is 140 cm, the position of the camera is at a height of 120 cm, and it is difficult to allow an upper part of the subject to be included in the captured image. Therefore, it is necessary to make a correction for the elevation angle of the photographing direction of the camera, reflective of the height of thewearer 2 on which thewearable terminal 20 is mounted. - A camera
posture calculation unit 131 in theposture detection unit 23 calculates correction information on the photographing direction of the camera, reflective of the height of thewearer 2, based on the camera position information registered in acamera posture database 132 in theposture detection unit 23. Then, the cameraposture calculation unit 131 adds the correction information to the posture difference information S13 and notifies thecamera control unit 14 of thus obtained posture difference information. As the camera position information, camera position information corresponding to the wearer is read from thecamera posture database 132 by using the identification information included in the wearer information. - The
camera control unit 14 controls the photographing direction of the camera based on: the posture difference information S13 that is calculated in theposture detection unit 23 and that represents the difference between the positions of the face images in the captured image and the reference image; and the correction information based on the camera position information. - The operation of the
wearable terminal 20 according to the second example embodiment will now be described with reference to the drawings.FIG. 9 is a flowchart illustrating the operation of thewearable terminal 20 according to the second example embodiment. - First, the
camera unit 11 of thewearable terminal 20 acquires the image captured by photographing themirror image 2′ of thewearer 2, reflected by themirror 4, by the camera (step P11) and notifies the image-processingunit 22 of the captured image S11. Then, the image-processingunit 22 of thewearable terminal 20 detects the face image of thewearer 2 from the captured image S11 of which the notification has been provided (step P12) and calculates the position of the face image in the captured image. - Specifically, the
face detection unit 121 of the image-processingunit 12 detects the face image from the captured image (step P12), and theface verification unit 122 verifies the face image stored in thedatabase 123A in advance against the extracted face image (step P13). The process of theface verification unit 122 returns to step P11 when the face images do not match with each other (No in step P14). When the face images match with each other (Yes in step P14), theface verification unit 122 sends the position information of the face image in the reference image and the wearer information, registered in thedatabase 123A, to the face coordinatecalculation unit 124. - The face coordinate
calculation unit 124 calculates the position of the face image in the captured image (step P15) and sends the wearer information, together with the position information of the face image in the reference image, to theposture detection unit 23. - The
posture detection unit 23 confirms whether the difference between the positions of the face image in the captured image and the face image in the reference image is present (step P16). When the difference between the positions is present (Yes in step P16), the posture difference information is sent to thecamera control unit 14. Based on the posture difference information, thecamera control unit 14 gives an instruction to a shift unit (not illustrated) of thecamera unit 11 to control the photographing direction of the camera (step P17). - The
wearable terminal 20 repeats step P11 to step P17 until the difference between the positions of the face image in the captured image and the face image in the reference image becomes absent. When the difference between the positions of the face images becomes absent (No in step P16), the photographing direction of the camera is controlled based on the correction information (step P18). - In the second example embodiment, the example in which the
face detection unit 121 of the image-processingunit 22 notifies the face coordinatecalculation unit 124 of the captured image S11 is described. However, the second example embodiment is not limited thereto. For example, two of theface detection unit 121 and face coordinatecalculation unit 124 of the image-processingunit 22 may also be notified of the captured image S11 from thecamera unit 11. In this case, the need for notifying the face coordinatecalculation unit 124 of the captured image S11 from theface detection unit 121 of the image-processingunit 22 is eliminated. - As above, the
wearable terminal 20 of the second example embodiment enables the photographing direction of the camera in the camera unit to be set to a predetermined direction, like the first example embodiment. The reason for this is because thewearable terminal 20 of the second example embodiment photographs themirror image 2′ of thewearer 2 by the camera, calculates the difference (horizontal direction: V4, vertical direction: V5) between the position of the face image in the captured image and the position of the face image in the reference image, and further shifts the photographing direction of the camera based on the calculated difference information such that the position of the face image in the captured image approaches the position of the face image in the reference image. - In addition, the wearable terminal of the second example embodiment enables the photographing direction of the camera to be controlled based on the camera position information associated with the height of the
wearer 2. Because the data of thewearer 2 is read from thedatabase 123A when the face images match with each other as a result of the verification of the face images, the protection of personal information is also facilitated. - A third example embodiment will now be described with reference to the drawings. The third example embodiment is an example in which a two-dimensional bar code is used for a subject for adjustment of the photographing direction of a camera. In the descriptions of the third example embodiment, the same configurations as those of the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted.
-
FIG. 10 is a view illustrating an example in which the photographing direction of the camera of a wearable terminal according to the third example embodiment is adjusted. The camera (not illustrated) of thewearable terminal 30 mounted on awearer 2 is a camera targeted for adjustment of the photographing direction thereof. The two-dimensional bar code 5 is photographed by the camera of thewearable terminal 30. -
FIG. 11 is a block diagram illustrating the configuration of thewearable terminal 30 according to the third example embodiment. As illustrated inFIG. 11 , thewearable terminal 30 includes acamera unit 11, an image-processingunit 32, aposture detection unit 13, and acamera control unit 14. - The
camera unit 11 of thewearable terminal 30 acquires an image captured by photographing the two-dimensional bar code 5 and notifies the image-processingunit 32 of the captured image S11. - The image-processing
unit 32 of thewearable terminal 30 generates image information S32 from the input captured image S11 and notifies theposture detection unit 13 of the image information S32. Specifically, the image-processingunit 32 includesinformation extraction unit 128 and coordinatecalculation unit 129. Theinformation extraction unit 128 extracts bar code information from an image of the two-dimensional bar code 5 in the input captured image S11. The bar code information of the two-dimensional bar code 5 includes the information of the size of the two-dimensional bar code 5 and the installation position (height) of the two-dimensional bar code 5. - The coordinate
calculation unit 129 calculates the position information of the two-dimensional bar code in the captured image and notifies theposture detection unit 13 of the position information, together with the bar code information, as the image information S32. The position information of the two-dimensional bar code can be defined by the coordinates of the image of the two-dimensional bar code in the captured image and by the respective distances (horizontal direction and vertical direction) between the left and top edges of the captured image and the left and top edges of the image of the two-dimensional bar code. - The
posture detection unit 13 of thewearable terminal 30 generates posture difference information S13 from the image position of the two-dimensional bar code in the captured image and the bar code information based on the input image information S32 and notifies thecamera control unit 14 of the posture difference information S13. - Based on the posture difference information S13, the
camera control unit 14 of thewearable terminal 30 instructs thecamera unit 11 such that the position of asetting image 3A in the captured image approaches the position of asetting image 3B in a reference image to control the photographing direction of the camera. - The operation of the
wearable terminal 30 according to the third example embodiment will now be described with reference to the drawings.FIG. 12 is a flowchart illustrating the operation of thewearable terminal 30 according to the third example embodiment. The same operations of the third example embodiment as those of the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted. - The image-processing
unit 32 of thewearable terminal 30 acquires the captured image of the two-dimensional bar code (step P21). Theinformation extraction unit 128 detects the two-dimensional bar code 5 from the input captured image S11 (step P22) and extracts the bar code information of the two-dimensional bar code 5 (step P23). The bar code information includes the information of the size of the two-dimensional bar code and the installation height of the two-dimensional bar code. - The coordinate
calculation unit 129 calculates the position of the two-dimensional bar code image from the two-dimensional bar code image in the captured image S11 (step P24). The coordinatecalculation unit 129 notifies theposture detection unit 13 of the calculated position information of the two-dimensional bar code image. - A camera
posture calculation unit 131 in theposture detection unit 13 calculates posture difference information from the position information of the two-dimensional bar code image as well as the size of the two-dimensional bar code and the installation height of the two-dimensional bar code, included in the bar code information. - Based on the posture difference information calculated in the
posture detection unit 13, thecamera control unit 14 controls a camera shift amount for instructing a shift unit (not illustrated) in thecamera unit 11 to eliminate a direction deviation to control the photographing direction of the camera of the camera unit 11 (step P26). - As above, the
wearable terminal 30 according to the third example embodiment enables the photographing direction of the camera in the camera unit to be set to a predetermined direction, like the third example embodiment. - In accordance with the
wearable terminal 30 according to the third example embodiment, the information of the installation height or size of the two-dimensional bar code can be obtained from the photographed two-dimensional bar code, and therefore, the storage capacity of the database can be reduced. - A fourth example embodiment will now be described with reference to the drawings. In the descriptions of the fourth example embodiment, the same configurations as those of the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted.
-
FIG. 13 is a block diagram illustrating the configuration of awearable terminal 40 according to the fourth example embodiment. As illustrated inFIG. 13 , thewearable terminal 40 includes acamera unit 11, an image-processingunit 12, aposture detection unit 13, acamera control unit 14, and asensor unit 15. - The
sensor unit 15 of thewearable terminal 40 is mounted on thecamera unit 11 and has the function of storing the photographing direction of a camera after the photographing direction of a camera has been controlled by thecamera unit 11. Specifically, thesensor unit 15 includes a triaxial gyro sensor or a 6-axis sensor. In thewearable terminal 40, a direction toward which the gyro sensor is directed can be registered as the adjusted photographing direction of the camera by determining the photographing direction of the camera by thecamera control unit 14 and by then activating the gyro function of thesensor unit 15. - For example, when the photographing direction of the camera of the
wearable terminal 40 deviates due to the motion of a wearer, thewearable terminal 40 can correct the photographing direction of the camera without re-capturing a predetermined image by using the direction information of the gyro sensor of thesensor unit 15. Thesensor unit 15 of the fourth example embodiment can be applied to all of the first example embodiment to the third example embodiment. - The operation of the
wearable terminal 40 according to the fourth example embodiment will now be described with reference to the drawings.FIG. 14 is a flowchart illustrating the operation of thewearable terminal 40 according to the fourth example embodiment. In the following description, the descriptions of the same operations as those of the first example embodiment are omitted. - The
posture detection unit 13 confirms whether a difference is present between the positions of asetting image 3A in a captured image and asetting image 3B in a reference image. When the difference between the positions is present (Yes in step P34), theposture detection unit 13 calculates posture difference information S13 and notifies thecamera control unit 14 of the posture difference information S13. Based on the posture difference information calculated in theposture detection unit 13, thecamera control unit 14 gives an instruction (shift amount information S14) to the camera unit 11 (step P35) to control the photographing direction of the camera. When the difference between the positions is absent (No in step P34), the gyro function of thesensor unit 15 is activated, and the photographing direction of the camera in a state in which the difference between the positions is absent is stored as the initial direction of the gyro sensor (step P36). - After the adjustment of the photographing direction of the camera by the
posture detection unit 13, thesensor unit 15 confirms whether the difference between the photographing direction of the camera and the initial direction of the gyro sensor is present (step P37). When the difference between the directions is present (Yes in step P37), thesensor unit 15 calculates direction difference information (angular difference) between the photographing direction and the initial direction and notifies thecamera control unit 14 of the direction difference information. Based on the direction difference information, thecamera control unit 14 generates the shift amount information S14 to eliminate the direction difference and gives an instruction to thecamera unit 11 to re-control the photographing direction of the camera (step P38). - As above, the
wearable terminal 40 according to the fourth example embodiment enables the photographing direction of the camera in the camera unit to be set to a predetermined direction, like the first example embodiment. In addition, thewearable terminal 40 according to the fourth example embodiment enables the photographing direction of the camera to be corrected without re-capturing a predetermined image. - A fifth example embodiment will now be described with reference to the drawings. The fifth example embodiment is an example in which a wearable terminal controls two camera units.
FIG. 15A is a top schematic view illustrating an example in which the front camera unit and back camera unit of the wearable terminal are mounted.FIG. 15B is a front schematic view illustrating an example in which the front camera unit of the wearable terminal is mounted.FIG. 15C is a back schematic view illustrating an example in which the front camera unit and back camera unit of the wearable terminal are worn. - As illustrated in
FIGS. 15A to C, thefront camera unit 16 of the wearable terminal is mounted on the right shoulder of awearer 2, and theback camera unit 17 is mounted on the upper portion of the back. The photographing direction of the camera of thefront camera unit 16 is a direction in which thewearer 2 photographs from the front, and the photographing direction of the camera of theback camera unit 17 is a direction in which thewearer 2 photographs from the back. In other words, the photographing directions of the cameras of thefront camera unit 16 and theback camera unit 17 are opposite in direction to each other. The elevation angles of the photographing direction of the camera of thefront camera unit 16 and the photographing direction of the camera of theback camera unit 17 are respectively symmetric with respect to the horizontal plane. -
FIG. 16 is a block diagram illustrating the configuration of the wearable terminal according to the fifth example embodiment. With regard to the configuration of thewearable terminal 50 according to the fifth example embodiment inFIG. 16 , the same configurations as those of thewearable terminal 10 according to the first example embodiment are denoted by the same reference characters, and the detailed descriptions thereof are omitted as appropriate. - The
wearable terminal 50 according to the fifth example embodiment includes thefront camera unit 16, an image-processingunit 12, aposture detection unit 13, acamera control unit 18, and theback camera unit 17. Theback camera unit 17 includes a camera (not illustrated) and a shift unit (not illustrated) for changing the photographing direction of the camera. - The
front camera unit 16 of thewearable terminal 50 of the fifth example embodiment acquires an image captured by photographing a predetermined subject by the camera (not illustrated). Then, thefront camera unit 16 notifies the image-processingunit 12 of the captured image S11. The image-processingunit 12 of thewearable terminal 50 generates verification image information S12 based on the captured image S11 and notifies theposture detection unit 13 of the verification image information S12. The configuration and operation of theposture detection unit 13 are the same as those of thewearable terminal 10 according to the first example embodiment, and therefore, the detailed descriptions thereof are omitted. Like thewearable terminal 10 of the first example embodiment, theposture detection unit 13 sends generated posture difference information S13 to thecamera control unit 18. - Based on the posture difference information S13, the
camera control unit 18 of thewearable terminal 50 gives an instruction to thefront camera unit 16 to control the photographing direction of the camera. Further, thecamera control unit 18 instructs theback camera unit 17 to be set to be opposite in direction to the photographing direction of thefront camera unit 16 to control the photographing direction of the camera. Theback camera unit 17 may be controlled simultaneously with or after the control of thefront camera unit 16. - In the fifth example embodiment described above, the example of application to the wearable terminal of the first example embodiment is described. However, application to the wearable terminals according to the second to fourth example embodiments is also acceptable.
- In addition to the effects of the first example embodiment, the photographing direction of a camera in another camera unit can be easily adjusted according to the fifth example embodiment, as described above. The reason for this is because the adjustment of the photographing direction of the other camera is allowed to be reflective of a symmetry property between the photographing direction of the camera of the
front camera unit 16 and the photographing direction of the camera of theback camera unit 17. - As above, the
wearable terminal 50 according to the fifth example embodiment enables the photographing direction of the camera in the camera unit to be set to the predetermined direction, like the first example embodiment. - In addition, the wearable terminal according to the fifth example embodiment enables the photographing direction of the camera of the
back camera unit 17 to be adjusted by giving an instruction that the photographing direction of the camera of theback camera unit 17 is set to be opposite in direction to the photographing direction of thefront camera unit 16 to control theback camera unit 17. -
FIG. 17 is a view illustrating a hardware configuration in which each control unit or each processing unit of thewearable terminals - As illustrated in
FIG. 17 , each control unit or each processing unit of thewearable terminals wearable terminals memory 903 and astorage device 904 such as a hard disk in which a program is stored. In addition, theCPU 901 is connected to aninput device 905 and anoutput device 906 via asystem bus 907. - The
CPU 901 runs an operating system to control the wearable terminals according to the first to fifth example embodiments. In addition, theCPU 901 read, for example, programs and data from a recording medium mounted in a drive device to thememory 903. - In addition, the
CPU 901 has, for example, the function of processing an information signal input from each function unit in each example embodiment and executes processing of various functions based on the programs. - The
storage device 904 is, for example, an optical disk, a flexible disk, a magneto-optical disk, an external hard disk, a semiconductor memory, or the like. A storage medium in part of thestorage device 904 is a non-volatile storage device, in which the programs are stored. The programs may also be downloaded from an external computer that is connected to a communication network and is not illustrated. - The
input device 905 is implemented by, for example, a mouse, a keyboard, a touch panel, or the like, and is used for input manipulation. - The
output device 906 is implemented by, for example, a display, and is used to output and confirm information or the like processed by theCPU 901. - As above, each example embodiment is implemented by the hardware configuration illustrated in
FIG. 17 . However, each implementation unit included in thewearable terminals - The disclosed subject matter is described above with reference to the example embodiments (and examples). However, the disclosed subject matter is not limited to the example embodiments (and examples) described above. Various modifications that can be understood by a person skilled in the art can be made to the constitutions and details of the disclosed subject matter within the scope of the disclosed subject matter.
- This application claims priority based on Japanese Patent Application No. 2014-245098, which was filed on Dec. 3, 2014, and of which the entire disclosure is incorporated herein.
-
- 2 Wearer
- 2′ Mirror image
- 3A Setting image
- 3B Setting image
- 4 Mirror
- 5 Two-dimensional bar code
- 10 Wearable terminal
- 11 Camera unit
- 12 Image-processing unit
- 13 Posture detection unit
- 14 Camera control unit
- 15 Sensor unit
- 16 Front camera unit
- 17 Back camera unit
- 18 Camera control unit
- 20 Wearable terminal
- 22 Image-processing unit
- 23 Posture detection unit
- 30 Wearable terminal
- 32 Image-processing unit
- 40 Wearable terminal
- 50 Wearable terminal
- 121 Face detection unit
- 122 Face verification unit
- 123 Database
- 124 Face coordinate calculation unit
- 125 Feature extraction unit
- 126 Feature verification unit
- 127 Coordinate calculation unit
- 128 Information extraction unit
- 129 Coordinate calculation unit
- 131 Camera posture calculation unit
- 132 Camera posture database
- 901 CPU
- 902 Communication I/F (communication interface)
- 903 Memory
- 904 Storage device
- 905 Input device
- 906 Output device
- 907 System bus
- S11 Captured image
- S12 Verification image information
- S13 Posture difference information
- S14 Shift amount information
- S22 Verification image information
- S32 Image information
Claims (10)
1. A direction control device comprising:
a camera unit that acquires an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction;
an image-processing unit that calculates a position of a first setting image that represents the subject in the captured image;
a posture detection unit that detects a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and
a camera control unit that shifts the photographing direction of the camera based on the difference.
2. The direction control device according to claim 1 ,
wherein the image-processing unit comprises:
a feature extraction unit that extracts a feature amount from the captured image;
a database in which the setting image is stored;
a feature verification unit that verifies the setting image stored in the database against the image feature extracted in the feature extraction unit to specify a feature amount of the captured image; and
a coordinate calculation unit that calculates the position of the verified setting image.
3. The direction control device according to claim 1 ,
wherein the image-processing unit comprises:
a face detection unit that detects a face from the captured image;
a database in which a predetermined face image which is the setting image is stored;
a face verification unit that verifies the predetermined face image stored in the database against a face image detected in the face detection unit to specify a face image of the captured image; and
a face coordinate calculation unit that calculates a position of the verified face image.
4. The direction control device according to claim 1 ,
wherein the image-processing unit comprises:
an image detection unit that detects a two-dimensional bar code from the captured image;
an information extraction unit that extracts information of the two-dimensional bar code; and
a coordinate calculation unit that calculates a position of the two-dimensional bar code in the captured image from the information extracted in the information extraction unit, wherein the information of the two-dimensional bar code comprises installation height information of the two-dimensional bar code and size information of the two-dimensional bar code.
5. The direction control device according to claim 1 ,
wherein the camera unit comprises a sensor that detects a posture of the camera;
wherein the sensor stores a state of absence of a positional deviation in the photographing direction of the camera as a direction toward which the sensor is directed; and
wherein the camera control unit controls the photographing direction of the camera by using the direction toward which the sensor is directed.
6. The direction control device according to claim 5 , wherein the sensor is a triaxial gyro sensor or a 6-axis sensor.
7. The direction control device according to claim 1 ,
wherein the camera unit comprises a front camera unit and a back camera unit; and
wherein the camera control unit controls a photographing direction of a camera in the back camera unit to be opposite to a photographing direction of a camera in the front camera unit when elevation angles of the photographing direction of the camera in the front camera unit and the photographing direction of the camera in the back camera unit are respectively symmetric with respect to a horizontal.
8. A wearable terminal comprising:
the direction control device according to claim 1 .
9. A direction control method comprising:
acquiring an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction;
calculating a position of a first setting image that represents the subject in the captured image;
detecting a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and
shifting the photographing direction of the camera based on the difference.
10. A non-transitory computer-readable recording medium storing a direction control program that causes a computer to execute:
acquiring an image captured by photographing a subject by a camera targeted for adjustment of a photographing direction;
calculating a position of a first setting image that represents the subject in the captured image;
detecting a difference between a position of a second setting image that represents the subject in a reference image registered in advance and the position of the first setting image; and
shifting the photographing direction of the camera based on the difference.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014-245098 | 2014-12-03 | ||
JP2014245098 | 2014-12-03 | ||
PCT/JP2015/005983 WO2016088368A1 (en) | 2014-12-03 | 2015-12-02 | Direction control device, direction control method and recording medium |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/005983 A-371-Of-International WO2016088368A1 (en) | 2014-12-03 | 2015-12-02 | Direction control device, direction control method and recording medium |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/379,977 Division US11102382B2 (en) | 2014-12-03 | 2019-04-10 | Direction control device, direction control method and recording medium |
Publications (1)
Publication Number | Publication Date |
---|---|
US20170272623A1 true US20170272623A1 (en) | 2017-09-21 |
Family
ID=56091330
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/532,157 Abandoned US20170272623A1 (en) | 2014-12-03 | 2015-12-02 | Direction control device, direction control method and recording medium |
US16/379,977 Active US11102382B2 (en) | 2014-12-03 | 2019-04-10 | Direction control device, direction control method and recording medium |
US17/236,394 Abandoned US20210243339A1 (en) | 2014-12-03 | 2021-04-21 | Direction control device, direction control method and recording medium |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/379,977 Active US11102382B2 (en) | 2014-12-03 | 2019-04-10 | Direction control device, direction control method and recording medium |
US17/236,394 Abandoned US20210243339A1 (en) | 2014-12-03 | 2021-04-21 | Direction control device, direction control method and recording medium |
Country Status (4)
Country | Link |
---|---|
US (3) | US20170272623A1 (en) |
JP (2) | JP6737183B2 (en) |
SG (2) | SG11201704378QA (en) |
WO (1) | WO2016088368A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109639979A (en) * | 2018-12-28 | 2019-04-16 | 北京阿法龙科技有限公司 | A kind of wearable device transmission method based on USB |
CN113542575A (en) * | 2020-04-15 | 2021-10-22 | 荣耀终端有限公司 | Device pose adjusting method, image shooting method and electronic device |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102360306B1 (en) * | 2018-11-23 | 2022-02-10 | 주식회사 엔텍로직 | Method and apparatus of inspecting calibration for gimbal camera and directional controller |
JP7275759B2 (en) * | 2019-03-28 | 2023-05-18 | セイコーエプソン株式会社 | OBJECT DETECTION METHOD, OBJECT DETECTION DEVICE, AND ROBOT SYSTEM |
US11134079B2 (en) * | 2019-05-21 | 2021-09-28 | International Business Machines Corporation | Cognitive behavioral and environmental access |
JP7099597B2 (en) * | 2020-09-30 | 2022-07-12 | 株式会社リコー | Information processing device, moving object, shooting system, shooting control method and program |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH063726A (en) * | 1992-06-17 | 1994-01-14 | Canon Inc | Photographing device |
JP2007019776A (en) | 2005-07-07 | 2007-01-25 | Matsushita Electric Ind Co Ltd | Imaging system |
JP4767052B2 (en) * | 2006-03-22 | 2011-09-07 | ダイハツ工業株式会社 | Optical axis deviation detector |
JP4932368B2 (en) * | 2006-07-26 | 2012-05-16 | Necカシオモバイルコミュニケーションズ株式会社 | Portable terminal device and program |
US7839431B2 (en) * | 2006-10-19 | 2010-11-23 | Robert Bosch Gmbh | Image processing system and method for improving repeatability |
US8154616B2 (en) | 2007-01-16 | 2012-04-10 | Panasonic Corporation | Data processing apparatus and method, and recording medium |
JP4260215B1 (en) | 2007-08-29 | 2009-04-30 | 任天堂株式会社 | Imaging device |
JP5212046B2 (en) | 2008-11-25 | 2013-06-19 | 株式会社ニコン | Digital camera, image processing apparatus, and image processing program |
JP4743283B2 (en) * | 2009-01-28 | 2011-08-10 | ソニー株式会社 | Information processing apparatus, information processing method, program, system, and imaging target generation apparatus |
JP4875117B2 (en) * | 2009-03-13 | 2012-02-15 | 株式会社東芝 | Image processing device |
JP5291605B2 (en) * | 2009-12-04 | 2013-09-18 | 日本放送協会 | Camera posture estimation apparatus and camera posture estimation program |
JP5857747B2 (en) | 2012-01-05 | 2016-02-10 | 富士通株式会社 | Operation setting method for a robot equipped with an imaging device. |
CN102547209B (en) * | 2012-02-06 | 2015-07-22 | 华为技术有限公司 | Method and device for controlling videoconferencing equipment, and videoconferencing system |
JP2013232722A (en) * | 2012-04-27 | 2013-11-14 | Nikon Corp | Electronic apparatus and imaging module |
JP2014212473A (en) * | 2013-04-19 | 2014-11-13 | 株式会社ニコン | Communication device and head-mounted display device |
US9253410B2 (en) * | 2013-04-25 | 2016-02-02 | Canon Kabushiki Kaisha | Object detection apparatus, control method therefor, image capturing apparatus, and storage medium |
JP5651221B2 (en) * | 2013-09-30 | 2015-01-07 | 株式会社キングジム | Symbol piece, image processing program, and image processing method |
US9412205B2 (en) * | 2014-08-25 | 2016-08-09 | Daqri, Llc | Extracting sensor data for augmented reality content |
-
2015
- 2015-12-02 JP JP2016562304A patent/JP6737183B2/en active Active
- 2015-12-02 WO PCT/JP2015/005983 patent/WO2016088368A1/en active Application Filing
- 2015-12-02 SG SG11201704378QA patent/SG11201704378QA/en unknown
- 2015-12-02 US US15/532,157 patent/US20170272623A1/en not_active Abandoned
- 2015-12-02 SG SG10201810925RA patent/SG10201810925RA/en unknown
-
2019
- 2019-04-10 US US16/379,977 patent/US11102382B2/en active Active
-
2020
- 2020-07-07 JP JP2020116771A patent/JP7140160B2/en active Active
-
2021
- 2021-04-21 US US17/236,394 patent/US20210243339A1/en not_active Abandoned
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109639979A (en) * | 2018-12-28 | 2019-04-16 | 北京阿法龙科技有限公司 | A kind of wearable device transmission method based on USB |
CN113542575A (en) * | 2020-04-15 | 2021-10-22 | 荣耀终端有限公司 | Device pose adjusting method, image shooting method and electronic device |
Also Published As
Publication number | Publication date |
---|---|
WO2016088368A1 (en) | 2016-06-09 |
JP2020184779A (en) | 2020-11-12 |
JP6737183B2 (en) | 2020-08-05 |
US20190238727A1 (en) | 2019-08-01 |
JP7140160B2 (en) | 2022-09-21 |
SG11201704378QA (en) | 2017-06-29 |
US11102382B2 (en) | 2021-08-24 |
SG10201810925RA (en) | 2019-01-30 |
US20210243339A1 (en) | 2021-08-05 |
JPWO2016088368A1 (en) | 2017-11-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11102382B2 (en) | Direction control device, direction control method and recording medium | |
US10157477B2 (en) | Robust head pose estimation with a depth camera | |
US10408912B2 (en) | Method and system for detecting that an object of interest has re-entered a field of view of an imaging device | |
US10755438B2 (en) | Robust head pose estimation with a depth camera | |
KR102111935B1 (en) | Display control apparatus, display control method, and program | |
US10169880B2 (en) | Information processing apparatus, information processing method, and program | |
WO2012096166A1 (en) | Image capturing system, camera control device for use therein, image capturing method, camera control method, and computer program | |
JP6364952B2 (en) | Information processing apparatus, information processing system, and information processing method | |
US9946146B2 (en) | Control apparatus configured to control projection of an image based on position information, projection information, and shape information, corresponding control method and corresponding storage medium | |
EP3062286B1 (en) | Optical distortion compensation | |
JP6562437B1 (en) | Monitoring device and monitoring method | |
CN109155055B (en) | Region-of-interest image generating device | |
KR102315525B1 (en) | Surveillance system and operation method thereof | |
US9880634B2 (en) | Gesture input apparatus, gesture input method, and program for wearable terminal | |
US20170316612A1 (en) | Authoring device and authoring method | |
JP2015226209A (en) | Information presentation system, stereo camera system and information presentation method | |
US20150146028A1 (en) | System and method of controlling imaging direction and angle of view of camera | |
US20160156839A1 (en) | Position detection device | |
EP3136724B1 (en) | Wearable display apparatus, information processing apparatus, and control method therefor | |
JP6689492B2 (en) | Terminal device, data processing system and program | |
US20230054623A1 (en) | Image processing method | |
US20220254109A1 (en) | Information processing apparatus, display system, display method, and non-transitory computer readable medium storing program | |
KR20130121416A (en) | Location recognizing apparatus for smart robot | |
KR102596487B1 (en) | Display Control System, Method and Computer Readable Record Medium Thereof | |
JP2016162164A (en) | Operation device and operation method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: NEC CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YACHIDA, SHOJI;HIROAKI, TOSHIHIKO;REEL/FRAME:042559/0266 Effective date: 20170519 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |