US20110013807A1 - Apparatus and method for recognizing subject motion using a camera - Google Patents

Apparatus and method for recognizing subject motion using a camera Download PDF

Info

Publication number
US20110013807A1
US20110013807A1 US12/837,111 US83711110A US2011013807A1 US 20110013807 A1 US20110013807 A1 US 20110013807A1 US 83711110 A US83711110 A US 83711110A US 2011013807 A1 US2011013807 A1 US 2011013807A1
Authority
US
United States
Prior art keywords
motion
previous
blocks
image frames
current image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/837,111
Other languages
English (en)
Inventor
Dong-Hyuk Lee
Mu-Sik Kwon
Jung-Rim Kim
Seong-taek Hwang
Sang-Wook Oh
Hyun-Soo Kim
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Assigned to SAMSUNG ELECTRONICS CO., LTD. reassignment SAMSUNG ELECTRONICS CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HWANG, SEONG-TAEK, KIM, HYUN-SOO, Kim, Jung-Rim, KWON, MU-SIK, LEE, DONG-HYUK, OH, SANG-WOOK
Publication of US20110013807A1 publication Critical patent/US20110013807A1/en
Priority to US14/447,130 priority Critical patent/US9400563B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Definitions

  • the present invention relates generally to an apparatus and method for detecting a subject's motions in captured images, and more particularly, to an apparatus and method for recognizing or detecting a user's hand motions in image frames received from a mobile camera, i.e., a camera embedded in a mobile terminal.
  • a mobile camera i.e., a camera embedded in a mobile terminal.
  • Mobile terminals also known as mobile communication terminals or portable terminals
  • mobile communication terminals or portable terminals which were originally developed for voice calls, have evolved into devices that provide many different types of services to users.
  • more recently developed mobile terminals also provide data services such as text messaging, photo and video services, and mobile banking service.
  • Users of mobile terminals with a camera may capture a variety of images with the camera.
  • the mobile terminal with a camera may recognize shapes or patterns in the images, and control an operation of a specific application based on the shape or pattern recognition results.
  • a conventional hand shape recognition method using a mobile camera predefines recognizable hand shapes or hand patterns, and detects hand shapes in images captured by the camera. More specifically, the mobile terminal searches a database in a memory for a predefined hand shape that best corresponds with the detected hand shape, and triggers an event associated with the search results.
  • the conventional hand shape recognition method defines various hand shapes, and diversifies the types of events corresponding to the defined hand shapes.
  • the conventional hand shape recognition method predefines diverse hand shapes corresponding to input signals, compares a hand shape in the current input image with pre-learned or stored hand shapes, and initiates an event based on the comparison results.
  • aspects of the present invention provide a hand motion-based input method and a hand motion recognition apparatus and method, which are robust to various disturbing factors and high in speed, and can be applied to a variety of mobile applications. This may be accomplished by considering only motion characteristics of a target close in color to the skin in a way, which is robust to diverse disturbing factors like the lighting and scale without separately detecting hand shapes.
  • a method for recognizing a subject motion using a camera in which each of the previous and current image frames received from the camera is split into multiple image blocks, motion blocks are detected among the image blocks based on a difference between previous and current pixel values, for each of the image frames, a motion center is detected based on positions of the motion blocks, for each of the image frames, and a subject motion appearing in the previous and current image frames is recognized based on the motion centers of the previous and current image frames.
  • an apparatus for recognizing a subject motion using a camera in which a motion block detector detects motion blocks in each of the previous and current image frames based on a difference between the previous and current pixel values, for each of the image frames received from the camera, and a momentum and direction determiner detects a motion center based on positions of the motion blocks for each of the image frames, and calculates a distance between the motion centers of the previous and current image frames.
  • FIG. 1 is a block diagram schematically illustrating a mobile terminal with a hand motion recognizer for recognizing hand motions in input images according to an embodiment of the present invention
  • FIG. 2 is a block diagram illustrating a detailed structure of a hand motion recognizer as illustrated in FIG. 1 ;
  • FIG. 3 is a flowchart illustrating a method for recognizing hand motions in input images according to an embodiment of the present invention
  • FIGS. 4A to 4D are diagrams illustrating steps of estimating directions of hand motions
  • FIGS. 5A to 6B are diagrams illustrating steps of determining strengths of momentums.
  • FIGS. 7A to 8B are diagrams illustrating steps of allocating events.
  • the present invention provides high-speed hand motion recognition and hand motion-based input method, which is robust to various disturbing factors, and can be applied to various mobile applications, by considering only motion characteristics of a target (i.e., a subject or an object in an image) close in color to the skin, which are robust to a variety of disturbing factors like lighting and scale, by recognizing hand motions (basically, a subject's motions appearing in images) instead of hand shapes in captured images.
  • hand motion-based input refers to using hand motion recognition results as user interfaces, which may be applied to various applications such as to view photos or play games.
  • FIG. 1 is a block diagram, which schematically illustrates a mobile terminal including a hand motion recognizer for recognizing hand motions in input images according to an embodiment of the present invention.
  • a mobile terminal 100 includes a camera 110 , an Image Signal Processor (ISP) 120 , a display 130 , a wireless communication unit 140 , a hand motion recognizer 200 , a controller 150 , and a memory unit 160 .
  • ISP Image Signal Processor
  • the mobile terminal 100 may further include a speaker, a microphone, and a user interface device like a keypad.
  • the camera 110 captures an image of a subject, and detects the captured image in an electrical signal.
  • the camera 110 may include a lens system having at least one lens to form an image of a subject, and an image sensor, such as a Charge-Coupled Device (CCD) image sensor and/or a Complementary Metal-Oxide Semiconductor (C-MOS) image sensor, for converting the image formed on the lens system to an electrical signal.
  • CCD Charge-Coupled Device
  • C-MOS Complementary Metal-Oxide Semiconductor
  • the ISP 120 under the control of the controller 150 , processes image signals received from the camera 110 or images stored in the memory unit 160 on a frame-by-frame basis, and outputs image frames that have been converted according to screen characteristics (size, quality, resolution, etc.) of the display 130 .
  • the display 130 displays image frames received from the ISP 120 on a screen.
  • a Liquid Crystal Display (LCD), a touch screen, or the like may be used as the display 130 .
  • the touch screen displays images under the control of the controller 150 . If a user input, such as a finger or a stylus pen, comes in contact with its surface, the touch screen generates a key contact interrupt and outputs user input information including input coordinates and input status to the controller 150 .
  • the wireless communication unit 140 receives wireless downlink signals over the air via an antenna, and outputs downlink data obtained by demodulating the wireless downlink signals, to the controller 150 . Also, the wireless communication unit 140 generates wireless uplink signals by modulating uplink data received from the controller 150 , and wirelessly transmits the generated wireless uplink signals into the air via the antenna.
  • the modulation and demodulation may be carried out using Code Division Multiple Access (CDMA), and may also be performed using Frequency Division Multiplexing (FDM) or Time Division Multiplexing (TDM).
  • CDMA Code Division Multiple Access
  • FDM Frequency Division Multiplexing
  • TDM Time Division Multiplexing
  • the hand motion recognizer 200 recognizes hand motions from image frames received from the ISP 120 , and outputs the recognition results to the controller 150 .
  • the memory unit 160 may store applications with various functions such as games, images for offering Graphical User Interfaces (GUIs) associated with the applications, databases regarding user information and documents, and background images (menu screens, idle screen, etc.) or other programs for operating the mobile terminal 100 .
  • GUIs Graphical User Interfaces
  • the controller 150 runs an application corresponding to user input information, and the application performs a program operation corresponding to the user input information.
  • the user inputs may include normal inputs made using keypads, touch screens, etc., and camera-based hand motion inputs. For example, if a user moves his hand toward the camera from side to side while a photo album application is running, the photo album application may replace the current photo displayed on the display 130 with the next photo. Such an action might trigger a “Flip Photo Album” event in reply to the hand motion causing the event processing result (i.e., flipping to the next photo) to be displayed on the display 130 .
  • FIG. 2 is a block diagram illustrating a hand motion recognizer 200
  • FIG. 3 is a flowchart illustrating a method for recognizing hand motions in input images according to an embodiment of the present invention.
  • the hand motion recognizer 200 includes a skin color detector 210 , a motion block detector 220 , a momentum & direction determiner 230 , and an event allocator 240 .
  • the hand motion recognition method includes a motion block extraction step S 110 , a motion center estimation step S 120 , an inter-motion frame speed & momentum estimation step S 130 , a momentum comparison step S 140 , an input direction and strength estimation step S 160 , and an event allocation step S 170 .
  • the skin color detector 210 detects skin color blocks (or motion candidate blocks) corresponding to a hand in image frames received from the ISP 120 , and the motion block detector 220 detects motion blocks among the skin color blocks.
  • the skin color detector 210 receives an image frame from the ISP 120 , and splits the image frame into blocks each having a predetermined number of pixels in order to reduce noises and computation.
  • This image splitting step may be performed on the entire image frame, or on an area of interest in the image frame displayed on the display 130 .
  • the area of interest may be determined by the user or the controller 150 . Also, the area of interest may be automatically set according to, for example, a default value stored in the memory unit 160 .
  • the image splitting step is a virtual step, and the image frame is split into, for example, N*M blocks and processed in the hand motion recognizer 200 on a block-by-block basis. For example, each of the blocks may have a size of 8*8 or 16*16 pixels.
  • the skin color detector 210 detects pixels (hereinafter, skin color pixels) close in color to the skin among the pixels included in each block, and determines a certain block as a skin color block if the number of the detected skin color pixels is greater than or equal to a predetermined ratio (e.g., 60 to 80%) of the total number of pixels in the block.
  • a predetermined ratio e.g. 60 to 80%
  • the skin color detector 210 can determine skin color pixels, using Equation (1) below.
  • a SkinColorMargin value may be set to 10 so that it may include a color value similar to that of the skin color, and the specific figures in Equation (1) are given by way of example.
  • the skin color detector 210 determines whether or not a color value of each pixel satisfies Equation (1) or if it falls within a threshold range.
  • the motion block detector 220 detects motion pixels among the pixels included in each skin color block, and determines the skin color block as a motion block if the number of the detected motion pixels is greater than or equal to a predetermined ratio of the total number of pixels in the block. This motion block detection step is performed on the current image frame and the previous image frame on the basis of the current image frame at a sampling time of each image frame.
  • the motion block detector 220 compares each pixel value (or a brightness value) of a skin color block in the current image frame with a pixel value of the same pixel in the previous image frame and determines the pixel to be a motion pixel if the difference between the pixel values is greater than or equal to a predetermined pixel threshold (e.g., 10-30 in the YCrCb format).
  • a predetermined pixel threshold e.g. 10-30 in the YCrCb format.
  • the motion block detector 220 determines a pertinent skin color block as a motion block if the number of motion pixels included in each skin color block is greater than or equal to a predetermined block threshold or a predetermined ratio (e.g., 60 to 80%) of the total number of pixels in the skin color block.
  • the reason for detecting motion blocks from among the skin color blocks in this embodiment is to extract only the target having a relatively large motion because the image frame may contain the user's face in addition to his or her moving hand. Therefore, the motion block estimation (or matching) step described above is optional depending on the use environment of the present invention, or the features of the application.
  • the skin color blocks are detected in order to exclude a motion of the background in the image frame.
  • the above-described skin color block detection step is also optional depending on the use environment of the present invention, or the features of the application.
  • motions of other targets may also be estimated depending on the input method of the application.
  • the momentum & direction determiner 230 estimates the central point (or motion center) for all motion blocks in each of the image frames.
  • a central point may be determined as the center of the whole distribution area of the motion blocks, the center of an area where the motion blocks are crowded, or the center given by a combination of both methods. For example, every area may be given a different weight depending on the density of motion blocks.
  • the momentum & direction determiner 230 calculates the speed and the momentum of a hand motion based on a distance between the motion centers of the current and previous image frames.
  • the hand motion recognition may be achieved by simply calculating the distance between the motion centers of the current and previous image frames. That is, a speed of the hand motion is expressed in the distance between the motion centers as described below, and in the case of a simple application, hand motion inputting may be completed by simply estimating the hand motion's speed. Estimating the momentum, strength, direction, and hand motion speed offers a variety of events according to the state of the hand motion.
  • Equation (2) sqrt( )represents a square root function, and the speed is expressed in a distance between motion centers.
  • a time difference between image frames being input to the hand motion recognizer 200 i.e., a sampling interval between the image frames, is constant, and the speed and momentum of the hand motion are meaningful as relative values for comparison, so the sampling interval is omitted in Equation (2).
  • the “speed” of the hand motion corresponds to the distance of the hand motion, or the magnitude of a motion vector.
  • Equation (3) A moment used to calculate a momentum of the hand motion can be written as Equation (3) below.
  • N MB current represents the number of motion blocks in the current image frame
  • N MB Previous represents the number of motion blocks in the previous image frame
  • Max( ) represents a function for calculating the maximum value among factors.
  • Max( ) is used above to calculate the momentum of the hand motion
  • an average function may be used in the alternative.
  • Equation (4) The momentum P calculated using the speed and moment of the hand motion can be written as Equation (4) below.
  • the momentum & direction determiner 230 compares the momentum of the hand motion with a predetermined momentum threshold. If the momentum is less than or equal to the threshold, the momentum & direction determiner 230 does not perform the input's direction & strength estimation step in step S 150 , determining that the hand motion is invalid. If the momentum exceeds the threshold, the momentum & direction determiner 230 performs the input direction and strength estimation step S 160 .
  • the mobile terminal would use the square root value (or an approximation thereof) to determine Speed, or the value of Speed squared may be substituted increase computational efficiency.
  • this comparison step may be optionally replaced by comparing only the speed.
  • the momentum & direction determiner 230 estimates direction and strength of the hand motion. For example, for (x1 ⁇ x2)>0, the momentum & direction determiner 230 may determine that the hand motion was made in the right direction (or +x direction), while for (x1 ⁇ x2) ⁇ 0, the momentum & direction determiner 230 may determine that the hand motion was made in the left direction (or -x direction). A decision on the up-down direction (y-axis direction) may be made in the same manner.
  • the momentum & direction determiner 230 may determine the direction on the x-axis as stated above, determining that the hand motion was made in the left-right direction (x-axis direction), while for (x1 ⁇ x2)/(y1 ⁇ y2) ⁇ 1, the momentum & direction determiner 230 may determine the direction on the y-axis as stated above, determining that the hand motion was made in the up-down direction (y-axis direction).
  • FIGS. 4A to 4D illustrate a process of estimating hand motion directions.
  • the arrows indicate the directions of hand motions, and motion blocks 610 to 640 are estimated based on the hand motions.
  • FIG. 4A illustrates an image frame of a user making an upward hand motion, and the motion blocks 610 corresponding thereto.
  • FIG. 4B illustrates an image frame of a user making a downward hand motion, and the motion blocks 620 corresponding thereto.
  • FIG. 4C illustrates an image frame of a user making a leftward hand motion, and the motion blocks 630 corresponding thereto.
  • FIG. 4D illustrates an image frame of a user making a rightward hand motion, and the motion blocks 640 corresponding thereto.
  • a group of motion blocks lean toward the direction in which the user made the hand motion, making it possible to estimate the direction of the hand motion by identifying the center of the motion block group.
  • the momentum & direction determiner 230 estimates the strength (or magnitude) of the momentum by comparing the momentum with a predetermined threshold range. For example, if the strength of the momentum is greater than the threshold and less than 10 in the momentum comparison step S 140 , the strength is determined as level-1 strength. If the strength of the moment is greater than or equal to 10 and less than 20, the strength is determined as level-2 strength. If the strength of the momentum is greater than or equal to 20 (another threshold corresponding to the top limit may be set), the strength is determined as level-3 strength.
  • FIGS. 5A to 6B are diagrams illustrating steps of determining strengths of momentums.
  • the arrows in the image frames represent the directions and strengths of the hand motions.
  • FIGS. 5A and 5B illustrate a situation where level-1 strength occurs as the user makes a hand motion from left to right at low speed. It is noted that a difference between the center of a motion block group 310 in the previous image frame illustrated in FIG. 5A and the center of a motion block group 320 in the current image frame illustrated in FIG. 5B is not so large.
  • FIGS. 6A and 6B illustrates a situation where level-3 strength occurs as the user makes a hand motion from left to right at high speed. It can be appreciated that a difference between the center of a motion block group 410 in the previous image frame illustrated in FIG. 6A and the center of a motion block group 420 in the current image frame illustrated in FIG. 6B is relatively large.
  • the event allocator 240 In the event allocation step S 170 , the event allocator 240 generates an event allocated to the direction and strength of the momentum, determined in the input direction and strength estimation step S 160 .
  • this event may signal a photo album application to “flip” the “pages” of an album.
  • the controller 150 may display the execution results of the Flip Photo Album event, i.e., the next photo, on the display 130 .
  • FIGS. 7A to 8B are diagrams illustrating steps of allocating events.
  • FIGS. 7A to 7D illustrate four successive image frames.
  • the arrows indicate the directions of hand motions, and motion blocks 510 to 540 are estimated based on the hand motions.
  • the event allocator 240 generates a Flip Photo Album event (i.e., a Page Up event) in reply to the leftward hand motion appearing in the image frames, and the execution results of the Flip Photo Album event are illustrated in FIGS. 8A and 8B . That is, the previous photo illustrated in FIG. 8A is replaced with the current photo illustrated in FIG. 8B .
  • a Flip Photo Album event i.e., a Page Up event
  • a Flip Photo Album event (i.e., a Page Down event) is generated to replace the photos in the opposite direction. For example, in a cartoon application, if a Page Up event occurs, the page is flipped from 10 to 11, whereas if a Page Down event occurs, the page is flipped from 10 to 9.
  • image blocks similar in color to the skin are set as interested motion candidate blocks
  • the motion blocks are extracted from the interested motion candidate blocks using the motion block detector
  • the center of the motion blocks is calculated
  • the speed and momentum of the hand motions are estimated based on a difference between the centers of motion blocks in different motion frames and the number of motion blocks.
  • the estimated data may be applied to a variety of mobile applications.
  • the estimated data can be applied to a mobile camera-based User Interface (UI) for viewing photos that is responsive to hand motions.
  • UI User Interface
  • the embodiments of the present invention can preset event timing, sense the position where a hand motion occurs at the cycle of the event timing, and allocate an event based thereon, thereby supporting hand-face guessing games like the Korean game Cham-Cham-Cham.
  • the present invention could also be used for Olympic-type games (broad jump, hurdle race, etc.) in which events are generated depending upon the frequency and timing of hand motions, and other sports games (tennis, ping-pong, etc.) in which events are generated depending on the strength and timing of hand motions.
  • the embodiments of the present invention can be applied to a variety of games and UIs that detect and determine up/down/left/right directions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)
US12/837,111 2009-07-17 2010-07-15 Apparatus and method for recognizing subject motion using a camera Abandoned US20110013807A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/447,130 US9400563B2 (en) 2009-07-17 2014-07-30 Apparatus and method for recognizing subject motion using a camera

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020090065434A KR20110007806A (ko) 2009-07-17 2009-07-17 카메라를 이용하여 손동작을 인식하는 장치 및 방법
KR10-2009-0065434 2009-07-17

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/447,130 Continuation US9400563B2 (en) 2009-07-17 2014-07-30 Apparatus and method for recognizing subject motion using a camera

Publications (1)

Publication Number Publication Date
US20110013807A1 true US20110013807A1 (en) 2011-01-20

Family

ID=42989657

Family Applications (2)

Application Number Title Priority Date Filing Date
US12/837,111 Abandoned US20110013807A1 (en) 2009-07-17 2010-07-15 Apparatus and method for recognizing subject motion using a camera
US14/447,130 Active US9400563B2 (en) 2009-07-17 2014-07-30 Apparatus and method for recognizing subject motion using a camera

Family Applications After (1)

Application Number Title Priority Date Filing Date
US14/447,130 Active US9400563B2 (en) 2009-07-17 2014-07-30 Apparatus and method for recognizing subject motion using a camera

Country Status (3)

Country Link
US (2) US20110013807A1 (ko)
EP (1) EP2280377B1 (ko)
KR (1) KR20110007806A (ko)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120093360A1 (en) * 2010-10-19 2012-04-19 Anbumani Subramanian Hand gesture recognition
US20130271618A1 (en) * 2012-04-13 2013-10-17 Samsung Electronics Co., Ltd. Camera apparatus and control method thereof
CN103809734A (zh) * 2012-11-07 2014-05-21 联想(北京)有限公司 一种电子设备的控制方法、控制器及电子设备
JP2015041381A (ja) * 2013-08-21 2015-03-02 株式会社リコー 移動物体検出方法及びシステム
WO2015057410A1 (en) * 2013-10-16 2015-04-23 Qualcomm Incorporated Z-axis determination in a 2d gesture system
US9357127B2 (en) 2014-03-18 2016-05-31 Google Technology Holdings LLC System for auto-HDR capture decision making
US9392322B2 (en) 2012-05-10 2016-07-12 Google Technology Holdings LLC Method of visually synchronizing differing camera feeds with common subject
US9413947B2 (en) 2014-07-31 2016-08-09 Google Technology Holdings LLC Capturing images of active subjects according to activity profiles
US9571727B2 (en) 2014-05-21 2017-02-14 Google Technology Holdings LLC Enhanced image capture
US9654700B2 (en) 2014-09-16 2017-05-16 Google Technology Holdings LLC Computational camera using fusion of image sensors
US9669260B2 (en) 2011-10-21 2017-06-06 Korea Institute Of Science And Technology Apparatus and method for tracking the position of each part of the body for golf swing analysis
CN106920250A (zh) * 2017-02-14 2017-07-04 华中科技大学 基于rgb‑d视频的机器人目标识别与定位方法及系统
US9729784B2 (en) 2014-05-21 2017-08-08 Google Technology Holdings LLC Enhanced image capture
US9774779B2 (en) 2014-05-21 2017-09-26 Google Technology Holdings LLC Enhanced image capture
US9813611B2 (en) 2014-05-21 2017-11-07 Google Technology Holdings LLC Enhanced image capture
CN107357414A (zh) * 2016-05-09 2017-11-17 株式会社理光 一种点击动作的识别方法及点击动作识别装置
US9936143B2 (en) 2007-10-31 2018-04-03 Google Technology Holdings LLC Imager module with electronic shutter
US20180129875A1 (en) * 2012-10-08 2018-05-10 Pixart Imaging Inc. Gesture identification with natural images
US10242241B1 (en) * 2010-11-09 2019-03-26 Open Invention Network Llc Advanced mobile communication device gameplay system
CN111932620A (zh) * 2020-07-27 2020-11-13 根尖体育科技（北京）有限公司 排球发球过网与否的判定方法及发球速度的获取方法
CN112232332A (zh) * 2020-12-17 2021-01-15 四川圣点世纪科技有限公司 一种基于视频序列的非接触式手掌检测方法
CN112381854A (zh) * 2020-11-13 2021-02-19 西安闻泰电子科技有限公司 基于图像的运动检测方法、装置、电子设备和存储介质
US11221681B2 (en) * 2017-12-22 2022-01-11 Beijing Sensetime Technology Development Co., Ltd Methods and apparatuses for recognizing dynamic gesture, and control methods and apparatuses using gesture interaction

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102166333B1 (ko) * 2013-11-21 2020-10-16 삼성전자주식회사 장문 영상 처리 방법 및 이를 위한 단말기
US9589207B2 (en) 2013-11-21 2017-03-07 Mo' Motion Ventures Jump shot and athletic activity analysis system
US10664690B2 (en) 2013-11-21 2020-05-26 Mo' Motion Ventures Jump shot and athletic activity analysis system
WO2016013702A1 (ko) * 2014-07-25 2016-01-28 엘지전자 주식회사 포장된 식품의 개봉여부 판단 및 개봉시간과 관련된 정보를 제공하는 착용형 디바이스 및 제어 방법
WO2017078767A1 (en) 2015-11-06 2017-05-11 Hewlett-Packard Development Company, L.P. Payoff information determination
KR20190139605A (ko) 2018-06-08 2019-12-18 전자부품연구원 제스처 인식 장치 및 방법
CN110796041B (zh) 2019-10-16 2023-08-18 Oppo广东移动通信有限公司 主体识别方法和装置、电子设备、计算机可读存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010042245A1 (en) * 1998-10-13 2001-11-15 Ryuichi Iwamura Remote control system
US20060013440A1 (en) * 1998-08-10 2006-01-19 Cohen Charles J Gesture-controlled interfaces for self-service machines and other applications
US20080259031A1 (en) * 2007-04-18 2008-10-23 Fujifilm Corporation Control apparatus, method, and program

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5048890B2 (ja) 1998-10-13 2012-10-17 ソニー エレクトロニクス インク 動作検知インターフェース
JP2001307107A (ja) 2000-04-21 2001-11-02 Sony Corp 画像処理装置および方法、並びに記録媒体
US7227526B2 (en) * 2000-07-24 2007-06-05 Gesturetek, Inc. Video-based image control system
US7613322B2 (en) * 2005-05-19 2009-11-03 Objectvideo, Inc. Periodic motion detection with applications to multi-grabbing
US7907117B2 (en) * 2006-08-08 2011-03-15 Microsoft Corporation Virtual controller for visual displays

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060013440A1 (en) * 1998-08-10 2006-01-19 Cohen Charles J Gesture-controlled interfaces for self-service machines and other applications
US20010042245A1 (en) * 1998-10-13 2001-11-15 Ryuichi Iwamura Remote control system
US20080259031A1 (en) * 2007-04-18 2008-10-23 Fujifilm Corporation Control apparatus, method, and program

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Assen et al. "Accurate object localization in gray level images using the center of gravity measure: accuracy versus precision", Dec. 2002, IEEE Transactions on Image Processing, vol. 11, iss. 12, p. 1379-1384. *
Chen et al., "Hand gesture recognition using a real-time tracking method and hidden Markov models", 1 Aug 2003, Image and Vision Computing, vol. 21, iss. 8, p. 745-758. *
Jones et al., "Statistical color models with application to skin detection", 25 Jun. 1999, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999, vol. 1, p. 274-280. *
Puri et al., "An efficient block-matching algorithm for motion-compensated coding", Apr. 1987, IEEE International Conference on ICASSP '87, vol. 12, p. 1063-1066. *
Rosin, "Thresholding for change detection", 7 Jan. 1998, Sixth International Conference on Computer Vision, 1998, p. 274-279. *

Cited By (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9936143B2 (en) 2007-10-31 2018-04-03 Google Technology Holdings LLC Imager module with electronic shutter
US20120093360A1 (en) * 2010-10-19 2012-04-19 Anbumani Subramanian Hand gesture recognition
US8768006B2 (en) * 2010-10-19 2014-07-01 Hewlett-Packard Development Company, L.P. Hand gesture recognition
US10242241B1 (en) * 2010-11-09 2019-03-26 Open Invention Network Llc Advanced mobile communication device gameplay system
US9669260B2 (en) 2011-10-21 2017-06-06 Korea Institute Of Science And Technology Apparatus and method for tracking the position of each part of the body for golf swing analysis
US20130271618A1 (en) * 2012-04-13 2013-10-17 Samsung Electronics Co., Ltd. Camera apparatus and control method thereof
US9654685B2 (en) * 2012-04-13 2017-05-16 Samsung Electronics Co., Ltd Camera apparatus and control method thereof
US9392322B2 (en) 2012-05-10 2016-07-12 Google Technology Holdings LLC Method of visually synchronizing differing camera feeds with common subject
US10198627B2 (en) * 2012-10-08 2019-02-05 Pixart Imaging Inc. Gesture identification with natural images
US10366281B2 (en) * 2012-10-08 2019-07-30 Pixart Imaging Inc. Gesture identification with natural images
US20180129875A1 (en) * 2012-10-08 2018-05-10 Pixart Imaging Inc. Gesture identification with natural images
CN103809734A (zh) * 2012-11-07 2014-05-21 联想(北京)有限公司 一种电子设备的控制方法、控制器及电子设备
JP2015041381A (ja) * 2013-08-21 2015-03-02 株式会社リコー 移動物体検出方法及びシステム
US9412012B2 (en) 2013-10-16 2016-08-09 Qualcomm Incorporated Z-axis determination in a 2D gesture system
WO2015057410A1 (en) * 2013-10-16 2015-04-23 Qualcomm Incorporated Z-axis determination in a 2d gesture system
US9357127B2 (en) 2014-03-18 2016-05-31 Google Technology Holdings LLC System for auto-HDR capture decision making
US10250799B2 (en) 2014-05-21 2019-04-02 Google Technology Holdings LLC Enhanced image capture
US11019252B2 (en) 2014-05-21 2021-05-25 Google Technology Holdings LLC Enhanced image capture
US9813611B2 (en) 2014-05-21 2017-11-07 Google Technology Holdings LLC Enhanced image capture
US11943532B2 (en) 2014-05-21 2024-03-26 Google Technology Holdings LLC Enhanced image capture
US9729784B2 (en) 2014-05-21 2017-08-08 Google Technology Holdings LLC Enhanced image capture
US11575829B2 (en) 2014-05-21 2023-02-07 Google Llc Enhanced image capture
US11290639B2 (en) 2014-05-21 2022-03-29 Google Llc Enhanced image capture
US9628702B2 (en) 2014-05-21 2017-04-18 Google Technology Holdings LLC Enhanced image capture
US9571727B2 (en) 2014-05-21 2017-02-14 Google Technology Holdings LLC Enhanced image capture
US9774779B2 (en) 2014-05-21 2017-09-26 Google Technology Holdings LLC Enhanced image capture
US9413947B2 (en) 2014-07-31 2016-08-09 Google Technology Holdings LLC Capturing images of active subjects according to activity profiles
US9654700B2 (en) 2014-09-16 2017-05-16 Google Technology Holdings LLC Computational camera using fusion of image sensors
CN107357414A (zh) * 2016-05-09 2017-11-17 株式会社理光 一种点击动作的识别方法及点击动作识别装置
CN106920250A (zh) * 2017-02-14 2017-07-04 华中科技大学 基于rgb‑d视频的机器人目标识别与定位方法及系统
US11221681B2 (en) * 2017-12-22 2022-01-11 Beijing Sensetime Technology Development Co., Ltd Methods and apparatuses for recognizing dynamic gesture, and control methods and apparatuses using gesture interaction
CN111932620A (zh) * 2020-07-27 2020-11-13 根尖体育科技（北京）有限公司 排球发球过网与否的判定方法及发球速度的获取方法
CN112381854A (zh) * 2020-11-13 2021-02-19 西安闻泰电子科技有限公司 基于图像的运动检测方法、装置、电子设备和存储介质
CN112232332A (zh) * 2020-12-17 2021-01-15 四川圣点世纪科技有限公司 一种基于视频序列的非接触式手掌检测方法

Also Published As

Publication number Publication date
US20140340307A1 (en) 2014-11-20
EP2280377A1 (en) 2011-02-02
EP2280377B1 (en) 2017-09-06
US9400563B2 (en) 2016-07-26
KR20110007806A (ko) 2011-01-25

Similar Documents

Publication Publication Date Title
US9400563B2 (en) Apparatus and method for recognizing subject motion using a camera
CN107831904B (zh) 电子设备及其操作方法
US10440284B2 (en) Determination of exposure time for an image frame
US20090174674A1 (en) Apparatus and methods for a touch user interface using an image sensor
WO2019080797A1 (zh) 一种活体检测方法、终端和存储介质
US9002061B2 (en) Image processing device, image processing method and computer-readable medium
CN108259758B (zh) 图像处理方法、装置、存储介质和电子设备
KR101620933B1 (ko) 제스쳐 인식 메커니즘을 제공하는 방법 및 장치
EP2725458B1 (en) Information processing device, input control method, and input control program
WO2021179856A1 (zh) 内容识别方法、装置、电子设备及存储介质
EP2261858B1 (en) Method and apparatus for determining presence of user's hand tremor or intentional motion
KR101503017B1 (ko) 모션 검출 방법 및 장치
CN110944163A (zh) 一种图像处理方法及电子设备
WO2019097690A1 (ja) 画像処理装置、制御方法及び制御プログラム
KR20100118317A (ko) 휴대단말의 제스처 인식 방법 및 시스템
CN110223239B (zh) 一种图像处理方法、终端及计算机可读存储介质
JP5861530B2 (ja) 利用者検知装置、方法、及びプログラム
WO2021197085A1 (zh) 屏下摄像头终端设备的误触提醒方法和装置
CN109492451B (zh) 一种编码图像识别方法及移动终端
CN114520866B (zh) 图像散景处理方法、电子设备和存储介质
JPWO2013145900A1 (ja) 情報処理装置、情報処理方法およびプログラム
CN117082178A (zh) 基于机器学习的多页扫描
KR20130123316A (ko) 사용자의 얼굴 분석 결과에 따라 이동 단말을 제어하는 장치 및 방법
JP5621927B2 (ja) 情報処理装置、入力制御方法及び入力制御プログラム
CN118470311A (zh) 一种目标对象识别方法及电子设备

Legal Events

Date Code Title Description
AS Assignment

Owner name: SAMSUNG ELECTRONICS CO., LTD., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LEE, DONG-HYUK;KWON, MU-SIK;KIM, JUNG-RIM;AND OTHERS;REEL/FRAME:024703/0429

Effective date: 20100709

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION