WO2012153228A1 - Camera control - Google Patents

Camera control Download PDF

Info

Publication number
WO2012153228A1
WO2012153228A1 PCT/IB2012/052150 IB2012052150W WO2012153228A1 WO 2012153228 A1 WO2012153228 A1 WO 2012153228A1 IB 2012052150 W IB2012052150 W IB 2012052150W WO 2012153228 A1 WO2012153228 A1 WO 2012153228A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
gesture
command
video data
control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2012/052150
Other languages
French (fr)
Inventor
Kong Qiao Wang
Suresh Chande
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nokia Inc
Original Assignee
Nokia Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Inc filed Critical Nokia Inc
Publication of WO2012153228A1 publication Critical patent/WO2012153228A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/61Control of cameras or camera modules based on recognised objects
    • H04N23/611Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body

Definitions

  • This invention relates generally to camera control on a terminal, particularly using gestures received independently of a touch-based interface of the terminal.
  • terminals particularly mobile communications terminals, to comprise one or more cameras.
  • a camera is assumed to mean a digital camera capable of generating image data representing a scene received by the camera's sensor.
  • the image data can be used to capture still images using a single frame of image data or to record a succession of frames as video data.
  • predetermined user gestures detected using the camera to one or more commands associated with the application.
  • a known photo-browsing application uses hand-waving gestures made in front of a terminal's front-facing camera to control how photographs are displayed on the user interface, a right-to-left gesture typically resulting in the application advancing through a sequence of photos.
  • Some terminals comprise both front- and rear-facing cameras.
  • Prior art applications which run on the terminals enable switching between the cameras by providing a dedicated 'swap' icon provided as part of the application's graphical user interface (GUI) which requires the user to touch the button on the GUI.
  • GUI graphical user interface
  • a first aspect of the invention provides apparatus comprising a gesture recognition system configured to detect one or more predetermined user gestures independent of any touch-based interface and to control at least one camera in response to detecting the or each predetermined user gesture.
  • the apparatus may be configured to disable an enabled camera in response to detecting a predetermined user gesture.
  • the apparatus may be configured to control first and second cameras, wherein the gesture recognition system is further configured to enable a currently-disabled camera in response to detecting the predetermined user gesture.
  • the gesture recognition system may be configured to receive video data from an enabled camera and to identify from the video data one or more predetermined user gestures.
  • the gesture recognition system may be configured to identify, from the received video data, a gesture represented by a motion vector associated with a foreground object's change of position between subsequent frames of video data, and to compare said motion vector with a set of predetermined reference motion vector to identify a corresponding control command for the at least one camera.
  • the gesture recognition system may be configured to receive motion signals from a motion sensor and to identify therefrom one or more predetermined user gestures corresponding to said movement.
  • the motion sensor may include at least one of an accelerometer and a gyroscope, the motion signal being generated based on at least one of a change in acceleration and a change in orientation of the apparatus.
  • the gesture control system may be configured to disable the display of video data from a currently selected camera in response to detection of a predetermined motion gesture and to enable the display of video data from the other, non-selected camera.
  • a second aspect of the invention provides apparatus, the apparatus having at least one processor and at least one memory having computer-readable code stored thereon which when executed controls the at least one processor:
  • a third aspect of the invention provides a method comprising:
  • the outputted command may be configured to disable a currently-enabled camera.
  • the outputted command may be configured to enable a currently-disabled camera.
  • Receiving gestural data may comprise receiving video data from the at least one camera and identifying from the video data one or more predetermined user gestures. Receiving gestural data may further comprise identifying a motion vector associated with a foreground object's change of position between subsequent frames of video data, and comparing said motion vector with a set of predetermined reference motion vectors to identify a corresponding control command for the or each camera.
  • Receiving gestural data may comprises receiving a signal from a motion sensor provided on the device, the signal being representative of movement of the device, and identifying therefrom one or more predetermined user gestures corresponding to the sensed movement.
  • the signal may be received from at least one of an
  • the signal being generated based on at least one of a change in acceleration and a change in orientation of the device.
  • the method may comprise, in response to detection of a predetermined motion gesture, disabling display of video data from a currently selected camera and enabling the display of video data from a non-selected camera.
  • Another aspect provides a computer program comprising instructions that when executed by a computer apparatus control it to perform any method above.
  • Another aspect provides a portable device comprising any of the apparatus above.
  • a further aspect of the invention provides a non-transitory computer-readable storage medium having stored thereon computer-readable code, which, when executed by computing apparatus, causes the computing apparatus to perform a method comprising:
  • Figure 1 is a perspective view of a mobile terminal embodying aspects of the invention
  • Figure 2 is a schematic diagram illustrating components of the Figure l mobile terminal and their interconnection
  • Figure 3 is a schematic diagram illustrating certain components shown in Figure 2 relevant to operation of a gesture recognition system of the invention
  • Figure 4 is a flow diagram indicating the generalised processing steps performed by the gesture recognition system shown in Figure 3;
  • Figure 5 is a perspective view of the mobile terminal shown in Figure 1 which is useful for understanding a first embodiment
  • Figure 6 shows a look-up-table employed by the gesture recognition system in the first embodiment
  • Figure 7 is a flow diagram indicating the processing steps performed by the gesture recognition system in the first embodiment
  • FIGS 8a and 8b are perspective views of the mobile terminal shown in Figure 1 employed in use according to the first embodiment
  • Figure 9 is a perspective view of the mobile terminal shown in Figure 1 which is useful for understanding a second embodiment
  • Figure 10 shows a look-up-table employed by the gesture recognition system in the second embodiment
  • Figure 11 is a flow diagram indicating the processing steps performed by the gesture recognition system in the second embodiment. Detailed Description of Preferred Embodiments
  • a terminal 100 is shown.
  • the exterior of the terminal 100 has a touch sensitive display 102, hardware keys 104, front and rear cameras 105a, 105b, a speaker 118 and a headphone port 120.
  • the front camera 105a is provided on a first side of the terminal 100, that is the same side as the touch sensitive display 102.
  • the rear camera 105b is provided on the opposite side of the terminal.
  • FIG. 2 shows a schematic diagram of the components of terminal 100.
  • the terminal 100 has a controller 106, a touch sensitive display 102 comprised of a display part 108 and a tactile interface part 110, the hardware keys 104, the front and rear cameras 105a, 105b, a memory 112, RAM 114, a speaker 118, the headphone port 120, a wireless communication module 122, an antenna 124, motion sensors in the form of a set of accelerometers and gyroscopes 130, and a battery 116.
  • the controller 106 is connected to each of the other components (except the battery 116) in order to control operation thereof.
  • the memory 112 may be a non-volatile memory such as read only memory (ROM) a hard disk drive (HDD) or a solid state drive (SSD).
  • the memory 112 stores, amongst other things, an operating system 126 and may store software applications 128.
  • the RAM 114 is used by the controller 106 for the temporary storage of data.
  • the operating system 126 may contain code which, when executed by the controller 106 in conjunction with RAM 114, controls operation of each of the hardware components of the terminal.
  • the controller 106 may take any suitable form. For instance, it may be a
  • microcontroller plural microcontrollers, a processor, or plural processors.
  • the terminal 100 may be a mobile telephone or a smartphone, a personal digital assistant (PDA), a portable media player (PMP), a portable computer or any other device capable of running software applications and providing audio outputs.
  • the terminal 100 may engage in cellular communications using the wireless communications module 122 and the antenna 124.
  • the wireless communications module 122 may be configured to communicate via several protocols such as GSM (Global System for Mobiles), CDMA (code division multiple access), UMTS (universal mobile telephone system), Bluetooth and IEEE 802.11 (Wi-Fi).
  • GSM Global System for Mobiles
  • CDMA code division multiple access
  • UMTS universal mobile telephone system
  • Bluetooth IEEE 802.11
  • the display part 108 of the touch sensitive display 102 is for displaying images and text to users of the terminal and the tactile interface part 110 is for receiving touch inputs from users.
  • the memory 112 may also store multimedia files such as music and video files.
  • a wide variety of software applications 128 may be installed on the terminal including web browsers, radio and music players, games and utility applications. Some or all of the software applications stored on the terminal may provide audio outputs. The audio provided by the applications may be converted into sound by the speaker(s) 118 of the terminal or, if headphones or speakers have been connected to the headphone port 120, by the headphones or speakers connected to the headphone port 120.
  • the terminal 100 may also be associated with external software application not stored on the terminal. These may be applications stored on a remote server device and may run partly or exclusively on the remote server device. These applications can be termed cloud-hosted applications.
  • the terminal 100 may be in communication with the remote server device in order to utilise the software application stored there. This may include receiving audio outputs provided by the external software application.
  • the hardware keys 104 are dedicated volume control keys or switches.
  • the hardware keys may for example comprise two adjacent keys, a single rocker switch or a rotary dial.
  • the hardware keys 104 are located on the side of the terminal 100.
  • Figure 3 shows a schematic diagram of certain components of the terminal 100 relevant to embodiments described herein.
  • a dedicated application 140 hereafter referred to as 'the gesture detection application'.
  • the gesture detection application is associated with operation of the front and rear cameras 105a, 105b independent of the touch sensitive display 102.
  • the gesture detection application 140 may be provided as an integral part of the terminal's operating system 126 or as a separate plug-in module to the operating system.
  • the gesture detection application 140 is associated with a gesture-to-command map 142, hereafter 'command map' which is a database storing a look up table (LUT) which corresponds one or more predefined reference gestures received through sensors of the terminal 100 to operating commands associated with the front and rear cameras 105a, 105b.
  • a gesture-to-command map 142 hereafter 'command map' which is a database storing a look up table (LUT) which corresponds one or more predefined reference gestures received through sensors of the terminal 100 to operating commands associated with the front and rear cameras 105a, 105b.
  • LUT look up table
  • the command map 142 stores one or more commands, which, when executed by the controller, causes switching of one or both cameras 105a, 105b between enabled and disabled modes, as well as swapping control between the cameras so that when one camera is enabled, the other is disabled.
  • enabling a particular one of the cameras 105a, 105b means making the controller 106 configured to receive image or video data from the enabled camera for output to the display 108 and also to enable capture of the transferred image or video data using a camera application (not shown) handling aspects such as zoom, capture and storage on the memory 112.
  • the gesture detection application 140 identifies gestures from, in a first embodiment, either of the front and rear cameras 105a, 105b and, in a second embodiment, the motion sensing accelerometers and/or gyroscopes 130. It will therefore be appreciated that camera control can be achieved independently of the touch sensitive display 120 and indeed of other hard keys provided on the terminal 100.
  • the general operating steps performed by the gesture detecting application 140 are as follows.
  • a first step 4.1 the gesture detecting application 140 is run, and in a second step 4.2 a first one of the cameras 105a, 105b, as a default camera, is enabled.
  • a third step 4.3 gestures received through one or more sensors of the terminal 100 operating independently of the touch sensitive display 120 are monitored.
  • a subsequent step 4.4 if a received gesture is matched with a reference gesture stored in the command map 142, it is mapped to its associated command in step 4.5 which is then executed in step 4.6 by the controller 4.6 to perform a predetermined camera function.
  • the front and rear cameras 105a, 105b are used to detect gestures received through an enabled one of the cameras, the gestures being in the form of hand movements. Hand movements are converted to image or, more particularly, video data for comparison with reference gestures stored in the command map 142.
  • the terminal 100 is shown with the rear camera 105b, in this case the default camera, enabled. Dotted lines indicate a rectangular field-of-view 160 representing the spatial area covered by the sensor of the rear camera 105b.
  • User gestures for controlling aspects of the camera's operation, through the gesture detection application 140, are in the form of hand waving movements 162.
  • Any one of the many known video processing methods for detecting and quantifying motion in a digital camera's field-of-view can be employed.
  • One example includes periodically establishing a background image for the frame based on predominately static pixel luminance values and thereafter detecting a foreground object based on detecting pixel values above a predetermined threshold compared with the background image.
  • Alternative methods can employ foreground object detecting algorithms that do not require a background image to be established.
  • the foreground object can thereafter be quantified and tracked in terms of its motion, e.g. as an inter-frame motion vector, to represent a gesture.
  • a first reference gesture #1 maps video data representative of a left-to-right hand-waving gesture to a camera_switch command, that is to alternate controller control between the front and rear cameras 105a, 105b.
  • a second reference gesture #2 maps video data representative of a left-to-right upwards hand-waving gesture to a camera_off command, that is to disable the currently enabled camera.
  • Other reference gestures and command mappings may be provided.
  • a first step 7.1 the gesture detecting application 140 is run.
  • the default, rear camera 105b is enabled or On'.
  • a third step 7.3 foreground objects received through the rear camera 105b are monitored.
  • a fourth step 7.4 if the motion of a foreground object is matched with one of the reference gestures in the command map 142, in a subsequent step 7.5, the corresponding camera command is retrieved. In this case, the switch_camera command is retrieved.
  • step 7.6 the gesture detection application 140 outputs the switch_camera command to the controller 106 which, in step 7.7, switches control to disable the rear camera 105b and enable the front camera 105a.
  • Figures 8a and 8b show an example of how the gesture detection application 140 can be advantageously employed.
  • the terminal 100 is shown running a proprietary presentation application which, in use, allows a user to generate slides and run a slideshow.
  • the terminal 100 is shown connected to a projector 170 for displaying the output 175' of the presentation application on a projector screen 174.
  • Certain types of terminal 100 include their own projector system, sometimes termed 'picoprojectors', for this purpose. Quite separate from the gesture recognition application 140, the
  • presentation application itself provides for gestural control of certain functions received through the front and rear cameras 105a, 105b, for example to advance forwards and return backwards through a series of slides.
  • Such control gestures need to be different from those employed by the gesture detection application 140 for controlling the cameras 105a, 105b.
  • the user When the user is making a presentation, they may initially enable the front camera 105a which is either the default camera or, if not, by waving their hand from right-to- left to cause the gesture detection application 140 to switch camera control from the rear camera 105b to the front camera. With the front camera 105a enabled, the user can operate the presentation application using the appropriate hand gestures to scroll through the slides. If at any time the user wishes to move in front of the terminal 100 to highlight something on the projector screen 174 by way of hand gestures, they will need to enable the rear camera 105b. Again, they may switch camera control using a right-to-left swipe gesture before the front camera 105a.
  • a further point to make is that, when one of the cameras 105a, 105b is enabled, there is a relatively large power consumption.
  • an enabled front camera 105a may typically run down a fully charged lOOOmAh battery in about an hour. So, the ability to switch the cameras 105a, 105b off when they are not needed is advantageous to save power and can be easily effected in the present embodiment using the relevant hand waving gesture.
  • the terminal 100 is connected to a holder on a car dashboard and the driver is using the front camera 105a to hold a hands-free conference call. If battery power is running low, the driver may wish to switch off the camera 105a and use voice-only
  • the driver avoids the need to locate and physically touch the relevant Off button terminal 100 by simply making the appropriate gesture in the camera's field-of-view.
  • Switching the front camera 105a back on may employ detection of a different gesture, perhaps based on motion, as will be introduced in the second embodiment described below.
  • the gesture detection application 140 is arranged to receive signals received not from the cameras 105a, 105b but from the accelerometers and/or gyroscopes 130 provided by the terminal 100.
  • accelerometers are able to detect and measure the amount and direction of acceleration as a vector quantity. They can also be used to measure orientation, although gyroscopes are better suited for this purpose.
  • either or both are employed to generate signals from which can be interpreted a gesture based on the sensed amount, direction and orientation of movement over a predetermined time frame, e.g. half a second.
  • these parameters are referred to collectively as motion parameters.
  • the command map 142 in this case stores a predetermined number of reference gestures which correspond to respective quantities of the motion parameters. Each reference gesture is mapped to a respective camera control command, as was the case for the first embodiment.
  • FIG. 9 there is shown the terminal 100 with dotted lines ⁇ , ⁇ , ⁇ respectively representing the principal three-dimensional axes of the terminal which are used by the accelerometers/gyroscopes 130 as reference axes. Also shown are arrows A,B,C representing respective orientation angles ⁇ , ⁇ , 6C of the reference axes ⁇ , ⁇ , ⁇ through which the terminal 100 can rotate in use. It will therefore be appreciated that, during movement, different values for amount, direction and orientation of movement can be stored against each of the three axes ⁇ , ⁇ , ⁇ to quantify and interpret a gesture.
  • movement corresponding to a wrist turnover action is quantified and stored as a reference gesture.
  • this gesture corresponds with a camera switch_camera command.
  • the reference gesture is shown pictorially, it will be appreciated that the above-mentioned motion parameters appropriate to a wrist-turnover motion will be stored, with a degree of tolerance allowed to account for appreciable differences in movement that will result from use by different people.
  • a first step 11.1 the gesture detecting application 140 is run.
  • a second step 11.2 the default, rear camera 105b, is enabled or On'.
  • the motion parameters received from the accelerometers/gyroscopes 130 are monitored.
  • a fourth step 11.4 if the motion parameters are matched with the reference gesture in the command map 142, in a subsequent step 11.5, the corresponding camera command is retrieved. In this case, the switch_camera command is retrieved.
  • the gesture detection application 140 outputs the switch_camera command to the controller 106 which, in step 11m g.7, switches control to disable the rear camera 105b and enable the front camera 105a.
  • the second embodiment avoids conflict problems that may arise in the first embodiment where both the gesture detection application 140 and a proprietary application use gestural information detected from one or more of the cameras 105a, 105b.
  • camera control is effected using a different set of movement sensors.
  • the rear camera 105b of a communications terminal will have a greater resolution and frame rate than that of the front camera 105a which is on the same side as the touch sensitive display 102. Therefore, use of the rear camera 105b may be preferred over the front camera 105a for certain tasks involving hand- movement detection, e.g. to control a proprietary application. Also, hybrid use of both front and rear cameras 105a, 105b may be preferred to differentiate between similar gestures or between basic and advanced gestures.
  • the gesture detection application 140 to switch between the cameras 105a, 105b, one might use the front camera 105a for handwave control of the image editing and image presentation views, and then switch to the rear camera 105b for thumbnail scrolling which is effected by the wrist turning action shown in Figure 9.
  • the devices described above provide for user control of one or more cameras through gestures independent of any touch-based interface, that is without the use of keys or a touch-screen. This means that application developers do not have to incorporate dedicated command buttons or icons into their GUI code to cater for touch-based camera control. Further, the or each camera can be controlled remotely from the terminal in certain situations.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Apparatus has at least one processor and at least one memory having computer-readable code stored thereon which when executed controls the at least one processor: to receive gestural data representing a user gesture made independently of any touch-based input interface of a device; to identify from the gestural data a corresponding camera command associated with the user gesture; and to output the identified camera command to control the at least one camera.

Description

Camera Control
Field of the Invention
This invention relates generally to camera control on a terminal, particularly using gestures received independently of a touch-based interface of the terminal.
Background to the Invention
It is commonplace for terminals, particularly mobile communications terminals, to comprise one or more cameras.
In the context of this application, a camera is assumed to mean a digital camera capable of generating image data representing a scene received by the camera's sensor. The image data can be used to capture still images using a single frame of image data or to record a succession of frames as video data.
It is known to use video data received by a camera to enable user control of applications running on a terminal. Applications store mappings relating
predetermined user gestures detected using the camera to one or more commands associated with the application. For example, a known photo-browsing application uses hand-waving gestures made in front of a terminal's front-facing camera to control how photographs are displayed on the user interface, a right-to-left gesture typically resulting in the application advancing through a sequence of photos.
Some terminals comprise both front- and rear-facing cameras. Prior art applications which run on the terminals enable switching between the cameras by providing a dedicated 'swap' icon provided as part of the application's graphical user interface (GUI) which requires the user to touch the button on the GUI.
Disadvantages exist in that developers have to incorporate a dedicated function and icon to effect touch-based control of the camera or cameras via a GUI, e.g. to enable/disable and/or swap between the front and rear cameras. Furthermore, the requirement for users to touch the interface can be problematic in situations where the user cannot hold or touch the terminal, for example when driving or giving a presentation, or where the user is using a rear-facing camera because this camera is on the opposite side to the touch-based interface. Summary of the Invention
A first aspect of the invention provides apparatus comprising a gesture recognition system configured to detect one or more predetermined user gestures independent of any touch-based interface and to control at least one camera in response to detecting the or each predetermined user gesture.
The apparatus may be configured to disable an enabled camera in response to detecting a predetermined user gesture. The apparatus may be configured to control first and second cameras, wherein the gesture recognition system is further configured to enable a currently-disabled camera in response to detecting the predetermined user gesture.
The gesture recognition system may be configured to receive video data from an enabled camera and to identify from the video data one or more predetermined user gestures. The gesture recognition system may be configured to identify, from the received video data, a gesture represented by a motion vector associated with a foreground object's change of position between subsequent frames of video data, and to compare said motion vector with a set of predetermined reference motion vector to identify a corresponding control command for the at least one camera.
The gesture recognition system may be configured to receive motion signals from a motion sensor and to identify therefrom one or more predetermined user gestures corresponding to said movement. The motion sensor may include at least one of an accelerometer and a gyroscope, the motion signal being generated based on at least one of a change in acceleration and a change in orientation of the apparatus.
The gesture control system may be configured to disable the display of video data from a currently selected camera in response to detection of a predetermined motion gesture and to enable the display of video data from the other, non-selected camera.
A second aspect of the invention provides apparatus, the apparatus having at least one processor and at least one memory having computer-readable code stored thereon which when executed controls the at least one processor:
to receive gestural data representing a user gesture made independently of any touch-based input interface of a device;
to identify from the gestural data a corresponding camera command associated with the user gesture; and to output the identified camera command to control the at least one camera.
A third aspect of the invention provides a method comprising:
receiving gestural data representing a user gesture made independently of any touch-based input interface of a device;
identifying from the gestural data a corresponding camera command associated with the user gesture; and
outputting the identified camera command to control the at least one camera.
The outputted command may be configured to disable a currently-enabled camera. The outputted command may be configured to enable a currently-disabled camera.
Receiving gestural data may comprise receiving video data from the at least one camera and identifying from the video data one or more predetermined user gestures. Receiving gestural data may further comprise identifying a motion vector associated with a foreground object's change of position between subsequent frames of video data, and comparing said motion vector with a set of predetermined reference motion vectors to identify a corresponding control command for the or each camera.
Receiving gestural data may comprises receiving a signal from a motion sensor provided on the device, the signal being representative of movement of the device, and identifying therefrom one or more predetermined user gestures corresponding to the sensed movement. The signal may be received from at least one of an
accelerometer and gyroscope, the signal being generated based on at least one of a change in acceleration and a change in orientation of the device.
The method may comprise, in response to detection of a predetermined motion gesture, disabling display of video data from a currently selected camera and enabling the display of video data from a non-selected camera.
Another aspect provides a computer program comprising instructions that when executed by a computer apparatus control it to perform any method above.
Another aspect provides a portable device comprising any of the apparatus above.
A further aspect of the invention provides a non-transitory computer-readable storage medium having stored thereon computer-readable code, which, when executed by computing apparatus, causes the computing apparatus to perform a method comprising:
receiving gestural data representing a user gesture made independently of any touch-based input interface of a device;
identifying from the gestural data a corresponding camera command associated with the user gesture; and
outputting the identified camera command to control the at least one camera.
Brief Description of the Drawings
Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
Figure 1 is a perspective view of a mobile terminal embodying aspects of the invention;
Figure 2 is a schematic diagram illustrating components of the Figure l mobile terminal and their interconnection;
Figure 3 is a schematic diagram illustrating certain components shown in Figure 2 relevant to operation of a gesture recognition system of the invention;
Figure 4 is a flow diagram indicating the generalised processing steps performed by the gesture recognition system shown in Figure 3;
Figure 5 is a perspective view of the mobile terminal shown in Figure 1 which is useful for understanding a first embodiment;
Figure 6 shows a look-up-table employed by the gesture recognition system in the first embodiment;
Figure 7 is a flow diagram indicating the processing steps performed by the gesture recognition system in the first embodiment;
Figures 8a and 8b are perspective views of the mobile terminal shown in Figure 1 employed in use according to the first embodiment;
Figure 9 is a perspective view of the mobile terminal shown in Figure 1 which is useful for understanding a second embodiment;
Figure 10 shows a look-up-table employed by the gesture recognition system in the second embodiment; and
Figure 11 is a flow diagram indicating the processing steps performed by the gesture recognition system in the second embodiment. Detailed Description of Preferred Embodiments
Referring firstly to Figure l, a terminal 100 is shown. The exterior of the terminal 100 has a touch sensitive display 102, hardware keys 104, front and rear cameras 105a, 105b, a speaker 118 and a headphone port 120.
The front camera 105a is provided on a first side of the terminal 100, that is the same side as the touch sensitive display 102. The rear camera 105b is provided on the opposite side of the terminal.
Figure 2 shows a schematic diagram of the components of terminal 100. The terminal 100 has a controller 106, a touch sensitive display 102 comprised of a display part 108 and a tactile interface part 110, the hardware keys 104, the front and rear cameras 105a, 105b, a memory 112, RAM 114, a speaker 118, the headphone port 120, a wireless communication module 122, an antenna 124, motion sensors in the form of a set of accelerometers and gyroscopes 130, and a battery 116. The controller 106 is connected to each of the other components (except the battery 116) in order to control operation thereof.
The memory 112 may be a non-volatile memory such as read only memory (ROM) a hard disk drive (HDD) or a solid state drive (SSD). The memory 112 stores, amongst other things, an operating system 126 and may store software applications 128. The RAM 114 is used by the controller 106 for the temporary storage of data. The operating system 126 may contain code which, when executed by the controller 106 in conjunction with RAM 114, controls operation of each of the hardware components of the terminal.
The controller 106 may take any suitable form. For instance, it may be a
microcontroller, plural microcontrollers, a processor, or plural processors.
The terminal 100 may be a mobile telephone or a smartphone, a personal digital assistant (PDA), a portable media player (PMP), a portable computer or any other device capable of running software applications and providing audio outputs. In some embodiments, the terminal 100 may engage in cellular communications using the wireless communications module 122 and the antenna 124. The wireless communications module 122 may be configured to communicate via several protocols such as GSM (Global System for Mobiles), CDMA (code division multiple access), UMTS (universal mobile telephone system), Bluetooth and IEEE 802.11 (Wi-Fi). The display part 108 of the touch sensitive display 102 is for displaying images and text to users of the terminal and the tactile interface part 110 is for receiving touch inputs from users.
As well as storing the operating system 126 and software applications 128, the memory 112 may also store multimedia files such as music and video files. A wide variety of software applications 128 may be installed on the terminal including web browsers, radio and music players, games and utility applications. Some or all of the software applications stored on the terminal may provide audio outputs. The audio provided by the applications may be converted into sound by the speaker(s) 118 of the terminal or, if headphones or speakers have been connected to the headphone port 120, by the headphones or speakers connected to the headphone port 120.
In some embodiments the terminal 100 may also be associated with external software application not stored on the terminal. These may be applications stored on a remote server device and may run partly or exclusively on the remote server device. These applications can be termed cloud-hosted applications. The terminal 100 may be in communication with the remote server device in order to utilise the software application stored there. This may include receiving audio outputs provided by the external software application.
In some embodiments, the hardware keys 104 are dedicated volume control keys or switches. The hardware keys may for example comprise two adjacent keys, a single rocker switch or a rotary dial. In some embodiments, the hardware keys 104 are located on the side of the terminal 100.
Figure 3 shows a schematic diagram of certain components of the terminal 100 relevant to embodiments described herein. Stored on the memory 112 is a dedicated application 140, hereafter referred to as 'the gesture detection application'. The gesture detection application is associated with operation of the front and rear cameras 105a, 105b independent of the touch sensitive display 102. The gesture detection application 140 may be provided as an integral part of the terminal's operating system 126 or as a separate plug-in module to the operating system. The gesture detection application 140 is associated with a gesture-to-command map 142, hereafter 'command map' which is a database storing a look up table (LUT) which corresponds one or more predefined reference gestures received through sensors of the terminal 100 to operating commands associated with the front and rear cameras 105a, 105b.
Specifically, the command map 142 stores one or more commands, which, when executed by the controller, causes switching of one or both cameras 105a, 105b between enabled and disabled modes, as well as swapping control between the cameras so that when one camera is enabled, the other is disabled. In this sense, enabling a particular one of the cameras 105a, 105b means making the controller 106 configured to receive image or video data from the enabled camera for output to the display 108 and also to enable capture of the transferred image or video data using a camera application (not shown) handling aspects such as zoom, capture and storage on the memory 112.
As will be described in greater detail below, the gesture detection application 140 identifies gestures from, in a first embodiment, either of the front and rear cameras 105a, 105b and, in a second embodiment, the motion sensing accelerometers and/or gyroscopes 130. It will therefore be appreciated that camera control can be achieved independently of the touch sensitive display 120 and indeed of other hard keys provided on the terminal 100.
Referring to Figure 4, the general operating steps performed by the gesture detecting application 140 are as follows. In a first step 4.1, the gesture detecting application 140 is run, and in a second step 4.2 a first one of the cameras 105a, 105b, as a default camera, is enabled. In a third step 4.3, gestures received through one or more sensors of the terminal 100 operating independently of the touch sensitive display 120 are monitored. In a subsequent step 4.4, if a received gesture is matched with a reference gesture stored in the command map 142, it is mapped to its associated command in step 4.5 which is then executed in step 4.6 by the controller 4.6 to perform a predetermined camera function.
A first embodiment will now be described in greater detail with reference to Figures 5 to 8. In this embodiment, the front and rear cameras 105a, 105b are used to detect gestures received through an enabled one of the cameras, the gestures being in the form of hand movements. Hand movements are converted to image or, more particularly, video data for comparison with reference gestures stored in the command map 142. Referring to Figure 5, the terminal 100 is shown with the rear camera 105b, in this case the default camera, enabled. Dotted lines indicate a rectangular field-of-view 160 representing the spatial area covered by the sensor of the rear camera 105b. User gestures for controlling aspects of the camera's operation, through the gesture detection application 140, are in the form of hand waving movements 162. Any one of the many known video processing methods for detecting and quantifying motion in a digital camera's field-of-view can be employed. One example includes periodically establishing a background image for the frame based on predominately static pixel luminance values and thereafter detecting a foreground object based on detecting pixel values above a predetermined threshold compared with the background image. Alternative methods can employ foreground object detecting algorithms that do not require a background image to be established. The foreground object can thereafter be quantified and tracked in terms of its motion, e.g. as an inter-frame motion vector, to represent a gesture.
Referring to Figure 6, a schematic representation of a command map 142 is shown. Here, a plurality of reference gestures, which in practice correspond to different foreground object motion vectors, are shown together with their corresponding camera commands. A first reference gesture #1 maps video data representative of a left-to-right hand-waving gesture to a camera_switch command, that is to alternate controller control between the front and rear cameras 105a, 105b. A second reference gesture #2 maps video data representative of a left-to-right upwards hand-waving gesture to a camera_off command, that is to disable the currently enabled camera. Other reference gestures and command mappings may be provided.
Referring to Figure 7, the operating steps performed by the gesture detection application 140 in accordance with the first embodiment are indicated. In a first step 7.1, the gesture detecting application 140 is run. In a second step 7.2, the default, rear camera 105b, is enabled or On'. In a third step 7.3, foreground objects received through the rear camera 105b are monitored. In a fourth step 7.4, if the motion of a foreground object is matched with one of the reference gestures in the command map 142, in a subsequent step 7.5, the corresponding camera command is retrieved. In this case, the switch_camera command is retrieved. In step 7.6, the gesture detection application 140 outputs the switch_camera command to the controller 106 which, in step 7.7, switches control to disable the rear camera 105b and enable the front camera 105a. Figures 8a and 8b show an example of how the gesture detection application 140 can be advantageously employed.
Referring to Figure 8a, the terminal 100 is shown running a proprietary presentation application which, in use, allows a user to generate slides and run a slideshow. The terminal 100 is shown connected to a projector 170 for displaying the output 175' of the presentation application on a projector screen 174. Certain types of terminal 100 include their own projector system, sometimes termed 'picoprojectors', for this purpose. Quite separate from the gesture recognition application 140, the
presentation application itself provides for gestural control of certain functions received through the front and rear cameras 105a, 105b, for example to advance forwards and return backwards through a series of slides. Such control gestures, for obvious reasons, need to be different from those employed by the gesture detection application 140 for controlling the cameras 105a, 105b.
When the user is making a presentation, they may initially enable the front camera 105a which is either the default camera or, if not, by waving their hand from right-to- left to cause the gesture detection application 140 to switch camera control from the rear camera 105b to the front camera. With the front camera 105a enabled, the user can operate the presentation application using the appropriate hand gestures to scroll through the slides. If at any time the user wishes to move in front of the terminal 100 to highlight something on the projector screen 174 by way of hand gestures, they will need to enable the rear camera 105b. Again, they may switch camera control using a right-to-left swipe gesture before the front camera 105a.
Referring to Figure 8b, when behind the terminal 100, the user's hand is captured with the rear camera's field of view and gestures are again monitored by both the gesture recognition algorithms being employed by the gestural detection application 128 and presentation applications. In this case, a pointing finger gesture is received through the rear camera 105b and detected by the presentation application which causes a pointer 178 to be projected over the slide onto the projector screen 174, substantially in alignment with the finger. The pointer 178 thereafter tracks movement of the finger over the displayed slide. When the user wishes to revert back to the front camera 105a, a left-to-right swipe gesture is made in the field-of- view of the rear camera 105b. This usage example demonstrates a further advantage in being able to control one or both cameras 105a, 105b remotely of the terminal 100 in that the user avoids disturbing the position of the terminal which should remain stationary in use;
otherwise the terminal will need to be re-aligned with the projector screen 174.
A further point to make is that, when one of the cameras 105a, 105b is enabled, there is a relatively large power consumption. In a typical device, an enabled front camera 105a may typically run down a fully charged lOOOmAh battery in about an hour. So, the ability to switch the cameras 105a, 105b off when they are not needed is advantageous to save power and can be easily effected in the present embodiment using the relevant hand waving gesture. Consider the situation where the terminal 100 is connected to a holder on a car dashboard and the driver is using the front camera 105a to hold a hands-free conference call. If battery power is running low, the driver may wish to switch off the camera 105a and use voice-only
communications. The driver avoids the need to locate and physically touch the relevant Off button terminal 100 by simply making the appropriate gesture in the camera's field-of-view. Switching the front camera 105a back on may employ detection of a different gesture, perhaps based on motion, as will be introduced in the second embodiment described below.
A second embodiment will now be described with reference to Figures 9 to 11. Here, the gesture detection application 140 is arranged to receive signals received not from the cameras 105a, 105b but from the accelerometers and/or gyroscopes 130 provided by the terminal 100. As will be appreciated, accelerometers are able to detect and measure the amount and direction of acceleration as a vector quantity. They can also be used to measure orientation, although gyroscopes are better suited for this purpose. In this embodiment, either or both are employed to generate signals from which can be interpreted a gesture based on the sensed amount, direction and orientation of movement over a predetermined time frame, e.g. half a second. For ease of explanation, these parameters are referred to collectively as motion parameters. The command map 142 in this case stores a predetermined number of reference gestures which correspond to respective quantities of the motion parameters. Each reference gesture is mapped to a respective camera control command, as was the case for the first embodiment.
Referring to Figure 9, there is shown the terminal 100 with dotted lines Χ,Υ,Ζ respectively representing the principal three-dimensional axes of the terminal which are used by the accelerometers/gyroscopes 130 as reference axes. Also shown are arrows A,B,C representing respective orientation angles ΘΑ, ΘΒ, 6C of the reference axes Χ,Υ,Ζ through which the terminal 100 can rotate in use. It will therefore be appreciated that, during movement, different values for amount, direction and orientation of movement can be stored against each of the three axes Χ,Υ,Ζ to quantify and interpret a gesture.
In the present use example, movement corresponding to a wrist turnover action, indicated in Figure 9, is quantified and stored as a reference gesture. Referring to Figure 10, which shows the command map 142, this gesture corresponds with a camera switch_camera command. Although the reference gesture is shown pictorially, it will be appreciated that the above-mentioned motion parameters appropriate to a wrist-turnover motion will be stored, with a degree of tolerance allowed to account for appreciable differences in movement that will result from use by different people.
Referring to Figure 11, the operating steps performed by the gesture detection application 140 in accordance with the second embodiment are indicated. In a first step 11.1, the gesture detecting application 140 is run. In a second step 11.2, the default, rear camera 105b, is enabled or On'. In a third step 11.3, the motion parameters received from the accelerometers/gyroscopes 130 are monitored. In a fourth step 11.4, if the motion parameters are matched with the reference gesture in the command map 142, in a subsequent step 11.5, the corresponding camera command is retrieved. In this case, the switch_camera command is retrieved. In step 11.6, the gesture detection application 140 outputs the switch_camera command to the controller 106 which, in step 11m g.7, switches control to disable the rear camera 105b and enable the front camera 105a.
In general, the second embodiment avoids conflict problems that may arise in the first embodiment where both the gesture detection application 140 and a proprietary application use gestural information detected from one or more of the cameras 105a, 105b. Here, camera control is effected using a different set of movement sensors.
A further practical use of the second embodiment will now be described. It will be appreciated that, in general, the rear camera 105b of a communications terminal will have a greater resolution and frame rate than that of the front camera 105a which is on the same side as the touch sensitive display 102. Therefore, use of the rear camera 105b may be preferred over the front camera 105a for certain tasks involving hand- movement detection, e.g. to control a proprietary application. Also, hybrid use of both front and rear cameras 105a, 105b may be preferred to differentiate between similar gestures or between basic and advanced gestures. Therefore, using a wrist turning action, as indicated in Figure 9, to effect switching between the front and rear cameras 105a, 105b offers advantages where the user is holding the terminal 100 and does not necessarily need to see the touch sensitive display 120. Taking the example of a proprietary application for viewing an image gallery, there may be provided three views, namely a thumbnail view, an image editing view and an image presentation view. Using only the front camera 105a for detecting both left-to-right and up-to- down handwaving gestures may be technically difficult in terms of differentiation given its more limited resolution and frame rate. Hence, by using the gesture detection application 140 to switch between the cameras 105a, 105b, one might use the front camera 105a for handwave control of the image editing and image presentation views, and then switch to the rear camera 105b for thumbnail scrolling which is effected by the wrist turning action shown in Figure 9.
Using both cameras 105a, 105b in this way will achieve greater recognition accuracy than just one of the cameras, particularly for handwaving or 'hovering' recognition applications.
It will be seen that the devices described above provide for user control of one or more cameras through gestures independent of any touch-based interface, that is without the use of keys or a touch-screen. This means that application developers do not have to incorporate dedicated command buttons or icons into their GUI code to cater for touch-based camera control. Further, the or each camera can be controlled remotely from the terminal in certain situations.
It will be appreciated that the above described embodiments are purely illustrative and are not limiting on the scope of the invention. Other variations and
modifications will be apparent to persons skilled in the art upon reading the present application.
Moreover, the disclosure of the present application should be understood to include any novel features or any novel combination of features either explicitly or implicitly disclosed herein or any generalization thereof and during the prosecution of the present application or of any application derived therefrom, new claims may be formulated to cover any such features and/or combination of such features.

Claims

Claims
1. Apparatus comprising a gesture recognition system configured to detect one or more predetermined user gestures independent of any touch-based interface and to control at least one camera in response to detecting the or each predetermined user gesture.
2. Apparatus according to claim l, configured to disable an enabled camera in response to detecting a predetermined user gesture.
3. Apparatus according to claim 2, configured to control first and second cameras, wherein the gesture recognition system is further configured to enable a currently-disabled camera in response to detecting the predetermined user gesture.
4. Apparatus according to any preceding claim, wherein the gesture recognition system is configured to receive video data from an enabled camera and to identify from the video data one or more predetermined user gestures.
5. Apparatus according to claim 4, wherein the gesture recognition system is configured to identify, from the received video data, a gesture represented by a motion vector associated with a foreground object's change of position between subsequent frames of video data, and to compare said motion vector with a set of predetermined reference motion vector to identify a corresponding control command for the at least one camera.
6. Apparatus according to any of claims 1 to 3, wherein the gesture recognition system is configured to receive motion signals from a motion sensor and to identify therefrom one or more predetermined user gestures corresponding to said movement.
7. Apparatus according to claim 6, wherein the motion sensor includes at least one of an accelerometer and a gyroscope, the motion signal being generated based on at least one of a change in acceleration and a change in orientation of the apparatus.
8. Apparatus as claimed in any preceding claim, wherein the gesture control system is configured to disable the display of video data from a currently selected camera in response to detection of a predetermined motion gesture and to enable the display of video data from the other, non-selected camera.
9. Apparatus, the apparatus having at least one processor and at least one memory having computer-readable code stored thereon which when executed controls the at least one processor:
to receive gestural data representing a user gesture made independently of any touch-based input interface of a device;
to identify from the gestural data a corresponding camera command associated with the user gesture; and
to output the identified camera command to control the at least one camera.
10. A method comprising:
receiving gestural data representing a user gesture made independently of any touch-based input interface of a device;
identifying from the gestural data a corresponding camera command associated with the user gesture; and
outputting the identified camera command to control the at least one camera.
11. A method according to claim 10, wherein the outputted command is configured to disable a currently-enabled camera.
12. A method according to claim 11, wherein the outputted command is configured to enable a currently-disabled camera.
13. A method according to any one of claims 10 to 12, wherein receiving gestural data comprises receiving video data from the at least one camera and identifying from the video data one or more predetermined user gestures.
14. A method according to claim 13, wherein receiving gestural data further comprises identifying a motion vector associated with a foreground object's change of position between subsequent frames of video data, and comparing said motion vector with a set of predetermined reference motion vectors to identify a corresponding control command for the or each camera.
15. A method according to any of claims 10 to 12, wherein receiving gestural data comprises receiving a signal from a motion sensor provided on the device, the signal being representative of movement of the device, and identifying therefrom one or more predetermined user gestures corresponding to the sensed movement.
16. A method according to claim 15, wherein the signal is received from at least one of an accelerometer and gyroscope, the signal being generated based on at least one of a change in acceleration and a change in orientation of the device.
17. A method according to any of claims 10 to 16, comprising, in response to detection of a predetermined motion gesture, disabling display of video data from a currently selected camera and enabling the display of video data from a non-selected camera.
18. A computer program comprising instructions that when executed by a computer apparatus control it to perform the method of any one of claims 10 to 17.
19. A portable device comprising apparatus as claimed in any of claims 1 to 10.
20. A non-transitory computer-readable storage medium having stored thereon computer-readable code, which, when executed by computing apparatus, causes the computing apparatus to perform a method comprising: receiving gestural data representing a user gesture made independently of any touch-based input interface of a device;
identifying from the gestural data a corresponding camera command associated with the user gesture; and
outputting the identified camera command to control the at least one camera.
21. A non-transitory computer-readable storage medium according to claim
20, wherein the computer-readable code when executed by the computing apparatus causes the computing apparatus to perform outputting the identified camera command to disable a currently-enabled camera.
22. A non-transitory computer-readable storage medium according to claim
21, wherein the computer-readable code when executed by the computing apparatus causes the computing apparatus to perform outputting the identified camera command to enable a currently-disabled camera.
PCT/IB2012/052150 2011-05-06 2012-04-30 Camera control Ceased WO2012153228A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US13/102,671 2011-05-06
US13/102,671 US20120281129A1 (en) 2011-05-06 2011-05-06 Camera control

Publications (1)

Publication Number Publication Date
WO2012153228A1 true WO2012153228A1 (en) 2012-11-15

Family

ID=47090001

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2012/052150 Ceased WO2012153228A1 (en) 2011-05-06 2012-04-30 Camera control

Country Status (2)

Country Link
US (1) US20120281129A1 (en)
WO (1) WO2012153228A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9912853B2 (en) 2014-07-31 2018-03-06 Microsoft Technology Licensing, Llc Switching between cameras of an electronic device
CN108430819A (en) * 2015-12-22 2018-08-21 歌乐株式会社 Vehicle device

Families Citing this family (83)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120326966A1 (en) * 2011-06-21 2012-12-27 Qualcomm Incorporated Gesture-controlled technique to expand interaction radius in computer vision applications
US10088924B1 (en) 2011-08-04 2018-10-02 Amazon Technologies, Inc. Overcoming motion effects in gesture recognition
KR101189633B1 (en) * 2011-08-22 2012-10-10 성균관대학교산학협력단 A method for recognizing ponter control commands based on finger motions on the mobile device and a mobile device which controls ponter based on finger motions
US9223415B1 (en) * 2012-01-17 2015-12-29 Amazon Technologies, Inc. Managing resource usage for task performance
US9075566B2 (en) 2012-03-02 2015-07-07 Microsoft Technoogy Licensing, LLC Flexible hinge spine
US9870066B2 (en) 2012-03-02 2018-01-16 Microsoft Technology Licensing, Llc Method of manufacturing an input device
US9134807B2 (en) 2012-03-02 2015-09-15 Microsoft Technology Licensing, Llc Pressure sensitive key normalization
JP5970937B2 (en) * 2012-04-25 2016-08-17 ソニー株式会社 Display control apparatus and display control method
KR101858604B1 (en) * 2012-04-30 2018-05-17 엘지전자 주식회사 Mobile terminal and control method thereof
US20130300590A1 (en) 2012-05-14 2013-11-14 Paul Henry Dietz Audio Feedback
US9438805B2 (en) * 2012-06-08 2016-09-06 Sony Corporation Terminal device and image capturing method
US9646200B2 (en) * 2012-06-08 2017-05-09 Qualcomm Incorporated Fast pose detector
US8957973B2 (en) 2012-06-11 2015-02-17 Omnivision Technologies, Inc. Shutter release using secondary camera
JP2013257686A (en) * 2012-06-12 2013-12-26 Sony Corp Projection type image display apparatus, image projecting method, and computer program
US20130335587A1 (en) * 2012-06-14 2013-12-19 Sony Mobile Communications, Inc. Terminal device and image capturing method
SE537553C2 (en) * 2012-08-03 2015-06-09 Crunchfish Ab Improved identification of a gesture
US9207759B1 (en) * 2012-10-08 2015-12-08 Edge3 Technologies, Inc. Method and apparatus for generating depth map from monochrome microlens and imager arrays
US8786767B2 (en) * 2012-11-02 2014-07-22 Microsoft Corporation Rapid synchronized lighting and shuttering
US9270889B2 (en) * 2013-01-25 2016-02-23 Htc Corporation Electronic device and camera switching method thereof
US9661221B2 (en) * 2013-03-15 2017-05-23 Qualcomm Incorporated Always-on camera sampling strategies
US10439972B1 (en) 2013-05-30 2019-10-08 Snap Inc. Apparatus and method for maintaining a message thread with opt-in permanence for entries
US9742713B2 (en) 2013-05-30 2017-08-22 Snap Inc. Apparatus and method for maintaining a message thread with opt-in permanence for entries
US9705831B2 (en) 2013-05-30 2017-07-11 Snap Inc. Apparatus and method for maintaining a message thread with opt-in permanence for entries
US9706034B2 (en) * 2013-06-03 2017-07-11 Yulong Computer Telecommunication Scientific (Shenzhen) Co., Ltd. Terminal and image file processing method
AU2014100583B4 (en) * 2013-06-09 2015-02-19 Apple Inc. Device, method, and graphical user interface for switching between camera interfaces
EP2814234A1 (en) * 2013-06-11 2014-12-17 Nokia Corporation Apparatus for controlling camera modes and associated methods
US9367137B2 (en) 2013-07-01 2016-06-14 Blackberry Limited Alarm operation by touch-less gesture
US9256290B2 (en) 2013-07-01 2016-02-09 Blackberry Limited Gesture detection using ambient light sensors
US9423913B2 (en) 2013-07-01 2016-08-23 Blackberry Limited Performance control of ambient light sensors
US9398221B2 (en) * 2013-07-01 2016-07-19 Blackberry Limited Camera control using ambient light sensors
US9489051B2 (en) 2013-07-01 2016-11-08 Blackberry Limited Display navigation using touch-less gestures
US9323336B2 (en) 2013-07-01 2016-04-26 Blackberry Limited Gesture detection using ambient light sensors
US9342671B2 (en) 2013-07-01 2016-05-17 Blackberry Limited Password by touch-less gesture
US9405461B2 (en) 2013-07-09 2016-08-02 Blackberry Limited Operating a device using touchless and touchscreen gestures
US20160162039A1 (en) * 2013-07-21 2016-06-09 Pointgrab Ltd. Method and system for touchless activation of a device
US9465448B2 (en) 2013-07-24 2016-10-11 Blackberry Limited Backlight for touchless gesture detection
US9304596B2 (en) 2013-07-24 2016-04-05 Blackberry Limited Backlight for touchless gesture detection
KR102047703B1 (en) * 2013-08-09 2019-11-22 엘지전자 주식회사 Mobile terminal and controlling method thereof
US9194741B2 (en) 2013-09-06 2015-11-24 Blackberry Limited Device having light intensity measurement in presence of shadows
CN105531623B (en) 2013-09-12 2018-08-14 麦克赛尔株式会社 Image recording device and storage medium
US9628699B2 (en) * 2013-11-29 2017-04-18 Intel Corporation Controlling a camera with face detection
CN104735340A (en) * 2013-12-24 2015-06-24 索尼公司 Spare camera function control
US9294666B2 (en) * 2013-12-27 2016-03-22 Panasonic Intellectual Property Corporation Of America Communication method
US9560254B2 (en) 2013-12-30 2017-01-31 Google Technology Holdings LLC Method and apparatus for activating a hardware feature of an electronic device
US9282283B2 (en) 2014-01-29 2016-03-08 Microsoft Technology Licensing, Llc Detecting patterns traced on a screen of a user device
US10082926B1 (en) * 2014-02-21 2018-09-25 Snap Inc. Apparatus and method for alternate channel communication initiated through a common message thread
WO2015131157A1 (en) * 2014-02-28 2015-09-03 Vikas Gupta Gesture operated wrist mounted camera system
US9749585B2 (en) 2014-03-17 2017-08-29 Microsoft Technology Licensing, Llc Highlighting unread messages
US10284813B2 (en) 2014-03-17 2019-05-07 Microsoft Technology Licensing, Llc Automatic camera selection
US9888207B2 (en) * 2014-03-17 2018-02-06 Microsoft Technology Licensing, Llc Automatic camera selection
US10178346B2 (en) 2014-03-17 2019-01-08 Microsoft Technology Licensing, Llc Highlighting unread messages
EP2942936B1 (en) 2014-05-06 2017-12-20 Nokia Technologies OY Zoom input and camera selection
US9276886B1 (en) 2014-05-09 2016-03-01 Snapchat, Inc. Apparatus and method for dynamically configuring application component tiles
US9916010B2 (en) * 2014-05-16 2018-03-13 Visa International Service Association Gesture recognition cloud command platform, system, method, and apparatus
US9537811B2 (en) 2014-10-02 2017-01-03 Snap Inc. Ephemeral gallery of ephemeral messages
US9396354B1 (en) 2014-05-28 2016-07-19 Snapchat, Inc. Apparatus and method for automated privacy protection in distributed images
US9113301B1 (en) 2014-06-13 2015-08-18 Snapchat, Inc. Geo-location based event gallery
US9225897B1 (en) 2014-07-07 2015-12-29 Snapchat, Inc. Apparatus and method for supplying content aware photo filters
US10055717B1 (en) 2014-08-22 2018-08-21 Snap Inc. Message processor with application prompts
KR102294599B1 (en) * 2014-09-02 2021-08-27 엘지전자 주식회사 Display device and controlling method thereof
US10824654B2 (en) 2014-09-18 2020-11-03 Snap Inc. Geolocation-based pictographs
US11216869B2 (en) 2014-09-23 2022-01-04 Snap Inc. User interface to augment an image using geolocation
US10284508B1 (en) 2014-10-02 2019-05-07 Snap Inc. Ephemeral gallery of ephemeral messages with opt-in permanence
US9385983B1 (en) 2014-12-19 2016-07-05 Snapchat, Inc. Gallery of messages from individuals with a shared interest
US10311916B2 (en) 2014-12-19 2019-06-04 Snap Inc. Gallery of videos set to an audio time line
US10133705B1 (en) 2015-01-19 2018-11-20 Snap Inc. Multichannel system
KR102662169B1 (en) 2015-03-18 2024-05-03 스냅 인코포레이티드 Geo-fence authorization provisioning
US10135949B1 (en) 2015-05-05 2018-11-20 Snap Inc. Systems and methods for story and sub-story navigation
US20160360118A1 (en) * 2015-06-04 2016-12-08 String Theory, Inc. Smartphone camera user interface
DE102015118571A1 (en) * 2015-10-30 2017-05-04 Kraussmaffei Technologies Gmbh Plastic injection molding machine with at least one camera
CN105824406A (en) * 2015-11-30 2016-08-03 维沃移动通信有限公司 Photographing method and terminal
US10354425B2 (en) 2015-12-18 2019-07-16 Snap Inc. Method and system for providing context relevant media augmentation
US9871962B2 (en) * 2016-03-04 2018-01-16 RollCall, LLC Movable user interface shutter button for camera
US9832368B1 (en) * 2016-05-31 2017-11-28 Motorola Mobility Llc Managing unintended camera clicks
CN106648102A (en) * 2016-12-26 2017-05-10 珠海市魅族科技有限公司 Method and system of controlling terminal equipment through non-touch gesture
US10581782B2 (en) 2017-03-27 2020-03-03 Snap Inc. Generating a stitched data stream
US10582277B2 (en) 2017-03-27 2020-03-03 Snap Inc. Generating a stitched data stream
US10880496B1 (en) 2019-12-30 2020-12-29 Snap Inc. Including video feed in message thread
US11756081B2 (en) * 2020-06-12 2023-09-12 International Business Machines Corporation Rendering privacy aware advertisements in mixed reality space
CN115484380B (en) 2021-06-16 2024-07-19 荣耀终端有限公司 Shooting method, graphical user interface and electronic device
CN115643485B (en) * 2021-11-25 2023-10-24 荣耀终端有限公司 Shooting method and electronic equipment
CN116193243B (en) 2021-11-25 2024-03-22 荣耀终端有限公司 Shooting method and electronic equipment
CN117692757A (en) * 2022-08-19 2024-03-12 英特尔公司 Automatic user adaptive triggering

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070004451A1 (en) * 2005-06-30 2007-01-04 C Anderson Eric Controlling functions of a handheld multifunction device
US20080089587A1 (en) * 2006-10-11 2008-04-17 Samsung Electronics Co.; Ltd Hand gesture recognition input system and method for a mobile phone
WO2008107733A1 (en) * 2007-03-07 2008-09-12 Sony Ericsson Mobile Communications Ab Method and system for a self timer function for a camera and camera equipped mobile radio terminal
EP1983730A1 (en) * 2007-04-18 2008-10-22 Samsung Electronics Co., Ltd. Portable electronic device adapted to change operation mode
US20090303176A1 (en) * 2008-06-10 2009-12-10 Mediatek Inc. Methods and systems for controlling electronic devices according to signals from digital camera and sensor modules
US20110076003A1 (en) * 2009-09-30 2011-03-31 Lg Electronics Inc. Mobile terminal and method of controlling the operation of the mobile terminal

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7433494B2 (en) * 2002-09-19 2008-10-07 Denso Corporation Moving body detecting apparatus
US8344325B2 (en) * 2009-05-22 2013-01-01 Motorola Mobility Llc Electronic device with sensing assembly and method for detecting basic gestures
TWI395483B (en) * 2009-05-25 2013-05-01 Visionatics Inc Motion object detection method using adaptive background model and computer program product thereof
KR20110010906A (en) * 2009-07-27 2011-02-08 삼성전자주식회사 Method and device for controlling electronic devices using user interaction
US20120242793A1 (en) * 2011-03-21 2012-09-27 Soungmin Im Display device and method of controlling the same

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070004451A1 (en) * 2005-06-30 2007-01-04 C Anderson Eric Controlling functions of a handheld multifunction device
US20080089587A1 (en) * 2006-10-11 2008-04-17 Samsung Electronics Co.; Ltd Hand gesture recognition input system and method for a mobile phone
WO2008107733A1 (en) * 2007-03-07 2008-09-12 Sony Ericsson Mobile Communications Ab Method and system for a self timer function for a camera and camera equipped mobile radio terminal
EP1983730A1 (en) * 2007-04-18 2008-10-22 Samsung Electronics Co., Ltd. Portable electronic device adapted to change operation mode
US20090303176A1 (en) * 2008-06-10 2009-12-10 Mediatek Inc. Methods and systems for controlling electronic devices according to signals from digital camera and sensor modules
US20110076003A1 (en) * 2009-09-30 2011-03-31 Lg Electronics Inc. Mobile terminal and method of controlling the operation of the mobile terminal

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9912853B2 (en) 2014-07-31 2018-03-06 Microsoft Technology Licensing, Llc Switching between cameras of an electronic device
CN108430819A (en) * 2015-12-22 2018-08-21 歌乐株式会社 Vehicle device

Also Published As

Publication number Publication date
US20120281129A1 (en) 2012-11-08

Similar Documents

Publication Publication Date Title
US20120281129A1 (en) Camera control
US9389691B2 (en) Devices and methods for establishing a communicative coupling in response to a gesture
US9338116B2 (en) Device and method for displaying and interacting with display objects
US10122917B2 (en) Systems and methods for capturing images from a lock screen
US12321545B2 (en) Device and method for processing user input
US9582168B2 (en) Apparatus, method and computer readable recording medium for displaying thumbnail image of panoramic photo
CA2849616C (en) Device and method for generating data for generating or modifying a display object
US20120326994A1 (en) Information processing apparatus, information processing method and program
CA2857232C (en) Actionable user input on displayed items
US20150063785A1 (en) Method of overlappingly displaying visual object on video, storage medium, and electronic device
KR102904536B1 (en) Electronic device and method for sharing screens
US9756251B2 (en) Digital device and method of controlling therefor
CA2854759A1 (en) Devices and methods for establishing a communicative coupling in response to a gesture
US20190310702A1 (en) Electronic device, control method for electronic device, and non-transitory computer readable medium
WO2013164351A1 (en) Device and method for processing user input
EP2660695B1 (en) Device and method for processing user input
CA2855162C (en) Device and method for displaying and interacting with display objects

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12783014

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12783014

Country of ref document: EP

Kind code of ref document: A1